Default mode

If nghttpx is invoked without--http2-proxy, it operates indefault mode. In this mode, it works as reverse proxy (gateway) forHTTP/3, HTTP/2 and HTTP/1 clients to backend servers. This is alsoknown as "HTTP/2 router".

By default, frontend connection is encrypted using SSL/TLS. Soserver's private key and certificate must be supplied to the commandline (or through configuration file). In this case, the frontendprotocol selection will be done via ALPN.

To turn off encryption on frontend connection, useno-tls keywordin--frontend option. HTTP/2 and HTTP/1 are available onthe frontend, and an HTTP/1 connection can be upgraded to HTTP/2 usingHTTP Upgrade. Starting HTTP/2 connection by sending HTTP/2 connectionpreface is also supported.

In order to receive HTTP/3 traffic, usequic parameter in--frontend option (.e.g,--frontend='*,443;quic')

nghttpx can listen on multiple frontend addresses. This is achievedby using multiple--frontend options. For each frontendaddress, TLS can be enabled or disabled.

By default, backend connections are not encrypted. To enable TLSencryption on backend connections, usetls keyword in--backend option. Using patterns andproto keyword in--backend option, backend application protocol can bespecified per host/request path pattern. It means that you can useboth HTTP/2 and HTTP/1 in backend connections at the same time. Notethat default backend protocol is HTTP/1.1. To use HTTP/2 in backend,you have to specifyh2 inproto keyword in--backendexplicitly.

The backend is supposed to be a Web server. For example, to makenghttpx listen to encrypted HTTP/2 requests at port 8443, and abackend Web server is configured to listen to HTTP requests at port8080 on the same host, run nghttpx command-line like this:

$ nghttpx -f0.0.0.0,8443 -b127.0.0.1,8080 /path/to/server.key /path/to/server.crt

Then an HTTP/2 enabled client can access the nghttpx server using HTTP/2. Forexample, you can send a GET request using nghttp:

$ nghttp -nv https://localhost:8443/

HTTP/2 proxy mode

If nghttpx is invoked with--http2-proxy (or its shorthand-s) option, it operates in HTTP/2 proxy mode. The supportedprotocols in frontend and backend connections are the same as indefaultmode. The difference is that this mode acts like a forward proxy andassumes the backend is an HTTP proxy server (e.g., Squid, Apache TrafficServer). HTTP/1 requests must include an absolute URI in request line.

By default, the frontend connection is encrypted. So this mode isalso called secure proxy.

To turn off encryption on the frontend connection, useno-tls keywordin--frontend option.

The backend must be an HTTP proxy server. nghttpx supports multiplebackend server addresses. It translates incoming requests to HTTPrequest to backend server. The backend server performs real proxywork for each request, for example, dispatching requests to the originserver and caching contents.

The backend connection is not encrypted by default. To enableencryption, usetls keyword in--backend option. Thedefault backend protocol is HTTP/1.1. To use HTTP/2 in backendconnection, use--backend option, and specifyh2 inproto keyword explicitly.

For example, to make nghttpx listen to encrypted HTTP/2 requests atport 8443, and a backend HTTP proxy server is configured to listen toHTTP/1 requests at port 8080 on the same host, run nghttpx command-linelike this:

$ nghttpx -s -f'*,8443' -b127.0.0.1,8080 /path/to/server.key /path/to/server.crt

At the time of this writing, Firefox 41 and Chromium v46 can usenghttpx as HTTP/2 proxy.

To make Firefox or Chromium use nghttpx as HTTP/2 proxy, user has tocreate proxy.pac script file like this:

functionFindProxyForURL(url,host){return"HTTPS SERVERADDR:PORT";}

SERVERADDR andPORT is the hostname/address and port of themachine nghttpx is running. Please note that both Firefox andChromium require valid certificate for secure proxy.

For Firefox, open Preference window and select Advanced then clickNetwork tab. Clicking Connection Settings button will show thedialog. Select "Automatic proxy configuration URL" and enter the pathto proxy.pac file, something like this:

file:///path/to/proxy.pac

For Chromium, use following command-line:

$ google-chrome --proxy-pac-url=file:///path/to/proxy.pac --use-npn

As HTTP/1 proxy server, Squid may work as out-of-box. Traffic serverrequires to be configured as forward proxy. Here is the minimumconfiguration items to edit:

CONFIG proxy.config.reverse_proxy.enabled INT 0CONFIG proxy.config.url_remap.remap_required INT 0

Consult Traffic serverdocumentationto know how to configure traffic server as forward proxy and itssecurity implications.

ALPN support

ALPN support requires OpenSSL >= 1.0.2.

Disable frontend SSL/TLS

The frontend connections are encrypted with SSL/TLS by default. Toturn off SSL/TLS, useno-tls keyword in--frontendoption. If this option is used, the private key and certificate arenot required to run nghttpx.

Enable backend SSL/TLS

The backend connections are not encrypted by default. To enableSSL/TLS encryption, usetls keyword in--backend option.

Enable SSL/TLS on memcached connection

By default, memcached connection is not encrypted. To enableencryption, usetls keyword in--tls-ticket-key-memcached.

Specifying additional server certificates

nghttpx accepts additional server private key and certificate pairsusing--subcert option. It can be used multiple times.

Specifying additional CA certificate

By default, nghttpx tries to read CA certificate from system. Butdepending on the system you use, this may fail or is not supported.To specify CA certificate manually, use--cacert option.The specified file must be PEM format and can contain multiplecertificates.

By default, nghttpx validates server's certificate. If you want toturn off this validation, knowing this is really insecure and what youare doing, you can use--insecure option to disablecertificate validation.

Read/write rate limit

nghttpx supports transfer rate limiting on frontend connections. Youcan do rate limit per frontend connection for reading and writingindividually.

To perform rate limit for reading, use--read-rate and--read-burst options. For writing, use--write-rate and--write-burst.

Please note that rate limit is performed on top of TCP and nothing todo with HTTP/2 flow control.

Rewriting location header field

nghttpx automatically rewrites location response header field if thefollowing all conditions satisfy:

• In the default mode (--http2-proxy is not used)

• --no-location-rewrite is not used

• URI in location header field is an absolute URI

• URI in location header field includes non empty host component.

• host (without port) in URI in location header field must match thehost appearing in:authority orhost header field.

When rewrite happens, URI scheme is replaced with the ones used infrontend, and authority is replaced with which appears in:authority, orhost request header field.:authorityheader field has precedence overhost.

Hot swapping

nghttpx supports hot swapping using signals. The hot swapping innghttpx is multi step process. First send USR2 signal to nghttpxprocess. It will do fork and execute new executable, using samecommand-line arguments and environment variables.

As of nghttpx version 1.20.0, that is all you have to do. The newmain process sends QUIT signal to the original process, when it isready to serve requests, to shut it down gracefully.

For earlier versions of nghttpx, you have to do one more thing. Atthis point, both current and new processes can accept requests. Togracefully shutdown current process, send QUIT signal to currentnghttpx process. When all existing frontend connections are done, thecurrent process will exit. At this point, only new nghttpx processexists and serves incoming requests.

If you want to just reload configuration file without executing newbinary, send SIGHUP to nghttpx main process.

Re-opening log files

When rotating log files, it is desirable to re-open log files afterlog rotation daemon renamed existing log files. To tell nghttpx tore-open log files, send USR1 signal to nghttpx process. It willre-open files specified by--accesslog-file and--errorlog-file options.

Multiple frontend addresses

nghttpx can listen on multiple frontend addresses. To specify them,just use--frontend (or its shorthand-f) optionrepeatedly. TLS can be enabled or disabled per frontend addressbasis. For example, to listen on port 443 with TLS enabled, and onport 80 without TLS:

frontend=*,443frontend=*,80;no-tls

Multiple backend addresses

nghttpx supports multiple backend addresses. To specify them, justuse--backend (or its shorthand-b) optionrepeatedly. For example, to use192.168.0.10:8080 and192.168.0.11:8080, use command-line like this:-b192.168.0.10,8080-b192.168.0.11,8080. In configuration file,this looks like:

backend=192.168.0.10,8080backend=192.168.0.11,8008

nghttpx can route request to different backend according to requesthost and path. For example, to route request destined to hostdoc.example.com to backend serverdocserv:3000, you can writelike so:

backend=docserv,3000;doc.example.com/

When you write this option in command-line, you should encloseargument with single or double quotes, since the character; has aspecial meaning in shell.

To route, request to request path/foo to backend server[::1]:8080, you can write like so:

backend=::1,8080;/foo

If the last character of path pattern is/, all request pathswhich start with that pattern match:

backend=::1,8080;/bar/

The request path/bar/buzz matches the/bar/.

You can use* at the end of the path pattern to make it wildcardpattern.* must match at least one character:

backend=::1,8080;/sample*

The request path/sample1/foo matches the/sample* pattern.

Of course, you can specify both host and request path at the sametime:

backend=192.168.0.10,8080;example.com/foo

We can use* in the left most position of host to achieve wildcardsuffix match. If* is the left most character, then the remainingstring should match the request host suffix.* must match atleast one character. For example,*.example.com matcheswww.example.com anddev.example.com, and does not matchexample.com andnghttp2.org. The exact match (without*)always takes precedence over wildcard match.

One important thing you have to remember is that we have to specifydefault routing pattern for so called "catch all" pattern. To write"catch all" pattern, just specify backend server address, withoutpattern.

Usually, host is the value ofHost header field. In HTTP/2, thevalue of:authority pseudo header field is used.

When you write multiple backend addresses sharing the same routingpattern, they are used as load balancing. For example, to use 2serversserv1:3000 andserv2:3000 for request hostexample.com and path/myservice, you can write like so:

backend=serv1,3000;example.com/myservicebackend=serv2,3000;example.com/myservice

You can also specify backend application protocol in--backend option usingproto keyword after pattern.Utilizing this allows ngttpx to route certain request to HTTP/2, otherrequests to HTTP/1. For example, to route requests to/ws/ inbackend HTTP/1.1 connection, and use backend HTTP/2 for otherrequests, do this:

backend=serv1,3000;/;proto=h2backend=serv1,3000;/ws/;proto=http/1.1

The default backend protocol is HTTP/1.1.

TLS can be enabled per pattern basis:

backend=serv1,8443;/;proto=h2;tlsbackend=serv2,8080;/ws/;proto=http/1.1

In the above case, connection to serv1 will be encrypted by TLS. Onthe other hand, connection to serv2 will not be encrypted by TLS.

Dynamic hostname lookup

By default, nghttpx performs backend hostname lookup at start up, orconfiguration reload, and keeps using them in its entire session. Tomake nghttpx perform hostname lookup dynamically, usednsparameter in--backend option, like so:

backend=foo.example.com,80;;dns

nghttpx will cache resolved addresses for certain period of time. Tochange this cache period, use--dns-cache-timeout.

Enable PROXY protocol

PROXY protocol can be enabled per frontend. In order to enable PROXYprotocol, useproxyproto parameter in--frontend option,like so:

frontend=*,443;proxyproto

nghttpx supports both PROXY protocol v1 and v2. AF_UNIX in PROXYprotocol version 2 is ignored.

Session affinity

Two kinds of session affinity are available: client IP, and HTTPCookie.

To enable client IP based affinity, specifyaffinity=ip parameterin--backend option. If PROXY protocol is enabled, then anaddress obtained from PROXY protocol is taken into consideration.

To enable HTTP Cookie based affinity, specifyaffinity=cookieparameter, and specify a name of cookie inaffinity-cookie-nameparameter. Optionally, a Path attribute can be specified inaffinity-cookie-path parameter:

backend=127.0.0.1,3000;;affinity=cookie;affinity-cookie-name=nghttpxlb;affinity-cookie-path=/

Secure attribute of cookie is set if client connection is protected byTLS.affinity-cookie-stickiness specifies the stickiness of thisaffinity. Ifloose is given, which is the default, removing oradding a backend server might break affinity. Whilestrict isgiven, removing the designated backend server breaks affinity, butadding new backend server does not cause breakage.

PSK cipher suites

nghttpx supports pre-shared key (PSK) cipher suites for both frontendand backend TLS connections. For frontend connection, use--psk-secrets option to specify a file which contains PSKidentity and secrets. The format of the file is<identity>:<hex-secret>, where<identity> is PSK identity, and<hex-secret> is PSK secret in hex, like so:

client1:9567800e065e078085c241d54a01c6c3f24b3bab71a606600f4c6ad2c134f3b9client2:b1376c3f8f6dcf7c886c5bdcceecd1e6f1d708622b6ddd21bda26ebd0c0bca99

nghttpx server accepts any of the identity and secret pairs in thefile. The default cipher suite list does not contain PSK ciphersuites. In order to use PSK, PSK cipher suite must be enabled byusing--ciphers option. The desired PSK cipher suite may belisted inHTTP/2 cipher block list. In order to usesuch PSK cipher suite with HTTP/2, disable HTTP/2 cipher block list byusing--no-http2-cipher-block-list option. But you shouldunderstand its implications.

At the time of writing, even if only PSK cipher suites are specifiedin--ciphers option, certificate and private key are stillrequired.

For backend connection, use--client-psk-secrets option tospecify a file which contains single PSK identity and secret. Theformat is the same as the file used by--psk-secretsdescribed above, but only first identity and secret pair is solelyused, like so:

client2:b1376c3f8f6dcf7c886c5bdcceecd1e6f1d708622b6ddd21bda26ebd0c0bca99

The default cipher suite list does not contain PSK cipher suites. Inorder to use PSK, PSK cipher suite must be enabled by using--client-ciphers option. The desired PSK cipher suite maybe listed inHTTP/2 cipher block list. In order to usesuch PSK cipher suite with HTTP/2, disable HTTP/2 cipher block list byusing--client-no-http2-cipher-block-list option. But youshould understand its implications.

TLSv1.3

As of nghttpx v1.34.0, if it is built with OpenSSL 1.1.1 or later, itsupports TLSv1.3. 0-RTT data is supported, but by default itsprocessing is postponed until TLS handshake completes to mitigatereplay attack. This costs extra round trip and reduces effectivenessof 0-RTT data.--tls-no-postpone-early-data makes nghttpxnot wait for handshake to complete before forwarding request includedin 0-RTT to get full potential of 0-RTT data. In this case, nghttpxaddsEarly-Data:1 header field when forwarding a request to abackend server. All backend servers should recognize this headerfield and understand that there is a risk for replay attack. SeeRFC8470 forEarly-Data headerfield.

nghttpx disables anti replay protection provided by OpenSSL. The antireplay protection of OpenSSL requires that a resumed request must hitthe same server which generates the session ticket. Therefore itmight not work nicely in a deployment where there are multiple nghttpxinstances sharing ticket encryption keys via memcached.

Because TLSv1.3 completely changes the semantics of cipher suitenaming scheme and structure, nghttpx provides the new option--tls13-ciphers and--tls13-client-ciphers tochange preferred cipher list for TLSv1.3.

WebSockets over HTTP/2

nghttpx supportsRFC 8441Bootstrapping WebSockets with HTTP/2 for both frontend and backendconnections. This feature is enabled by default and no configurationis required.

WebSockets over HTTP/3 is also supported.

HTTP/3

nghttpx supports HTTP/3 if it is built with HTTP/3 support enabled.HTTP/3 support is experimental.

In order to listen UDP port to receive HTTP/3 traffic,--frontend option must havequic parameter:

frontend=*,443;quic

The above example makes nghttpx receive HTTP/3 traffic on UDPport 443.

nghttpx does not support HTTP/3 on backend connection.

Hot swapping (SIGUSR2) or configuration reload (SIGHUP) require eBPFprogram. Without eBPF, old worker processes keep getting HTTP/3traffic and do not work as intended. The QUIC keying material toencrypt Connection ID must be set with--frontend-quic-secret-file and must provide the existingkeys in order to keep the existing connections alive during reload.

The construction of Connection ID closely follows Block Cipher CIDAlgorithm described inQUIC-LB draft.A Connection ID that nghttpx generates is always 17 bytes long. Ituses first 3 bits as a configuration ID. The remaining bits in thefirst byte are reserved and random. The next 4 bytes are server ID.The next 4 bytes are used to route UDP datagram to a correctSO_REUSEPORT socket. The remaining bytes are randomly generated.The server ID and the next 12 bytes are encrypted with AES-ECB. Thekey is derived from the keying materials stored in a file specified by--frontend-quic-secret-file. The first 2 bits of keyingmaterial in the file is used as a configuration ID. The remainingbits and following 3 bytes are reserved and unused. The next 32 bytesare used as an initial secret. The remaining 32 bytes are used as asalt. The encryption key is generated byHKDF with SHA256 andthese keying materials andconnectionidencryptionkey as info.

In order announce that HTTP/3 endpoint is available, you shouldspecify alt-svc header field. For example, the following options sendalt-svc header field in HTTP/1.1 and HTTP/2 response:

altsvc=h3,443,,,ma=3600http2-altsvc=h3,443,,,ma=3600

Migration from nghttpx v1.18.x or earlier

As of nghttpx v1.19.0,--ciphers option only changes cipherlist for frontend TLS connection. In order to change cipher list forbackend connection, use--client-ciphers option.

Similarly,--no-http2-cipher-block-list option only disablesHTTP/2 cipher block list for frontend connection. In order to disableHTTP/2 cipher block list for backend connection, use--client-no-http2-cipher-block-list option.

--accept-proxy-protocol option was deprecated. Instead, useproxyproto parameter in--frontend option to enablePROXY protocol support per frontend.

Migration from nghttpx v1.8.0 or earlier

As of nghttpx 1.9.0,--frontend-no-tls and--backend-no-tlshave been removed.

To disable encryption on frontend connection, useno-tls keywordin--frontend potion:

frontend=*,3000;no-tls

The TLS encryption is now disabled on backend connection in all modesby default. To enable encryption on backend connection, usetlskeyword in--backend option:

backend=127.0.0.1,8080;tls

As of nghttpx 1.9.0,--http2-bridge,--client and--client-proxy options have been removed. These functionality canbe used using combinations of options.

Use following option instead of--http2-bridge:

backend=<ADDR>,<PORT>;;proto=h2;tls

Use following options instead of--client:

frontend=<ADDR>,<PORT>;no-tlsbackend=<ADDR>,<PORT>;;proto=h2;tls

Use following options instead of--client-proxy:

http2-proxy=yesfrontend=<ADDR>,<PORT>;no-tlsbackend=<ADDR>,<PORT>;;proto=h2;tls

We also removed--backend-http2-connections-per-worker option. Itwas present because previously the number of backend h2 connection wasstatically configured, and defaulted to 1. Now the number of backendh2 connection is increased on demand. We know the maximum number ofconcurrent streams per connection. When we push as many request asthe maximum concurrency to the one connection, we create another newconnection so that we can distribute load and avoid delay the requestprocessing. This is done automatically without any configuration.