httputil
packagestandard libraryThis package is not in the latest version of its module.
Details
Validgo.mod file
The Go module system was introduced in Go 1.11 and is the official dependency management solution for Go.
Redistributable license
Redistributable licenses place minimal restrictions on how software can be used, modified, and redistributed.
Tagged version
Modules with tagged versions give importers more predictable builds.
Stable version
When a project reaches major version v1 it is considered stable.
- Learn more about best practices
Repository
Links
Documentation¶
Overview¶
Package httputil provides HTTP utility functions, complementing themore common ones in the net/http package.
Index¶
- Variables
- func DumpRequest(req *http.Request, body bool) ([]byte, error)
- func DumpRequestOut(req *http.Request, body bool) ([]byte, error)
- func DumpResponse(resp *http.Response, body bool) ([]byte, error)
- func NewChunkedReader(r io.Reader) io.Reader
- func NewChunkedWriter(w io.Writer) io.WriteCloser
- type BufferPool
- type ClientConndeprecated
- func (cc *ClientConn) Close() error
- func (cc *ClientConn) Do(req *http.Request) (*http.Response, error)
- func (cc *ClientConn) Hijack() (c net.Conn, r *bufio.Reader)
- func (cc *ClientConn) Pending() int
- func (cc *ClientConn) Read(req *http.Request) (resp *http.Response, err error)
- func (cc *ClientConn) Write(req *http.Request) error
- type ProxyRequest
- type ReverseProxy
- type ServerConndeprecated
Examples¶
Constants¶
This section is empty.
Variables¶
var (// Deprecated: No longer used.ErrPersistEOF = &http.ProtocolError{ErrorString: "persistent connection closed"}// Deprecated: No longer used.ErrClosed = &http.ProtocolError{ErrorString: "connection closed by user"}// Deprecated: No longer used.ErrPipeline = &http.ProtocolError{ErrorString: "pipeline error"})
var ErrLineTooLong =internal.ErrLineTooLong
ErrLineTooLong is returned when reading malformed chunked datawith lines that are too long.
Functions¶
funcDumpRequest¶
DumpRequest returns the given request in its HTTP/1.x wirerepresentation. It should only be used by servers to debug clientrequests. The returned representation is an approximation only;some details of the initial request are lost while parsing it intoanhttp.Request. In particular, the order and case of header fieldnames are lost. The order of values in multi-valued headers is keptintact. HTTP/2 requests are dumped in HTTP/1.x form, not in theiroriginal binary representations.
If body is true, DumpRequest also returns the body. To do so, itconsumes req.Body and then replaces it with a newio.ReadCloserthat yields the same bytes. If DumpRequest returns an error,the state of req is undefined.
The documentation forhttp.Request.Write details which fieldsof req are included in the dump.
Example¶
package mainimport ("fmt""io""log""net/http""net/http/httptest""net/http/httputil""strings")func main() {ts := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {dump, err := httputil.DumpRequest(r, true)if err != nil {http.Error(w, fmt.Sprint(err), http.StatusInternalServerError)return}fmt.Fprintf(w, "%q", dump)}))defer ts.Close()const body = "Go is a general-purpose language designed with systems programming in mind."req, err := http.NewRequest("POST", ts.URL, strings.NewReader(body))if err != nil {log.Fatal(err)}req.Host = "www.example.org"resp, err := http.DefaultClient.Do(req)if err != nil {log.Fatal(err)}defer resp.Body.Close()b, err := io.ReadAll(resp.Body)if err != nil {log.Fatal(err)}fmt.Printf("%s", b)}
Output:"POST / HTTP/1.1\r\nHost: www.example.org\r\nAccept-Encoding: gzip\r\nContent-Length: 75\r\nUser-Agent: Go-http-client/1.1\r\n\r\nGo is a general-purpose language designed with systems programming in mind."
funcDumpRequestOut¶
DumpRequestOut is likeDumpRequest but for outgoing client requests. Itincludes any headers that the standardhttp.Transport adds, such asUser-Agent.
Example¶
package mainimport ("fmt""log""net/http""net/http/httputil""strings")func main() {const body = "Go is a general-purpose language designed with systems programming in mind."req, err := http.NewRequest("PUT", "http://www.example.org", strings.NewReader(body))if err != nil {log.Fatal(err)}dump, err := httputil.DumpRequestOut(req, true)if err != nil {log.Fatal(err)}fmt.Printf("%q", dump)}
Output:"PUT / HTTP/1.1\r\nHost: www.example.org\r\nUser-Agent: Go-http-client/1.1\r\nContent-Length: 75\r\nAccept-Encoding: gzip\r\n\r\nGo is a general-purpose language designed with systems programming in mind."
funcDumpResponse¶
DumpResponse is like DumpRequest but dumps a response.
Example¶
package mainimport ("fmt""log""net/http""net/http/httptest""net/http/httputil")func main() {const body = "Go is a general-purpose language designed with systems programming in mind."ts := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {w.Header().Set("Date", "Wed, 19 Jul 1972 19:00:00 GMT")fmt.Fprintln(w, body)}))defer ts.Close()resp, err := http.Get(ts.URL)if err != nil {log.Fatal(err)}defer resp.Body.Close()dump, err := httputil.DumpResponse(resp, true)if err != nil {log.Fatal(err)}fmt.Printf("%q", dump)}
Output:"HTTP/1.1 200 OK\r\nContent-Length: 76\r\nContent-Type: text/plain; charset=utf-8\r\nDate: Wed, 19 Jul 1972 19:00:00 GMT\r\n\r\nGo is a general-purpose language designed with systems programming in mind.\n"
funcNewChunkedReader¶
NewChunkedReader returns a new chunkedReader that translates the data read from rout of HTTP "chunked" format before returning it.The chunkedReader returnsio.EOF when the final 0-length chunk is read.
NewChunkedReader is not needed by normal applications. The http packageautomatically decodes chunking when reading response bodies.
funcNewChunkedWriter¶
func NewChunkedWriter(wio.Writer)io.WriteCloser
NewChunkedWriter returns a new chunkedWriter that translates writes into HTTP"chunked" format before writing them to w. Closing the returned chunkedWritersends the final 0-length chunk that marks the end of the stream but doesnot send the final CRLF that appears after trailers; trailers and the lastCRLF must be written separately.
NewChunkedWriter is not needed by normal applications. The httppackage adds chunking automatically if handlers don't set aContent-Length header. Using NewChunkedWriter inside a handlerwould result in double chunking or chunking with a Content-Lengthlength, both of which are wrong.
Types¶
typeBufferPool¶added ingo1.6
A BufferPool is an interface for getting and returning temporarybyte slices for use byio.CopyBuffer.
typeClientConndeprecated
type ClientConn struct {// contains filtered or unexported fields}
ClientConn is an artifact of Go's early HTTP implementation.It is low-level, old, and unused by Go's current HTTP stack.We should have deleted it before Go 1.
Deprecated: Use Client or Transport in packagenet/http instead.
funcNewClientConndeprecated
funcNewProxyClientConndeprecated
func (*ClientConn)Close¶
func (cc *ClientConn) Close()error
Close callsClientConn.Hijack and then also closes the underlying connection.
func (*ClientConn)Hijack¶
func (cc *ClientConn) Hijack() (cnet.Conn, r *bufio.Reader)
Hijack detaches theClientConn and returns the underlying connection as wellas the read-side bufio which may have some left over data. Hijack may becalled before the user or Read have signaled the end of the keep-alivelogic. The user should not call Hijack whileClientConn.Read or ClientConn.Write is in progress.
func (*ClientConn)Pending¶
func (cc *ClientConn) Pending()int
Pending returns the number of unanswered requeststhat have been sent on the connection.
func (*ClientConn)Read¶
Read reads the next response from the wire. A valid response might bereturned together with anErrPersistEOF, which means that the remoterequested that this be the last request serviced. Read can be calledconcurrently withClientConn.Write, but not with another Read.
func (*ClientConn)Write¶
func (cc *ClientConn) Write(req *http.Request)error
Write writes a request. AnErrPersistEOF error is returned if the connectionhas been closed in an HTTP keep-alive sense. If req.Close equals true, thekeep-alive connection is logically closed after this request and the opposingserver is informed. An ErrUnexpectedEOF indicates the remote closed theunderlying TCP connection, which is usually considered as graceful close.
typeProxyRequest¶added ingo1.20
type ProxyRequest struct {// In is the request received by the proxy.// The Rewrite function must not modify In.In *http.Request// Out is the request which will be sent by the proxy.// The Rewrite function may modify or replace this request.// Hop-by-hop headers are removed from this request// before Rewrite is called.Out *http.Request}
A ProxyRequest contains a request to be rewritten by aReverseProxy.
func (*ProxyRequest)SetURL¶added ingo1.20
func (r *ProxyRequest) SetURL(target *url.URL)
SetURL routes the outbound request to the scheme, host, and base pathprovided in target. If the target's path is "/base" and the incomingrequest was for "/dir", the target request will be for "/base/dir".To route requests without joining the incoming path,set r.Out.URL directly.
SetURL rewrites the outbound Host header to match the target's host.To preserve the inbound request's Host header (the default behaviorofNewSingleHostReverseProxy):
rewriteFunc := func(r *httputil.ProxyRequest) {r.SetURL(url)r.Out.Host = r.In.Host}
func (*ProxyRequest)SetXForwarded¶added ingo1.20
func (r *ProxyRequest) SetXForwarded()
SetXForwarded sets the X-Forwarded-For, X-Forwarded-Host, andX-Forwarded-Proto headers of the outbound request.
- The X-Forwarded-For header is set to the client IP address.
- The X-Forwarded-Host header is set to the host name requestedby the client.
- The X-Forwarded-Proto header is set to "http" or "https", dependingon whether the inbound request was made on a TLS-enabled connection.
If the outbound request contains an existing X-Forwarded-For header,SetXForwarded appends the client IP address to it. To append to theinbound request's X-Forwarded-For header (the default behavior ofReverseProxy when using a Director function), copy the headerfrom the inbound request before calling SetXForwarded:
rewriteFunc := func(r *httputil.ProxyRequest) {r.Out.Header["X-Forwarded-For"] = r.In.Header["X-Forwarded-For"]r.SetXForwarded()}
typeReverseProxy¶
type ReverseProxy struct {// Rewrite must be a function which modifies// the request into a new request to be sent// using Transport. Its response is then copied// back to the original client unmodified.// Rewrite must not access the provided ProxyRequest// or its contents after returning.//// The Forwarded, X-Forwarded, X-Forwarded-Host,// and X-Forwarded-Proto headers are removed from the// outbound request before Rewrite is called. See also// the ProxyRequest.SetXForwarded method.//// Unparsable query parameters are removed from the// outbound request before Rewrite is called.// The Rewrite function may copy the inbound URL's// RawQuery to the outbound URL to preserve the original// parameter string. Note that this can lead to security// issues if the proxy's interpretation of query parameters// does not match that of the downstream server.//// At most one of Rewrite or Director may be set.Rewrite func(*ProxyRequest)// Director is a function which modifies// the request into a new request to be sent// using Transport. Its response is then copied// back to the original client unmodified.// Director must not access the provided Request// after returning.//// By default, the X-Forwarded-For header is set to the// value of the client IP address. If an X-Forwarded-For// header already exists, the client IP is appended to the// existing values. As a special case, if the header// exists in the Request.Header map but has a nil value// (such as when set by the Director func), the X-Forwarded-For// header is not modified.//// To prevent IP spoofing, be sure to delete any pre-existing// X-Forwarded-For header coming from the client or// an untrusted proxy.//// Hop-by-hop headers are removed from the request after// Director returns, which can remove headers added by// Director. Use a Rewrite function instead to ensure// modifications to the request are preserved.//// Unparsable query parameters are removed from the outbound// request if Request.Form is set after Director returns.//// At most one of Rewrite or Director may be set.Director func(*http.Request)// The transport used to perform proxy requests.// If nil, http.DefaultTransport is used.Transporthttp.RoundTripper// FlushInterval specifies the flush interval// to flush to the client while copying the// response body.// If zero, no periodic flushing is done.// A negative value means to flush immediately// after each write to the client.// The FlushInterval is ignored when ReverseProxy// recognizes a response as a streaming response, or// if its ContentLength is -1; for such responses, writes// are flushed to the client immediately.FlushIntervaltime.Duration// ErrorLog specifies an optional logger for errors// that occur when attempting to proxy the request.// If nil, logging is done via the log package's standard logger.ErrorLog *log.Logger// BufferPool optionally specifies a buffer pool to// get byte slices for use by io.CopyBuffer when// copying HTTP response bodies.BufferPoolBufferPool// ModifyResponse is an optional function that modifies the// Response from the backend. It is called if the backend// returns a response at all, with any HTTP status code.// If the backend is unreachable, the optional ErrorHandler is// called without any call to ModifyResponse.//// Hop-by-hop headers are removed from the response before// calling ModifyResponse. ModifyResponse may need to remove// additional headers to fit its deployment model, such as Alt-Svc.//// If ModifyResponse returns an error, ErrorHandler is called// with its error value. If ErrorHandler is nil, its default// implementation is used.ModifyResponse func(*http.Response)error// ErrorHandler is an optional function that handles errors// reaching the backend or errors from ModifyResponse.//// If nil, the default is to log the provided error and return// a 502 Status Bad Gateway response.ErrorHandler func(http.ResponseWriter, *http.Request,error)}
ReverseProxy is an HTTP Handler that takes an incoming request andsends it to another server, proxying the response back to theclient.
1xx responses are forwarded to the client if the underlyingtransport supports ClientTrace.Got1xxResponse.
Hop-by-hop headers (seeRFC 9110, section 7.6.1), includingConnection, Proxy-Connection, Keep-Alive, Proxy-Authenticate,Proxy-Authorization, TE, Trailer, Transfer-Encoding, and Upgrade,are removed from client requests and backend responses.The Rewrite function may be used to add hop-by-hop headers to the request,and the ModifyResponse function may be used to remove them from the response.
Example¶
package mainimport ("fmt""io""log""net/http""net/http/httptest""net/http/httputil""net/url")func main() {backendServer := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {fmt.Fprintln(w, "this call was relayed by the reverse proxy")}))defer backendServer.Close()rpURL, err := url.Parse(backendServer.URL)if err != nil {log.Fatal(err)}frontendProxy := httptest.NewServer(&httputil.ReverseProxy{Rewrite: func(r *httputil.ProxyRequest) {r.SetXForwarded()r.SetURL(rpURL)},})defer frontendProxy.Close()resp, err := http.Get(frontendProxy.URL)if err != nil {log.Fatal(err)}b, err := io.ReadAll(resp.Body)if err != nil {log.Fatal(err)}fmt.Printf("%s", b)}
Output:this call was relayed by the reverse proxy
funcNewSingleHostReverseProxy¶
func NewSingleHostReverseProxy(target *url.URL) *ReverseProxy
NewSingleHostReverseProxy returns a newReverseProxy that routesURLs to the scheme, host, and base path provided in target. If thetarget's path is "/base" and the incoming request was for "/dir",the target request will be for /base/dir.
NewSingleHostReverseProxy does not rewrite the Host header.
To customize the ReverseProxy behavior beyond whatNewSingleHostReverseProxy provides, use ReverseProxy directlywith a Rewrite function. The ProxyRequest SetURL methodmay be used to route the outbound request. (Note that SetURL,unlike NewSingleHostReverseProxy, rewrites the Host headerof the outbound request by default.)
proxy := &ReverseProxy{Rewrite: func(r *ProxyRequest) {r.SetURL(target)r.Out.Host = r.In.Host // if desired},}
func (*ReverseProxy)ServeHTTP¶
func (p *ReverseProxy) ServeHTTP(rwhttp.ResponseWriter, req *http.Request)
typeServerConndeprecated
type ServerConn struct {// contains filtered or unexported fields}
ServerConn is an artifact of Go's early HTTP implementation.It is low-level, old, and unused by Go's current HTTP stack.We should have deleted it before Go 1.
Deprecated: Use the Server in packagenet/http instead.
funcNewServerConndeprecated
func (*ServerConn)Close¶
func (sc *ServerConn) Close()error
Close callsServerConn.Hijack and then also closes the underlying connection.
func (*ServerConn)Hijack¶
func (sc *ServerConn) Hijack() (net.Conn, *bufio.Reader)
Hijack detaches theServerConn and returns the underlying connection as wellas the read-side bufio which may have some left over data. Hijack may becalled before Read has signaled the end of the keep-alive logic. The usershould not call Hijack whileServerConn.Read orServerConn.Write is in progress.
func (*ServerConn)Pending¶
func (sc *ServerConn) Pending()int
Pending returns the number of unanswered requeststhat have been received on the connection.
func (*ServerConn)Read¶
func (sc *ServerConn) Read() (*http.Request,error)
Read returns the next request on the wire. AnErrPersistEOF is returned ifit is gracefully determined that there are no more requests (e.g. after thefirst request on an HTTP/1.0 connection, or after a Connection:close on aHTTP/1.1 connection).
func (*ServerConn)Write¶
Write writes resp in response to req. To close the connection gracefully, set theResponse.Close field to true. Write should be considered operational untilit returns an error, regardless of any errors returned on theServerConn.Read side.