Responses

When an HTTP client talks HTTP to a server, the server responds with an HTTPresponse message or curl considers it an error and returns 52 with the errormessageEmpty reply from server.

Size of an HTTP response

An HTTP response has a certain size and curl needs to figure it out. There areseveral different ways to signal the end of an HTTP response but the mostbasic way is to use theContent-Length: header in the response and with thatspecify the exact number of bytes in the response body.

Some early HTTP server implementations had problems with file sizes greaterthan 2GB and wrongly managed to send Content-Length: headers with negativesizes or otherwise just plain wrong data. curl can be told to ignore theContent-Length: header completely with--ignore-content-length. Doing so mayhave some other negative side-effects but should at least let you get thedata.

HTTP response codes

An HTTP transfer gets a 3 digit response code back in the first response line.The response code is the server's way of giving the client a hint about howthe request was handled.

It is important to note that curl does not consider it an error even if theresponse code would indicate that the requested document could not bedelivered (or similar). curl considers a successful sending and receiving ofHTTP to be good.

The first digit of the HTTP response code is a kind of error class:

• 1xx: transient response, more is coming
• 2xx: success
• 3xx: a redirect
• 4xx: the client asked for something the server could not or would not deliver
• 5xx: there is a problem in the server

Remember that you can use curl's--write-out option to extract the responsecode. See the--write-out section.

To make curl return an error for response codes >= 400, you need to use--fail or--fail-with-body. Then curl exits with error code 22 for suchoccurrences.

CONNECT response codes

Since there can be an HTTP request and a separate CONNECT request in the samecurl transfer, we often separate the CONNECT response (from the proxy) fromthe remote server's HTTP response.

The CONNECT is also an HTTP request so it gets response codes in the samenumeric range and you can use--write-out to extract that code as well.

Chunked transfer encoding

An HTTP 1.1 server can decide to respond with a chunked encoded response, afeature that was not present in HTTP 1.0.

When receiving a chunked response, there is no Content-Length: for theresponse to indicate its size. Instead, there is aTransfer-Encoding: chunked header that tells curl there is chunked data coming and then in theresponse body, the data comes in a series of chunks. Every individual chunkstarts with the size of that particular chunk (in hexadecimal), then a newlineand then the contents of the chunk. This is repeated over and over until theend of the response, which is signaled with a zero sized chunk. The point ofthis response encoding is for the client to be able to figure out when theresponse has ended even though the server did not know the full size before itstarted to send it. This is usually the case when the response is dynamic andgenerated at the point when the request comes.

Clients like curl decode the chunks and do not show the chunk sizes to users.

Gzipped transfers

Responses over HTTP can be sent in compressed format. This is most commonlydone by the server when it includes aContent-Encoding: gzip in the responseas a hint to the client. Compressed responses make a lot of sense when eitherstatic resources are sent (that were compressed previously) or even in runtimewhen there is more CPU power available than bandwidth. Sending a much smalleramount of data is often preferred.

You can ask curl to both ask for compressed contentand automatically andtransparently uncompress gzipped data when receiving content encoded gzip (orin fact any other compression algorithm that curl understands) by using--compressed:

curl --compressed http://example.com/

Transfer encoding

A less common feature used with transfer encoding is compression.

Compression in itself is common. Over time the dominant and web compatible wayto do compression for HTTP has become to useContent-Encoding as describedin the section above. HTTP was originally intended and specified to allowtransparent compression as a transfer encoding, and curl supports thisfeature.

The client then simply asks the server to do compression transfer encoding andif acceptable, it responds with a header indicating that it does and curl thentransparently decompresses that data on arrival. A curl user asks for acompressed transfer encoding with--tr-encoding:

curl --tr-encoding http://example.com/

It should be noted that not many HTTP servers in the wild support this.

Pass on transfer encoding

In some situations you may want to use curl as a proxy or other in-betweensoftware. In those cases, curl's way to deal with transfer-encoding headersand then decoding the actual data transparently may not be desired, if the endreceiveralso expects to do the same.

You can then ask curl to pass on the received data, without decoding it. Thatmeans passing on the sizes in the chunked encoding format or the compressedformat when compressed transfer encoding is used etc.

curl --raw http://example.com/