My browser shows something else

A common use case is using curl to get a URL that you can get in your browserwhen you paste the URL in the browser's address bar.

A browser getting a URL as input does so much more and in so many differentways than curl that what curl shows in your terminal output is probably not atall what you see in your browser window.

Client differences

Curl only gets exactly what you ask it to get and it never parses the actualcontent—the data—that the server delivers. A browser gets data and itactivates different parsers depending on what kind of content it thinks itgets. For example, if the data is HTML it parses it to display a webpage andpossibly download other sub resources such as images, JavaScript and CSSfiles. When curl downloads HTML it just gets that single HTML resource, evenif it, when parsed by a browser, would trigger a whole busload of moredownloads. If you want curl to download any sub-resources as well, you need topass those URLs to curl and ask it to get those, just like any other URLs.

Clients also differ in how they send their requests, and some aspects of arequest for a resource include, for example, format preferences, asking forcompressed data, or just telling the server from which previous page we are"coming from". curl's requests differ a little or a lot from how your browsersends its requests.

Server differences

The server that receives the request and delivers data is often set up to act incertain ways depending on what kind of client it thinks communicates with it.Sometimes it is as innocent as trying to deliver the best content for theclient, sometimes it is to hide some content for some clients or even to tryto work around known problems in specific browsers. Then there are also, ofcourse, various kinds of login systems that might rely on HTTP authentication orcookies or the client being from the pre-validated IP address range.

Sometimes getting the same response from a server using curl as the responseyou get with a browser ends up really hard work. Users then typically recordtheir browser sessions with the browser's networking tools and then comparethat recording with recorded data from curl's--trace-ascii option andproceed to modify curl's requests (often with-H / --header) until theserver starts to respond the same to both.

This type of work can be both time consuming and tedious. You should always dothis with permission from the server owners or admins.

Intermediaries' fiddlings

Intermediaries are proxies, explicit or implicit ones. Some environments forceyou to use one or you may choose to use one for various reasons, but there arealso the transparent ones that intercept your network traffic silently andproxy it for you no matter what you want.

Proxies are "middle men" that terminate the traffic and then act on yourbehalf to the remote server. This can introduce all sorts of explicitfiltering and "saving" you from certain content or even "protecting" theremote server from what data you try to send to it, but even more so itintroduces another software's view on how the protocol works and what theright things to do are.

Interfering intermediaries are often the cause of lots of headaches andmysteries down to downright malicious modifications of content.

We strongly encourage you to use HTTPS or other means to verify that thecontents you are downloading or uploading are really the data that the remoteserver has sent to you and that your precious bytes end up verbatim at theintended destination.