This page describes how App Engine applicationsuse the URL Fetch service toissue HTTP and HTTPS requests and receive responses.To see code samples demonstrating how to issue HTTP and HTTPS requests from yourApp Engine application, seeIssuing HTTP(S) Requests.

If you haveset up Serverless VPC Accessor if you use theSockets API, youneed tostop URL Fetch from handling requests.URL Fetch causes requests to your VPC network or to theSockets API to fail. After you disable URL Fetch, the standard Python librarywill handle HTTP requests. If you need the features provided by URL Fetchfor specific requests, you can use theurlfetch library directly for thosespecific requests.

This API is supported for first-generation runtimes and can be used whenupgrading to corresponding second-generation runtimes. If you are updating to the App Engine Python 3 runtime, refer to themigration guide to learn about your migration options for legacy bundled services.

Requests

App Engine uses the URL Fetch service to issue outbound requests.In Python, you can use thehttplib,urllib, andurllib2 libraries to make HTTP requests;in an App Engine application, each library will perform these requestsby using the URL Fetch service. You can also use theurlfetchlibrary directly.

Request protocols

An application can fetch a URL using either HTTP or HTTPS. The protocol thatshould be used is inferred by looking at the protocol in the target URL.

The URL to be fetched can use any port number in the followingranges:

• 80-90
• 440-450
• 1024-65535.

If the port is not mentioned in the URL, the port is implied by theprotocol. HTTP requests occur on port80, and HTTPS requests occuron port443.

Request methods

If you issue requests through the URL Fetch service, you can use any of thefollowing HTTP methods:

• GET
• POST
• PUT
• HEAD
• DELETE
• PATCH

A request can include HTTP headers and, forPOST,PUT, andPATCHrequests, a payload.

Request proxying

Note that the URL Fetch service uses an HTTP/1.1 compliant proxy to fetch the result.

To prevent an application from causing an endless recursion of requests, arequest handler is not allowed to fetch its own URL. It is stillpossible to cause an endless recursion with other means, soexercise caution if your application can be made to fetch requests forURLs supplied by the user.

Request headers

Your application can set HTTP headers for the outgoing request.

When sending an HTTPPOST request, if aContent-Type header isnot set explicitly, the header is set tox-www-form-urlencoded.This is the content type used by web forms.

For security reasons, the following headers cannot be modifiedby the application:

• Content-Length
• Host
• Vary
• Via
• X-Appengine-Inbound-Appid
• X-Forwarded-For
• X-ProxyUser-IP

These headers are set to accurate values by App Engine asappropriate. For example, App Engine calculates theContent-Length header from the request data and adds it tothe request prior to sending.

The following headers indicate the application ID of the requesting app:

• User-Agent. This header can be modified, but App Engine willappend an identifier string to allow servers to identify AppEngine requests. The appended string has the format"AppEngine-Google; (+http://code.google.com/appengine; appid: APPID)",whereAPPID is your app's identifier.
• X-Appengine-Inbound-Appid. This header cannot be modified, and isadded automatically if the request is sent via the URL Fetchservice when the follow redirectsparameter is set toFalse.

Request timeouts

You can set a deadline, or timeout, for a request. By default, thetimeout for a request is 10 seconds.

You can send synchronous requests and asynchronous requests. The followingbehavior applies to the URL Fetch API:

• Synchronous requests: The fetch call waits until the remotehost returns a result, and then returns control to the application.If the maximum wait time for the fetch call is exceeded, thecall raises an exception.
• Asynchronous requests: The URL Fetch service starts the request,then returns immediately with an object. The application can performother tasks while the URL is being fetched. When the applicationneeds the results, it calls a method on the object, which waits forthe request to finish if necessary, then returns the result. If anyURL Fetch requests are pending when the request handler exits, theapplication server waits for all remaining requests to either returnor reach their deadline before returning a response to the user.

Secure connections and HTTPS

Your application can fetch a URL securely by using HTTPS to connect tosecure servers. Request and response data are transmitted over thenetwork in encrypted form.

By default, the URL Fetch proxy validates the host it contacts. This behaviorallows the API to detect man-in-the-middle attacksbetween App Engine and the remote host when using HTTPS.

Note: Theurllib module doesn't provide methods to validate hosts.

Responses

If you use the URL Fetch API, note that the URL Fetch service returns allresponse data, including the response, code, headers, and body.

By default, if the URL Fetch service receives a response with aredirect code, the service will follow the redirect. The service willfollow up to five redirect responses, then return the final resource.You caninstruct the URL Fetch service to not follow redirectsand instead return a redirect response to the application.

Note: The URL Fetch API imposes responsesize limits. If the incomingresponse exceeds the maximum response sizelimit, the URL Fetch service raises an exception. See theQuotas and limits section for details.

You can instruct the URL Fetch API to truncate the response instead ofraising an exception. If you usetheurllib,urllib2, orhttpliblibraries to fetch URLs, the response is always truncated, and noexception is raised.

Using URL Fetch on the development server

When your application is running on the App Engine development server on yourcomputer, calls to the URL Fetch service are handled locally. Thedevelopment server fetches URLs by contacting remote hosts directlyfrom your computer, using whatever network configuration your computeris using to access the Internet.

When testing the features of your application that fetch URLs, make surethat your computer can access the remote hosts.

Quotas and limits for URL Fetch

For information about URL Fetch service quotas, seeQuotas. To see the current quota usage ofyour application, go to the Quota Details page in theGoogle Cloud console.

Go to the Quota Details page

In addition, the following limits apply to the use of theURL Fetch service:

Limit	Amount
Request size	10 megabytes
Request header size	16 KB (Note that this limits the maximum length of the URL that can be specified in the header)
Response size	32 megabytes

What's next

Run code samples and get guidance on how to issue requests from yourapplication inIssuing HTTP(S) Requests.