The Remote API library allows any Python client to access services available toApp Engine applications.

For example, if your App Engine application uses Datastore orGoogle Cloud Storage, a Python client could access those storage resources usingthe Remote API.

You can use the Remote API to access your application's data store from an apprunning on your local machine, or from a local interactive Remote API shell. TheRemote API interacts with real services, so this access does use quota andbillable resources.

Enabling Remote API access in your app

The easiest way to enable the Remote API for your application is to use thebuiltins directive in theapp.yaml file for your app, which specifies thedefault URL/_ah/remote_api/. However, you can instead use theurldirective in that same file to specify some other URL.

runtime:python27api_version:1threadsafe:truebuiltins:-remote_api:on

-url:/some-URL/*script:google.appengine.ext.remote_api.handler.application

Make sure you deploy your application to App Engine after making thischange.

Using the Remote API shell

The Python SDK includes a Remote API shell, which allows you to invoke Pythoncommands on App Engine services used by your application. You don't need tosupply any additional authentication, because this automatically uses the samecredentials you used to upload the app to App Engine.

To start the Remote API shell:

1. Invoke the following command from a terminal window on your local machine:

   SDK-INSTALL-DIRECTORY/remote_api_shell.py -sYOUR-PROJECT-ID.REGION_ID.r.appspot.com

   Replace[SDK-INSTALL-DIRECTORY] with the path to the App Engine SDKfor Python, and[YOUR-PROJECT-ID] with your project ID.

2. In the interactive shell that is displayed, invoke the Python commands youwant to run. For example, if your application uses Datastore, you couldinvoke the following ndb query to fetch 10 records:

    >>>fromgoogle.appengine.extimportndb >>> >>># Fetch 10 keys from the datastore >>>ndb.Query().fetch(10,keys_only=True)

1. Enable the remote API.

2. Export thePYTHONPATH environment variable for your Python directory, forexample:

   exportPYTHONPATH=/usr/somedir/v3/bin/python2.7

   Replace that path with the actual values for your python location.

3. Add your App Engine SDK for Python location toPYTHONPATH:

   exportGAE_SDK_ROOT="/usr/local/home/mydir/google_appengine"exportPYTHONPATH=${GAE_SDK_ROOT}:${PYTHONPATH}

   Replace the SDK path shown above with your actual path to the App EngineSDK.

4. In your client code, importdev_appserver and calldev_appserver.fix_sys_path() to ensure all of the App Engine SDK modulesimport correctly:

   try:importdev_appserverdev_appserver.fix_sys_path()

5. Add the followingremote_api_stub code to your application, making sure youpass it your project ID in your code:

   remote_api_stub.ConfigureRemoteApiForOAuth('{}.appspot.com'.format(project_id),'/_ah/remote_api')

   If you don't use the default URL/_ah/remote_api for the Remote API, you'llhave to change the code above to reflect the URL you are using. For thedefinition and documentation forremote_api_stub.ConfigureRemoteApiForOAuth, see the SDK file[SDK-INSTALL-DIRECTORY]/google/appengine/ext/remote_api/remote_api_stub.py.

6. Add any needed App Engine imports and Python code to access the desired AppEngine services. The following sample code accesses the project's data store:

   importargparsetry:importdev_appserverdev_appserver.fix_sys_path()exceptImportError:print('Please make sure the App Engine SDK is in your PYTHONPATH.')raisefromgoogle.appengine.extimportndbfromgoogle.appengine.ext.remote_apiimportremote_api_stubdefmain(project_id):remote_api_stub.ConfigureRemoteApiForOAuth('{}.appspot.com'.format(project_id),'/_ah/remote_api')# List the first 10 keys in the datastore.keys=ndb.Query().fetch(10,keys_only=True)forkeyinkeys:print(key)if__name__=='__main__':parser=argparse.ArgumentParser(description=__doc__,formatter_class=argparse.RawDescriptionHelpFormatter)parser.add_argument('project_id',help='Your Project ID.')args=parser.parse_args()main(args.project_id)

7. With your application deployed to App Engine, start your Remote API client:

   pythonyour-client.pyYOUR-PROJECT-ID

   Replacingyour-client.py with your client module, andYOUR-PROJECT-IDwith your project ID. This assumes your client accepts project ID as thecommand-line input, following theclient.py code sample.

Limitations and best practices

The remote_api module goes to great lengths to make sure that as far aspossible, it behaves exactly like the native App Engine datastore. In somecases, this means doing things that are less efficient than they mightotherwise be. When using remote_api, here's a few things to keep in mind:

Every datastore request requires a round-trip

Because you're accessing the datastore over HTTP, there's a bit more overheadand latency than when you access it locally. In order to speed things up anddecrease load, try to limit the number of round-trips you do by batchinggets and puts, and fetching batches of entities from queries. This is goodadvice not just for remote_api, but for using the datastore in general,because a batch operation is only considered to be a single Datastoreoperation.For example, instead of this:

forkeyinkeys:rec=key.get()rec.foo=barrec.put()

you can do this:

records=ndb.get_multi(keys)forrecinrecords:rec.foo=barndb.put_multi(records)

Both examples have the same effect, but the latter requires only tworoundtrips in total, while the former requires two roundtrips for eachentity.

Requests to remote_api use quota

Because the remote_api operates over HTTP, every datastore call you make incursquota usage for HTTP requests, bytes in and out, as well as the usualdatastore quota you would expect. Bear this in mind if you're usingremote_api to do bulk updates.

1 MB API limits apply

As when running natively, the 1MB limit on API requests and responses stillapplies. If your entities are particularly large, you may need to limit thenumber you fetch or put at a time to keep below this limit. This conflictswith minimising round-trips, unfortunately, so the best advice is to use thelargest batches you can without going over the request or response sizelimitations. For most entities, this is unlikely to be an issue, however.

Avoid iterating over queries

One common pattern with datastore access is the following:

q=MyModel.query()forentityinq:# Do something with entity

When you do this, the SDK fetches entities from the datastore in batches of20, fetching a new batch whenever it uses up the existing ones. Because eachbatch has to be fetched in a separate request by remote_api, it's unable todo this as efficiently. Instead, remote_api executes an entirely new queryfor each batch, using the offset functionality to get further into theresults.

If you know how many entities you need, you can do the whole fetch in onerequest by asking for the number you need:

entities=MyModel.query().fetch(100)forentityinentities:# Do something with entity

If you don't know how many entities you will want, you can usecursorsto efficiently iterate over large result sets. This alsoallows you to avoid the 1000 entity limit imposed on normal datastorequeries.

Transactions are less efficient

In order to implement transactions via remote_api, it accumulatesinformation on entities fetched inside the transaction, along with copies ofentities that were put or deleted inside the transaction. When thetransaction is committed, it sends all of this information off to the AppEngine server, where it has to fetch all the entities that were used in thetransaction again, verify that they have not been modified, then put anddelete all the changes the transaction made and commit it. Ifthere's a conflict, the server rolls back the transaction and notifies theclient end, which then has to repeat the process all over again.

This approach works, and exactly duplicates the functionality provided bytransactions on the local datastore, but is rather inefficient. By all meansuse transactions where they are necessary, but try to limit the number andcomplexity of the transactions you execute in the interest of efficiency.