Query cursors allow an application to retrieve a query's results in convenientbatches, and are recommended over using integer offsets for pagination.SeeQueries for more information on structuring queries for your app.

Query cursors

Query cursors allow an application to retrieve a query's results in convenientbatches without incurring the overhead of a query offset. After performing aretrieval operation, the application can obtain acursor, which is an opaque base64-encoded string marking the index position ofthe last result retrieved. The application can save this string, for example inDatastore, in Memcache, in a Task Queue task payload, or embedded ina web page as an HTTPGET orPOST parameter, and can then use the cursor asthe starting point for a subsequent retrieval operation to obtain the next batchof results from the point where the previous retrieval ended. A retrieval canalso specify an end cursor, to limit the extent of the result set returned.

Offsets versus cursors

Although Datastore supports integer offsets, you should avoidusing them. Instead, use cursors. Using an offset only avoids returning theskipped entities to your application, but these entities are still retrievedinternally. The skipped entities do affect the latency of the query, and yourapplication is billed for the read operations required to retrieve them. Usingcursors instead of offsets lets you avoid all these costs.

Query cursor example

In Python, an application obtains a cursor after retrieving query results bycalling theQuery object'scursor() method. Toretrieve additional results from the point of the cursor, the applicationprepares a similar query with the same entity kind, filters, and sort orders,and passes the cursor to the query'swith_cursor() method before performing theretrieval:

fromgoogle.appengine.apiimportmemcachefromgoogle.appengine.extimportdb# class Person(db.Model): ...# Start a query for all Person entitiespeople=Person.all()# If the application stored a cursor during a previous request, use itperson_cursor=memcache.get('person_cursor')ifperson_cursor:people.with_cursor(start_cursor=person_cursor)# Iterate over the resultsforpersoninpeople:# Do something# Get updated cursor and store it for next timeperson_cursor=people.cursor()memcache.set('person_cursor',person_cursor)

• A cursor can be used only by the same application that performed the originalquery, and only to continue the same query. To use the cursor in a subsequentretrieval operation, you must reconstitute the original query exactly,including the same entity kind, ancestor filter, property filters, and sortorders. It is not possible to retrieve results using a cursor without settingup the same query from which it was originally generated.
• Because the!= andIN operators areimplemented with multiple queries, queries that use them do not supportcursors.
• Cursors don't always work as expected with a query that uses an inequalityfilter or a sort order on a property with multiple values. The de-duplicationlogic for such multiple-valued properties does not persist between retrievals,possibly causing the same result to be returned more than once.
• New App Engine releases might change internal implementation details,invalidating cursors that depend on them. If an application attempts to use acursor that is no longer valid, DatastoreraisesaBadRequestError exception.

Cursors and data updates

The cursor's position is defined as the location in the result list after thelast result returned. A cursor is not a relative position in the list(it's not an offset); it's a marker to which Datastore can jumpwhen starting an index scan for results. If the results for a query changebetween uses of a cursor, the query notices only changes that occur in resultsafter the cursor. If a new result appears before the cursor's position for thequery, it will not be returned when the results after the cursor are fetched.Similarly, if an entity is no longer a result for a query but had appearedbefore the cursor, the results that appear after the cursor do not change. Ifthe last result returned is removed from the result set, the cursor still knowshow to locate the next result.

When retrieving query results, you can use both a start cursor and an end cursorto return a continuous group of results from Datastore. Whenusing a start and end cursor to retrieve the results, you are not guaranteedthat the size of the results will be the same as when you generated the cursors.Entities may be added or deleted from Datastore between thetime the cursors are generated and when they are used in a query.

What's next?

• Learn how to specify what a query returns and further control queryresults.
• Learn thecommon restrictions for queries onDatastore.
• Understand data consistency and how data consistencyworks with different types of queries on Datastore.
• Learn thebasic syntax and structure of queries forDatastore.