Using Pull Queues in Python 2 Stay organized with collections Save and categorize content based on your preferences.
This page provides an overview of pull queues in the App Enginestandard environment.
This API is supported for first-generation runtimes and can be used whenupgrading to corresponding second-generation runtimes. If you are updating to the App Engine Python 3 runtime, refer to themigration guide to learn about your migration options for legacy bundled services.Inpush queues tasks are delivered to a worker servicebased on the queue's configuration. Inpull queues the worker service mustaskthe queue for tasks. The queue responds by allowing that worker unique accessto process the task for a specified period of time, which is called alease.
Using pull queues, you can also group related tasks using tags and then configure yourworker to pull multiple tasks with a certain tag all at once. This process iscalledbatching.
If a worker cannot process a task before its lease expires, it can either renew thelease or let it expire, at which point another worker can acquire it. Once thework associated with a task is complete, the worker must delete it.
Using pull queues requires your code to handle some functions that are automatedin push queues:
- Scaling your workers
- Your code needs to scale the number of workers based onprocessing volume. If your code does not handle scaling, you riskwasting computing resources if there are no tasks to process; you also risklatency if you have too many tasks to process.
- Deleting the tasks
- Your code also needs to explicitly delete tasks after processing.In push queues, App Engine deletes the tasks for you. If your worker doesnot delete pull queue tasks after processing, another worker will re-processthe task. This wastes computing resources and risks errors if tasks arenotidempotent.
Pull queues in the App Engine standard environment are created by setting a property in aconfiguration file called.queue.yaml
Pull queue workflow
Workers that process tasks from pull queues must be defined within a service that runsin the App Engine standard environment.
The workflow is as follows:
- Youcreate a pull queue, using
.queue.yaml - Youcreate tasks and add them to the queue.
- The worker you have createdleases the task, usingTaskQueue.
- App Engine sends task data to the worker in the lease response.
- The worker processes the task. If the task fails to execute before the leaseexpires, the worker can modify the lease duration. If the lease expires, thetask will be available to be leased to another worker.
- After a task is processed successfully, theworker deletes it.
What's next
- Learn how tocreate pull queues.
Except as otherwise noted, the content of this page is licensed under theCreative Commons Attribution 4.0 License, and code samples are licensed under theApache 2.0 License. For details, see theGoogle Developers Site Policies. Java is a registered trademark of Oracle and/or its affiliates.
Last updated 2025-12-15 UTC.