This page describes how to create and customize a push queue, and how toexamine the contents of a queue.

Using a queue configuration file to create queues

To process a task, you must add it to a push queue. App Engine provides adefault push queue, nameddefault, which is configured andready to use with default settings. If you want, you can just add all your tasksto the default queue, without having to create and configure other queues.

To add queues or change the default configuration, edit the queue configuration filefor your application, which you upload to App Engine. You can create up to 100queues. Queues cannot be created dynamically.

This queue configuration file defines two queues:

queue:- name: queue-blue  target: v2.task-module  rate: 5/s- name: queue-red  rate: 1/s

To upload the file:

gcloud app deploy queue.yaml

There are many other parameters that can be added to the configuration file tocustomize the behavior of a push queue. For more information, see thequeue configuration file reference.

Defining the push queue processing rate

You can control the rate at which tasks are processed in each of your queues bydefining other directives, such asrate,bucket_size,andmax_concurrent_requests.

The task queue usestoken buckets tocontrol the rate of task execution. Each named queue has a token bucket thatholds tokens, up to the maximum specified by thebucket_size, or a maximum of5 tokens if you don't specify the bucket size.

Each time your application executes a task, a token is removed from the bucket.Your app continues processing tasks in the queue until the queue's bucket runsout of tokens. App Engine refills the bucket with new tokens continuously basedon therate that you specified for the queue.

If your queue contains tasks to process, and the queue's bucket contains tokens,App Engine simultaneously processes as many tasks as there are tokens.This can lead to bursts of processing, consuming system resources and competingwith user-serving requests.

If you want to prevent too many tasks from running at once or to preventdatastore contention, usemax_concurrent_requests.

The following sample shows how to setmax_concurrent_requests to limittasks and also shows how to adjust the bucket size and rate based on yourapplication's needs and available resources:

queue:- name: queue-blue  rate: 20/s  bucket_size: 40  max_concurrent_requests: 10

Setting storage limits for all queues

You can use your queue configuration file to define the total amount of storage that task datacan consume over all queues. To define the total storage limit, include anelement namedtotal_storage_limitat the top level:

# Set the total storage limit for all queues to 120MBtotal_storage_limit: 120Mqueue:- name: queue-blue  rate: 35/s

The value is a number followed by a unit:B for bytes,K for kilobytes,Mfor megabytes,G for gigabytes,T for terabytes. For example,100Kspecifies a limit of 100 kilobytes. If adding a task would cause the queue toexceed its storage limit, the call to add the task will fail. The default limitis500M (500 megabytes) for free apps. For billed apps there is no limit untilyou explicitly set one. You can use this limit to protect your app from aforkbomb programming error in which each task adds multiple other tasks during itsexecution.

If your app is receiving errors for insufficient quota when addingtasks, increasing the total storage limit can help. If you are using thisfeature, we strongly recommend setting a limit that corresponds to the storagerequired for several days' worth of tasks. This allows the queues to betemporarily backed up and continue to accept new taskswhile working through the backlog while still being protected from a fork bombprogramming error.

Monitoring queues in the Google Cloud console

1. In the Google Cloud console, go to the Cloud Tasks page.

   Go toCloud Tasks

   Note that if you go to the App EngineTask queue page, therewill be instructions that guide you to the Cloud Tasks page.This update in the Google Cloud console does not change how Task queuesfunction.

2. Enable the Cloud Tasks API.

3. Once you're in the Cloud Tasks page, you will see a list of allof the queues in the application. Clicking on a the name of a queue brings uptheQueue Details page, which shows all of the tasks in the selected queue.

What's next

Learn aboutcreating tasks.