Task queue functions take advantage of GoogleCloud Tasksto help your app run time-consuming, resource-intensive, or bandwidth-limitedtasks asynchronously, outside your main application flow.

For example, imagine that you want to create backups of a large set of imagefiles that are currently hosted on an API with a rate limit. In order to be aresponsible consumer of that API, you need to respect their rate limits. Plus,this kind of long-running job could be vulnerable to failure due to timeouts andmemory limits.

To mitigate this complexity, you can write a task queue function that sets basictask options likescheduleTime, anddispatchDeadline, and then hands thefunction off to a queue inCloud Tasks. TheCloud Tasksenvironment is designed specifically to ensure effective congestion control andretry policies for these kinds of operations.

The Firebase SDK forCloud Functions for Firebase v3.20.1 and higher interoperateswithFirebase Admin SDK v10.2.0 and higher to support task queue functions.

Using task queue functions with Firebase can result in charges forCloud Tasks processing. SeeCloud Tasks pricingfor more information.

Create task queue functions

To use task queue functions, follow this workflow:

1. Write a task queue function using theFirebase SDK forCloud Functions.
2. Test your function by triggering it with an HTTP request.
3. Deploy your function with theFirebase CLI. When deploying your taskqueue function for the first time, the CLI will create atask queue inCloud Taskswith options (rate limiting and retry) specified in your source code.
4. Add tasks to the newly created task queue, passing along parameters to set upan execution schedule if needed. You can achieve this by writing the codeusing theAdmin SDK and deploying it toCloud Functions for Firebase.

Write task queue functions

Code samples in this section are based on an app that setsup a service that backs up all images from NASA'sAstronomy Picture of the Day.To get started, import the required modules:

// Dependencies for task queue functions.const{onTaskDispatched}=require("firebase-functions/tasks");const{onRequest,HttpsError}=require("firebase-functions/https");const{getFunctions}=require("firebase-admin/functions");const{logger}=require("firebase-functions");// Dependencies for image backup.constpath=require("path");const{initializeApp}=require("firebase-admin/app");const{getStorage}=require("firebase-admin/storage");const{GoogleAuth}=require("google-auth-library");
index.js

# Dependencies for task queue functions.fromgoogle.cloudimporttasks_v2importrequestsfromfirebase_functions.optionsimportRetryConfig,RateLimits,SupportedRegion# Dependencies for image backup.fromdatetimeimportdatetime,timedeltaimportjsonimportpathlibfromurllib.parseimporturlparsefromfirebase_adminimportinitialize_app,storage,functionsfromfirebase_functionsimporthttps_fn,tasks_fn,paramsimportgoogle.authfromgoogle.auth.transport.requestsimportAuthorizedSession
main.py

UseonTaskDispatchedoron_task_dispatchedfor task queue functions. When writing a task queue functionyou can set per-queue retry and rate- limiting configuration.

Configure task queue functions

Task queue functions come with a powerful set of configuration settingsto precisely control rate limits and retry behavior of a task queue:

exports.backupapod=onTaskDispatched({retryConfig:{maxAttempts:5,minBackoffSeconds:60,},rateLimits:{maxConcurrentDispatches:6,},},async(req)=>{
index.js

@tasks_fn.on_task_dispatched(retry_config=RetryConfig(max_attempts=5,min_backoff_seconds=60),rate_limits=RateLimits(max_concurrent_dispatches=10))defbackupapod(req:tasks_fn.CallableRequest)->str:"""Grabs Astronomy Photo of the Day (APOD) using NASA's API."""
main.py

• retryConfig.maxAttempts=5: Each task in the task queue is automaticallyretried up to 5 times. This helps mitigate transient errors like networkerrors or temporary service disruption of a dependent, external service.

• retryConfig.minBackoffSeconds=60: Each task is retried at least 60 secondsapart from each attempt. This provides a large buffer between each attemptso we don't rush to exhaust the 5 retry attempts too quickly.

• rateLimits.maxConcurrentDispatch=6: At most 6 tasks are dispatched at agiven time. This helps ensure a steady stream of requests to the underlyingfunction and helps reduce the number of active instances and cold starts.

Test task queue functions

For most cases, theCloud Functions emulator is the best way to test taskqueue functions. See the Emulator Suite documentation to learn how toinstrument your app for task queue functions emulation.

Additionally, task queue functions are exposed as simpleHTTP functions in theFirebase Local Emulator Suite.You can test an emulated task function by sending an HTTP POSTrequest with a JSON data payload:

#starttheLocalEmulatorSuitefirebaseemulators:start#triggertheemulatedtaskqueuefunctioncurl\-XPOST#AnHTTPPOSTrequest...-H"content-type: application/json"\#...withaJSONbodyhttp://localhost:$PORT/$PROJECT_ID/$REGION/$NAME \ # ... to function url-d'{"data":{...somedata....}}'#...withJSONencodeddata

Deploy task queue functions

Deploy task queue function using theFirebase CLI:

$firebasedeploy--onlyfunctions:backupapod

When deploying a task queue function for the first time, the CLI creates atask queue inCloud Tasks with options (rate limiting and retry) specified in your source code.

If you encounter permissions errors when deploying functions, make sure that theappropriateIAM rolesare assigned to the user running the deployment commands.

Enqueue task queue functions

Task queue functions can be enqueued inCloud Tasks from a trustedserver environment likeCloud Functions for Firebase using theFirebase Admin SDK forNode.js or Google Cloud libraries for Python.If you're new to theAdmin SDKs, seeAdd Firebase to a server to get started.

A typical flow creates a new task, enqueues it inCloud Tasks, and sets the configuration for the task:

exports.enqueuebackuptasks=onRequest(async(_request,response)=>{constqueue=getFunctions().taskQueue("backupapod");consttargetUri=awaitgetFunctionUrl("backupapod");constenqueues=[];for(leti=0;i<=BACKUP_COUNT;i+=1){constiteration=Math.floor(i/HOURLY_BATCH_SIZE);// Delay each batch by N * hourconstscheduleDelaySeconds=iteration*(60*60);constbackupDate=newDate(BACKUP_START_DATE);backupDate.setDate(BACKUP_START_DATE.getDate()+i);// Extract just the date portion (YYYY-MM-DD) as string.constdate=backupDate.toISOString().substring(0,10);enqueues.push(queue.enqueue({date},{scheduleDelaySeconds,dispatchDeadlineSeconds:60*5,// 5 minutesuri:targetUri,}),);}awaitPromise.all(enqueues);response.sendStatus(200);});
index.js

@https_fn.on_request()defenqueuebackuptasks(_:https_fn.Request)->https_fn.Response:"""Adds backup tasks to a Cloud Tasks queue."""task_queue=functions.task_queue("backupapod")target_uri=get_function_url("backupapod")foriinrange(BACKUP_COUNT):batch=i//HOURLY_BATCH_SIZE# Delay each batch by N hoursschedule_delay=timedelta(hours=batch)schedule_time=datetime.now()+schedule_delaydispatch_deadline_seconds=60*5# 5 minutesbackup_date=BACKUP_START_DATE+timedelta(days=i)body={"data":{"date":backup_date.isoformat()[:10]}}task_options=functions.TaskOptions(schedule_time=schedule_time,dispatch_deadline_seconds=dispatch_deadline_seconds,uri=target_uri)task_queue.enqueue(body,task_options)returnhttps_fn.Response(status=200,response=f"Enqueued{BACKUP_COUNT} tasks")
main.py

• The sample code tries to spread out execution oftasks by associating a delay of Nth minutes for the Nth task. Thistranslates to triggering ~ 1 task/minute. Note that you can also usescheduleTime (Node.js) orschedule_time (Python) if you wantCloud Tasks to trigger a task at a specific time.

• The sample code sets the maximum amount of timeCloud Tasks will waitfor a task to complete.Cloud Tasks will retrythe task following the retryconfiguration of the queue or until this deadline is reached. In the sample,the queue is configured to retry the task up to 5 times, but the task isautomatically cancelled if the whole process (including retry attempts)takes more than 5 minutes.

Logs fromCloud Tasks contain useful diagnostic information like thestatus of the request associated with a task. By default, logs fromCloud Tasks are turned off due to the large volume of logs it canpotentially generate on your project. We recommend you turn on the debug logswhile you are actively developing and debugging your task queue functions. SeeTurning onlogging.

IAM Permissions

You may seePERMISSION DENIED errors when enqueueing tasks or whenCloud Tasks tries to invoke your task queue functions. Ensure that yourproject has the following IAM bindings:

• The identity used to enqueue tasks toCloud Tasks needscloudtasks.tasks.create IAM permission.

  In the sample, this is theApp Engine default service account

gcloudprojectsadd-iam-policy-binding$PROJECT_ID\--member=serviceAccount:${PROJECT_ID}@appspot.gserviceaccount.com\--role=roles/cloudtasks.enqueuer

• The identity used to enqueue tasks toCloud Tasks needs permissionto use the service account associated with a task inCloud Tasks.

  In the sample, this is theApp Engine default service account.

SeeGoogle Cloud IAM documentationfor instructions on how to add theApp Engine default service accountas a user of theApp Engine default service account.

• The identity used to trigger the task queue function needscloudfunctions.functions.invoke permission.

  In the sample, this is theApp Engine default service account

gcloudfunctionsadd-iam-policy-binding$FUNCTION_NAME\--region=us-central1\--member=serviceAccount:${PROJECT_ID}@appspot.gserviceaccount.com\--role=roles/cloudfunctions.invoker