Create custom query quotas

This document describes how to set or modify custom query quotas to control costs.To learn how BigQuery analysts can estimate and control costs,seeEstimate and control costs.

If you have multiple BigQuery projects and users, you canmanage costs by requesting a custom quota that specifies a limit on the amountof data processed per day. Daily quotas are reset at midnight Pacific Time.

Custom quota is proactive, so you can't run an 11 TB queryif you have a 10 TB quota. Creating a custom quota on processed data lets youcontrol costs at the project level or at the user level.

To set custom cost controls, you can update one or both of the followingquery quotas:

• QueryUsagePerDay: Project-level custom quotas limit the aggregate usage of all users in thatproject.

• QueryUsagePerUserPerDay: User-level custom quota is separately applied to all users andservice accountswithin a project. Regardless of the per user limit, the total usage for allusers in the project combined can never exceed the query usage per day limit.

Note: You can't assign a custom quota to an individual user or service account.

The default limit for theQueryUsagePerDay quota is 200 Tebibytes (TiB) ofdata processed per project per day. The default limit for theQueryUsagePerUserPerDay is unlimited. To check your current limits, seetheQuotas page.You canchange the limits anytime - custom overridessupersede the default limits.

Query usage quotas apply only to theon-demand query pricing model.

For more information about BigQuery quotas that you can set, seeQuotas and limits.

Required role

To get the permission that you need to change your quota, ask your administrator to grant you the Quota Administrator (role/servicemanagement.quotaAdmin) IAM role on your project. For more information about granting roles, seeManage access to projects, folders, and organizations.

This predefined role contains the serviceusage.quotas.update permission, which is required to change your quota.

You might also be able to get this permission withcustom roles or otherpredefined roles.

Set or modify custom quotas

You can set a custom quota or modify an existing custom quota for any quotadisplayed on theQuotas & System Limits page of the Google Cloud console. Whenyou request a lower quota, the change takes effect within a few minutes. If yourequest a higher quota, your request goes through an approval process, which cantake more time. For more information, seeRequest a quotaadjustment.

Note: Custom quotas are approximate. The custom quotas feature provides anadditional safeguard against excessive spending, but is not designed to strictlylimit bytes processed. BigQuery might occasionally run a querythat exceeds a quota. For greater price consistency, consider areservation.

To set or update a custom cost control, such as limiting the amount ofBigQuery data that can be processed each day, do the following:

1. In the Google Cloud console, open theIAM & Admin >Quotas & System Limits page:

   Go to Quotas & System Limits

2. Use theService filter in theFilter search box to filter for the BigQuery API.

3. Select the quotas that you want to adjust. For example, to limit the amount of data that be queried per day at both the project level and user level, selectQuery usage per day andQuery usage per day per user. You might need to page through the list to find them. After you select a quota, a toolbar appears.

4. In the toolbar, clickeditEdit. TheQuota changes dialog opens.

5. IfUnlimited is selected, deselect it.

6. Enter the quota value in TiB that you want in theNew value field.

7. ClickDone.

8. ClickSubmit request.

For more information about viewing and managing quotas, seeView and manage quotas.

Note: Query usage is an accumulation of billed bytes. The query usage value on theQuotas & System Limits page might not match the billed bytes for the same period obtained from theINFORMATION_SCHEMA.JOBS view if some of the queries are charged at a higher rate compared to regularon-demand pricing. For example, regular query on-demand pricing might be charged at X $ per TiB, whereas another query operation, such as creating a logistic regression model, might be charged at a value of 50X $ per TiB, 50 times more than the regular price. In this case, theINFORMATION_SCHEMA.JOBS view might return 100 GiB of billed bytes, but the query usage reported on theQuotas & System Limits page would show 5 TiB of usage, 50 times more, because values for query usage are normalized against the regular on-demand pricing.

Returned error messages

After you set a custom quota, BigQuery returns an error when youexceed it:

• If you exceed a project-level custom quota, BigQuery returnstheusageQuotaExceedederror:

  Custom quota exceeded: Your usage exceeded the custom quota forQueryUsagePerDay, which is set by your administrator. For more information,see https://cloud.google.com/bigquery/cost-controls

• If the user exceeds a user-level custom quota, BigQuery returnsausageQuotaExceedederror with a different error message:

  Custom quota exceeded: Your usage exceeded the custom quota forQueryUsagePerUserPerDay, which is set by your administrator. For moreinformation, see https://cloud.google.com/bigquery/cost-controls

You can run your query from another project that has access to your datasetsand that doesn't have a custom quota or hasn't yet exceeded it.

Example

Suppose you set the following custom quotas for a project with 10 users,one of which is a service account:

• Project level: 50 TB per day
• User level: 10 TB per day

Project-level custom quotas limit the aggregate usage of all users in thatproject. User-level custom quotas are separately applied to each user orservice accountwithin a project.

The following table describes the remaining quota as the 10 usersrun queries throughout the day.

Usage	Remaining quota
Each of the 10 users queries 4 TBs	Project level: 10 TBs remain. User level: 6 TBs per user remain, but only up to 10 TBs total.
The service account queries another 6 TBs	Project level: 4 TBs remain. User level: The service account can no longer use BigQuery. 6 TBs per user remain for the other users, but only up to 4 TBs total.
One user queries another 4 TBs	Project level: 0 TBs remain. User level: Various TBs remain, but no one can use BigQuery because the project-level quota has been exceeded.

With no remaining quota, BigQuery stops working for everyone inthat project.

What's next

• Learn aboutBigQuery pricing.
• Learn how toestimate and control costs.
• Learn how to analyzeBigQuery audit logsto monitor query costs and BigQuery usage.

• To learn about billing, alerts, and visualizing data, see the following topics:

  • Create, edit, or delete budgets and budget alerts
  • Export Cloud Billing data to BigQuery
  • Visualize your costs with Looker Studio