Service limits

This page provides details on the limits that apply to Google Security Operations.You can request API limit increases by contactingCloud Customer Care.

Backstory API quotas

The Backstory API quotas are enforced at the server level using an interceptor.Each service which integrates with the Backstory API must specify theappropriate quota server key to enable quota enforcement.

The following table lists the enforced quotas.The abbreviations are:

• QPS - Queries Per Second
• QPM - Queries Per Minute
• QPH - Queries Per Hour
• QPD - Queries Per Day

See additional API documentation:

• Detection Engine API
• BigQuery Access API
• Search API
• Feed Management API
• Forwarder Management API

Chronicle API and Google Cloud project quotas

You can view quotas for the Chronicle API and your Google Cloud project usingthe Google Cloud console. SeeView quotas in theGoogle Cloud console for moreinformation.

Dashboard data sources supported

Dashboards include the following data sources, each with its own query time limit and YARA-L prefix:

For search, the quota is per user per hour, but for dashboards it is perGoogle SecOps instance. For more information about dashboards,seeDashboards.

Data ingestion burst limits

Data ingestion burst limits restrict the amount of data a customer can send to Google SecOps. These limits ensure fairness and prevent issues for other customers caused by ingestion spikes from a single customer. Burst limits ensure that customer data ingestion operates smoothly. You can adjust them using a support ticket. To apply burst limits, Google SecOps uses the following classifications based on ingestion volume:

The following guidelines apply to burst limits:

• When your burst limit is reached, configure ingestion sources tobufferadditional data. Don't configure them to drop data.

  • For pull-based ingestion, such as Google Cloud and API feeds, systemsautomatically buffer ingestion and require no further configuration.
  • For push-based ingestion methods, such as forwarders, webhooks, and APIingestion, configure the systems to automatically resend data when theburst limit is reached. For systems like Bindplane and Cribl, set upbuffering to handle data overflow efficiently.

• Before you reach your burst limit, you can increase it.

• To determine if you are near your burst limit, seeView burst limitusage.

Data table limits

• Maximum number of data tables for a Google SecOps account:1,000.

• Only theCSV file type is supported for uploads.

• Thelimitson the number ofin statements when referencing a reference list in a queryalso apply toin statements in a data table.

• Maximum number ofin statements in a query: 10.

• Maximum number ofin statements in a query forString andNumber datatype columns: 10.

• Maximum number ofin statements with regular expression operators: 5.

• Maximum number ofin statements with CIDR operators: 5.

• Maximum columns per data table: 1,000.

• Maximum rows per data table: 10 million.

• Maximum number of rows you can delete from a data table at one time: 49.

• Maximum aggregate limit of data volume across data tables in a account: 1TB.

• Maximum display limit in web page for data table rows in text and tableeditor view: 10,000 rows.

• Maximum row limit: 10 million rows when uploading a file through the web interface.

• Maximum file size: 10 GB when uploading a file via the API for data table creation.

• Placeholders aren't allowed in the setup section.

• Unmapped columns of a data table with data type set tostring can only bejoined with string fields of UDM event or UDM entity.

• Use only unmapped columns in a data table with a data type set tocidr orregex for CIDR or regular expression.

• Data table lookups: Regular expression wildcards aren't supported and searchterms are limited to 100 characters.

Joins

• Fetching all event samples for detections isn't supported when using data table joins with events.

• Unlike entities and UDM, data tables don't support placeholders. This meansyou can't:

  • Apply one set of filters to a data table and join it with a UDM entity.

  • Apply a different set of filters to the same data table while joining it with another UDM placeholder.

  For example, a data table nameddt with 3 columns:my_hostname,org, andmy_email and with the following rule:

  events:$e1.principal.hostname =  %dt.my_hostname%dt.org ="hr"$e2.principal.email =  %dt.my_email%dt.org !="hr"

All filters on a data table are applied first, and then the filtered rows from the data table are joined with UDM. In this case, the contradictory filters (%dt.org ="hr" and %dt.org !="hr") on thedt table result in an empty data table, which is then joined with bothe1 ande2.

Use data tables with rules

The following limitations apply to data tables when used with rules.

Run frequency

Real-time run frequency isn't supported for rules with data tables.

Output to data tables

• any andall modifiers aren't supported for repeated field columns indata tables.

• Array indexing isn't supported for repeated fields columns in data tables.

• You can only export outcome variables to a data table. You can't exportevent path or data table columns directly.

• Column lists must include the primary key columns for data tables.

• You can have a maximum of 20 outcomes.

• If a data table doesn't exist, a new table is created with the defaultstring data type for all columns, following the order specified.

• Only one rule can write to a data table at a time. If a rule tries to writeto a data table that another rule is already writing to, the rulecompilation fails.

• There's no guarantee that a producer rule can add rows to a data tablebefore a consumer rule for that data table starts.

• A single rule has a limit on the number of outcomes rows. A maximum 10,000-row limitapplies over the result and persisted data and to data tables.

• If a row with the same primary key already exists in the data table,it's non-primary key columns are replaced with the new values.

Entity enrichment from data tables

• You can apply only one enrichment operation (eitheroverride,append, orexclude) to a single entity graph variable.

• Each enrichment operation can use only one data table.

• You can define a maximum of two enrichment operations of any type in thesetup section of a YARA-L rule.

In the following example, anoverride operation is applied to the entity graph variable$g1 and anappend operation is applied to the entity graph variable$g2.

    setup:    graph_override($g1.graph.entity.user.userid = %table1.myids)    graph_append [$g2, %table1]

In the preceding example, the same data table (table1) is used to enhance different entity graphs. You can also use different data tables to enhance the different entity graphs, as follows:

    setup:    graph_override($g1.graph.entity.user.userid = %table1.myids)    graph_append [$g2, %table2]

Use data tables with Search

The following limitations apply to data tables when used with Search.

• You can't run search queries on data tables using the Chronicle API. Queries are only supported through the web interface.

• A single query execution can output a maximum of 1 million rows to a data table or 1 GB, whichever limit comes first.

• Search output to a data table skips event rows if they exceed 5 MB.

• Entity enrichment is not supported with Search.

• Data tables are not supported for customer-managed encryption keys (CMEK) users.

• Writes are limited to 6 per minute per customer.

• API support is not available Search-related data table operations.

• Statistics queries aren't supported with data table joins.

• Data table and data table joins are only supported with UDM events, and not with entities.

  Supported:%datatable1.column1 = %datatable2.column1Not supported:graph.entity.hostname = %sample.test

• You can't include amatch variable in statistics query in theexport section of a statistics query.

  For example, the following is not supported:

  match:      principal.hostname  export:      %sample.write_row(      row: principal.hostname    )

Ingestion rate

When the data ingestion rate for a tenant reaches a certain threshold,Google Security Operations dynamically adjusts the ingestion rate to ensure availabilityfor new data feeds. The ingestion volume and tenant's usage history determines the threshold.For information on volume of data which can be ingested into Google SecOpsby a single customer, seeBurst limits.

A reference list is a generic list of values which can be used to analyze your data.For more information, seeReference lists.

String lists

String lists have the following limits:

• Maximum list size: 6MB
• Maximum length of any single list content line: 5000 characters

Regular expression lists

Regular expression lists have the following size limits:

• Maximum list size: 0.1MB
• Maximum number of lines: 100
• Maximum length of each content line: 5000 characters

CIDR lists

CIDR lists have the following size limits:

• Maximum list size: 0.1MB
• Maximum number of lines: 150
• Maximum length of each content line: 5000 characters

Rule limits

Google Security Operations has the following limitations with regards to rule detections:

• Each rule version has a limit of 10,000 detections per day. This limitresets at midnight UTC.

  For example, a rule version might generate 9,900 detections by 3 PM UTC onJanuary 1. If all these detections are recorded for January 1, it generatesonly 100 more detections that day. On January 2, the rule version cangenerate 10,000 new detections.

• If the rule version is updated, the limit is reset and the rule can againgenerate 10,000 detections in that same day.

  For example, if a rule version produces 9,900 detections by 3 PM UTC onJanuary 1, and all of these detections have a detection time on January 1,it generates only 100 more detections for that day. If rule version isupdated at 4 PM on January 1, the rule version can generate 10,000detections with a detection time on January 1 until the end of day. OnJanuary 2, the rule version can generate another 10,000 new detections.

• TheRules Dashboard can display up to 50 MB of detection data. If thetotal size of the detections exceeds this limit, the interface shows amessage indicating that the data is incomplete. This means the systemgenerated more detections than the interface can display, but the detectionsstill exist and are not lost.

• Running a retrohunt after updating the reference list doesn't reset theexisting detections limits or generate new ones. If the existing detectionlimit has already been reached, no new detections are generated.

• Retrohunts limitations:

  • Run a maximum of 3 concurrent retrohunt jobs for each Google SecOps instance or tenant.
  • The combined text size of all rules must not exceed 1 MB.
  • If you run multiple retrohunts in parallel, the system allocates resources from the same Google SecOps instance. This can leadto slower performance or delays in job completion.

Rule quotas

Google SecOps enforces capacity limits on detection rules toensure consistent system performance and query speed. For details aboutrule quotas, seeUnderstand rule quotas.

Search limits

When conducting searches, the following factors can limit the number ofresults returned:

• Maximum search results: 1M events. When results exceed 1M, only 1Mresults are shown.

• Use search settings to specify a lower limit: By default,Google SecOps limits the number of events displayed to 30K.You can change the limit to any value between 1 and 1M from the searchsettings on theResults page.

• Search results are limited to 10K: If your search returns more than10,000 results, the console displays only the first 10,000. This limitation doesn't alter the total number of returned events.