This document describes, at a high level, the Logging query language thatyou use to query and filter Cloud Logging data.

For in-depth information about the Logging query language design, seetheGoogle API formal specifications for filtering.

For examples of common queries you might want to use, seeSample queries using the Logs Explorer.

Overview

You can use the Logging query language in the Logs Explorer in theGoogle Cloud console, theLogging API,or thecommand-line interface.You can use the Logging query language to query data and to write filters tocreatesinks andlog-based metrics.

A query is a Boolean expression that specifies a subset of all the log entriesin your selected Google Cloud resource, such as a Google Cloud projector folder.

You can build queries based on theLogEntryindexed field using the logical operatorsAND andOR.Using theresource.type field in the following examples, theLogging query language grammar looks like this:

• Simple restriction:resource.type = "k8s_cluster"

• Conjunctive restriction:resource.type = "k8s_cluster" AND severity = "ERROR"

  Note: When a query contains multiple statements,you can either specify how those statements are joined or rely on Cloud Logging implicitlyadding the conjunctive restriction,AND, between the statements. For example,suppose a query or filter dialog contains two statements,resource.type = "gce_instance" andseverity >= "ERROR".The actual query isresource.type = "gce_instance" AND severity >= "ERROR".Cloud Logging supports both disjunctive restrictions,OR, and conjunctiverestrictions,AND. When you useOR statements, we recommend that yougroup the clauses with parentheses.

• Disjunctive restriction:resource.type = "k8s_cluster" OR resource.type = "gce_instance"

  • Alternatively:resource.type = ("k8s_cluster" OR "gce_instance")

• Complex conjunctive/disjunctive expression:resource.type = "k8s_cluster" AND (severity = "ERROR" OR "error")

The following is an example of a query:

resource.type = "gce_instance" ANDseverity >= "ERROR" ANDNOT textPayload:robot

This query matches log entries from Compute Engine that have severity valuesof at leastERROR and whosetextPayload field doesn't contain the stringrobot anywhere inside it. String comparisons aren't case sensitive. The namesresource,severity, andtextPayload are defined in theLogEntry type.

Syntax notation

The following sections provide an overview of the Logging query languagesyntax, and discuss in detail how queries are structured and how matching isperformed. Some of the examples usecomments to provide explanatorytext.

Note the following:

• The length of a query can't exceed 20,000 characters.

• The Logging query language is case-insensitive, with the exceptionof regular expressions and logical operators, likeAND andOR.Logical operators must be capitalized.

Syntax summary

The Logging query language syntax can be thought of in terms ofqueriesandcomparisons.

Aquery is a string containing anexpression:

expression = ["NOT"] comparison { ("AND" | "OR") ["NOT"] comparison }

Acomparison is either a single value or a Boolean expression:

"The cat in the hat"resource.type = "k8s_cluster"

The first line is an example of a comparison that is a single value. Thesetypes of comparisons areglobal restrictions. Each field of a log entry iscompared to the value by implicitly using thehas operator. For thisexample, if any field in aLogEntry, or if its payload, contains the phrase"The cat in the hat", then the comparison is successful.

The second line is an example of a comparison that is a Boolean expression ofthe form[FIELD_NAME] [OP] [VALUE]. A comparison has three components:

• [FIELD_NAME] is a field in a log entry. For example,resource.type.

• [OP] is a comparison operator. For example,=.

• [VALUE] is a number, string, function, or parenthesized expression.For example,"k8s_cluster". For JSON null values, useNULL_VALUE.

Boolean operators

The Boolean operatorsAND andOR areshort-circuit operators.TheNOT operator has the highest precedence, followed byOR andANDin that order. For example, the following two expressions are equivalent:

"a" OR NOT "b" AND NOT "c" OR "d"("a" OR (NOT "b")) AND ((NOT "c") OR "d")

You can omit theAND operator between comparisons. You can also replacetheNOT operator with the- (minus) operator. For example, thefollowing two queries are the same:

a="b" AND c="d" AND NOT e="f"a="b" c="d" -e="f"

This documentation always usesAND andNOT.

For all filters except filters used bylog views,you can useAND,OR, andNOT operators. Log views only supportAND andNOT operations.

To combineAND andOR rules in the same expression, you must nest therules using parentheses. If you don't use parentheses, your query might notwork as intended.

Boolean operators always need to be capitalized. Lowercaseand,or,andnot are parsed as search terms.

Comparisons

Comparisons have the following form:

[FIELD_NAME] [OP] [VALUE]

A comparison has three components:

• [FIELD_NAME]: is the field path identifier of a field in a log entry.The following are examples of these identifiers:

  resource.typeresource.labels.zoneresource.labels.project_idinsertIdjsonPayload.httpRequest.protocollabels."compute.googleapis.com/resource_id"

  If a component of a field path identifier has special characters,then the component needs to be double-quoted. For example,compute.googleapis.com/resource_id must be double quoted becauseit contains a forward slash/.

  For details, seefield path identifiersin this document.

• [OP]: is a comparison operator, one of the following:

  =           -- equal!=          -- not equal>< >=<=   -- numeric ordering:           -- "has" matches any substring in the log entry field=~          -- regular expression search for a pattern!~          -- regular expression search not for a pattern

To learn how to search log entries using regular expressions, seeUsing regular expressions.

• [VALUE]: is a number, string, function, orparenthesized expression.Strings are used to represent arbitrary text, plus Boolean, enumeration,and byte-string values. The[VALUE] is converted to the field's type priorto the comparison. For JSON null values, useNULL_VALUE.

To filter for a JSON null value, use the following syntax:

jsonPayload.field = NULL_VALUE      -- includes "field" with null valueNOT jsonPayload.field = NULL_VALUE  -- excludes "field" with null value

If[VALUE] is a parenthesized Boolean combination of comparisons,then the field name and the comparison operator are applied to each element.For example:

jsonPayload.cat = ("longhair" OR "shorthair")jsonPayload.animal : ("nice" AND "pet")

The first comparison checks that the fieldcat has the value "longhair" or"shorthair". The second checks that the value of the fieldanimal containsboth of the words "nice" and "pet", in any order.

All log entries are instances of typeLogEntry. Theidentifier that is (or begins) the left-hand side of a comparison must be afield defined in theLogEntry type. For details on the possible identifiersand their values, see theLogEntry type.

Here is the current list of log entry fields. Each field is followed by thenext level of names for that field, if applicable:

• httpRequest: {cacheFillBytes,cacheHit,cacheLookup,cacheValidatedWithOriginServer,latency,protocol,referer,remoteIp,requestMethod,requestSize,requestUrl,responseSize,serverIp,status,userAgent }
• insertId
• jsonPayload {variable }
• labels {variable }
• logName
• metadata {systemLabels,userLabels }
• operation{id,producer,first,last }
• protoPayload {@type,variable }
• receiveTimestamp
• resource {type,labels }
• severity
• sourceLocation: {file,line,function }
• spanId
• textPayload
• timestamp
• trace

Following are examples of field path identifiers you can use in yourcomparisons:

• resource.type: If your first path identifier isresource, then the nextidentifier must be a field in theMonitoredResourcetype.

• httpRequest.latency: If your first path identifier ishttpRequest,then the next identifier must be a field in theHttpRequesttype.

• labels.[KEY] If your first path identifier islabels, then the nextidentifier,[KEY], must be one of the keys from the key-value pairsappearing in thelabels field.

• logName: Since thelogName field is a string, you can't follow it byany subfield names.

When you querymap orstruct fields, you must preserve theirkeys' letter case and formatting in your expression.

For example,jsonPayload is a struct field, so a field name nested insidejsonPayload likejsonPayload.end_time differs fromjsonPayload.endTime. Similarly, for a map field likelabels, the label keylabels.env_name is different thanlabels.envName. In contrast, whenquerying the regular protocol buffer fieldprotoPayload, youdon't need to preserve case.

For information on theLogEntry field types, see thegoogle.logging.v2 reference.

Special characters

If aLogEntry field contains special characters, the log field must be quoted.For example:

jsonPayload.":name":applejsonPayload."foo.bar":applejsonPayload."\"foo\"":apple

For the list of special characters, see thestring section inValues and conversions.

For more information on using field path identifiers that reference objects orarrays, seeObject and array types in this document.

Monitored resource types

For faster queries, specify a monitored resource type. For a list of resourcetypes, seeMonitored resource types.

For example, Compute Engine VMs use the resource typegce_instanceand Amazon EC2 instances useaws_ec2_instance. The following example showshow to limit your queries to both type of VMs:

resource.type = ("gce_instance" OR "aws_ec2_instance")

The monitored resource type values in logs are indexed. Using substring matchesfor them results in slower queries.

Missing fields

If you use a field name in a query, and that field doesn't appear in a logentry, then the field ismissing,undefined, ordefaulted:

• If the field is part of the log entry's payload (jsonPayloadorprotoPayload), or if it is in a label in thelabels section ofthe log entry, then the field ismissing. Using a missing fieldwon't display an error, but all comparisons using missing fields failsilently.

  Examples:jsonPayload.nearest_store,protoPayload.name.nickname

• If the field is defined in theLogEntrytype, then the field isdefaulted. Comparisons are performed as ifthe field were present and had its default value.

  Examples:httpRequest.remoteIp,trace,operation.producer

• Otherwise, the field isundefined, which is an error that is detectedbefore the query is used.

  Examples:thud,operation.thud,textPayload.thud

To test if a missing or defaulted field exists without testing for a particularvalue in the field, use the:* comparison. For example, the followingcomparison succeeds if the fieldoperation.id is explicitly present in a logentry:

operation.id:*

Note the behavior of the following queries:

• When you use the BooleanNOT operator on a missing field, the result isTRUE:

  -- Returns TRUENOT missingField=foo

• When you use the not equal comparison operator!= on a missing field, theresult isFALSE:

  -- Returns FALSEmissingField!=foo

Object and array types

Each log entry field can hold ascalar,object, orarray.

• Ascalar field stores a single value, like174.4 or-1.Astring is also considered a scalar. Fields that can be converted to (orfrom) a string, such asDuration andTimestamp are also scalar types.

• An object type stores a collection of named values, like the followingJSON value:

  {"age": 24, "height": 67}

  You can refer to value inside an object. For example, ifjsonPayload.xcontained the preceding value, thenjsonPayload.x.age would have thevalue24.

• An array field stores a list of values—all of the sametype. For example, a field holding measurements might have an arrayof numbers:

  {8.5, 9, 6}

  When comparisons are performed and[FIELD_NAME] is an array field, eachmember of the array is compared to[VALUE] and the results are joinedtogether using theOR operator. For example, ifjsonPayload.shoeSizeis an array field that stores{8.5, 9, 6}, the comparison:

  jsonPayload.shoeSize < 7

  is equivalent to:

  8.5 < 7 OR 9 < 7 OR 6 < 7

  In this example, the overall comparison evaluates to successful.

Values and conversions

The first step in evaluating a comparison is to convert the right-hand sidevalue to the type of the log entry field. Scalar field types are permitted incomparisons, along with two additional types whose values are represented asstrings:Duration andTimestamp. For a list of scalar types, see thescalar protocol buffer types list. The following table explainswhat values can be converted to the log field types:

If an attempted conversion fails, then the comparison fails.

When a conversion requires a string, you can also use a number or unquoted textif they don't contain special characters such as spaces and operators.Similarly, when a conversion requires a number, you can use a string whosecontent is a number.

The typesintNN anduintNN represent integer types of various sizes, such asint32 anduint64. When writing a value to be converted to a 64-bit integertype, you write the value as a string, such as "9223372036854775807".

Types of log fields

Here is how the type of a log entry field is determined:

• Log fields defined in the typeLogEntry, and in the componenttype areprotocol buffer fields. Protocolbuffer fields have explicit types.

• Log fields that are part ofprotoPayload objects are alsoprotocol bufferfields and have explicit types. The name of the protocol buffer type isstored in the field"@type" ofprotoPayload. For more information,see theJSON mapping.

  When you are filtering on a field that is associated with theAnymessage type, thevalue field is automatically traversed. Therefore,don't include it in the query. For more information, seeTroubleshooting.

• Log fields inside ofjsonPayload have types that are inferred from thefield's value when the log entry is received:

  • Fields whose values are unquoted numbers have typedouble.
  • Fields whose values aretrue orfalse have typebool.
  • Fields whose values are strings have typestring.

  Long (64-bit) integers are stored in string fields, because they can't berepresented exactly asdouble values.

• TheDuration andTimestamp types are recognized only inprotocol buffer fields.Elsewhere, those values are stored in string fields.

Comments

Comments start with two dashes (--), and any text following the dashes isignored until the end of the line. Comments can be placed at the beginning of afilter, in between terms, and at the end of a filter.

You might use comments for the following cases:

• To annotate your complex filters with information about what a clause does:

   -- All of our target users are emitted by Compute Engine instances. resource.type = "gce_instance" -- Looking for logs from "alex". jsonPayload.targetUser = "alex"

• To quickly enable or disable a clause by adding or removing the commentprefix:

   resource.type = "gce_instance" -- jsonPayload.targetUser = "alex" jsonPayload.targetUser = "kiran" -- jsonPayload.targetUser = "sasha"

• All numeric types: Equality and inequality have their normal meaning fornumbers.
• bool: Equality means the same Boolean value. Inequality is defined bytrue>false.
• enum: Equality means the same enumeration value. Inequality uses the underlying numeric values of the enumeration literals.
• Duration: Equality means the same duration length. Inequality is based onthe length of the duration. Example: as durations,"1s">"999ms".
• Timestamp: Equality means the same instant in time. Ifa andb areTimestamp values,a <b meansa is earlier in time thanb.
• bytes: Operands are compared byte by byte, left-to-right.
• string: Comparisons ignore letter case. Specifically, both operands arefirst normalized usingNFKC_CF Unicode normalizationand then use lexicographic comparisons. However, regular expression searchesaren't normalized. For more information on searching log entries usingregular expressions, seeUsing regular expressions.

The substring operator (:) is applicable tostring andbytes, and ishandled like equality except that the right-hand operand need only equal somepart of the left-hand field. Substring matches on indexed fields don't takeadvantage of log indexes.

Global restrictions

If the comparison consists of a single value, it is called aglobal restriction. Logging uses thehas (:) operator to determineif any field in a log entry, or if its payload, contains the global restriction.If it does, then the comparison succeeds.

The simplest query written in terms of a global restriction is asingle value:

"The Cat in The Hat"

You can combine global restrictions using theAND andOR operators for amore interesting query. For example, if you want to display all log entriesthat have a field that containscat and a field that contains eitherhatorbat, write the query as:

("cat" AND ("hat" OR "bat"))

In this case, there are three global restrictions:cat,hat andbat. Theseglobal restrictions are applied separately and the results are combined, justas if the expression had been written without parentheses.

A global restriction is one way to query your logs for a particular value.For example, if you are looking in your activity log for entries containing anymention ofGCE_OPERATION_DONE, you can use the following query:

logName = "projects/my-project-id/logs/compute.googleapis.com%2Factivity_log" AND"GCE_OPERATION_DONE"

Instead of using global restrictions, which can be slow, we recommend thatyou use thebuilt-inSEARCH functionand query indexed fields. For more information,seeFinding log entries quickly in this document.

Functions

You can use built-in functions as global restrictions in queries:

function = identifier ( [ argument { , argument } ] )

whereargument is a value, field name, or aparenthesized expression.The functions are described in the following sections.

log_id

Thelog_id function is useful when creating a customlog view on alog bucket that contains log entries frommany Google Cloud projects, folders, or organizations.

The single argument to thelog_id function is a non-URL-encoded log ID:

log_id(non-URL-encoded log ID)

For example, the following query returns all activity audit logs:

log_id("cloudaudit.googleapis.com/activity")

In the previous query, the argument consists only ofalphanumeric characters and special characters, (/,_,-,.).

Thecast function accepts two parameters; theLogEntry field tobe casted, and the data type in which the field is converted to:

cast([FIELD], [TYPE][, OPTION])

The parameters of the previous expression are defined as follows:

• [FIELD]: The name of a field in the log entry, such aslogName orjsonPayload.a_field.

• [TYPE]: The data type, such asSTRING,INT64,FLOAT64,BOOL.

  • TIMESTAMP, orDURATION: Some data types offer additionaloptions, such as specifying anIANA Timezone Databasetimezone for theTIMESTAMP data type.

For example, the following query casts thetimestamp field into a STRINGand specifies theAmerica/New_York timezone:

cast(timestamp, STRING, TIME_ZONE("America/New_York")) =~ "^2025-04-02.*"

regexp_extract

Use theregexp_extract function to find the first substring that matches aregular expression:

REGEXP_EXTRACT([FIELD], [REGULAR_EXPRESSION])

In the previous expression, the fields are defined as follows:

• [FIELD]: The name of a field in the log entry, such aslogName orjsonPayload.a_field.
• [REGULAR_EXPRESSION]: TheRE2 regular expression that must containone capture group ((...)). A non-capturing group(?:...) must be used ifadditional grouping is required for the regular expression. Multiple capturegroups or no capture groups result in an error.

You can chain thecast andregexp_extract functions:

CAST(REGEXP_EXTRACT(CAST(timestamp, STRING), "\\d+:\\d+:(\\d+)"), INT64)< 30

The previous example casts thetimestamp field as a string. The regularexpression captures the value of the seconds field from thetimestamp string,and then casts it into an integer to perform a numeric comparison. The queryresult lists a log entry when the seconds field of its timestamp is less than30.

source

Thesource function matches log entries from a particular resource inthe organizations, folders, and Google Cloud projects hierarchy.

Thesource function doesn't match child resources. For example, usingsource(folders/folder_123) matches logs from thefolder_123 resource,and not logs from the Google Cloud project resources withinfolder_123.

To query for logs at a particular resource level, use the following syntax:

source(RESOURCE_TYPE/RESOURCE_ID)

sample

Thesample function selects a fraction of the total number of log entries:

sample([FIELD], [FRACTION])

[FIELD] is the name of a field in the log entry, such aslogName orjsonPayload.a_field. The value of the field determines whether the log entryis in the sample. The field type must be a string or numeric value.Setting[FIELD] toinsertId is a good choice, because every log entry hasa different value for that field.

[FRACTION] is the fraction of log entries that have values for[FIELD] toinclude. It is a number greater than 0.0 and no greater than 1.0. For example,if you specify0.01, then the sample contains roughly one percent of all logentries that have values for[FIELD]. If[FRACTION] is 1, then all the logentries that have values for[FIELD] are chosen.

Example: The following query returns 25 percent of the log entriesfrom logsyslog:

logName = "projects/my-project/logs/syslog" AND sample(insertId, 0.25)

Details:A deterministic algorithm, based on hashing, is used to determine if a log entryis included, or excluded, from the sample. The accuracyof the resulting sample is dependent on the distribution of the hashed values.If the hashed values aren't uniformly distributed,then the resulting sample can be skewed.In the worst case, when[FIELD] always contains the same value,the resulting sample contains either the[FRACTION] of all log entries or nolog entries.

If[FIELD] does appear in a log entry, then:

• A hash of the value is computed.
• The hashed value, which is a number, is divided by the maximum possible hashed value.
• If the resulting fraction is less than or equal to[FRACTION], the log entry is included in the sample; otherwise it is excluded from the sample.

If[FIELD] doesn't appear in a log entry, then:

• If[FIELD] is part of the log entry's payload orlabels sections,the log entry isn't selected for the sample, even if[FRACTION] is 1.
• Otherwise, the log entry is treated as if[FIELD] is in the log entryand the value of[FIELD] is the default value. The default value isdetermined by theLogEntrytype. For more information on missing and defaulted fields, seeMissing fields in this document.

To exclude log entries with defaulted fields from the sample, use thefield-exists operator,:*. The following query produces a 1 percent sample oflog entries that have explicitly supplied a value forfield:

field:* AND sample(field, 0.01)

ip_in_net

Theip_in_net function determines if an IP address in a log entry is containedin a subnet. You might use this to tell if a request comes from an internal orexternal source. For example:

ip_in_net([FIELD], [SUBNET])

[FIELD] is a string-valued field in the log entry that contains an IP addressor range. The field can be repeating, in which case only one of the repeatedfields has to have an address or range contained in the subnet.

[SUBNET] is a string constant for an IP address or range. It is an error if[SUBNET] isn't a legal IP address or range, as described later in thissection.

Example: The following query tests an IP address in the payload of logentries from the logmy_log:

logName = "projects/my_project/logs/my_log" ANDip_in_net(jsonPayload.realClientIP, "10.1.2.0/24")

Details: If, in a log entry,[FIELD] is missing, defaulted, or it does notcontain a legal IP address or range, then the function returns false. For moreinformation on missing and defaulted fields, seeMissing fields in this document.

Examples of the supported IP addresses and ranges follow:

• IPv4:10.1.2.3
• IPv4 subnet:10.1.2.0/24
• CIDR IPv6:1234:5678:90ab:cdef:1234:5678:90ab:cdef
• CIDR IPv6 subnet:1:2::/48

SEARCH function

SEARCH([query])SEARCH([field], [query])

When theSEARCH function is processed, thequery string is processed bya text analyzer that splits the string into tokens. Cloud Logging alwaysperforms case-insensitive comparisons, even for tokenswrapped with backticks. This behavior differs from that of BigQuery,which preserves case in tokens wrapped with backticks.For information about the analyzer rules, see the BigQuery documentText analyzer rules.

When constructing a search, consider the following:

• Tokens are case-insensitive. The following functions produce the sameresults:

  SEARCH("world")SEARCH("World")

  The previous functions match a log entry when a single field contains thetoken "world". BecauseSEARCH performs exact matches and not substringmatches, the previous functions don't match a field whose value is"worldwide".

• If you don't specify the field to search, then theSEARCH functionmatches a log entry when that log entry contains all tokens. However,the order of tokens doesn't matter and the tokens aren't required tobe found in the same field of the log entry.

  The following functions produce the same results, and they match a log entrythat contains the tokens "hello" and "world":

  SEARCH("hello world")SEARCH("World hello")

• If you specify the field to search, then theSEARCH function onlysearches that field. A match occurs when that field contains all tokens;however, the order of tokens doesn't matter.

  The following functions produce a match only when thetextPayload fieldcontains the tokens "hello" and "world":

  SEARCH(textPayload, "hello world")

• To impose a case-insensitive but exact match on a phrase,enclose the phrase in backticks.For example, the following functions match the string "hello world":

  SEARCH("`hello world`")SEARCH("`Hello World`")SEARCH("`HELLO WORLD`")

  Because backticks are used in the following functions, they produce differentresults:

  SEARCH("`hello world`")SEARCH("`world hello`")

The Logging query language supports different ways that you can searchyour log data. When searching for a string, it is more efficient to use theSEARCH function than to perform a global search or a substring search.However, you can't use theSEARCH function to match non-text fields.For guidance on performing search operations, seeMinimize global and substring searches.

Searching by time

In the interface, you can set specific limits on thedate and time of log entries to show. For example, if you add the followingconditions to your query, the results pane displays exactly the log entriesin the indicated 30-minute period and you won't be able to scroll outsideof that date range:

timestamp >= "2023-11-29T23:00:00Z"timestamp<= "2023-11-29T23:30:00Z"

When writing a query with a timestamp, you must use dates and times in theformat shown previously.

You can also search log entries usingtimestamp shortcuts. For example,you can enter a date with a comparison operator to get all log entries after acertain day:

timestamp > "2023-11-29"

You can use regular expressions to build queries and create filters forsinks, metrics, and wherever log filters are used. For example, you can useregular expressions in theLogs Explorer and withGoogle Cloud CLI.

A regular expression is a sequence of characters that define a search. TheLogging query language uses the RE2 syntax. For a complete explanation ofthe RE2 syntax, see theRE2 wiki on GitHub.

Regular expression queries have the following characteristics:

• Only fields of the string type can be matched with a regular expression.

• String normalization isn't performed; for example,kubernetes isn'tconsidered the same asKUBERNETES. For more information,see theComparison operators section.

• Queries are case sensitive and aren't anchored by default.

• Boolean operators can be used between multiple regular expressions on theright side of the regular expression comparison operator,=~ and!~.

A regular expression query has the following structure:

Match a pattern:

jsonPayload.message =~ "regular expression pattern"

Does not match a pattern:

jsonPayload.message !~ "regular expression pattern"

The=~ and!~ changes the query to a regular expression query, and thepattern you're trying to match must be within double quotation marks. To queryfor patterns thatcontain double quotation marks, escape them using abackslash.

Finding log entries quickly

To find log entries efficiently, do the following:

• Query using indexed fields.
• Minimize the number of log entries that must be searched.

Use indexed fields

Logging always indexes the followingLogEntry fields:

• resource.type
• resource.labels.*
• logName
• severity
• timestamp
• insertId
• operation.id
• trace
• httpRequest.status
• labels.*
• split.uid

You can also addcustom indexed fields toany log bucket or make use of theSEARCH function to accelerate your query.

Optimize your queries

Make your searches faster by reducing the number of logs, the number of logentries, or the time span of your searches. Even better, you can reduce allthree.

Example: Use theSEARCH function

Instead of doing global or substring searches, you can use theSEARCHfunction to make use of indexes to optimize your queries.

Example: Use the right log name

Specify the log containing the log entries you're interested in. Be sure youknow the actual log name by inspecting one of your log entries. For example, theresults pane shows that the Compute Engine section contains a log named"activity". On closer inspection of the Admin Activity audit log entries, the logis actually named "cloudaudit.googleapis.com/activity".

The following comparison isincorrect. It doesn't match anything because ituses the wrong log name:

logName = "projects/my-project-id/logs/activity"   -- WRONG!

The following comparison iscorrect. It chooses log entries from theAdmin Activity audit log entries. You must URL-encode the log name, as shown:

logName = "projects/my-project-id/logs/cloudaudit.googleapis.com%2Factivity"

Example: Choose the right log entries

If you know that the log entries you want are coming from a particular VMinstance, then specify it. Check for the right label names by inspecting one ofthe log entries that you want to search for. In the following example,instance_id is one of the indexed labels:

resource.type = "gce_instance" ANDresource.labels.instance_id = "6731710280662790612"logName = "projects/my-project-id/logs/cloudaudit.googleapis.com%2Factivity"

Example: Choose the right time period

Specify a time period to search in. A quick way of determining usefultimestamps in RFC 3339 format is to use the GNU/Linuxdate command:

$ date --rfc-3339=s2023-06-27 17:39:00-04:00$ date --rfc-3339=s --date="3 hours ago"2023-06-27 14:40:00-04:00$ date --rfc-3339=s --date="5 hours ago"2023-06-27 12:40:00-04:00

Use the values of these timestamps in the following queries. To create atimestamp acceptable to Logging, replace the space between thedate and time with the letterT.

For example, to search within the last three hours:

timestamp >= "2023-06-27T14:40:00-04:00"

As another example, to search between three and five hours ago:

timestamp >= "2023-06-27T12:40:00-04:00" ANDtimestamp<= "2023-06-27T14:40:00-04:00"

Minimize disjunctive restrictions

Queries that use only conjunctive restrictions,AND, can make better useof indexes. You can use disjunctive restriction,OR, but these queries canbe slow and cannot make use of indexes.

For example,SEARCH("foo") AND SEARCH("bar") makes use of indexes and will befaster than a query likeSEARCH("foo") OR SEARCH("bar").

Minimize global and substring searches

Avoid the temptation to take shortcuts when typing queries.

Example: Don't use global searches

If you're searching for a log entry with "Hello Kitty" in the payload:

• Don't use a global search. For one reason, they are all substring searches:

     "Hello Kitty"   -- THIS CAUSES A SLOW SEARCH!

• Do limit the search to a single field, even if you must keep the substring search:

     textPayload:"Hello Kitty"

• Do use an equality test if you can:

     textPayload = "Hello Kitty"

• Do reference individual fields in a payload, if your log entries have structured payloads:

     jsonPayload.my_favorite_cat = "Hello Kitty"

• Do use an indexed field to restrict the search:

     logName = "projects/my-project_id/logs/somelog" AND   jsonPayload.my_favorite_cat = "Hello Kitty"

• Do use theSEARCH function and specify the complete text to match. TheSEARCH function performs a case-insensitive match:

    SEARCH("Hello Kitty")

  Don't use theSEARCH function and specify partial text. For example, the following function doesn't match "Hello Kitty".

    SEARCH("Hello Kit")

Searching examples

The log entries shown are the ones that match a query.If theJump to time menu contains a value, thenthe display scrolls to that point in time. Here are some query examples:

resource.type=k8s_cluster

Finds all Google Kubernetes Engine log entries. For a list of resource types,seeMonitored resource list.

As you type, the query pane suggests completions for fieldslikeresource.type.

resource.type=k8s_cluster AND logName:request_log

Finds log entries for Google Kubernetes Engine clusters from log names containingrequest_log. Note several things:

• The= operator is exact equality. The resource type must beexactly"k8s_cluster" except for letter case.
• The: operator means "has". ThelogName field must containrequest_log, in any letter case. The actual log name is muchlonger. Using: might result in slower searches.
• The two comparisons are joined byAND. You can also useOR, butANDis assumed if you leave out the operator.

resource.type = (gce_instance OR aws_ec2_instance) AND severity >= ERROR

Finds log entries with either of two resource types: Compute Engine VMinstance or AWS EC2 VM instance. The log entries must haveseverity of atleastERROR, which is equivalent to selectingERROR in the queryinterface's severity menu.

logName = "projects/[PROJECT_ID]/logs/cloudaudit.googleapis.com%2Factivity"

Finds all the Admin Activity audit log entries in the project[PROJECT_ID].Audit logs all use the same log name in a project, but have differentresource types. The log ID,cloudaudit.googleapis.com/activity must beURL-encoded in the log name. Using equality in the comparison speeds up thesearch. For more information, seeUnderstanding audit logs.

unicorn

Finds log entries containingunicorn in any field, in any letter case. Asearch term that isn't part of a field comparison is an "all fields" query.

unicorn phoenix

Finds log entries that containunicorn in some field andphoenix insome field.

textPayload:(unicorn phoenix)

Finds log entries whosetextPayload field contains bothunicorn andphoenix in any order—theAND is implicit between the two words.

textPayload:"unicorn phoenix"

Finds log entries whosetextPayload field contains the string"unicorn phoenix".

NOT textPayload: "unicorn phoenix"

Finds log entries whosetextPayload field doesnot contain the string"unicorn phoenix". This type of query reduces unwanted log entries.

timestamp >= "2023-11-29T23:00:00Z" timestamp <= "2023-11-29T23:30:00Z"

Finds log entries within a 30-minute period.

Troubleshooting

Syntax issues

If you have problems with your queries' expressions, check thefollowing:

• Your query obeys the syntax rules, with matched parentheses and quotationmarks.

• Your log entry field names are correctly spelled.

• Boolean operations are in uppercase letters (AND,OR,NOT).

• Ensure that you're usingNULL_VALUEto represent JSON null values.

• Boolean expressions as global restrictions or as the right-hand side ofcomparisons should be parenthesized for clarity. For example, thefollowing two queries look the same, but are not:

  insertId = "ABC-1" OR "ABC-2"  -- ERROR!?insertId = ("ABC-1" OR "ABC-2")

• Unquoted text must not contain any special characters. When in doubt, adddouble quotation marks. For example, in the following, the first comparisonis illegalbecause of the embedded substring operator (:). The comparison must bewritten with quotation marks:

  insertId = abc:def  -- ILLEGAL!insertId = "abc:def"

  Note: If you use the search bar and if you don't wrap the search terms indouble quotes, then theSEARCH functionis used. TheSEARCH function performs a search alongtoken boundaries. For example, if you enterworld in the search bar, thenLogging runs the querySEARCH("world"). The previousquery matchesworld but it doesn't matchworlds.

• TheGoogle Cloud CLI requiresthe query to be in double quotes. To use double quotes for escaping specialcharacters using thegcloud logging command, wrap the entire query withsingle quotes instead:

  gcloud logging read 'resource.type=gce_instance AND jsonPayload.message="Stopped Unattended Upgrades Shutdown."'gcloud logging read 'timestamp>="2020-06-17T21:00:00Z"'

• When you are filtering on a field that is associated with theAnymessage type, thevalue field is automatically traversed. Therefore,don't includevalue in the query.

  For example, theStatusfield in anAuditLogmessage has adetails field that is of typegoogle.protobuf.Any.To query thedetails field, omit thevalue field when specifying thefilter:

  • Do

    protoPayload.status.details.conditionNotMet.userVisibleMessage =~ "Specified reservation.*"

  • Don't

    protoPayload.status.details.value.conditionNotMet.userVisibleMessage =~ "Specified reservation.*"