Movatterモバイル変換

[0]ホーム
URL:

Filter pattern syntax for metric filters, subscription filters, filter log events, and Live Tail - Amazon CloudWatch Logs
DocumentationAmazon CloudWatchUser Guide
Supported regular expressionsMatch terms using regular expressionsMatch terms in unstructured log eventsMatch terms in JSON log eventsMatch terms in space-delimited log events

Filter pattern syntax for metric filters, subscription filters, filter log events, and Live Tail

With CloudWatch Logs, you can usemetric filters to transform log data into actionable metrics,subscription filters to route log events to other AWS services,filter log events to search for log events, andLive Tail to interactively view your logs in real-time as they are ingested.

Filter patterns make up the syntax that metric filters, subscription filters, log events, and Live Tail use to match terms in log events. Terms can be words, exact phrases, or numeric values. Regular expressions (regex) can be used to create standalone filter patterns, or can be incorporated with JSON and space-delimited filter patterns.

Create filter patterns with the terms that you want to match. Filter patterns only return the log events that contain the terms you define. You can test filter patterns in the CloudWatch console.

Supported regular expressions (regex) syntax

When using regex to search and filter log data, you must surround your expressions with%.

Filter patterns with regex can only include the following:

  • Alphanumeric characters – An alphanumeric character is a character that is either a letter (from A to Z or a to z) or a digit (from 0 to 9).

  • Supported symbol characters – These include: ':', '_', '#', '=', '@','/', ';', ',', and '-'. For example,%something!% would be rejected since '!' is not supported.

  • Supported operators – These include: '^', '$', '?', '[', ']', '{', '}', '|', '\', '*', '+', and '.'.

The( and) operators are not supported. You cannot use parentheses to define a subpattern.

Multi-byte characters are not supported.

Usage of supported operators

Using filter patterns to match terms with a regular expression (regex)

Using filter patterns to match terms in unstructured log events

The following examples contain code snippets that show how you can use filter patterns to match terms in unstructured log events.

Example: Match a single term

The following code snippet shows an example of a single-term filter pattern that returns all log events where messages contain the wordERROR.

ERROR

This filter pattern matches log event messages, such as the following:

Example: Match multiple terms

The following code snippet shows an example of a multiple-term filter pattern that returns all log events where messages contain the wordsERROR andARGUMENTS.

ERROR ARGUMENTS

The filter returns log event messages, such as the following:

This filter pattern doesn't return the following log event messages because they don't contain both of the terms specified in the filter pattern.

Example: Match optional terms

You can use pattern matching to create filter patterns that return log events containing optional terms. Place a question mark ("?") before the terms that you want to match. The following code snippet shows an example of a filter pattern that returns all log events where messages contain the wordERROR or the wordARGUMENTS.

?ERROR ?ARGUMENTS

This filter pattern matches log event messages, such as the following:

Example: Match exact phrases

The following code snippet shows an example of a filter pattern that returns log events where messages contain the exact phraseINTERNAL SERVER ERROR.

"INTERNAL SERVER ERROR"

This filter pattern returns the following log event message:

Example: Include and exclude terms

You can create filter patterns that return log events where messages include some terms and exclude other terms. Place a minus symbol ("-") before the terms that you want to exclude. The following code snippet shows an example of a filter pattern that returns log events where messages include the termERROR and exclude the termARGUMENTS.

ERROR -ARGUMENTS

This filter pattern returns log event messages, such as the following:

This filter pattern doesn't return the following log event messages because they contain the wordARGUMENTS.

Example: Match everything

You can match everything in your log events with double quotation marks. The following code snippet shows an example of a filter pattern that returns all log events.

" "

Using filter patterns to match terms in JSON log events

The following describes how to write the syntax for filter patterns that match JSON terms containing strings and numeric values.

Writing filter patterns that match strings

You can create filter patterns to match strings in JSON log events. The following code snippet shows an example of the syntax for string-based filter pattern.

{PropertySelectorEqualityOperatorString }

Enclose filter patterns in curly braces ("{}"). String-based filter patterns must contain the following parts:

The following code snippet contains an example of a filter pattern showing how you can format a filter pattern to match a JSON term with a string.

{ $.eventType = "UpdateTrail" }
Writing filter patterns that match numeric values

You can create filter patterns to match numeric values in JSON log events. The following code snippet shows an example of the syntax for filter patterns that match numeric values.

{PropertySelectorNumericOperatorNumber }

Enclose filter patterns in curly braces ("{}"). Filter patterns that match numeric values must have the following parts:

The following code snippet contains examples showing how you can format filter patterns to match JSON terms with numeric values.

// Filter pattern with greater than symbol{ $.bandwidth > 75 }      // Filter pattern with less than symbol{ $.latency < 50 }// Filter pattern with greater than or equal to symbol{ $.refreshRate >= 60 } // Filter pattern with less than or equal to symbol{ $.responseTime <= 5 }// Filter pattern with equal sign{ $.errorCode = 400} // Filter pattern with not equal sign{ $.errorCode != 500 }// Filter pattern with scientific notation and plus symbol{ $.number[0] = 1e-3 } // Filter pattern with scientific notation and minus symbol{ $.number[0] != 1e+3 }

The following examples contain code snippets that show how filter patterns can match terms in a JSON log event.

JSON log event

{      "eventType": "UpdateTrail",      "sourceIPAddress": "111.111.111.111",      "arrayKey": [            "value",            "another value"      ],      "objectList": [{             "name": "a",             "id": 1           },{             "name": "b",             "id": 2           }      ],      "SomeObject": null,      "cluster.name": "c"}
Example: Filter pattern that matches string values

This filter pattern matches the string"UpdateTrail" in the property"eventType".

{ $.eventType = "UpdateTrail" }
Example: Filter pattern that matches string values (IP address)

This filter pattern contains a wild card and matches the property"sourceIPAddress" because it doesn't contain a number with the prefix"123.123.".

{ $.sourceIPAddress != 123.123.* }
Example: Filter pattern that matches a specific array element with a string value

This filter pattern matches the element"value" in the array"arrayKey".

{ $.arrayKey[0] = "value" }
Example: Filter pattern that matches a string using regex

This filter pattern matches the string"Trail" in the property"eventType".

{ $.eventType = %Trail% }
Example: Filter pattern that uses a wildcard to match values of any element in the array using regex

The filter pattern contain regex which matches the element"value" in the array"arrayKey".

{ $.arrayKey[*] = %val.{2}% }
Example: Filter pattern that uses a wildcard to match values of any element with a specific prefix and subnet using regex (IP address)

This filter pattern contains regex which matches the element"111.111.111.111" in the property"sourceIPAddress".

{ $.* = %111\.111\.111\.1[0-9]{1,2}% }
Example: Filter pattern that matches a JSON property with a period (.) in the key
{ $.['cluster.name'] = "c" }
Example: Filter pattern that matches JSON logs using IS

You can create filter patterns that match fields in JSON logs with theIS variable. TheIS variable can match fields that contain the valuesNULL,TRUE, orFALSE. The following filter pattern returns JSON logs where the value ofSomeObject isNULL.

{ $.SomeObject IS NULL }
Example: Filter pattern that matches JSON logs using NOT EXISTS

You can create filter patterns with theNOT EXISTS variable to return JSON logs that don't contain specific fields in the log data. The following filter pattern usesNOT EXISTS to return JSON logs that don't contain the fieldSomeOtherObject.

{ $.SomeOtherObject NOT EXISTS }

You can use the logical operators AND ("&&") and OR ("||") in filter patterns to create compound expressions that match log events where two or more conditions are true. Compound expressions support the use of parentheses ("()") and the following standard order of operations: () > && > ||. The following examples contain code snippets that show how you can use filter patterns with compound expressions to match terms in a JSON object.

JSON object

{    "user":{        "id": 1,         "email": "John.Stiles@example.com"    },    "users": [{         "id": 2,         "email": "John.Doe@example.com"        },{         "id": 3,         "email": "Jane.Doe@example.com"        }    ],    "actions": [        "GET",        "PUT",        "DELETE"    ],    "coordinates": [        [0, 1, 2],        [4, 5, 6],        [7, 8, 9]    ]}
Example: Expression that matches using AND (&&)

This filter pattern contains a compound expression that matches"id" in"user" with a numeric value of1 and"email" in the first element of the"users" array with the string"John.Doe@example.com".

{ ($.user.id = 1) && ($.users[0].email = "John.Doe@example.com") }
Example: Expression that matches using OR (||)

This filter pattern contains a compound expression that matches"email" in"user" with the string"John.Stiles@example.com".

{ $.user.email = "John.Stiles@example.com" || $.coordinates[0][1] = "nonmatch" && $.actions[2] = "nonmatch" }
Example: Expression that doesn't match using AND (&&)

This filter pattern contains a compound expression that doesn't find a match because the expression doesn't match the third action in"actions".

{ ($.user.email = "John.Stiles@example.com" || $.coordinates[0][1] = "nonmatch") && $.actions[2] = "nonmatch" }
Example: Expression that doesn't match using OR (||)

This filter pattern contains a compound expression that doesn't find a match because the expression doesn't match the first property in"users" or the third action in"actions".

{ ($.user.id = 2 && $.users[0].email = "nonmatch") || $.actions[2] = "GET" }

Using filter patterns to match terms in space-delimited log events

You can create filter patterns to match terms in space-delimited log events. The following provides an example space-delimited log event and describes how to write the syntax for filter patterns that match terms in the space-delimited log event.

Example: Space-delimited log event

The following code snippet shows a space-delimited log event that contains seven fields:ip,user,username,timestamp,request,status_code, andbytes.

                                    127.0.0.1 Prod frank [10/Oct/2000:13:25:15 -0700] "GET /index.html HTTP/1.0" 404 1534
Writing filter patterns that match terms in a space-delimited log event

To create a filter pattern that matches terms in a space-delimited log event, enclose the filter pattern in brackets ("[]"), and specify fields with names that are separated by commas (","). The following filter pattern parses seven fields.

                                    [ip=%127\.0\.0\.[1-9]%, user, username, timestamp, request =*.html*, status_code = 4*, bytes]

You can use numeric operators ( >, <, =, !=, >=, or <=) and the asterisk (*) as a wild card or regex to give your filter pattern conditions. In the example filter pattern,ip uses regex that matches IP address range 127.0.0.1 - 127.0.0.9,request contains a wildcard that states it must extract a value with.html, andstatus_code contains a wildcard that states it must extract a value beginning with4.

If you don't know the number of fields that you're parsing in a space-delimited log event, you can use ellipsis (...) to reference any unnamed field. Elipsis can reference as many fields as needed. The following example shows a filter pattern with ellipsis that represent the first four unnamed fields shown in the previous example filter pattern.

[..., request =*.html*, status_code = 4*, bytes]

You also can use the logical operators AND (&&) and OR (||) to create compound expressions. The following filter pattern contains a compound expression that states the value ofstatus_code must be404 or410.

[ip, user, username, timestamp, request =*.html*, status_code = 404 || status_code = 410, bytes]

You can use pattern matching to create space-delimited filter patterns that match terms in a specific order. Specify the order of your terms with indicators. Usew1 to represent your first term andw2 and so on to represent the order of your subsequent terms. Place commas (",") between your terms. The following examples contain code snippets that show how you can use pattern matching with space-delimited filter patterns.

Space-delimited log event

INFO 09/25/2014 12:00:00 GET /service/resource/67 1200INFO 09/25/2014 12:00:01 POST /service/resource/67/part/111 1310WARNING 09/25/2014 12:00:02 Invalid user requestERROR 09/25/2014 12:00:02 Failed to process request
Example: Match terms in order

The following space-delimited filter pattern returns log events where the first word in the log events isERROR.

[w1=ERROR, w2]
Example: Match terms with AND (&&) and OR (||)

You can use the logical operators AND ("&&") and OR ("||") to create space-delimited filter patterns that contain conditions. The following filter pattern returns log events where the first word in the events isERROR orWARNING.

[w1=ERROR || w1=WARNING, w2]
Example: Exclude terms from matches

You can create space-delimited filter patterns that return log events excluding one or more terms. Place a not equal symbol ("!=") before the term or terms that you want to exclude. The following code snippet shows an example of a filter pattern that returns log events where the first words aren'tERROR andWARNING.

[w1!=ERROR && w1!=WARNING, w2]
Example: Match the top level item in a resource URI

The following code snippet shows an example of a filter pattern that matches the top level item in a resource URI using regex.

                                    [logLevel, date, time, method, url=%/service/resource/[0-9]+$%, response_time]
Example: Match the child level item in a resource URI

The following code snippet shows an example of a filter pattern that matches the child level item in a resource URI using regex.

                                    [logLevel, date, time, method, url=%/service/resource/[0-9]+/part/[0-9]+$%, response_time]
Log recursion prevention
Enable logging from AWS services
[8]ページ先頭
©2009-2025 Movatter.jp