Movatterモバイル変換

[0]ホーム
URL:

Loading
  1. Elastic Docs/
  2. Deploy and manage/
  3. Monitoring/
  4. Logging

Slow query and index logging

The slow log records search and indexing operations that exceed time thresholds you define.You can use slow logs to investigate, analyze or audit heavy operations, or troubleshoot your cluster’s historical search and indexing performance.

Slow logs report task duration at theshard level for searches, and at theindex level for indexing, but might not encompass the full task execution time observed on the client. For example, slow logs don’t surface HTTP network delays or the impact oftask queues. For more information about the higher-level operations affecting response times, refer toReading and writing documents.

Slow log thresholds can be enabled for these logging levels (in order of increasing verbosity):

  • WARN
  • INFO
  • DEBUG
  • TRACE

You can mimic setting log level thresholds by disabling more verbose levels.

Because logging every event or operation generates a high volume of log entries, slow logs are deactivated by default (all thresholds are set to-1). Activate only when needed and avoid setting low thresholds in production.

Refer toslow log settings to learn more about configuration options you can adjust to capture search and indexing details.

Events that meet the specified threshold are emitted intoElasticsearch logging under thefileset.name ofslowlog. These logs can be viewed in the following locations:

  • IfElasticsearch monitoring is enabled, fromStack Monitoring. Slow log events have alogger value ofindex.search.slowlog orindex.indexing.slowlog.
  • From the local Elasticsearch service logs directory. Slow log files have a suffix of_index_search_slowlog.json or_index_indexing_slowlog.json.

Refer tothis video for a walkthrough of setting and reviewing slow logs.

Depending on the settings you configure, slow logs can record:

  • the operation (searching or indexing)
  • phase for searches (query or fetch)
  • how long the operation took
  • number of hits
  • which shard or index is affected
  • optional metadata (such as the_source request body oruser.* fields)
Tip

If a call was initiated with anX-Opaque-ID header, then the ID is automatically included in Search slow logs in theelasticsearch.slowlog.id field. SeeX-Opaque-Id HTTP header for details and best practices.

The following are examples of a search and an indexing operation in the slow log respectively:

{  "@timestamp": "2024-12-21T12:42:37.255Z",  "auth.type": "REALM",  "ecs.version": "1.2.0",  "elasticsearch.cluster.name": "distribution_run",  "elasticsearch.cluster.uuid": "Ui23kfF1SHKJwu_hI1iPPQ",  "elasticsearch.node.id": "JK-jn-XpQ3OsDUsq5ZtfGg",  "elasticsearch.node.name": "node-0",  "elasticsearch.slowlog.id": "tomcat-123",  "elasticsearch.slowlog.message": "[index6][0]",  "elasticsearch.slowlog.search_type": "QUERY_THEN_FETCH",  "elasticsearch.slowlog.source": "{\"query\":{\"match_all\":{\"boost\":1.0}}}",  "elasticsearch.slowlog.stats": "[]",  "elasticsearch.slowlog.took": "747.3micros",  "elasticsearch.slowlog.took_millis": 0,  "elasticsearch.slowlog.total_hits": "1 hits",  "elasticsearch.slowlog.total_shards": 1,  "event.dataset": "elasticsearch.index_search_slowlog",  "fileset.name" : "slowlog",  "log.level": "WARN",  "log.logger": "index.search.slowlog.query",  "process.thread.name": "elasticsearch[runTask-0][search][T#5]",  "service.name": "ES_ECS",  "user.name": "elastic",  "user.realm": "reserved"}
{  "@timestamp" : "2024-12-11T22:34:22.613Z",  "auth.type": "REALM",  "ecs.version": "1.2.0",  "elasticsearch.cluster.name" : "41bd111609d849fc9bf9d25b5df9ce96",  "elasticsearch.cluster.uuid" : "BZTn4I9URXSK26imlia0QA",  "elasticsearch.index.id" : "3VfGR7wRRRKmMCEn7Ii58g",  "elasticsearch.index.name": "my-index-000001",  "elasticsearch.node.id" : "GGiBgg21S3eqPDHzQiCMvQ",  "elasticsearch.node.name" : "instance-0000000001",  "elasticsearch.slowlog.id" : "RCHbt5MBT0oSsCOu54AJ",  "elasticsearch.slowlog.source": "{\"key\":\"value\"}"  "elasticsearch.slowlog.took" : "0.01ms",  "event.dataset": "elasticsearch.index_indexing_slowlog",  "fileset.name" : "slowlog",  "log.level" : "TRACE",  "log.logger" : "index.indexing.slowlog.index",  "service.name" : "ES_ECS",  "user.name": "elastic",  "user.realm": "reserved"}

You enable slow logs by configuring thresholds. Thresholds can be aggressive, such as0ms to log everything, or conservative, such as5s.

You can enable slow logging at the following levels:

To view the current slow log settings, use theget index settings API:

GET _all/_settings?expand_wildcards=all&filter_path=*.settings.index.*.slowlog

To enable slow logging for a single index, use theupdate indices settings API:

Search slow logs emit per shard. They must be enabled separately for the shard’squery and fetch search phases.

PUT /my-index-000001/_settings{  "index.search.slowlog.threshold.query.warn": "10s",  "index.search.slowlog.threshold.query.info": "5s",  "index.search.slowlog.threshold.query.debug": "2s",  "index.search.slowlog.threshold.query.trace": "500ms",  "index.search.slowlog.threshold.fetch.warn": "1s",  "index.search.slowlog.threshold.fetch.info": "800ms",  "index.search.slowlog.threshold.fetch.debug": "500ms",  "index.search.slowlog.threshold.fetch.trace": "200ms",  "index.search.slowlog.include.user": true}
  1. You can use theindex.search.slowlog.include.user setting for search operations or theindex.indexing.slowlog.include.user setting for indexing operations to appenduser.* andauth.type fields to slow log entries. These fields contain information about the user who triggered the request.

For more information about slow log settings, refer toslow log settings.

Indexing slow logs emit per index document.

PUT /my-index-000001/_settings{  "index.indexing.slowlog.threshold.index.warn": "10s",  "index.indexing.slowlog.threshold.index.info": "5s",  "index.indexing.slowlog.threshold.index.debug": "2s",  "index.indexing.slowlog.threshold.index.trace": "500ms",  "index.indexing.slowlog.source": "1000",  "index.indexing.slowlog.reformat": true,  "index.indexing.slowlog.include.user": true}

  1. You can use theindex.search.slowlog.include.user setting for search operations or theindex.indexing.slowlog.include.user setting for indexing operations to appenduser.* andauth.type fields to slow log entries. These fields contain information about the user who triggered the request.

  2. Slow logs can record the_source of documents involved in slow queries. Use this setting only while actively troubleshooting as it can significantly increase log size and might expose sensitive data.

For more information about slow log settings, refer toslow log settings.

To adjust slow log settings across all indices (cluster-wide), use the following settings in yourlog4j2.properties configuration file:

index.search.slowlog.threshold.query.warn: 10sindex.search.slowlog.threshold.query.info: 5sindex.search.slowlog.threshold.query.debug: 2sindex.search.slowlog.threshold.query.trace: 500msindex.search.slowlog.threshold.fetch.warn: 1sindex.search.slowlog.threshold.fetch.info: 800msindex.search.slowlog.threshold.fetch.debug: 500msindex.search.slowlog.threshold.fetch.trace: 200msindex.search.slowlog.include.user: true
index.indexing.slowlog.threshold.index.warn: 10sindex.indexing.slowlog.threshold.index.info: 5sindex.indexing.slowlog.threshold.index.debug: 2sindex.indexing.slowlog.threshold.index.trace: 500msindex.indexing.slowlog.source: 1000index.indexing.slowlog.reformat: trueindex.indexing.slowlog.include.user: true

Logging slow requests can be resource intensive to your Elasticsearch cluster depending on the qualifying traffic’s volume. For example, emitted logs might increase the index disk usage of yourElasticsearch monitoring cluster.

To reduce the impact of slow logs, consider the following:

  • Enable slow logs only when troubleshooting.
  • Enable slow logs against specific indices rather than across all indices.
  • Set high thresholds to reduce the number of logged events.

If you aren’t sure how to start investigating traffic issues, consider enabling thewarn threshold with a high30s threshold at the index level using theupdate indices settings API:

PUT /*/_settings{  "index.search.slowlog.include.user": true,  "index.search.slowlog.threshold.fetch.warn": "30s",  "index.search.slowlog.threshold.query.warn": "30s"}
PUT /*/_settings{  "index.indexing.slowlog.include.user": true,  "index.indexing.slowlog.threshold.index.warn": "30s"}

Slow log thresholds being met does not guarantee cluster performance issues. Slow logs can provide helpful data to diagnose upstream traffic patterns or sources to resolve client-side issues. For example, you can use data included inX-Opaque-ID, the_source request body, oruser.* fields to identify the source of your issue. This is similar to troubleshootinglive expensive tasks.

If you’re experiencing search performance issues, then you might want to consider investigating searches flagged for their query durations using theprofile API. You can then use the profiled query to investigate optimization options using thequery profiler. This type of investigation should usually take place in a non-production environment.

Slow logging checks each event against the reporting threshold when the event is complete. This means that it can’t report if events triggercircuit breaker errors. If you suspect circuit breaker errors, then you should also consider enablingaudit logging, which logs events before they are executed.

To learn about other ways to optimize your search and indexing requests, refer totune for search speed andtune for indexing speed.

[8]ページ先頭
©2009-2026 Movatter.jp