The_source field contains the original JSON document body that was passed at index time. The_source field itself is not indexed (and thus is not searchable), but it is stored so that it can be returned when executingfetch requests, likeget orsearch.

If disk usage is important to you, then consider the following options:

• Usingsynthetic_source, which reconstructs source content at the time of retrieval instead of storing it on disk. This shrinks disk usage, at the cost of slower access to_source inGet andSearch queries.
• Disabling the_source field completely. This shrinks disk usage but disables features that rely on_source.

Though very handy to have around, the source field takes up a significant amount of space on disk. Instead of storing source documents on disk exactly as you send them, Elasticsearch can reconstruct source content on the fly upon retrieval. To enable this feature, use the valuesynthetic for the index settingindex.mapping.source.mode:

PUT idx{  "settings": {    "index": {      "mapping": {        "source": {          "mode": "synthetic"        }      }    }  }}

While this on-the-fly reconstruction isgenerally slower than saving the source documents verbatim and loading them at query time, it saves a lot of storage space. Additional latency can be avoided by not loading_source field in queries when it is not needed.

Synthetic_source is supported by all field types. Depending on implementation details, field types have different properties when used with synthetic_source.

Most field types construct synthetic_source using existing data, most commonlydoc_values andstored fields. For these field types, no additional space is needed to store the contents of_source field. Due to the storage layout ofdoc_values, the generated_source field undergoesmodifications compared to the original document.

For all other field types, the original value of the field is stored as is, in the same way as the_source field in non-synthetic mode. In this case there are no modifications and field data in_source is the same as in the original document. Similarly, malformed values of fields that useignore_malformed orignore_above need to be stored as is. This approach is less storage efficient since data needed for_source reconstruction is stored in addition to other data required to index the field (likedoc_values).

Some field types have additional restrictions. These restrictions are documented in thesynthetic_source section of the field type’sdocumentation.

Synthetic source is not supported insource-only snapshot repositories. To store indices that use synthetic_source, choose a different repository type.

When synthetic_source is enabled, retrieved documents undergo some modifications compared to the original JSON.

Synthetic_source arrays are moved to leaves. For example:

PUT idx/_doc/1{  "foo": [    {      "bar": 1    },    {      "bar": 2    }  ]}

Will become:

{  "foo": {    "bar": [1, 2]  }}

This can cause some arrays to vanish:

PUT idx/_doc/1{  "foo": [    {      "bar": 1    },    {      "baz": 2    }  ]}

Will become:

{  "foo": {    "bar": 1,    "baz": 2  }}

Synthetic source names fields as they are named in the mapping. When used withdynamic mapping, fields with dots (.) in their names are, by default, interpreted as multiple objects, while dots in field names are preserved within objects that havesubobjects disabled. For example:

PUT idx/_doc/1{  "foo.bar.baz": 1}

Will become:

{  "foo": {    "bar": {      "baz": 1    }  }}

This impacts how source contents can be referenced inscripts. For instance, referencing a script in its original source form will return null:

"script": { "source": """  emit(params._source['foo.bar.baz'])  """ }

Instead, source references need to be in line with the mapping structure:

"script": { "source": """  emit(params._source['foo']['bar']['baz'])  """ }

or simply

"script": { "source": """  emit(params._source.foo.bar.baz)  """ }

The followingfield APIs are preferable as, in addition to being agnostic to the mapping structure, they make use of docvalues if available and fall back to synthetic source only when needed. This reduces source synthesizing, a slow and costly operation.

"script": { "source": """  emit(field('foo.bar.baz').get(null))   """ }"script": { "source": """  emit($('foo.bar.baz', null))   """ }

Synthetic_source fields are sorted alphabetically. TheJSON RFC defines objects as "an unordered collection of zero or more name/value pairs" so applications shouldn’t care but without synthetic_source the original ordering is preserved and some applications may, counter to the spec, do something with that ordering.

Range field values (e.g.long_range) are always represented as inclusive on both sides with bounds adjusted accordingly. Seeexamples.

Values ofgeo_point fields are represented in synthetic_source with reduced precision. Seeexamples.

It is possible to avoid synthetic source modifications for a particular object or field, at extra storage cost. This is controlled through paramsynthetic_source_keep with the following option:

• none: synthetic source diverges from the original source as described above (default).
• arrays: arrays of the corresponding field or object preserve the original element ordering and duplicate elements. The synthetic source fragment for such arrays is not guaranteed to match the original source exactly, e.g. array[1, 2, [5], [[4, [3]]], 5] may appear as-is or in an equivalent format like[1, 2, 5, 4, 3, 5]. The exact format may change in the future, in an effort to reduce the storage overhead of this option.
• all: the source for both singleton instances and arrays of the corresponding field or object gets recorded. When applied to objects, the source of all sub-objects and sub-fields gets captured. Furthermore, the original source of arrays gets captured and appears in synthetic source with no modifications.

For instance:

PUT idx_keep{  "settings": {    "index": {      "mapping": {        "source": {          "mode": "synthetic"        }      }    }  },  "mappings": {    "properties": {      "path": {        "type": "object",        "synthetic_source_keep": "all"      },      "ids": {        "type": "integer",        "synthetic_source_keep": "arrays"      }    }  }}

PUT idx_keep/_doc/1{  "path": {    "to": [      { "foo": [3, 2, 1] },      { "foo": [30, 20, 10] }    ],    "bar": "baz"  },  "ids": [ 200, 100, 300, 100 ]}

returns the original source, with no array deduplication and sorting:

{  "path": {    "to": [      { "foo": [3, 2, 1] },      { "foo": [30, 20, 10] }    ],    "bar": "baz"  },  "ids": [ 200, 100, 300, 100 ]}

The option for capturing the source of arrays can be applied at index level, by settingindex.mapping.synthetic_source_keep toarrays. This applies to all objects and fields in the index, except for the ones with explicit overrides ofsynthetic_source_keep set tonone. In this case, the storage overhead grows with the number and sizes of arrays present in source of each document, naturally.

The following field types support synthetic source using data fromdoc_values orstored fields, and require no additional storage space to construct the_source field.

• aggregate_metric_double
• annotated-text
• binary
• boolean
• byte
• date
• date_nanos
• dense_vector
• double
• flattened
• float
• geo_point
• half_float
• histogram
• integer
• ip
• keyword
• long
• range types
• scaled_float
• short
• text
• version
• wildcard

Though very handy to have around, the source field does incur storage overhead within the index. For this reason, it can be disabled as follows:

PUT my-index-000001{  "mappings": {    "_source": {      "enabled": false    }  }}

An expert-only feature is the ability to prune the contents of the_source field after the document has been indexed, but before the_source field is stored.

Theincludes/excludes parameters (which also accept wildcards) can be used as follows:

PUT logs{  "mappings": {    "_source": {      "includes": [        "*.count",        "meta.*"      ],      "excludes": [        "meta.description",        "meta.other.*"      ]    }  }}PUT logs/_doc/1{  "requests": {    "count": 10,    "foo": "bar"  },  "meta": {    "name": "Some metric",    "description": "Some metric description",    "other": {      "foo": "one",      "baz": "two"    }  }}GET logs/_search{  "query": {    "match": {      "meta.other.foo": "one"    }  }}

1. These fields will be removed from the stored_source field.
2. We can still search on this field, even though it is not in the stored_source.