Text field type
A field to index full-text values, such as the body of an email or the description of a product. These fields areanalyzed, that is they are passed through ananalyzer to convert the string into a list of individual terms before being indexed. The analysis process allows Elasticsearch to search for individual wordswithin each full text field. Text fields are not used for sorting and seldom used for aggregations (although thesignificant text aggregation is a notable exception).
text fields are best suited for unstructured but human-readable content. If you need to index unstructured machine-generated content, seeMapping unstructured content.
If you need to index structured content such as email addresses, hostnames, status codes, or tags, it is likely that you should rather use akeyword field.
Below is an example of a mapping for a text field:
PUT my-index-000001{ "mappings": { "properties": { "full_name": { "type": "text" } } }}
Sometimes it is useful to have both a full text (text) and a keyword (keyword) version of the same field: one for full text search and the other for aggregations and sorting. This can be achieved withmulti-fields.
The following parameters are accepted bytext fields:
analyzer- Theanalyzer which should be used for the
textfield, both at index-time and at search-time (unless overridden by thesearch_analyzer). Defaults to the default index analyzer, or thestandardanalyzer. eager_global_ordinals- Should global ordinals be loaded eagerly on refresh? Accepts
trueorfalse(default). Enabling this is a good idea on fields that are frequently used for (significant) terms aggregations. fielddata- Can the field use in-memory fielddata for sorting, aggregations, or scripting? Accepts
trueorfalse(default). fielddata_frequency_filter- Expert settings which allow to decide which values to load in memory when
fielddatais enabled. By default all values are loaded. fields- Multi-fields allow the same string value to be indexed in multiple ways for different purposes, such as one field for search and a multi-field for sorting and aggregations, or the same string value analyzed by different analyzers.
index- Should the field be searchable? Accepts
true(default) orfalse. index_options- What information should be stored in the index, for search and highlighting purposes. Defaults to
positions. index_prefixes- If enabled, term prefixes of between 2 and 5 characters are indexed into a separate field. This allows prefix searches to run more efficiently, at the expense of a larger index.
index_phrases- If enabled, two-term word combinations (shingles) are indexed into a separate field. This allows exact phrase queries (no slop) to run more efficiently, at the expense of a larger index. Note that this works best when stopwords are not removed, as phrases containing stopwords will not use the subsidiary field and will fall back to a standard phrase query. Accepts
trueorfalse(default). norms- Whether field-length should be taken into account when scoring queries. Accepts
true(default) orfalse. position_increment_gap- The number of fake term position which should be inserted between each element of an array of strings. Defaults to the
position_increment_gapconfigured on the analyzer which defaults to100.100was chosen because it prevents phrase queries with reasonably large slops (less than 100) from matching terms across field values. store- Whether the field value should be stored and retrievable separately from the
_sourcefield. Acceptstrueorfalse(default). search_analyzer- The
analyzerthat should be used at search time on thetextfield. Defaults to theanalyzersetting. search_quote_analyzer- The
analyzerthat should be used at search time when a phrase is encountered. Defaults to thesearch_analyzersetting. similarity- Which scoring algorithm orsimilarity should be used. Defaults to
BM25. term_vector- Whether term vectors should be stored for the field. Defaults to
no. meta- Metadata about the field.
text fields can use akeyword sub-field to supportsynthetic_source without storing values of the text field itself.
In this case, the synthetic source of thetext field will have the samemodifications as akeyword field.
These modifications can impact usage oftext fields:
- Reordering text fields can have an effect onphrase andspan queries. See the discussion about
position_increment_gapfor more details. You can avoid this by making sure theslopparameter on the phrase queries is lower than theposition_increment_gap. This is the default. - Handling of
nullvalues is different.textfields ignorenullvalues, butkeywordfields support replacingnullvalues with a value specified in thenull_valueparameter. This replacement is represented in synthetic source.
For example:
PUT idx{ "settings": { "index": { "mapping": { "source": { "mode": "synthetic" } } } }, "mappings": { "properties": { "text": { "type": "text", "fields": { "kwd": { "type": "keyword", "null_value": "NA" } } } } }}PUT idx/_doc/1{ "text": [ null, "the quick brown fox", "the quick brown fox", "jumped over the lazy dog" ]}
Will become:
{ "text": [ "NA", "jumped over the lazy dog", "the quick brown fox" ]}If thetext field setsstore totrue then the sub-field is not used and no modifications are applied. For example:
PUT idx{ "settings": { "index": { "mapping": { "source": { "mode": "synthetic" } } } }, "mappings": { "properties": { "text": { "type": "text", "store": true, "fields": { "raw": { "type": "keyword" } } } } }}PUT idx/_doc/1{ "text": [ "the quick brown fox", "the quick brown fox", "jumped over the lazy dog" ]}
Will become:
{ "text": [ "the quick brown fox", "the quick brown fox", "jumped over the lazy dog" ]}text fields are searchable by default, but by default are not available for aggregations, sorting, or scripting. If you try to sort, aggregate, or access values from atext field using a script, you’ll see an exception indicating that field data is disabled by default on text fields. To load field data in memory, setfielddata=true on your field.
Loading field data in memory can consume significant memory.
Field data is the only way to access the analyzed tokens from a full text field in aggregations, sorting, or scripting. For example, a full text field likeNew York would get analyzed asnew andyork. To aggregate on these tokens requires field data.
It usually doesn’t make sense to enable fielddata on text fields. Field data is stored in the heap with thefield data cache because it is expensive to calculate. Calculating the field data can cause latency spikes, and increasing heap usage is a cause of cluster performance issues.
Most users who want to do more with text fields usemulti-field mappings by having both atext field for full text searches, and an unanalyzedkeyword field for aggregations, as follows:
PUT my-index-000001{ "mappings": { "properties": { "my_field": { "type": "text", "fields": { "keyword": { "type": "keyword" } } } } }}
- Use the
my_fieldfield for searches. - Use the
my_field.keywordfield for aggregations, sorting, or in scripts.
You can enable fielddata on an existingtext field using theupdate mapping API as follows:
PUT my-index-000001/_mapping{ "properties": { "my_field": { "type": "text", "fielddata": true } }}
- The mapping that you specify for
my_fieldshould consist of the existing mapping for that field, plus thefielddataparameter.
Fielddata filtering can be used to reduce the number of terms loaded into memory, and thus reduce memory usage. Terms can be filtered byfrequency:
The frequency filter allows you to only load terms whose document frequency falls between amin andmax value, which can be expressed an absolute number (when the number is bigger than 1.0) or as a percentage (eg0.01 is1% and1.0 is100%). Frequency is calculatedper segment. Percentages are based on the number of docs which have a value for the field, as opposed to all docs in the segment.
Small segments can be excluded completely by specifying the minimum number of docs that the segment should contain withmin_segment_size:
PUT my-index-000001{ "mappings": { "properties": { "tag": { "type": "text", "fielddata": true, "fielddata_frequency_filter": { "min": 0.001, "max": 0.1, "min_segment_size": 500 } } } }}