The highest tagged major version isv9.

bulk

package

v8.19.1Latest Latest

This package is not in the latest version of its module.

Go to latest Published: Dec 12, 2025 License:Apache-2.0Imports:14Imported by:10

Details

Valid go.mod file
Redistributable license
Tagged version
Stable version
Learn more about best practices

Repository

github.com/elastic/go-elasticsearch

Documentation ¶

Overview¶

Bulk index or delete documents.Perform multiple `index`, `create`, `delete`, and `update` actions in asingle request.This reduces overhead and can greatly increase indexing speed.

If the Elasticsearch security features are enabled, you must have thefollowing index privileges for the target data stream, index, or index alias:

* To use the `create` action, you must have the `create_doc`, `create`,`index`, or `write` index privilege. Data streams support only the `create`action.* To use the `index` action, you must have the `create`, `index`, or `write`index privilege.* To use the `delete` action, you must have the `delete` or `write` indexprivilege.* To use the `update` action, you must have the `index` or `write` indexprivilege.* To automatically create a data stream or index with a bulk API request, youmust have the `auto_configure`, `create_index`, or `manage` index privilege.* To make the result of a bulk operation visible to search using the`refresh` parameter, you must have the `maintenance` or `manage` indexprivilege.

Automatic data stream creation requires a matching index template with datastream enabled.

The actions are specified in the request body using a newline delimited JSON(NDJSON) structure:

```action_and_meta_data\noptional_source\naction_and_meta_data\noptional_source\n....action_and_meta_data\noptional_source\n```

The `index` and `create` actions expect a source on the next line and havethe same semantics as the `op_type` parameter in the standard index API.A `create` action fails if a document with the same ID already exists in thetargetAn `index` action adds or replaces a document as necessary.

NOTE: Data streams support only the `create` action.To update or delete a document in a data stream, you must target the backingindex containing the document.

An `update` action expects that the partial doc, upsert, and script and itsoptions are specified on the next line.

A `delete` action does not expect a source on the next line and has the samesemantics as the standard delete API.

NOTE: The final line of data must end with a newline character (`\n`).Each newline character may be preceded by a carriage return (`\r`).When sending NDJSON data to the `_bulk` endpoint, use a `Content-Type` headerof `application/json` or `application/x-ndjson`.Because this format uses literal newline characters (`\n`) as delimiters,make sure that the JSON actions and sources are not pretty printed.

If you provide a target in the request path, it is used for any actions thatdon't explicitly specify an `_index` argument.

A note on the format: the idea here is to make processing as fast aspossible.As some of the actions are redirected to other shards on other nodes, only`action_meta_data` is parsed on the receiving node side.

Client libraries using this protocol should try and strive to do somethingsimilar on the client side, and reduce buffering as much as possible.

There is no "correct" number of actions to perform in a single bulk request.Experiment with different settings to find the optimal size for yourparticular workload.Note that Elasticsearch limits the maximum size of a HTTP request to 100mb bydefault so clients must ensure that no request exceeds this size.It is not possible to index a single document that exceeds the size limit, soyou must pre-process any such documents into smaller pieces before sendingthem to Elasticsearch.For instance, split documents into pages or chapters before indexing them, orstore raw binary data in a system outside Elasticsearch and replace the rawdata with a link to the external system in the documents that you send toElasticsearch.

**Client suppport for bulk requests**

Some of the officially supported clients provide helpers to assist with bulkrequests and reindexing:

* Go: Check out `esutil.BulkIndexer`* Perl: Check out `Search::Elasticsearch::Client::5_0::Bulk` and`Search::Elasticsearch::Client::5_0::Scroll`* Python: Check out `elasticsearch.helpers.*`* #"#pkg-index" title="Go to Index" aria-label="Go to Index">¶

Constants¶

This section is empty.

Variables¶

View Source

var ErrBuildPath =errors.New("cannot build path, check for missing path parameters")

ErrBuildPath is returned in case of missing parameters within the build of the request.

Functions¶

This section is empty.

Types¶

typeBulk ¶

type Bulk struct {// contains filtered or unexported fields}

funcNew ¶

func New(tpelastictransport.Interface) *Bulk