Repository files navigation

Provides a major mode for editing ES query examples. Better highlighting andindention than sh-mode or js-mode.

It is intended to be a mixture of the three modes as well as mimicing some ofthe highlighting from Kibana's Dev Tools Console.

You can also think of it asKibana's Console-envyfor Emacs users.

es-mode is available in the MELPA repository. Do this, if MELPA isn't already inyour sources:

(require'package)(add-to-list'package-archives             '("MELPA"."https://melpa.org/packages/" ))

Then runM-x package-refresh-contents to load the contents of the newrepository, andM-x package-install RET es-mode RET to installes-mode.

(add-to-list'load-path"/path/to/es-mode-dir")(autoload'es-mode"es-mode.el""Major mode for editing Elasticsearch queries"t)(add-to-list'auto-mode-alist '("\\.es$". es-mode))

You can now open a file with an.es extension andes-mode willautomatically load..

• Highlighting for builtin queries, facets, aggregations, special paramaters
• Acompany-mode backend for completing ESqueries
• Better indenting than sh-mode (indents like js-mode)
• Sending the queries as a http-request to Elasticsearch endpoints.
• Navigate via goto-(next|previous)-request withC-c C-n andC-c C-p (whenusing parameters)
• An Elasticsearch Command Center mode for monitoring clusters
• Hooks for responses, seees-response-success-functions andes-response-failure-functions
• Support for "es" mode blocks in org-mode
• Pass the resulting JSON throughjq to returnonly the values you want in org-mode
• Displays warning headers from Elasticsearch in the results buffers fordeprecated features
• Yasnippet support with built-insnippets for common operations
• Ability to copy a request as acurl request, or othertype of request (customizinges-copy-as-fn)
• Support for kibana-like triple quoting (""") for request bodies

If you would like to enabledhs-minor-mode automatically on ES resultsbuffers, use the following:

(add-hook'es-result-mode-hook'hs-minor-mode)

You can specify requests with two different formats:

In the document, specify parameters similar to Console, like so:

POST /myindex/_search?pretty{"query": {"match_all": {}  }}

HittingC-c C-c anywhere on the parameter or body of the request will executethe request, opening a response buffer. The base-url can be configured bycustomizing thees-default-url var.

You also don't have to provide the leading "/", similar to Console (I personallythink the leading "/" looks better though), like this:

POST myindex/_search?pretty{"query": {"match_all": {}  }}

If you do not want to specify?pretty every time, you can customize thees-always-pretty-print var (defaults tonil).

Without any parameters, you can specify a request:

{"query": {"match_all": {}  }}

With the request region highlighted or inside the query structure, hitC-c C-cto execute it. The first time you do this you will be prompted for the URL andHTTP method. You can also set the URL withC-c C-u and the method withC-c C-m.

This is deprecated in favor of the console-like syntax, as it increasescomplexity for maintaining es-mode.

If your ES endpoint has TLS enabled with self signed certificates (this is insecure!) you'll need to makecurl ignore those. There are two ways:

1. Specify--insecure in the Emacs variablerequest-curl-options.
2. Create a~/.curlrc containing--insecure (seethe curl man page).

Seetest.es,test2.es, andall.org, here's a screenshot from my theme:

And here's an example of the completion of queries/filters:

One of the main reasons I started this was better highlighting and indention fororg-babel. So add the snippet below to your .emacs:

(org-babel-do-load-languages 'org-babel-load-languages '((elasticsearch . t)))

And then you will be able to hitC-c C-c on code like this in your org-modefile:

#+BEGIN_SRC esPOST /_search?pretty{  "query": {    "match_all": {}  }}#+END_SRC

OR (without parameters):

#+BEGIN_SRC es :method POST :url localhost:9200/_search?pretty{  "query": {    "match_all": {}  }}#+END_SRC

org-mode uses the arguments:url and:method to know where and howto send a query. If they are not present org-mode will usees-default-url andes-default-request-method instead.

Tangling these blocks will produce<filename>.es, if you specify the filenamewith:tangle foo.sh, es-mode will instead create a curl request for the bodyof the request.

Es-mode supports rudimentary table creation from aggregations using the:tablify header parameter. For example, consider the following aggregations:

#+BEGIN_SRC es :tablify prices :results raw tablePOST /test/doc/_search{  "aggs" : {    "prices" : {      "histogram" : {        "field" : "price",        "interval" : 20,        "min_doc_count": 0      }    }  },  "size": 0}#+END_SRC#+RESULTS:| key | document count ||-----+----------------||   0 |              4 ||  20 |              0 ||  40 |              1 ||  60 |              2 ||  80 |              2 || 100 |              1 || 120 |              0 || 140 |              0 || 160 |              2 |

Note that the "tablify" argument must be the name of the aggregation to betablified, in this example, "prices" is the name of the argument.

This also works forterms aggregations:

#+BEGIN_SRC es :tablify my_terms_agg :results raw tablePOST /test/doc/_search{  "aggs" : {    "my_terms_agg" : {      "terms" : {        "field" : "type"      }    }  },  "size": 0}#+END_SRC#+RESULTS:| key      | document count ||----------+----------------|| eggplant |              5 || foo      |              4 || widget   |              2 || cog      |              1 |

If you are using org-mode 8.3.1 or later, you can generate pretty ASCII graphsfrom org-mode usingorgtbl-ascii-plotlike so (hitC-c C-c on theTBLFM line to generate the graph):

#+RESULTS:| key | document count |            ||-----+----------------+------------||   0 |              4 | WWWWWWWWWl ||  20 |              0 |            ||  40 |              1 | WWc        ||  60 |              2 | WWWWV      ||  80 |              2 | WWWWV      || 100 |              1 | WWc        || 120 |              0 |            || 140 |              0 |            || 160 |              2 | WWWWV      |#+TBLFM: $3='(orgtbl-ascii-draw $2 0 5)Or:#+RESULTS:| key      | document count |              ||----------+----------------+--------------|| eggplant |              5 | WWWWWWWWWWWW || foo      |              4 | WWWWWWWWWl   || widget   |              2 | WWWWV        || cog      |              1 | WWc          |#+TBLFM: $3='(orgtbl-ascii-draw $2 0 5)

Be sure to pass the correct minimum and maximum values for the table (in thisexample, 0 and 5) to theorgtbl-ascii-draw method.

In org-mode you can also reduce the size of results by passing them through thejq command-line tool. For example, compare the output of these two different orgblocks:

#+BEGIN_SRC esGET /{}#+END_SRC#+RESULTS:#+begin_example{  "status" : 200,  "name" : "Everyman",  "version" : {    "number" : "1.3.2",    "build_hash" : "dee175dbe2f254f3f26992f5d7591939aaefd12f",    "build_timestamp" : "2014-08-13T14:29:30Z",    "build_snapshot" : false,    "lucene_version" : "4.9"  },  "tagline" : "You Know, for Search"}#+end_example

And the same thing, but passed through thejq tool, extracting the "name" and"version.number" fields:

#+BEGIN_SRC es :jq .name, .version.numberGET /{}#+END_SRC#+RESULTS:: "Everyman": "1.3.2"

You can use this to return only a certain hit, or the score of a hit, etc,easily, so you can format the output as desired. See thefull jq manual for how to use jq.

es-mode usesjq in thePATH, however, if you want to specify an absolutepath you can customize thees-jq-path var as you like.

jq will only be run if the response is an HTTP 20[0-9].

es-mode includes support for variable substitution in org-babelsource blocks. The variable references in the body should be in theform${var-name}.

#+BEGIN_SRC es :var index="theindex"POST /${index}/_search?pretty{  "query": {    "match_all": {}  }}#+END_SRC

Vars can also be used to use the results from other org-babel blocks.

In the example below, the first source block searches the indexchild-docs for documents from the past month andjq is used toselect theparent-ids from the hits that are returned.

The second source block takes theparent-ids and binds it to thevariableids in the header of the source block (:var ids=parent-ids). When the code is run${ids} is replaced with theJSON array prior to executing the search request against theparent-docs index.

#+NAME: parent-ids#+BEGIN_SRC es :jq "[.hits.hits[]._source.\"parent-id\"]"  GET /child-docs/_search?pretty  {    "query": {        {"range": {"time": {"gte": "now-1M/d"}}}    },    "_source": "parent-id"  }#+END_SRC#+RESULTS: parent-ids#+begin_example[  "id1",  "id2",  "id3"]#+end_example#+BEGIN_SRC es :var ids=parent-ids  GET /parent-docs/_search?pretty  {    "query": {      {"ids": {"values": ${ids}}}    }  }#+END_SRC#+RESULTS:#+begin_example{  "took" : 227,  "timed_out" : false,  "_shards" : {    "total" : 4,    "successful" : 4,    "failed" : 0  },  "hits" : {    "total" : 3,    "max_score" : 5.3673162,    "hits" : [      {        "_index" : "parent-docs",        "_type" : "t",        "_id" : "id1",        "_score" : 5.3673162,        "_source" : {          ...        }      },      {        "_index" : "parent-docs",        "_type" : "t",        "_id" : "id2",        "_score" : 5.3673162,        "_source" : {          ...        }      },      {        "_index" : "parent-docs",        "_type" : "t",        "_id" : "id3",        "_score" : 5.3673162,        "_source" : {           ...        }      }    ]  }}#+end_example

es-mode includes a mode called the "Elasticsearch Command Center", which ismeant for monitoring your cluster. This provides a graphical representation ofwhat's happening in the cluster.

To invoke it, simply doM-x es-command-center. ES-CC will automaticallyrefresh ates-cc-refresh-interval seconds, check outM-x customize-group es-cc to see all of the customization options.

Here's a screenshot of what it looks like:

This is my first major mode for Emacs, feedback is welcome, especially pullrequests that show me what I'm doing wrong.