Elastic Stack settings
Elastic Stack settings allow you to customize Elasticsearch, Kibana, and other Elastic Stack products to suit your needs.
In Elastic Cloud Serverless, cluster-level settings and node-level settings are not required by end users, and are fully managed by Elastic.
Certainproject settings allow you to customize your project’s performance and general data retention, and enable or disable project features.
The available Elastic Stack settings differ depending on your deployment type.
Elasticsearch settings can be found in theElasticsearch configuration reference.
| Deployment type | Applicable settings |
|---|---|
| Self managed | All Elasticsearch settings can be applied to a self-managed cluster. |
| Elastic Cloud Enterprise Elastic Cloud Hosted | Settings supported on Elastic Cloud Enterprise and Elastic Cloud Hosted are indicated by an Elastic Cloud icon ( Elastic Cloud Hosted and Elastic Cloud Enterprise block the configuration of certain settings that could break your cluster if misconfigured. If a setting is not supported, you will get an error message when you try to save. We suggest changing one setting with each save, so you know which one is not supported. |
| Elastic Cloud on Kubernetes | Most Elasticsearch settings can be applied to an ECK-managed Elasticsearch cluster. Some settings are managed by ECK. It is not recommended to change these managed settings. For a complete list, refer toSettings managed by ECK. |
Kibana settings can be found in theKibana configuration reference.
| Deployment type | Applicable settings |
|---|---|
| Self managed Elastic Cloud on Kubernetes | All Kibana settings can be applied to a self-managed or ECK instance. |
| Elastic Cloud Enterprise Elastic Cloud Hosted | Settings supported on Elastic Cloud Enterprise and Elastic Cloud Hosted are indicated by an Elastic Cloud icon ( Elastic Cloud Hosted and Elastic Cloud Enterprise block the configuration of certain settings that could break your cluster if misconfigured. If a setting is not supported, you will get an error message when you try to save. We suggest changing one setting with each save, so you know which one is not supported. |
| Elastic Cloud on Kubernetes | Most Elasticsearch settings can be applied to an ECK-managed Elasticsearch cluster. Some settings are managed by ECK. It is not recommended to change these managed settings. For a complete list, refer toSettings managed by ECK. |
For APM and Enterprise Search, refer to the product's documentation:
The way that you configure your Elastic Stack settings is determined by your deployment type.
- Dynamic Elasticsearch cluster settings can also be updated using Elasticsearch'supdate cluster settings API. However, Elastic Cloud Hosted and Elastic Cloud Enterprise don’t reject unsafe setting changes made using this API, and should be used with caution in these contexts.
- If a feature requires both standard
elasticsearch.ymlsettings andsecure settings, configure the secure settings first. Updating standard user settings can trigger a cluster rolling restart in self managed clusters and ECH and ECE deployments. If the required secure settings are not yet in place, the nodes might fail to start. Adding secure settings does not trigger a restart.
For Elastic Cloud Hosted and Elastic Cloud Enterprise deployments, you edit Elastic Stack settings through the Elastic Cloud Console or ECE Cloud UI. These settings are internally mapped to the appropriate YAML configuration files, such aselasticsearch.yml andkibana.yml, and they affect all users of that cluster.
Elastic Cloud Hosted and Elastic Cloud Enterprise block the configuration of certain settings that could break your cluster if misconfigured. If a setting is not supported, you will get an error message when you try to save. We suggest changing one setting with each save, so you know which one is not supported.
Log in to theElastic Cloud Console or ECECloud UI.
On the home page, find your deployment.
TipIf you have many deployments, you can instead go to theHosted deployments (Elastic Cloud Hosted) orDeployments (Elastic Cloud Enterprise) page. On that page, you can narrow your deployments by name, ID, or choose from several other filters.
SelectManage.
- Under the deployment's name in the navigation menu, selectEdit.
- Look for theManage user settings and extensions andEdit user settings links for each deployment, and select the one corresponding to the component you want to update, such as Elasticsearch or Kibana.
- Apply the necessary settings in theUsers Settings tab of the editor and selectBack when finished.
- SelectSave to apply the changes to the deployment. Saving your changes initiates a configuration plan change that restarts the affected components for you.
For further details and examples, refer to the resource for your deployment type:
Stack settings are defined as part of your resource specification.
Elasticsearch settings that are typically defined in theelasticsearch.yml configuration file can be specified for a set of nodes in thespec.nodeSets[?].config section of the Elasticsearch resource manifest.
Some settings are managed by ECK. It is not recommended to change these managed settings. For a complete list, refer toSettings managed by ECK.
spec: nodeSets: - name: masters count: 3 config: # On Elasticsearch versions before 7.9.0, replace the node.roles configuration with the following: # node.master: true node.roles: ["master"] xpack.ml.enabled: true - name: data count: 10 config: # On Elasticsearch versions before 7.9.0, replace the node.roles configuration with the following: # node.master: false # node.data: true # node.ingest: true # node.ml: true # node.transform: true node.roles: ["data", "ingest", "ml", "transform"]ECK parses Elasticsearch configuration and normalizes it to YAML. Consequently, some Elasticsearch configuration schema are impossible to express with ECK and, therefore, must be set usingdynamic cluster settings. For example:
spec: nodeSets: - name: data # ... config: cluster.max_shards_per_node: 1000 cluster.max_shards_per_node.frozen: 1000 # ...- This won't work because cluster.max_shards_per_node is defined as a scalar value on the previous line
You can add your own Kibana settings to thespec.config section.
The following example demonstrates how to set theelasticsearch.requestHeadersWhitelist configuration option.
apiVersion: kibana.k8s.elastic.co/v1kind: Kibanametadata: name: kibana-samplespec: version: 9.3.0 count: 1 elasticsearchRef: name: "elasticsearch-sample" config: elasticsearch.requestHeadersWhitelist: - authorizationThe method and location where you can update your Elastic Stack settings depends on the component and installation method.
Most settings can be changed on a running cluster using theCluster update settings API.
You can also set Elasticsearch settings inelasticsearch.yml. Some settings require a cluster restart. To learn more, refer toDynamic and static Elasticsearch settings.
To learn more about configuring Elasticsearch in a self-managed environment, refer toConfigure Elasticsearch.
| Installation method | Default location |
|---|---|
Archive distribution (tar.gz orzip) | $ES_HOME/config (override) |
| Package distribution (Debian or RPM) | /etc/elasticsearch (override) |
| Docker | /usr/share/elasticsearch/config/ (Learn more) |
To learn more about configuring Kibana in a self-managed environment, refer toConfigure Kibana.
| Installation method | Default location |
|---|---|
Archive distribution (tar.gz orzip) | $KIBANA_HOME/config (override) |
| Package distribution (Debian or RPM) | /etc/kibana (override) |
| Docker | /usr/share/kibana/config/ (Learn more) |
For APM and Enterprise Search, refer to the product's documentation:
Theelasticsearch.yml configuration format isYAML. Here is an example of changing the path of the data and logs directories in Elasticsearch:
path: data: /var/lib/elasticsearch logs: /var/log/elasticsearchSettings can also be flattened as follows:
path.data: /var/lib/elasticsearchpath.logs: /var/log/elasticsearchIn YAML, you can format non-scalar values as sequences:
discovery.seed_hosts: - 192.168.1.10:9300 - 192.168.1.11 - seeds.mydomain.comThough less common, you can also format non-scalar values as arrays:
discovery.seed_hosts: ["192.168.1.10:9300", "192.168.1.11", "seeds.mydomain.com"]Environment variables referenced with the${...} notation within theelasticsearch.yml configuration file will be replaced with the value of the environment variable. For example:
node.name: ${HOSTNAME}network.host: ${ES_NETWORK_HOST}Values for environment variables must be simple strings. Use a comma-separated string to provide values that the component will parse as a list. For example, Elasticsearch will split the following string into a list of values for the${HOSTNAME} environment variable:
export HOSTNAME="host1,host2"By default, configuration validation will fail if an environment variable used in the config file is not present when the component starts. This behavior can be changed by using a default value for the environment variable, using the${MY_ENV_VAR:defaultValue} syntax.
Some settings are sensitive, and relying on filesystem permissions to protect their values is not sufficient. For this use case, Elasticsearch and Kibana provide secure keystores to store sensitive configuration values such as passwords, API keys, and tokens.
Secure settings are often referred to askeystore settings, since they must be added to the product-specific keystore rather than the standardelasticsearch.yml orkibana.yml files. Unlike regular settings, they are encrypted and protected at rest, and they cannot be read or modified through the usual configuration files or environment variables.
If a feature requires both standardelasticsearch.yml settings andsecure settings, configure the secure settings first. Updating standard user settings can trigger a cluster rolling restart in self managed clusters and ECH and ECE deployments. If the required secure settings are not yet in place, the nodes might fail to start. Adding secure settings does not trigger a restart.
To learn how to interact with secure settings, refer toSecure your settings.
Elasticsearch cluster and node settings can be categorized based on how they are configured:
You can configure and update dynamic settings on a running cluster using thecluster update settings API. You can also configure dynamic settings locally on an unstarted or shut down node usingelasticsearch.yml.
Updates made using the cluster update settings API can bepersistent, which apply across cluster restarts, ortransient, which reset after a cluster restart. You can also reset transient or persistent settings by assigning them anull value using the API.
If you configure the same setting using multiple methods, Elasticsearch applies the settings in following order of precedence:
- Transient setting
- Persistent setting
elasticsearch.ymlsetting- Default setting value
For example, you can apply a transient setting to override a persistent setting orelasticsearch.yml setting. However, a change to anelasticsearch.yml setting will not override a defined transient or persistent setting.
We no longer recommend using transient cluster settings. Use persistent cluster settings instead. If a cluster becomes unstable, transient settings can clear unexpectedly, resulting in a potentially undesired cluster configuration.
In a self-managed cluster, you should use thecluster update settings API to configure dynamic cluster settings, and only useelasticsearch.yml for static cluster settings and node settings. The API doesn’t require a restart and ensures a setting’s value is the same on all nodes.
Elastic Cloud Hosted and Elastic Cloud Enterprise don’t reject unsafe setting changes made using this API, and should be used with caution in these contexts.
Static settings can only be configured on an unstarted or shut down node usingelasticsearch.yml.
Static settings must be set on every relevant node in the cluster.
elasticsearch.yml should contain settings which are node-specific (such asnode.name and paths), or settings which a node requires in order to be able to join a cluster, such ascluster.name andnetwork.host.