--- title: Elasticsearch monitoring integration source: https://docs.newrelic.com/docs/infrastructure/host-integrations/host-integrations-list/elasticsearch/elasticsearch-integration --- Our Elasticsearch integration collects and sends inventory and metrics from your [Elasticsearch](https://www.elastic.co/) cluster to our platform, where you can see the health of your Elasticsearch environment. We collect metrics at the cluster, node, and index level so you can more easily find the source of any problems. > #### 💡 TIP > > New OpenTelemetry option available! Consider using our [Elasticsearch OpenTelemetry integration](https://docs.newrelic.com/docs/opentelemetry/integrations/elasticsearch/elasticsearch-otel-integration-overview) instead. It offers vendor-neutral monitoring, enhanced performance, and future-proof architecture using industry-standard OpenTelemetry protocols. To install the Elasticsearch monitoring integration, run through the following steps: 1. [Install and activate the integration](#install). 2. [Configure the integration](#config). 3. [Find and use data](#find-and-use). 4. Optionally, see [Elasticsearch's configuration settings](https://docs.newrelic.com/docs/infrastructure/host-integrations/host-integrations-list/elasticsearch/elasticsearch-config). ## Compatibility and requirements [#req] ### Elasticsearch versions [#elasticsearch-versions] Our integration is compatible with Elasticsearch 7.x through 8.X. ### Elasticsearch user The integration requires an Elasticsearch user with `monitor` privileges for the entire cluster and for all indices. For details, see [Elasticsearch user authorization](https://www.elastic.co/guide/en/elasticsearch/reference/current/authorization.html). ### Supported operating systems [#supported-os] - Windows ![Windows](https://docs.newrelic.com/images/os_icon_windows.webp "Windows") - Linux ![Linux](https://docs.newrelic.com/images/os_icon_linux.webp "Linux") For a comprehensive list of specific Windows and Linux versions, check the table of [compatible operating systems](https://docs.newrelic.com/docs/infrastructure/install-infrastructure-agent/get-started/requirements-infrastructure-agent/#operating-systems). ### System requirements [#system-reqs] - A New Relic account. Don't have one? [Sign up for free!](https://newrelic.com/signup) No credit card required. - If Elasticsearch is not running on Kubernetes or Amazon ECS, [install the infrastructure agent](https://docs.newrelic.com/docs/infrastructure/install-infrastructure-agent/get-started/install-infrastructure-agent-new-relic) on a host that's running Elasticsearch. Otherwise: - If running on ![Kubernetes](https://docs.newrelic.com/images/os_icon_k8.webp "Kubernetes")Kubernetes, see [these requirements](https://docs.newrelic.com/docs/monitor-service-running-kubernetes#requirements). - If running on ![ECS](https://docs.newrelic.com/images/os_icon_ecs.webp "ECS")Amazon ECS, see [these requirements](https://docs.newrelic.com/docs/integrations/host-integrations/host-integrations-list/monitor-services-running-amazon-ecs). ## Install and activate the integration [#install] To install the Elasticsearch integration, follow the instructions for your environment: ### Linux installation [#linux] 1. Follow the instructions for [installing an integration](https://docs.newrelic.com/docs/install-integrations-package), and replace the `INTEGRATION_FILE_NAME` variable with `nri-elasticsearch`. 2. Change directory to the integrations configuration folder by running: ```shell cd /etc/newrelic-infra/integrations.d ``` 3. Copy the sample configuration file by running: ```shell sudo cp elasticsearch-config.yml.sample elasticsearch-config.yml ``` 4. Edit the `elasticsearch-config.yml` configuration file with your favorite editor. Check out some [configuration file examples](#examples). 5. To enable automatic Elasticsearch error log parsing and forwarding, copy (or rename) the `elasticsearch-log.yml.example` file to `elasticsearch-log.yml`. No need to restart the agent. **Example** ```shell sudo cp /etc/newrelic-infra/logging.d/elasticsearch-log.yml.example /etc/newrelic-infra/logging.d/elasticsearch-log.yml ``` ### Other environments [#other-env] ** Windows installation** 1. Download the `nri-elasticsearch` .MSI installer image from: 2. To install from the Windows command prompt, run: ``` msiexec.exe /qn /i PATH\TO\nri-elasticsearch-amd64.msi ``` 3. In the integration's directory, `C:\Program Files\New Relic\newrelic-infra\integrations.d\`, create a copy of the sample configuration file by running: ``` cp elasticsearch-config.yml.sample elasticsearch-config.yml ``` 4. Edit the `elasticsearch-config.yml`file as described in [elasticsearch-config.yml sample files](#examples). ** {' '} Amazon ECS installation ** See [Monitor service running on ECS](https://docs.newrelic.com/docs/integrations/host-integrations/host-integrations-list/monitor-services-running-amazon-ecs). ** Kubernetes installation** See [Monitor service running on Kubernetes](https://docs.newrelic.com/docs/monitor-service-running-kubernetes). Additional notes: - **Advanced:** Integrations are also available in [tarball format](https://docs.newrelic.com/docs/integrations/host-integrations/installation/install-host-integrations-built-new-relic#tarball) to allow for install outside of a package manager. - **On-host integrations do not automatically update.** For best results, regularly [update the integration package](https://docs.newrelic.com/docs/integrations/host-integrations/installation/update-infrastructure-host-integration-package) and [the infrastructure agent](https://docs.newrelic.com/docs/infrastructure/new-relic-infrastructure/installation/update-infrastructure-agent). ## Configure the integration [#config] There are several ways to configure the integration, depending on how it was installed: - If enabled via ![Kubernetes](https://docs.newrelic.com/images/os_icon_k8.webp "Kubernetes")Kubernetes, see [Monitor services running on Kubernetes](https://docs.newrelic.com/docs/monitor-service-running-kubernetes). - If enabled via ![ECS](https://docs.newrelic.com/images/os_icon_ecs.webp "ECS")Amazon ECS, see [Monitor services running on ECS](https://docs.newrelic.com/docs/integrations/host-integrations/host-integrations-list/monitor-services-running-amazon-ecs). - If installed on-host, edit the config in the integration's YAML config file, `elasticsearch-config.yml`. An integration's YAML-format configuration is where you can place required login credentials and configure how data is collected. Which options you change depend on your setup and preference. The configuration file has common settings applicable to all integrations, such as `interval`, `timeout`, `inventory_source`. To read all about these common settings, refer to our [configuration format](https://docs.newrelic.com/docs/create-integrations/infrastructure-integrations-sdk/specifications/host-integrations-newer-configuration-format/#configuration-basics) document. > #### ⚠️ IMPORTANT > > If you are still using our legacy configuration or definition files, check the [standard configuration format](https://docs.newrelic.com/docs/create-integrations/infrastructure-integrations-sdk/specifications/host-integrations-standard-configuration-format/). ## elasticsearch-config.yml sample files [#examples] **Basic configuration** This is the basic configuration used to collect metrics and inventory from your localhost: ````yaml integrations: - name: nri-elasticsearch env: HOSTNAME: localhost PORT: 9200 USERNAME: elasticsearch_user PASSWORD: elasticsearch_password CONFIG_PATH: /etc/elasticsearch/elasticsearch.yml interval: 15s labels: environment: production inventory_source: config/elasticsearch ``` ```` **Basic configuration with different metric and inventory intervals** This configuration collects metrics every 15 seconds and inventory every 60 seconds: ````yaml integrations: - name: nri-elasticsearch env: METRICS: true HOSTNAME: localhost PORT: 9200 USERNAME: elasticsearch_user PASSWORD: elasticsearch_password REMOTE_MONITORING: true interval: 15s labels: environment: production - name: nri-elasticsearch env: INVENTORY: true HOSTNAME: localhost PORT: 9200 USERNAME: elasticsearch_user PASSWORD: elasticsearch_password CONFIG_PATH: /etc/elasticsearch/elasticsearch.yml interval: 60s labels: environment: production inventory_source: config/elasticsearch ``` ```` **Environment variables replacement** In this configuration we are using the environment variable `ELASTIC_HOST` to populate the HOSTNAME setting of the integration: ````yaml integrations: - name: nri-elasticsearch env: METRICS: "true" HOSTNAME: {{ELASTIC_HOST}} PORT: 9200 USERNAME: elasticsearch_user PASSWORD: elasticsearch_password interval: 15s labels: env: production role: load_balancer ``` ```` **Cluster monitoring** The integration is capable of collecting all metrics from the cluster just connecting to one of the nodes, using a basic configuration showed before all the cluster metrics can be scraped. Another way to configure the integration is to configure the scrape of all instances but activating the `MASTER_ONLY` parameter which makes the integration only collect metrics if the scraped instance is master. This allows to share the same config for all the nodes and avoid data duplication. This is how the config file of each of the nodes should look like: ````yaml integrations: - name: nri-elasticsearch env: HOSTNAME: localhost PORT: 9200 USERNAME: elasticsearch_user PASSWORD: elasticsearch_password MASTER_ONLY: "true" CONFIG_PATH: /etc/elasticsearch/elasticsearch.yml interval: 15s labels: environment: production inventory_source: config/elasticsearch ``` ```` ## Configuration options for the integration [#config-options] For more on how to find and use your data, see [Elasticsearch's configuration settings](https://docs.newrelic.com/docs/infrastructure/host-integrations/host-integrations-list/elasticsearch/elasticsearch-config). ## Find and use data [#find-and-use] Data from this service is reported to an [integration dashboard](https://docs.newrelic.com/docs/integrations/new-relic-integrations/getting-started/infrastructure-integration-dashboards-charts). Elasticsearch data is attached to the following [event types](https://docs.newrelic.com/docs/using-new-relic/data/understand-data/new-relic-data-types#events-new-relic): - [`ElasticsearchClusterSample`](#cluster-metrics) - [`ElasticsearchNodeSample`](#node-metrics) - [`ElasticsearchCommonSample`](#common-metrics) - [`ElasticsearchIndexSample`](#index-metrics) You can [query this data](https://docs.newrelic.com/docs/using-new-relic/data/understand-data/query-new-relic-data) for troubleshooting purposes or to create custom charts and dashboards. For more on how to find and use your data, see how to [understand integration data](https://docs.newrelic.com/docs/infrastructure/integrations/find-use-infrastructure-integration-data). ## Metrics collected by the integration [#metrics] The Elasticsearch integration collects the following metrics. Each metric name is prefixed with a category indicator and a period, such as `cluster.` or `shards.`. **ElasticsearchClusterSample event** | Metric | Description | | ---------------------- | ------------------------------------------------------------------ | | `cluster.dataNodes` | The number of data nodes in the cluster. | | `cluster.nodes` | The number of nodes in the cluster. | | `cluster.status` | The Elasticsearch cluster health: `red`, `yellow`, or `green`. | | `shards.active` | The number of active shards in the cluster. | | `shards.initializing` | The number of shards that are currently initializing. | | `shards.primaryActive` | The number of active primary shards in the cluster. | | `shards.relocating` | The number of shards that are relocating from one node to another. | | `shards.unassigned` | The number of shards that are unassigned to a node. | **ElasticsearchNodeSample event** | Metric | Description | | ------------------------------------------------------------- | --------------------------------------------------------------------------------------------------- | | `activeSearches` | The number of active searches. | | `activeSearchesInMilliseconds` | The time spent on the search fetch. | | `breakers.estimatedSizeFieldDataCircuitBreakerInBytes` | The estimated size of the field data circuit breaker, in bytes. | | `breakers.estimatedSizeParentCircuitBreakerInBytes` | The estimated size of the parent circuit breaker, in bytes. | | `breakers.estimatedSizeRequestCircuitBreakerInBytes` | The estimated size of the request circuit breaker, in bytes. | | `breakers.fieldDataCircuitBreakerTripped` | The number of times the field data circuit breaker has tripped. | | `breakers.parentCircuitBreakerTripped` | The number of times the parent circuit breaker has tripped. | | `breakers.requestCircuitBreakerTripped` | The number of times the request circuit breaker has tripped. | | `cache.cacheSizeIDInBytes` | The size of the id cache, in bytes. | | `flush.indexFlushDisk` | The number of index flushes to disk since start. | | `flush.timeFlushIndexDiskInSeconds` | The time spent flushing the index to disk. | | `fs.bytesAvailableJVMInBytes` | Bytes available to this Java virtual machine on this file store, in bytes. | | `fs.bytesReadsInBytes` | The total bytes read from the file store, in bytes. | | `fs.bytesUserIoOperationsInBytes` | The total bytes used for all I/O operations on the file store, in bytes. | | `fs.iOOperations` | The total I/O operations on the file store. | | `fs.reads` | The total number of reads from the file store. | | `fs.totalSizeInBytes` | The total size of the file store, in bytes. | | `fs.unallocatedBytesInBytes` | The total number of unallocated bytes in the file store, in bytes. | | `fs.writes` | The total number of writes to the file store. | | `fs.writesInBytes` | The total bytes written to the file store, in bytes. | | `get.currentRequestsRunning` | The number of get requests currently running. | | `get.requestsDocumentExists` | The number of get requests where the document existed. | | `get.requestsDocumentExistsInMilliseconds` | The time spent on get requests where the document existed. | | `get.requestsDocumentMissing` | The number of get requests where the document was missing. | | `get.requestsDocumentMissingInMilliseconds` | The time spent on get requests where the document was missing. | | `get.timeGetRequestsInMilliseconds` | The time spent on get requests. | | `get.totalGetRequests` | The number of get requests. | | `http.currentOpenConnections` | The number of current open HTTP connections. | | `http.openedConnections` | The number of opened HTTP connections. | | `indexing.docsCurrentlyDeleted` | The number of documents currently being deleted from an index. | | `indexing.documentsCurrentlyIndexing` | The number of documents currently being indexed to an index. | | `indexing.documentsIndexed` | The number of documents indexed to an index. | | `indexing.timeDeletingDocumentsInMilliseconds` | The time spent deleting documents from an index. | | `indexing.timeIndexingDocumentsInMilliseconds` | The time spent indexing documents to an index. | | `indexing.totalDocumentsDeleted` | The number of documents deleted from an index. | | `indices.indexingOperationsFailed` | The number of failed indexing operations. | | `indices.indexingWaitedThrottlingInMilliseconds` | The time indexing waited due to throttling. | | `indices.memoryQueryCacheInBytes` | The memory used by the query cache, in bytes. | | `indices.numberIndices` | The number of documents across all primary shards assigned to the node. | | `indices.queryCacheEvictions` | The number of query cache evictions. | | `indices.queryCacheHits` | The number of query cache hits. | | `indices.queryCacheMisses` | The number of query cache misses. | | `indices.recoveryOngoingShardSource` | The number of ongoing recoveries for which a shard serves as a source. | | `indices.recoveryOngoingShardTarget` | The number of ongoing recoveries for which a shard serves as a target. | | `indices.recoveryWaitedThrottlingInMilliseconds` | The total time recoveries waited due to throttling. | | `indices.requestCacheEvictions` | The number of request cache evictions. | | `indices.requestCacheHits` | The number of request cache hits. | | `indices.requestCacheMemoryInBytes` | The memory used by the request cache, in bytes. | | `indices.requestCacheMisses` | The number of request cache misses. | | `indices.segmentsIndexShard` | The number of segments in an index shard. | | `indices.segmentsMaxMemoryIndexWriterInBytes` | The maximum memory used by the index writer, in bytes. | | `indices.segmentsMemoryUsedDocValuesInBytes` | The memory used by doc values, in bytes. | | `indices.segmentsMemoryUsedFixedBitSetInBytes` | The memory used by fixed bit set, in bytes. | | `indices.segmentsMemoryUsedIndexSegmentsInBytes` | The memory used by index segments, in bytes. | | `indices.segmentsMemoryUsedIndexWriterInBytes` | The memory used by the index writer, in bytes. | | `indices.segmentsMemoryUsedNormsInBytes` | The memory used by norm, in bytes. | | `indices.segmentsMemoryUsedSegmentVersionMapInBytes` | The memory used by the segment version map, in bytes. | | `indices.segmentsMemoryUsedStoredFieldsInBytes` | The memory used by stored fields, in bytes. | | `indices.segmentsMemoryUsedTermsInBytes` | The memory used by terms, in bytes. | | `indices.segmentsMemoryUsedTermVectorsInBytes` | The memory used by term vectors, in bytes. | | `indices.translogOperations` | The number of operations in the transaction log. | | `indices.translogOperationsInBytes` | The size of the transaction log, in bytes. | | `jvm.gc.collections` | The number of garbage collections run by the JVM. | | `jvm.gc.collectionsInMilliseconds` | The time spent on garbage collection in the JVM. | | `jvm.gc.concurrentMarkSweep` | The number of concurrent mark & sweep GCs in the JVM. | | `jvm.gc.concurrentMarkSweepInMilliseconds` | The time spent on concurrent mark & sweep GCs in the JVM. | | `jvm.gc.majorCollectionsOldGenerationObjects` | The number of major GCs in the JVM that collect old generation objects. | | `jvm.gc.majorCollectionsOldGenerationObjectsInMilliseconds` | The time spent in major GCs in the JVM that collect old generation objects. | | `jvm.gc.minorCollectionsYoungGenerationObjects` | The number of minor GCs in the JVM that collects young generation objects. | | `jvm.gc.minorCollectionsYoungGenerationObjectsInMilliseconds` | The time spent in minor GCs in the JVM that collects young generation objects. | | `jvm.gc.parallelNewCollections` | The number of parallel new GCs in the JVM. | | `jvm.gc.parallelNewCollectionsInMilliseconds` | The time spent on parallel new GCs in the JVM. | | `jvm.mem.heapCommittedInBytes` | The amount of memory guaranteed to be available to the JVM heap, in bytes. | | `jvm.mem.heapMaxInBytes` | The maximum amount of memory that can be used by the JVM heap, in bytes. | | `jvm.mem.heapUsed` | The percentage of memory currently used by the JVM heap as a value between `0` and `1`. | | `jvm.mem.heapUsedInBytes` | The amount of memory currently used by the JVM heap, in bytes. | | `jvm.mem.maxOldGenerationHeapInBytes` | The maximum amount of memory that can be used by the old generation heap, in bytes. | | `jvm.mem.maxSurvivorSpaceInBytes` | The maximum amount of memory that can be used by the survivor space, in bytes. | | `jvm.mem.maxYoungGenerationHeapInBytes` | The maximum amount of memory that can be used by the young generation heap, in bytes. | | `jvm.mem.nonHeapCommittedInBytes` | The amount of memory guaranteed to be available to JVM non-heap, in bytes. | | `jvm.mem.nonHeapUsedInBytes` | The amount of memory currently used by the JVM non-heap, in bytes. | | `jvm.mem.usedOldGenerationHeapInBytes` | The amount of memory currently used by the old generation heap, in bytes. | | `jvm.mem.usedSurvivorSpaceInBytes` | The amount of memory currently used by the survivor space, in bytes. | | `jvm.mem.usedYoungGenerationHeapInBytes` | The amount of memory currently used by the young generation heap, in bytes. | | `jvm.ThreadsActive` | The number of active threads in the JVM. | | `jvm.ThreadsPeak` | The peak number of threads used by the JVM. | | `merges.currentActive` | The number of currently active segment merges. | | `merges.docsSegmentsMerging` | The number of documents across segments currently being merged. | | `merges.docsSegmentMerges` | The number of documents across all merged segments. | | `merges.mergedSegmentsInBytes` | The size of all merged segments, in bytes. | | `merges.segmentMerges` | The number of segment merges. | | `merges.sizeSegmentsMergingInBytes` | The size of the segments currently being merged, in bytes. | | `merges.totalSegmentMergingInMilliseconds` | The time spent on segment merging. | | `openFD` | The number of opened file descriptors associated with the current process, or`-1` if not supported. | | `queriesTotal` | The number of queries. | | `refresh.total` | The number of index refreshes. | | `refresh.totalInMilliseconds` | The time spent on index refreshes. | | `searchFetchCurrentlyRunning` | The number of search fetches currently running. | | `searchFetches` | The number of search fetches. | | `sizeStoreInBytes` | The size of the store, in bytes. | | `threadpool.bulk.Queue` | The number of queued threads in the bulk pool. | | `threadpool.bulkActive` | The number of active threads in the bulk pool. | | `threadpool.bulkRejected` | The number of rejected threads in the bulk pool. | | `threadpool.bulkThreads` | The number of threads in the bulk pool. | | `threadpool.fetchShardStartedQueue` | The number of queued threads in the fetch shard started pool. | | `threadpool.fetchShardStartedRejected` | The number of rejected threads in the fetch shard started pool. | | `threadpool.fetchShardStartedThreads` | The number of threads in the fetch shard started pool. | | `threadpool.fetchShardStoreActive` | The number of active threads in the fetch shard store pool. | | `threadpool.fetchShardStoreQueue` | The number of queued threads in the fetch shard store pool. | | `threadpool.fetchShardStoreRejected` | The number of rejected threads in the fetch shard store pool. | | `threadpool.fetchShardStoreThreads` | The number of threads in the fetch shard store pool. | | `threadpool.flushActive` | The number of active threads in the flush queue. | | `threadpool.flushQueue` | The number of queued threads in the flush pool. | | `threadpool.flushRejected` | The number of rejected threads in the flush pool. | | `threadpool.flushThreads` | The number of threads in the flush pool. | | `threadpool.forceMergeActive` | The number of active threads for force merge operations. | | `threadpool.forceMergeQueue` | The number of queued threads for force merge operations. | | `threadpool.forceMergeRejected` | The number of rejected threads for force merge operations. | | `threadpool.forceMergeThreads` | The number of threads for force merge operations. | | `threadpool.genericActive` | The number of active threads in the generic pool. | | `threadpool.genericQueue` | The number of queued threads in the generic pool. | | `threadpool.genericRejected` | The number of rejected threads in the generic pool. | | `threadpool.genericThreads` | The number of threads in the generic pool. | | `threadpool.getActive` | The number of active threads in the get pool. | | `threadpool.getQueue` | The number of queued threads in the get pool. | | `threadpool.getRejected` | The number of rejected threads in the get pool. | | `threadpool.getThreads` | The number of threads in the get pool. | | `threadpool.listenerActive` | The number of active threads in the listener pool.(Deprecated on Elasticsearch 8) | | `threadpool.listenerQueue` | The number of queued threads in the listener pool.(Deprecated on Elasticsearch 8) | | `threadpool.listenerRejected` | The number of rejected threads in the listener pool.(Deprecated on Elasticsearch 8) | | `threadpool.listenerThreads` | The number of threads in the listener pool.(Deprecated on Elasticsearch 8) | | `threadpool.managementActive` | The number of active threads in the management pool. | | `threadpool.managementQueue` | The number of queued threads in the management pool. | | `threadpool.managementRejected` | The number of rejected threads in the management pool. | | `threadpool.managementThreads` | The number of threads in the management pool. | | `threadpool.mergeActive` | The number of active threads in the merge pool. | | `threadpool.mergeQueue` | The number of queued threads in the merge pool. | | `threadpool.mergeRejected` | The number of rejected threads in the merge pool. | | `threadpool.mergeThreads` | The number of threads in the merge pool. | | `threadpool.percolateActive` | The number of active threads in the percolate pool. | | `threadpool.percolateQueue` | The number of queued threads in the percolate pool. | | `threadpool.percolateRejected` | The number of rejected threads in the percolate pool. | | `threadpool.percolateThreads` | The number of threads in the percolate pool. | | `threadpool.refreshActive` | The number of active threads in the refresh pool. | | `threadpool.refreshQueue` | The number of queued threads in the refresh pool. | | `threadpool.refreshRejected` | The number of rejected threads in the refresh pool. | | `threadpool.refreshThreads` | The number of threads in the refresh pool. | | `threadpool.searchActive` | The number of active threads in the search pool. | | `threadpool.searchQueue` | The number of queued threads in the search pool. | | `threadpool.searchRejected` | The number of rejected threads in the search pool. | | `threadpool.searchThreads` | The number of threads in the search pool. | | `threadpool.snapshotActive` | The number of active threads in the snapshot pool. | | `threadpool.snapshotQueue` | The number of queued threads in the snapshot pool. | | `threadpool.snapshotRejected` | The number of rejected threads in the snapshot pool. | | `threadpool.snapshotThreads` | The number of threads in the snapshot pool. | | `threadpool.activeFetchShardStarted` | The number of active threads in the fetch shard started pool. | | `transport.connectionsOpened` | The number of connections opened for cluster communication. | | `transport.packetsReceived` | The number of packets received in cluster communication. | | `transport.packetsReceivedInBytes` | The size of data received in cluster communication, in bytes. | | `transport.packetsSent` | The number of packets sent in cluster communication. | | `transport.packetsSentInBytes` | The size of data sent in cluster communication, in bytes. | **ElasticsearchCommonSample event** | Metric | Description | | ----------------------------------------------------- | -------------------------------------------------------------------------------------- | | `primaries.docsDeleted` | The number of documents deleted from the primary shards. | | `primaries.docsnumber` | The number of documents in the primary shards. | | `primaries.flushesTotal` | The number of index flushes to disk from the primary shards since start. | | `primaries.flushTotalTimeInMilliseconds` | The time spent flushing the index to disk from the primary shards. | | `primaries.get.documentsExist` | The number of get requests on primary shards where the document existed. | | `primaries.get.documentsExistInMilliseconds` | The time spent on get requests from the primary shards where the document existed. | | `primaries.get.documentsMissing` | The number of get requests from the primary shards where the document was missing. | | `primaries.get.documentsMissingInMilliseconds` | The time spent on get requests from the primary shards where the document was missing. | | `primaries.get.requests` | The number of get requests from the primary shards. | | `primaries.get.requestsCurrent` | The number of get requests currently running on the primary shards. | | `primaries.get.requestsInMilliseconds` | The time spent on get requests from the primary shards. | | `primaries.index.docsCurrentlyDeleted` | The number of documents currently being deleted from an index on the primary shards. | | `primaries.index.docsCurrentlyDeletedInMilliseconds` | The time spent deleting documents from an index on the primary shards. | | `primaries.index.docsCurrentlyIndexing` | The number of documents currently being indexed to an index on the primary shards. | | `primaries.index.docsCurrentlyIndexingInMilliseconds` | The time spent indexing documents to an index on the primary shards. | | `primaries.index.docsDeleted` | The number of documents deleted from an index on the primary shards. | | `primaries.index.docsTotal` | The number of documents indexed to an index on the primary shards. | | `primaries.indexRefreshesTotal` | The number of index refreshes on the primary shards. | | `primaries.indexRefreshesTotalInMilliseconds` | The time spent on index refreshes on the primary shards. | | `primaries.merges.current` | The number of currently active segment merges on the primary shards. | | `primaries.merges.docsSegmentsCurrentlyMerged` | The number of documents across segments currently being merged on the primary shards. | | `primaries.merges.docsTotal` | The number of documents across all merged segments on the primary shards. | | `primaries.merges.SegmentsCurrentlyMergedInBytes` | The size of the segments currently being merged on the primary shards, in bytes. | | `primaries.merges.SegmentsTotal` | The number of segment merges on the primary shards. | | `primaries.merges.segmentsTotalInBytes` | The size of all merged segments on the primary shards, in bytes. | | `primaries.merges.segmentsTotalInMilliseconds` | The time spent on segment merging on the primary shards. | | `primaries.queriesInMilliseconds` | The time spent querying on the primary shards. | | `primaries.queriesTotal` | The number of queries to the primary shards. | | `primaries.queryActive` | The number of currently active queries on the primary shards. | | `primaries.queryFetches` | The number of query fetches currently running on the primary shards. | | `primaries.queryFetchesInMilliseconds` | The time spent on query fetches on the primary shards. | | `primaries.queryFetchesTotal` | The number of query fetches on the primary shards. | | `primaries.sizeInBytes` | The size of all the primary shards, in bytes. | **ElasticsearchIndexSample event** | Metric | Description | | ------------------------------- | -------------------------------------------------------------------- | | `index.docs` | The number of documents in the index. | | `index.docsDeleted` | The number of deleted documents in the index. | | `index.health` | The status of the index: `red`, `yellow`, or `green`. | | `index.primaryShards` | The number of primary shards in the index. | | `index.primaryStoreSizeInBytes` | The store size of primary shards in the index. | | `index.replicaShards` | The number of replica shards in the index. | | `index.storeSizeInBytes` | The store size of primary and replica shards in the index, in bytes. | The Elasticsearch integration is open source software. That means you can [browse its source code](https://github.com/newrelic/nri-elasticsearch) and send improvements, or create your own fork and build it. ### Elasticsearch instance settings [#instance-settings] You can use the `env` section of the `elasticsearch-config.yml` file, to define specific settings related to Elasticsearch. These settings control the connection to your Elasticsearch instance, as well as other security settings and features. The Elasticsearch integration collects both Metrics and Inventory information. In the table, use the **Applies to** column for the settings available to each collection: | Setting | Description | Default | Applies to | | ---------------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------- | -------------------------------------- | ---------- | | **HOSTNAME** | Hostname or IP where Elasticsearch is running. | localhost | M/I | | **LOCAL_HOSTNAME** | Hostname or IP of the Elasticsearch node from which to collect inventory data. Should only be set if you do not wish to collect inventory data against localhost. | localhost | M | | **PORT** | Port on which Elasticsearch is listening. | 9200 | M/I | | **USERNAME** | Username for accessing Elasticsearch Node. | N/A | M/I | | **PASSWORD** | Password for the given user. | N/A | M/I | | **CLUSTER_ENVIRONMENT** | A way to further specify which cluster we are gathering data for, example: 'staging'. | N/A | M/I | | **USE_SSL** | Use SSL when communicating with the Elasticsearch node. | false | M/I | | **CA_BUNDLE_FILE** | Location of SSL certificate on the host. Only required if `USE_SSL` is true. | N/A | M/I | | **CA_BUNDLE_DIR** | Alternative Certificate Authority bundle directory. | N/A | M/I | | **SSL_ALTERNATIVE_HOSTNAME** | Alternative server hostname that the integration will accept as valid for the purposes of SSL negotiation. | N/A | M/I | | **TLS_INSECURE_SKIP_VERIFY** | Skip the verification of the server's certificate chain and hostname. | false | M/I | | **CONFIG_PATH** | Path to the Elasticsearch configuration file. | /etc/elasticsearch/ elasticsearch.yml | I | | **TIMEOUT** | Timeout for API requests, in seconds. | 30 | M/I | | **COLLECT_INDICES** | Signals whether to collect indices metrics or not. | true | M | | **COLLECT_PRIMARIES** | Signals whether to collect primaries metrics or not. | true | M | | **INDICES_REGEX** | Filter which indices are collected. | N/A | M | | **MASTER_ONLY** | Collect cluster metrics on the elected master only. | false | M | | **METRICS** | Set to `true` to enable metrics-only collection. | false | | | **INVENTORY** | Set to `true` to enable inventory-only collection. | false | | The values for these settings can be defined in several ways: - Adding the value directly in the config file. This is the most common way. - Replacing the values from environment variables using the `{{ }}` notation. Read more about [using environment variable passthroughs with on-host integrations](https://docs.newrelic.com/docs/infrastructure/install-infrastructure-agent/configuration/configure-infrastructure-agent/#passthrough) or see the example for [environment variables replacement](https://docs.newrelic.com/docs/infrastructure/host-integrations/host-integrations-list/elasticsearch/elasticsearch-integration#envvar-replacement). This requires infrastructure agent v1.14.0+. - Using secrets management. Use this to protect sensitive information, such as passwords that would be exposed in plain text on the configuration file. For more information, see [secrets management](https://docs.newrelic.com/docs/integrations/host-integrations/installation/secrets-management). ### Labels and custom attributes [#labels] You can further decorate your metrics using labels. Labels allow you to add key-value pair attributes to your metrics, which you can then use to query, filter, or group your metrics on. The [basic configuration elasticsearch-config.yml sample file](https://docs.newrelic.com/docs/infrastructure/host-integrations/host-integrations-list/elasticsearch/elasticsearch-integration#basic-config) includes usage of labels. However, as they are not mandatory, you can remove, modify, or add new ones of your choice. ```yaml labels: env: production role: load_balancer ``` ## Inventory data [#inventory] The Elasticsearch integration captures the configuration parameters of the Elasticsearch node, as specified in the [YAML config file](https://docs.newrelic.com/docs/infrastructure/host-integrations/host-integrations-list/elasticsearch/elasticsearch-integration/#basic-config). It also collects node configuration information from the `\_nodes/\_local` endpoint. The data is available on the [Infrastructure inventory page](https://docs.newrelic.com/docs/infrastructure/new-relic-infrastructure/infrastructure-ui-pages/infrastructure-inventory-page-search-your-entire-infrastructure), under the `config/elasticsearch` source. For more about inventory data, see how to [understand integration data](https://docs.newrelic.com/docs/infrastructure/integrations-getting-started/getting-started/understand-integration-data-data-types#inventory-data).