---
title: Monitor Elasticsearch on Kubernetes with OpenTelemetry
source: https://docs.newrelic.com/docs/opentelemetry/integrations/elasticsearch/elasticsearch-otel-integration-k8-install
---

Monitor your Elasticsearch clusters in Kubernetes by deploying the OpenTelemetry Collector with automatic pod discovery. This integration uses the [elasticsearchreceiver](https://github.com/open-telemetry/opentelemetry-collector-contrib/tree/main/receiver/elasticsearchreceiver) and [receivercreator](https://github.com/open-telemetry/opentelemetry-collector-contrib/tree/main/receiver/receivercreator) to automatically discover and monitor Elasticsearch pods without manual configuration.

To get started, select the collector distribution that best fits your Kubernetes environment:

You can choose between three collector options:

-   **NRDOT:** New Relic Distribution of OpenTelemetry
-   **OTel Collector Contrib:** Standard OpenTelemetry Collector with community-contributed components
-   **Prometheus Receiver:** For environments already running a [Prometheus Elasticsearch exporter](https://github.com/prometheus-community/elasticsearch_exporter)

## Installation options [#installation-options]

Choose the collector distribution that matches your needs:

### NRDOT Collector

#### Before you begin [#prereq-nrdot]

Before deploying the NRDOT collector on Kubernetes, ensure you have:

**Required access privileges:**

-   Your New Relic license key
-   kubectl access to your Kubernetes cluster
-   Elasticsearch cluster admin privileges with `monitor` or `manage` cluster privilege (see [Elasticsearch security privileges documentation](https://www.elastic.co/guide/en/elasticsearch/reference/current/security-privileges.html) for details)

**System requirements:**

-   **Elasticsearch version 7.16 or higher** - This integration requires a modern Elasticsearch cluster
-   **Kubernetes cluster** - A running Kubernetes cluster where Elasticsearch is deployed
-   **Network connectivity** - Outbound HTTPS (port 443) to New Relic's [OTLP ingest endpoint](https://docs.newrelic.com/docs/opentelemetry/best-practices/opentelemetry-otlp)

**Elasticsearch pod requirements:**

-   **Pod labels (Required)** - Each Elasticsearch pod must have the label `app: elasticsearch` for automatic discovery to work. Without this label, the collector will not detect or monitor your pods.

> #### ⚠️ IMPORTANT
>
> **How to add labels to Elasticsearch pods:**
>
> If you're using a StatefulSet or Deployment for Elasticsearch, add the label in the pod template:
>
> ```yaml
> apiVersion: apps/v1
> kind: StatefulSet
> metadata:
>   name: elasticsearch
> spec:
>   template:
>     metadata:
>       labels:
>         app: elasticsearch  # Required for auto-discovery
>     spec:
>       containers:
>       - name: elasticsearch
>         image: docker.elastic.co/elasticsearch/elasticsearch:8.x.x
> ```
>
> For existing pods without labels, update your StatefulSet/Deployment and restart the pods:
>
> ```bash
> kubectl label pods -l <your-existing-selector> app=elasticsearch -n <namespace>
> ```
>
> You can verify labels are set correctly:
>
> ```bash
> kubectl get pods -n <namespace> --show-labels
> ```

#### Choose your installation method [#nrdot-choose-install]

You can install the NRDOT Collector using Kubernetes manifests or Helm charts. Choose the method that best fits your workflow:

**Manifest install:**

-   More control over individual Kubernetes resources
-   Requires completing the base Kubernetes OpenTelemetry manifest installation first
-   Best for customized deployments

**Helm install:**

-   Simpler deployment with single command
-   Easier to manage and upgrade
-   Best for standard deployments

Proceed to the next step for detailed instructions for your chosen method.

#### Install and configure NRDOT Collector [#nrdot-install-configure]

**Manifest install**

##### Complete base Kubernetes OpenTelemetry manifest installation [#nrdot-base-install]

Before configuring Elasticsearch monitoring, complete the base [Kubernetes OpenTelemetry manifest installation](https://docs.newrelic.com/docs/kubernetes-pixie/k8s-otel/install/#manifest-install).

##### Update collector image to NRDOT [#nrdot-update-image]

In both [`deployment.yaml`](https://github.com/newrelic/helm-charts/blob/master/charts/nr-k8s-otel-collector/examples/k8s/rendered/deployment.yaml) and [`daemonset.yaml`](https://github.com/newrelic/helm-charts/blob/master/charts/nr-k8s-otel-collector/examples/k8s/rendered/daemonset.yaml) files in your local `rendered` directory, update the image to:

```yaml
image: newrelic/nrdot-collector:latest
```

##### Configure Elasticsearch monitoring [#nrdot-configure-es]

Update the [`deployment-configmap.yaml`](https://github.com/newrelic/helm-charts/blob/master/charts/nr-k8s-otel-collector/examples/k8s/rendered/deployment-configmap.yaml) for Elasticsearch monitoring.

You have two options:

**Option 1: Replace all existing OpenTelemetry components** (recommended for Elasticsearch-only monitoring)

Replace the entire `deployment-configmap.yaml` content with the Elasticsearch-specific configuration:

```yaml
apiVersion: v1
kind: ConfigMap
metadata:
  name: nr-k8s-otel-collector-deployment-config
data:
  config.yaml: |
    extensions:
      health_check:
        endpoint: 0.0.0.0:13133
      k8s_observer:
        auth_type: serviceAccount
        observe_pods: true
        observe_nodes: true

    receivers:
      jaeger: null
      zipkin: null
      prometheus: null
      otlp: null
      receiver_creator/elasticsearch:
        watch_observers: [k8s_observer]
        receivers:
          elasticsearch:
            rule: type == "pod" && labels["app"] == "elasticsearch"
            config:
              endpoint: 'http://`endpoint`:9200'
              collection_interval: 15s
              metrics:
                elasticsearch.os.cpu.usage:
                  enabled: true
                elasticsearch.cluster.data_nodes:
                  enabled: true
                elasticsearch.cluster.health:
                  enabled: true
                elasticsearch.cluster.in_flight_fetch:
                  enabled: true
                elasticsearch.cluster.nodes:
                  enabled: true
                elasticsearch.cluster.pending_tasks:
                  enabled: true
                elasticsearch.cluster.shards:
                  enabled: true
                elasticsearch.cluster.state_update.time:
                  enabled: true
                elasticsearch.index.documents:
                  enabled: true
                elasticsearch.index.operations.merge.current:
                  enabled: true
                elasticsearch.index.operations.time:
                  enabled: true
                elasticsearch.node.cache.count:
                  enabled: true
                elasticsearch.node.cache.evictions:
                  enabled: true
                elasticsearch.node.cache.memory.usage:
                  enabled: true
                elasticsearch.node.shards.size:
                  enabled: true
                elasticsearch.node.cluster.io:
                  enabled: true
                elasticsearch.node.documents:
                  enabled: true
                elasticsearch.node.disk.io.read:
                  enabled: true
                elasticsearch.node.disk.io.write:
                  enabled: true
                elasticsearch.node.fs.disk.available:
                  enabled: true
                elasticsearch.node.fs.disk.total:
                  enabled: true
                elasticsearch.node.http.connections:
                  enabled: true
                elasticsearch.node.ingest.documents.current:
                  enabled: true
                elasticsearch.node.ingest.operations.failed:
                  enabled: true
                elasticsearch.node.open_files:
                  enabled: true
                elasticsearch.node.operations.completed:
                  enabled: true
                elasticsearch.node.operations.current:
                  enabled: true
                elasticsearch.node.operations.get.completed:
                  enabled: true
                elasticsearch.node.operations.time:
                  enabled: true
                elasticsearch.node.shards.reserved.size:
                  enabled: true
                elasticsearch.index.shards.size:
                  enabled: true
                elasticsearch.os.cpu.load_avg.1m:
                  enabled: true
                elasticsearch.os.cpu.load_avg.5m:
                  enabled: true
                elasticsearch.os.cpu.load_avg.15m:
                  enabled: true
                elasticsearch.os.memory:
                  enabled: true
                jvm.gc.collections.count:
                  enabled: true
                jvm.gc.collections.elapsed:
                  enabled: true
                jvm.memory.heap.max:
                  enabled: true
                jvm.memory.heap.used:
                  enabled: true
                jvm.memory.heap.utilization:
                  enabled: true
                jvm.threads.count:
                  enabled: true
                elasticsearch.index.segments.count:
                  enabled: true
                elasticsearch.index.operations.completed:
                  enabled: true
                elasticsearch.node.script.cache_evictions:
                  enabled: false
                elasticsearch.node.cluster.connections:
                  enabled: false
                elasticsearch.node.pipeline.ingest.documents.preprocessed:
                  enabled: false
                elasticsearch.node.thread_pool.tasks.queued:
                  enabled: false
                elasticsearch.cluster.published_states.full:
                  enabled: false
                jvm.memory.pool.max:
                  enabled: false
                elasticsearch.node.script.compilation_limit_triggered:
                  enabled: false
                elasticsearch.node.shards.data_set.size:
                  enabled: false
                elasticsearch.node.pipeline.ingest.documents.current:
                  enabled: false
                elasticsearch.cluster.state_update.count:
                  enabled: false
                elasticsearch.node.fs.disk.free:
                  enabled: false
                jvm.memory.nonheap.used:
                  enabled: false
                jvm.memory.pool.used:
                  enabled: false
                elasticsearch.node.translog.size:
                  enabled: false
                elasticsearch.node.thread_pool.threads:
                  enabled: false
                elasticsearch.cluster.state_queue:
                  enabled: false
                elasticsearch.node.translog.operations:
                  enabled: false
                elasticsearch.memory.indexing_pressure:
                  enabled: false
                elasticsearch.node.ingest.documents:
                  enabled: false
                jvm.classes.loaded:
                  enabled: false
                jvm.memory.heap.committed:
                  enabled: false
                elasticsearch.breaker.memory.limit:
                  enabled: false
                elasticsearch.indexing_pressure.memory.total.replica_rejections:
                  enabled: false
                elasticsearch.breaker.memory.estimated:
                  enabled: false
                elasticsearch.cluster.published_states.differences:
                  enabled: false
                jvm.memory.nonheap.committed:
                  enabled: false
                elasticsearch.node.translog.uncommitted.size:
                  enabled: false
                elasticsearch.node.script.compilations:
                  enabled: false
                elasticsearch.node.pipeline.ingest.operations.failed:
                  enabled: false
                elasticsearch.indexing_pressure.memory.limit:
                  enabled: false
                elasticsearch.breaker.tripped:
                  enabled: false
                elasticsearch.indexing_pressure.memory.total.primary_rejections:
                  enabled: false
                elasticsearch.node.thread_pool.tasks.finished:
                  enabled: false

    processors:
      memory_limiter:
        check_interval: 60s
        limit_mib: ${env:NEW_RELIC_MEMORY_LIMIT_MIB}
      cumulativetodelta: {}
      attributes/cluster_state_aggregate:
        include:
          match_type: strict
          metric_names:
            - elasticsearch.cluster.state_update.time
        actions:
          - key: type
            action: delete
          - key: state
            action: delete
      filter/critical_operations:
        metrics:
          datapoint:
            - 'attributes["operation"] == "query" or attributes["operation"] == "index" or attributes["operation"] == "get" or attributes["operation"] == "merge" or attributes["operation"] == nil'
      resource/cluster:
        attributes:
          - key: k8s.cluster.name
            value: "elasticsearch-cluster"
            action: insert
      resource/cluster_name_override:
        attributes:
          - key: elasticsearch.cluster.name
            value: "elasticsearch-cluster"
            action: upsert
      resourcedetection:
        detectors: [env, system]
        system:
          resource_attributes:
            host.name:
              enabled: true
            host.id:
              enabled: false
            os.type:
              enabled: true
      batch:
        timeout: 30s
        send_batch_size: 2048
        send_batch_max_size: 4096
      attributes/cardinality_reduction:
        actions:
          - key: process.pid
            action: delete
          - key: process.parent_pid
            action: delete
          - key: k8s.pod.uid
            action: delete
      transform/metadata_nullify:
        metric_statements:
          - context: metric
            statements:
              - set(description, "")
              - set(unit, "")

    exporters:
      otlp_http:
        endpoint: "${env:NEWRELIC_OTLP_ENDPOINT}"
        headers:
          api-key: "${env:NEWRELIC_LICENSE_KEY}"
        compression: gzip
        timeout: 30s
        retry_on_failure:
          enabled: true
          initial_interval: 5s
          max_interval: 30s
          max_elapsed_time: 300s

    service:
      extensions: [health_check, k8s_observer]
      pipelines:
        traces: null
        logs: null
        metrics: null
        metrics/elasticsearch:
          receivers: [receiver_creator/elasticsearch]
          processors: [memory_limiter, resourcedetection, resource/cluster, resource/cluster_name_override, attributes/cardinality_reduction, filter/critical_operations, attributes/cluster_state_aggregate, cumulativetodelta, transform/metadata_nullify, batch]
          exporters: [otlp_http]
```

**Option 2: Merge with existing configuration** (if monitoring multiple services)

Add the Elasticsearch receiver and processors to your existing configuration. Refer to the sections above and merge them with your current `deployment-configmap.yaml`.

> #### 💡 TIP
>
> **Customize for your environment:**
>
> **Required changes:**
>
> -   **Pod label rule** - The rule `labels["app"] == "elasticsearch"` must match your pod labels
> -   **Cluster name** - Replace `elasticsearch-cluster` in both `resource/cluster` and `resource/cluster_name_override` with your unique cluster name
>
> **Optional changes:**
>
> -   **Port** - Update `9200` if Elasticsearch runs on a different port
> -   **Authentication** - Add credentials if your Elasticsearch cluster is secured (see example below)

**For secured Elasticsearch clusters:**

```yaml
receivers:
  receiver_creator/elasticsearch:
    watch_observers: [k8s_observer]
    receivers:
      elasticsearch:
        rule: type == "pod" && labels["app"] == "elasticsearch"
        config:
          endpoint: 'https://`endpoint`:9200'
          username: "your_elasticsearch_username"
          password: "your_elasticsearch_password"
          tls:
            insecure_skip_verify: false
```

##### Apply the updated manifests [#nrdot-apply-manifests]

Apply the updated manifests and restart the deployment:

```bash
kubectl apply -n newrelic -R -f rendered
kubectl rollout restart deployment nr-k8s-otel-collector-deployment -n newrelic
```

**Helm install**

##### Create Kubernetes secret for credentials [#nrdot-helm-create-secret]

Create a Kubernetes secret to store your New Relic credentials securely:

1.  Create the namespace:

    ```bash
    kubectl create namespace newrelic
    ```

2.  Create the secret:

    ```bash
    kubectl create secret generic newrelic-licenses \
      --from-literal=NEWRELIC_LICENSE_KEY=YOUR_LICENSE_KEY_HERE \
      --from-literal=NEWRELIC_OTLP_ENDPOINT=https://otlp.nr-data.net:4318 \
      --from-literal=NEW_RELIC_MEMORY_LIMIT_MIB=100 \
      -n newrelic
    ```

    Update the values:

    -   Replace `YOUR_LICENSE_KEY_HERE` with your actual New Relic license key
    -   Replace `https://otlp.nr-data.net:4318` with your region's endpoint (refer to [OTLP endpoint documentation](https://docs.newrelic.com/docs/opentelemetry/best-practices/opentelemetry-otlp/#configure-endpoint-port-protocol))
    -   Replace `100` with your desired memory limit in MiB for the collector

##### Configure Elasticsearch monitoring [#nrdot-helm-configure-es]

Create a `values.yaml` file for NRDOT Collector Elasticsearch monitoring:

```yaml
mode: deployment

image:
  repository: newrelic/nrdot-collector
  tag: latest
  pullPolicy: IfNotPresent

resources:
  limits:
    cpu: 500m
    memory: 512Mi
  requests:
    cpu: 200m
    memory: 256Mi

ports: {}

extraEnvs:
  - name: NEWRELIC_LICENSE_KEY
    valueFrom:
      secretKeyRef:
        name: newrelic-licenses
        key: NEWRELIC_LICENSE_KEY
  - name: NEWRELIC_OTLP_ENDPOINT
    valueFrom:
      secretKeyRef:
        name: newrelic-licenses
        key: NEWRELIC_OTLP_ENDPOINT
  - name: NEW_RELIC_MEMORY_LIMIT_MIB
    valueFrom:
      secretKeyRef:
        name: newrelic-licenses
        key: NEW_RELIC_MEMORY_LIMIT_MIB
  - name: K8S_CLUSTER_NAME
    value: "elasticsearch-cluster"

clusterRole:
  create: true
  rules:
    - apiGroups: [""]
      resources: ["pods", "nodes", "nodes/stats", "nodes/proxy"]
      verbs: ["get", "list", "watch"]
    - apiGroups: ["apps"]
      resources: ["replicasets"]
      verbs: ["get", "list", "watch"]

config:
  extensions:
    health_check:
      endpoint: 0.0.0.0:13133
    k8s_observer:
      auth_type: serviceAccount
      observe_pods: true
      observe_nodes: true

  receivers:
    jaeger: null
    zipkin: null
    prometheus: null
    otlp: null
    receiver_creator/elasticsearch:
      watch_observers: [k8s_observer]
      receivers:
        elasticsearch:
          rule: type == "pod" && labels["app"] == "elasticsearch"
          config:
            endpoint: 'http://`endpoint`:9200'
            collection_interval: 15s
            metrics:
              elasticsearch.os.cpu.usage:
                enabled: true
              elasticsearch.cluster.data_nodes:
                enabled: true
              elasticsearch.cluster.health:
                enabled: true
              elasticsearch.cluster.in_flight_fetch:
                enabled: true
              elasticsearch.cluster.nodes:
                enabled: true
              elasticsearch.cluster.pending_tasks:
                enabled: true
              elasticsearch.cluster.shards:
                enabled: true
              elasticsearch.cluster.state_update.time:
                enabled: true
              elasticsearch.index.documents:
                enabled: true
              elasticsearch.index.operations.merge.current:
                enabled: true
              elasticsearch.index.operations.time:
                enabled: true
              elasticsearch.node.cache.count:
                enabled: true
              elasticsearch.node.cache.evictions:
                enabled: true
              elasticsearch.node.cache.memory.usage:
                enabled: true
              elasticsearch.node.shards.size:
                enabled: true
              elasticsearch.node.cluster.io:
                enabled: true
              elasticsearch.node.documents:
                enabled: true
              elasticsearch.node.disk.io.read:
                enabled: true
              elasticsearch.node.disk.io.write:
                enabled: true
              elasticsearch.node.fs.disk.available:
                enabled: true
              elasticsearch.node.fs.disk.total:
                enabled: true
              elasticsearch.node.http.connections:
                enabled: true
              elasticsearch.node.ingest.documents.current:
                enabled: true
              elasticsearch.node.ingest.operations.failed:
                enabled: true
              elasticsearch.node.open_files:
                enabled: true
              elasticsearch.node.operations.completed:
                enabled: true
              elasticsearch.node.operations.current:
                enabled: true
              elasticsearch.node.operations.get.completed:
                enabled: true
              elasticsearch.node.operations.time:
                enabled: true
              elasticsearch.node.shards.reserved.size:
                enabled: true
              elasticsearch.index.shards.size:
                enabled: true
              elasticsearch.os.cpu.load_avg.1m:
                enabled: true
              elasticsearch.os.cpu.load_avg.5m:
                enabled: true
              elasticsearch.os.cpu.load_avg.15m:
                enabled: true
              elasticsearch.os.memory:
                enabled: true
              jvm.gc.collections.count:
                enabled: true
              jvm.gc.collections.elapsed:
                enabled: true
              jvm.memory.heap.max:
                enabled: true
              jvm.memory.heap.used:
                enabled: true
              jvm.memory.heap.utilization:
                enabled: true
              jvm.threads.count:
                enabled: true
              elasticsearch.index.segments.count:
                enabled: true
              elasticsearch.index.operations.completed:
                enabled: true
              elasticsearch.node.script.cache_evictions:
                enabled: false
              elasticsearch.node.cluster.connections:
                enabled: false
              elasticsearch.node.pipeline.ingest.documents.preprocessed:
                enabled: false
              elasticsearch.node.thread_pool.tasks.queued:
                enabled: false
              elasticsearch.cluster.published_states.full:
                enabled: false
              jvm.memory.pool.max:
                enabled: false
              elasticsearch.node.script.compilation_limit_triggered:
                enabled: false
              elasticsearch.node.shards.data_set.size:
                enabled: false
              elasticsearch.node.pipeline.ingest.documents.current:
                enabled: false
              elasticsearch.cluster.state_update.count:
                enabled: false
              elasticsearch.node.fs.disk.free:
                enabled: false
              jvm.memory.nonheap.used:
                enabled: false
              jvm.memory.pool.used:
                enabled: false
              elasticsearch.node.translog.size:
                enabled: false
              elasticsearch.node.thread_pool.threads:
                enabled: false
              elasticsearch.cluster.state_queue:
                enabled: false
              elasticsearch.node.translog.operations:
                enabled: false
              elasticsearch.memory.indexing_pressure:
                enabled: false
              elasticsearch.node.ingest.documents:
                enabled: false
              jvm.classes.loaded:
                enabled: false
              jvm.memory.heap.committed:
                enabled: false
              elasticsearch.breaker.memory.limit:
                enabled: false
              elasticsearch.indexing_pressure.memory.total.replica_rejections:
                enabled: false
              elasticsearch.breaker.memory.estimated:
                enabled: false
              elasticsearch.cluster.published_states.differences:
                enabled: false
              jvm.memory.nonheap.committed:
                enabled: false
              elasticsearch.node.translog.uncommitted.size:
                enabled: false
              elasticsearch.node.script.compilations:
                enabled: false
              elasticsearch.node.pipeline.ingest.operations.failed:
                enabled: false
              elasticsearch.indexing_pressure.memory.limit:
                enabled: false
              elasticsearch.breaker.tripped:
                enabled: false
              elasticsearch.indexing_pressure.memory.total.primary_rejections:
                enabled: false
              elasticsearch.node.thread_pool.tasks.finished:
                enabled: false

  processors:
    memory_limiter:
      check_interval: 60s
      limit_mib: ${env:NEW_RELIC_MEMORY_LIMIT_MIB}
    cumulativetodelta: {}
    attributes/cluster_state_aggregate:
      include:
        match_type: strict
        metric_names:
          - elasticsearch.cluster.state_update.time
      actions:
        - key: type
          action: delete
        - key: state
          action: delete
    filter/critical_operations:
      metrics:
        datapoint:
          - 'attributes["operation"] == "query" or attributes["operation"] == "index" or attributes["operation"] == "get" or attributes["operation"] == "merge" or attributes["operation"] == nil'
    resource/cluster:
      attributes:
        - key: k8s.cluster.name
          value: "${env:K8S_CLUSTER_NAME}"
          action: insert
    resource/cluster_name_override:
      attributes:
        - key: elasticsearch.cluster.name
          value: "${env:K8S_CLUSTER_NAME}"
          action: upsert
    resourcedetection:
      detectors: [env, system]
      system:
        resource_attributes:
          host.name:
            enabled: true
          host.id:
            enabled: true
          os.type:
            enabled: true
    batch:
      timeout: 30s
      send_batch_size: 2048
      send_batch_max_size: 4096
    attributes/cardinality_reduction:
      actions:
        - key: process.pid
          action: delete
        - key: process.parent_pid
          action: delete
        - key: k8s.pod.uid
          action: delete
    transform/metadata_nullify:
      metric_statements:
        - context: metric
          statements:
            - set(description, "")
            - set(unit, "")

  exporters:
    otlp_http:
      endpoint: "${env:NEWRELIC_OTLP_ENDPOINT}"
      headers:
        api-key: "${env:NEWRELIC_LICENSE_KEY}"
      compression: gzip
      timeout: 30s
      retry_on_failure:
        enabled: true
        initial_interval: 5s
        max_interval: 30s
        max_elapsed_time: 300s

  service:
    extensions: [health_check, k8s_observer]
    pipelines:
      traces: null
      logs: null
      metrics: null
      metrics/elasticsearch:
        receivers: [receiver_creator/elasticsearch]
        processors: [memory_limiter, resourcedetection, resource/cluster, resource/cluster_name_override, attributes/cardinality_reduction, filter/critical_operations, attributes/cluster_state_aggregate, cumulativetodelta, transform/metadata_nullify, batch]
        exporters: [otlp_http]
```

> #### 💡 TIP
>
> **Customize for your environment:**
>
> **Required changes:**
>
> -   **Pod label rule** - Update `labels["app"] == "elasticsearch"` to match your pod labels
> -   **Cluster name** - Replace `elasticsearch-cluster` in `extraEnvs.K8S_CLUSTER_NAME` with your unique cluster name
>
> **Optional changes:**
>
> -   **Port** - Update `9200` if Elasticsearch runs on a different port
> -   **Authentication** - Add credentials if secured (see below)

**For secured Elasticsearch clusters:**

```yaml
receivers:
  receiver_creator/elasticsearch:
    watch_observers: [k8s_observer]
    receivers:
      elasticsearch:
        rule: type == "pod" && labels["app"] == "elasticsearch"
        config:
          endpoint: 'https://`endpoint`:9200'
          username: "your_elasticsearch_username"
          password: "your_elasticsearch_password"
          tls:
            insecure_skip_verify: false
```

##### Install with Helm [#nrdot-helm-install]

Install the NRDOT Collector using Helm with your `values.yaml` configuration:

```bash
helm repo add open-telemetry https://open-telemetry.github.io/opentelemetry-helm-charts
helm repo update
helm upgrade --install elasticsearch-nrdot-collector open-telemetry/opentelemetry-collector \
  --namespace newrelic \
  --create-namespace \
  -f values.yaml
```

#### Verify deployment and data collection [#nrdot-verify-deployment]

Verify that the NRDOT collector is running and collecting Elasticsearch data:

1.  Check that the collector pods are running:

    ```bash
    kubectl get pods -n newrelic --watch
    ```

    **For manifest install:** You should see pods with names like `nr-k8s-otel-collector-deployment-<hash>` in a `Running` state.

    **For Helm install:** You should see pods with names like `elasticsearch-nrdot-collector-<hash>` in a `Running` state.

2.  Check the collector logs for any errors:

    **For manifest install:**

    ```bash
    kubectl logs -n newrelic -l app.kubernetes.io/name=nr-k8s-otel-collector -f
    ```

    **For Helm install:**

    ```bash
    kubectl logs -n newrelic -l app.kubernetes.io/name=opentelemetry-collector -f
    ```

    Look for successful connections to Elasticsearch pods and New Relic. If you see errors, refer to the [troubleshooting guide](https://docs.newrelic.com/docs/opentelemetry/integrations/elasticsearch/troubleshooting).

3.  Run an NRQL query in New Relic to confirm data is arriving (replace `elasticsearch-cluster` with your cluster name):

    ```sql
    FROM Metric
    SELECT *
    WHERE metricName LIKE 'elasticsearch.%'
      AND instrumentation.provider = 'opentelemetry'
      AND k8s.cluster.name = 'elasticsearch-cluster'
    SINCE 10 minutes ago
    ```

### OTel Collector Contrib

#### Before you begin [#prereq-otel]

Before deploying the OTel Collector Contrib on Kubernetes, ensure you have:

**Required access privileges:**

-   Your New Relic license key
-   kubectl access to your Kubernetes cluster
-   Elasticsearch cluster admin privileges with `monitor` or `manage` cluster privilege (see [Elasticsearch security privileges documentation](https://www.elastic.co/guide/en/elasticsearch/reference/current/security-privileges.html) for details)

**System requirements:**

-   **Elasticsearch version 7.16 or higher** - This integration requires a modern Elasticsearch cluster
-   **Kubernetes cluster** - A running Kubernetes cluster where Elasticsearch is deployed
-   **Helm 3.0 or higher** - [Helm](https://helm.sh/docs/intro/install/) installed on your system
-   **Network connectivity** - Outbound HTTPS (port 443) to New Relic's [OTLP ingest endpoint](https://docs.newrelic.com/docs/opentelemetry/best-practices/opentelemetry-otlp)

**Elasticsearch pod requirements:**

-   **Pod labels (Required)** - Each Elasticsearch pod must have the label `app: elasticsearch` for automatic discovery to work. Without this label, the collector will not detect or monitor your pods.

> #### ⚠️ IMPORTANT
>
> **How to add labels to Elasticsearch pods:**
>
> If you're using a StatefulSet or Deployment for Elasticsearch, add the label in the pod template:
>
> ```yaml
> apiVersion: apps/v1
> kind: StatefulSet
> metadata:
>   name: elasticsearch
> spec:
>   template:
>     metadata:
>       labels:
>         app: elasticsearch  # Required for auto-discovery
>     spec:
>       containers:
>       - name: elasticsearch
>         image: docker.elastic.co/elasticsearch/elasticsearch:8.x.x
> ```
>
> For existing pods without labels, update your StatefulSet/Deployment and restart the pods:
>
> ```bash
> kubectl label pods -l <your-existing-selector> app=elasticsearch -n <namespace>
> ```
>
> You can verify labels are set correctly:
>
> ```bash
> kubectl get pods -n <namespace> --show-labels
> ```

#### Create Kubernetes secret for credentials [#create-secret]

Create a Kubernetes secret to store your New Relic credentials securely:

1.  Create the namespace:

```bash
kubectl create namespace newrelic
```

2.  Create the secret:

```bash
kubectl create secret generic newrelic-licenses \
  --from-literal=NEWRELIC_LICENSE_KEY=YOUR_LICENSE_KEY_HERE \
  --from-literal=NEWRELIC_OTLP_ENDPOINT=https://otlp.nr-data.net:4318 \
  --from-literal=NEW_RELIC_MEMORY_LIMIT_MIB=100 \
  -n newrelic
```

Update the values:

-   Replace `YOUR_LICENSE_KEY_HERE` with your actual New Relic license key
-   Replace `https://otlp.nr-data.net:4318` with your region's endpoint (refer to [OTLP endpoint documentation](https://docs.newrelic.com/docs/opentelemetry/best-practices/opentelemetry-otlp/#configure-endpoint-port-protocol))
-   Replace `100` with your desired memory limit in MiB for the collector (default: 100 MiB). Adjust based on your environment's needs

#### Configure Elasticsearch monitoring [#configure-es-k8s]

Create a `values.yaml` file to configure the OpenTelemetry Collector for Elasticsearch monitoring:

> #### 💡 TIP
>
> **Customize for your environment:** Update the following values in the configuration:
>
> **Required changes:**
>
> -   **Pod label rule** - The rule `labels["app"] == "elasticsearch"` must match your pod labels. If your Elasticsearch pods use different labels (e.g., `app: es-cluster`), update the rule accordingly:
>     ```yaml
>     rule: type == "pod" && labels["app"] == "es-cluster"
>     ```
> -   **Cluster name** - Replace `elasticsearch-cluster` with a unique name to identify your cluster in New Relic. This name will be used to create and identify your Elasticsearch entities in the New Relic UI. Choose a name that's unique across your New Relic account (e.g., `prod-es-k8s`, `staging-elasticsearch`)
>
> **Optional changes:**
>
> -   **Port** - Update `9200` if Elasticsearch runs on a different port
> -   **Authentication** - Add credentials if your Elasticsearch cluster is secured

```yaml
mode: deployment

image:
  repository: otel/opentelemetry-collector-contrib
  pullPolicy: IfNotPresent

command:
  name: otelcol-contrib

resources:
  limits:
    cpu: 500m
    memory: 512Mi
  requests:
    cpu: 200m
    memory: 256Mi

extraEnvs:
  - name: NEWRELIC_LICENSE_KEY
    valueFrom:
      secretKeyRef:
        name: newrelic-licenses
        key: NEWRELIC_LICENSE_KEY
  - name: NEWRELIC_OTLP_ENDPOINT
    valueFrom:
      secretKeyRef:
        name: newrelic-licenses
        key: NEWRELIC_OTLP_ENDPOINT
  - name: NEW_RELIC_MEMORY_LIMIT_MIB
    valueFrom:
      secretKeyRef:
        name: newrelic-licenses
        key: NEW_RELIC_MEMORY_LIMIT_MIB
  - name: K8S_CLUSTER_NAME
    value: "elasticsearch-cluster"

clusterRole:
  create: true
  rules:
    - apiGroups: [""]
      resources: ["pods", "nodes", "nodes/stats", "nodes/proxy"]
      verbs: ["get", "list", "watch"]
    - apiGroups: ["apps"]
      resources: ["replicasets"]
      verbs: ["get", "list", "watch"]

config:
    extensions:
      health_check:
        endpoint: 0.0.0.0:13133
      k8s_observer:
        auth_type: serviceAccount
        observe_pods: true
        observe_nodes: true

    receivers:
      receiver_creator/elasticsearch:
        watch_observers: [k8s_observer]
        receivers:
          elasticsearch:
            rule: type == "pod" && labels["app"] == "elasticsearch"
            config:
              endpoint: 'http://`endpoint`:9200'
              collection_interval: 30s
              metrics:
                elasticsearch.os.cpu.usage:
                  enabled: true
                elasticsearch.cluster.data_nodes:
                  enabled: true
                elasticsearch.cluster.health:
                  enabled: true
                elasticsearch.cluster.in_flight_fetch:
                  enabled: true
                elasticsearch.cluster.nodes:
                  enabled: true
                elasticsearch.cluster.pending_tasks:
                  enabled: true
                elasticsearch.cluster.shards:
                  enabled: true
                elasticsearch.cluster.state_update.time:
                  enabled: true
                elasticsearch.index.documents:
                  enabled: true
                elasticsearch.index.operations.merge.current:
                  enabled: true
                elasticsearch.index.operations.time:
                  enabled: true
                elasticsearch.node.cache.count:
                  enabled: true
                elasticsearch.node.cache.evictions:
                  enabled: true
                elasticsearch.node.cache.memory.usage:
                  enabled: true
                elasticsearch.node.shards.size:
                  enabled: true
                elasticsearch.node.cluster.io:
                  enabled: true
                elasticsearch.node.documents:
                  enabled: true
                elasticsearch.node.disk.io.read:
                  enabled: true
                elasticsearch.node.disk.io.write:
                  enabled: true
                elasticsearch.node.fs.disk.available:
                  enabled: true
                elasticsearch.node.fs.disk.total:
                  enabled: true
                elasticsearch.node.http.connections:
                  enabled: true
                elasticsearch.node.ingest.documents.current:
                  enabled: true
                elasticsearch.node.ingest.operations.failed:
                  enabled: true
                elasticsearch.node.open_files:
                  enabled: true
                elasticsearch.node.operations.completed:
                  enabled: true
                elasticsearch.node.operations.current:
                  enabled: true
                elasticsearch.node.operations.get.completed:
                  enabled: true
                elasticsearch.node.operations.time:
                  enabled: true
                elasticsearch.node.shards.reserved.size:
                  enabled: true
                elasticsearch.index.shards.size:
                  enabled: true
                elasticsearch.os.cpu.load_avg.1m:
                  enabled: true
                elasticsearch.os.cpu.load_avg.5m:
                  enabled: true
                elasticsearch.os.cpu.load_avg.15m:
                  enabled: true
                elasticsearch.os.memory:
                  enabled: true
                jvm.gc.collections.count:
                  enabled: true
                jvm.gc.collections.elapsed:
                  enabled: true
                jvm.memory.heap.max:
                  enabled: true
                jvm.memory.heap.used:
                  enabled: true
                jvm.memory.heap.utilization:
                  enabled: true
                jvm.threads.count:
                  enabled: true
                elasticsearch.index.segments.count:
                  enabled: true
                elasticsearch.index.operations.completed:
                  enabled: true
                elasticsearch.node.script.cache_evictions:
                  enabled: false
                elasticsearch.node.cluster.connections:
                  enabled: false
                elasticsearch.node.pipeline.ingest.documents.preprocessed:
                  enabled: false
                elasticsearch.node.thread_pool.tasks.queued:
                  enabled: false
                elasticsearch.cluster.published_states.full:
                  enabled: false
                jvm.memory.pool.max:
                  enabled: false
                elasticsearch.node.script.compilation_limit_triggered:
                  enabled: false
                elasticsearch.node.shards.data_set.size:
                  enabled: false
                elasticsearch.node.pipeline.ingest.documents.current:
                  enabled: false
                elasticsearch.cluster.state_update.count:
                  enabled: false
                elasticsearch.node.fs.disk.free:
                  enabled: false
                jvm.memory.nonheap.used:
                  enabled: false
                jvm.memory.pool.used:
                  enabled: false
                elasticsearch.node.translog.size:
                  enabled: false
                elasticsearch.node.thread_pool.threads:
                  enabled: false
                elasticsearch.cluster.state_queue:
                  enabled: false
                elasticsearch.node.translog.operations:
                  enabled: false
                elasticsearch.memory.indexing_pressure:
                  enabled: false
                elasticsearch.node.ingest.documents:
                  enabled: false
                jvm.classes.loaded:
                  enabled: false
                jvm.memory.heap.committed:
                  enabled: false
                elasticsearch.breaker.memory.limit:
                  enabled: false
                elasticsearch.indexing_pressure.memory.total.replica_rejections:
                  enabled: false
                elasticsearch.breaker.memory.estimated:
                  enabled: false
                elasticsearch.cluster.published_states.differences:
                  enabled: false
                jvm.memory.nonheap.committed:
                  enabled: false
                elasticsearch.node.translog.uncommitted.size:
                  enabled: false
                elasticsearch.node.script.compilations:
                  enabled: false
                elasticsearch.node.pipeline.ingest.operations.failed:
                  enabled: false
                elasticsearch.indexing_pressure.memory.limit:
                  enabled: false
                elasticsearch.breaker.tripped:
                  enabled: false
                elasticsearch.indexing_pressure.memory.total.primary_rejections:
                  enabled: false
                elasticsearch.node.thread_pool.tasks.finished:
                  enabled: false

    processors:
      memory_limiter:
        check_interval: 60s
        limit_mib: ${env:NEW_RELIC_MEMORY_LIMIT_MIB}
      cumulativetodelta: {}
      resource/cluster:
        attributes:
          - key: k8s.cluster.name
            value: "${env:K8S_CLUSTER_NAME}"
            action: insert
      resource/cluster_name_override:
        attributes:
          - key: elasticsearch.cluster.name
            value: "${env:K8S_CLUSTER_NAME}"
            action: upsert
      resourcedetection:
        detectors: [env, system]
        system:
          resource_attributes:
            host.name:
              enabled: true
            host.id:
              enabled: true
            os.type:
              enabled: true
      batch:
        timeout: 10s
        send_batch_size: 1024
      attributes/cardinality_reduction:
        actions:
          - key: process.pid
            action: delete
          - key: process.parent_pid
            action: delete
          - key: k8s.pod.uid
            action: delete
      transform/metadata_nullify:
        metric_statements:
          - context: metric
            statements:
              - set(description, "")
              - set(unit, "")

    exporters:
      otlphttp:
        endpoint: "${env:NEWRELIC_OTLP_ENDPOINT}"
        headers:
          api-key: "${env:NEWRELIC_LICENSE_KEY}"

    service:
      extensions: [health_check, k8s_observer]
      pipelines:
        metrics/elasticsearch:
          receivers: [receiver_creator/elasticsearch]
          processors: [memory_limiter, resourcedetection, resource/cluster, resource/cluster_name_override, attributes/cardinality_reduction, cumulativetodelta, transform/metadata_nullify, batch]
          exporters: [otlphttp]
```

> #### 💡 TIP
>
> **For secured Elasticsearch clusters:** If your Elasticsearch cluster requires authentication, add credentials to the receiver configuration:
>
> ```yaml
> receiver_creator/elasticsearch:
>   watch_observers: [k8s_observer]
>   receivers:
>     elasticsearch:
>       rule: type == "pod" && labels["app"] == "elasticsearch"
>       config:
>         endpoint: 'https://`endpoint`:9200'
>         username: "your_elasticsearch_username"
>         password: "your_elasticsearch_password"
>         tls:
>           insecure_skip_verify: false
> ```
>
> Store credentials securely using Kubernetes secrets rather than hardcoding them in the values file.

#### Install with Helm [#install-helm]

Install the OpenTelemetry Collector using Helm with your `values.yaml` configuration:

```bash
helm repo add open-telemetry https://open-telemetry.github.io/opentelemetry-helm-charts
helm repo update
helm upgrade --install elasticsearch-otel-collector open-telemetry/opentelemetry-collector \
  --namespace newrelic \
  --create-namespace \
  -f values.yaml
```

#### Verify deployment and data collection [#verify-deployment]

Verify that the OpenTelemetry Collector is running and collecting Elasticsearch data:

1.  Check that the collector pods are running:

    ```bash
    kubectl get pods -n newrelic --watch
    ```

    You should see pods with names like `elasticsearch-otel-collector-<hash>` in a `Running` state.

2.  Check the collector logs for any errors:

    ```bash
    kubectl logs -n newrelic -l app.kubernetes.io/name=opentelemetry-collector -f
    ```

    Look for successful connections to Elasticsearch pods and New Relic. If you see errors, refer to the [troubleshooting guide](https://docs.newrelic.com/docs/opentelemetry/integrations/elasticsearch/troubleshooting).

3.  Run an NRQL query in New Relic to confirm data is arriving (replace `elasticsearch-cluster` with your cluster name):

    ```sql
    FROM Metric
    SELECT *
    WHERE metricName LIKE 'elasticsearch.%'
      AND instrumentation.provider = 'opentelemetry'
      AND k8s.cluster.name = 'elasticsearch-cluster'
    SINCE 10 minutes ago
    ```

### Prometheus Receiver

Use this approach if you already have a [Prometheus Elasticsearch exporter](https://github.com/prometheus-community/elasticsearch_exporter) running in your Kubernetes cluster, or if you're migrating from a Prometheus-based monitoring stack.

> #### 💡 TIP
>
> **Recommended:** If you don't already have a Prometheus exporter running, use the **NRDOT Collector** or **OTel Collector Contrib** tabs instead. They connect directly to the Elasticsearch API without needing an additional exporter component.

#### Deploy the Elasticsearch exporter [#deploy-exporter-k8s]

If you don't already have the exporter running, deploy it using Helm:

```bash
helm repo add prometheus-community https://prometheus-community.github.io/helm-charts
helm repo update

helm install elasticsearch-exporter prometheus-community/prometheus-elasticsearch-exporter \
  --namespace monitoring \
  --create-namespace \
  --set es.uri=http://elasticsearch.default.svc.cluster.local:9200
```

Replace `elasticsearch.default.svc.cluster.local:9200` with your Elasticsearch service address.

Verify the exporter is running:

```bash
kubectl get pods -n monitoring -l app=prometheus-elasticsearch-exporter
```

#### Create the credentials Secret [#create-secret-prom-k8s]

```bash
kubectl create secret generic newrelic-credentials \
  --namespace monitoring \
  --from-literal=NEWRELIC_LICENSE_KEY=YOUR_NEWRELIC_LICENSE_KEY \
  --from-literal=NEWRELIC_OTLP_ENDPOINT=https://otlp.nr-data.net:4318
```

Replace `YOUR_NEWRELIC_LICENSE_KEY` with your license key.

> #### 💡 TIP
>
> For EU accounts, use `NEWRELIC_OTLP_ENDPOINT=https://otlp.eu01.nr-data.net:4318`

#### Create the collector ConfigMap [#create-configmap-prom-k8s]

Create a ConfigMap with the collector configuration. This works with both **NRDOT** (`newrelic/nrdot-collector`) and **OTel Collector Contrib** (`otel/opentelemetry-collector-contrib`) container images. The configuration scrapes metrics from the Elasticsearch exporter and translates Prometheus metric names to OpenTelemetry-compatible names that power the New Relic Elasticsearch dashboard.

**Full Prometheus receiver ConfigMap**

Save the following as `otel-collector-prometheus-es.yaml` and apply with `kubectl apply -f otel-collector-prometheus-es.yaml`:

```yaml
apiVersion: v1
kind: ConfigMap
metadata:
  name: otel-collector-prometheus-es
  namespace: monitoring
data:
  config.yaml: |
    receivers:
      prometheus/elasticsearch:
        config:
          scrape_configs:
            - job_name: 'elasticsearch'
              scrape_interval: 15s
              static_configs:
                - targets: ['elasticsearch-exporter-prometheus-elasticsearch-exporter.monitoring.svc.cluster.local:9114']

    processors:
      memory_limiter:
        check_interval: 60s
        limit_mib: 100

      metricstransform/prom_to_otel:
        transforms:
          - include: ^elasticsearch_cluster_health_number_of_data_nodes$
            match_type: regexp
            action: update
            new_name: elasticsearch.cluster.data_nodes
          - include: ^elasticsearch_cluster_health_status$
            match_type: regexp
            action: update
            new_name: elasticsearch.cluster.health
            operations:
              - action: update_label
                label: color
                new_label: status
          - include: ^elasticsearch_cluster_health_number_of_in_flight_fetch$
            match_type: regexp
            action: update
            new_name: elasticsearch.cluster.in_flight_fetch
          - include: ^elasticsearch_cluster_health_number_of_nodes$
            match_type: regexp
            action: update
            new_name: elasticsearch.cluster.nodes
          - include: ^elasticsearch_cluster_health_number_of_pending_tasks$
            match_type: regexp
            action: update
            new_name: elasticsearch.cluster.pending_tasks
          - include: ^elasticsearch_cluster_health_active_shards$
            match_type: regexp
            action: update
            new_name: elasticsearch.cluster.shards
            operations:
              - action: add_label
                new_label: state
                new_value: active
          - include: ^elasticsearch_cluster_health_unassigned_shards$
            match_type: regexp
            action: update
            new_name: elasticsearch.cluster.shards
            operations:
              - action: add_label
                new_label: state
                new_value: unassigned
          - include: ^elasticsearch_indices_docs$
            match_type: regexp
            action: insert
            new_name: elasticsearch.node.documents
            operations:
              - action: add_label
                new_label: state
                new_value: active
          - include: ^elasticsearch_indices_docs$
            match_type: regexp
            action: insert
            new_name: elasticsearch.index.documents
            operations:
              - action: add_label
                new_label: state
                new_value: active
              - action: add_label
                new_label: aggregation
                new_value: primary_shards
              - action: add_label
                new_label: elasticsearch.index.name
                new_value: _all
          - include: ^elasticsearch_indices_fielddata_evictions$
            match_type: regexp
            action: update
            new_name: elasticsearch.node.cache.evictions
            operations:
              - action: add_label
                new_label: cache_name
                new_value: fielddata
          - include: ^elasticsearch_indices_query_cache_evictions$
            match_type: regexp
            action: update
            new_name: elasticsearch.node.cache.evictions
            operations:
              - action: add_label
                new_label: cache_name
                new_value: query
          - include: ^elasticsearch_transport_rx_size_bytes_total$
            match_type: regexp
            action: update
            new_name: elasticsearch.node.cluster.io
            operations:
              - action: add_label
                new_label: direction
                new_value: received
          - include: ^elasticsearch_transport_tx_size_bytes_total$
            match_type: regexp
            action: update
            new_name: elasticsearch.node.cluster.io
            operations:
              - action: add_label
                new_label: direction
                new_value: sent
          - include: ^elasticsearch_filesystem_data_available_bytes$
            match_type: regexp
            action: update
            new_name: elasticsearch.node.fs.disk.available
          - include: ^elasticsearch_filesystem_data_size_bytes$
            match_type: regexp
            action: update
            new_name: elasticsearch.node.fs.disk.total
          - include: ^elasticsearch_indices_indexing_index_total$
            match_type: regexp
            action: update
            new_name: elasticsearch.node.operations.completed
            operations:
              - action: add_label
                new_label: operation
                new_value: index
          - include: ^elasticsearch_indices_search_query_total$
            match_type: regexp
            action: update
            new_name: elasticsearch.node.operations.completed
            operations:
              - action: add_label
                new_label: operation
                new_value: query
          - include: ^elasticsearch_indices_get_total$
            match_type: regexp
            action: update
            new_name: elasticsearch.node.operations.completed
            operations:
              - action: add_label
                new_label: operation
                new_value: get
          - include: ^elasticsearch_indices_merges_total$
            match_type: regexp
            action: update
            new_name: elasticsearch.node.operations.completed
            operations:
              - action: add_label
                new_label: operation
                new_value: merge
          - include: ^elasticsearch_indices_indexing_index_time_seconds_total$
            match_type: regexp
            action: update
            new_name: elasticsearch.node.operations.time
            operations:
              - action: add_label
                new_label: operation
                new_value: index
              - action: experimental_scale_value
                experimental_scale: 1000
          - include: ^elasticsearch_indices_search_query_time_seconds$
            match_type: regexp
            action: update
            new_name: elasticsearch.node.operations.time
            operations:
              - action: add_label
                new_label: operation
                new_value: query
              - action: experimental_scale_value
                experimental_scale: 1000
          - include: ^elasticsearch_os_cpu_percent$
            match_type: regexp
            action: update
            new_name: elasticsearch.os.cpu.usage
          - include: ^elasticsearch_jvm_gc_collection_seconds_count$
            match_type: regexp
            action: update
            new_name: jvm.gc.collections.count
          - include: ^elasticsearch_jvm_memory_max_bytes$
            match_type: regexp
            action: update
            new_name: jvm.memory.heap.max
          - include: ^elasticsearch_jvm_memory_used_bytes$
            match_type: regexp
            action: update
            new_name: jvm.memory.heap.used
          - include: ^elasticsearch_thread_pool_threads_count$
            match_type: regexp
            action: update
            new_name: jvm.threads.count
            operations:
              - action: aggregate_labels
                label_set: [name, cluster]
                aggregation_type: sum
          - include: ^elasticsearch_indices_segments_count$
            match_type: regexp
            action: update
            new_name: elasticsearch.index.segments.count

      filter/allowed_metrics:
        metrics:
          include:
            match_type: strict
            metric_names:
              - elasticsearch.cluster.data_nodes
              - elasticsearch.cluster.health
              - elasticsearch.cluster.in_flight_fetch
              - elasticsearch.cluster.nodes
              - elasticsearch.cluster.pending_tasks
              - elasticsearch.cluster.shards
              - elasticsearch.index.documents
              - elasticsearch.index.segments.count
              - elasticsearch.node.cache.evictions
              - elasticsearch.node.cluster.io
              - elasticsearch.node.documents
              - elasticsearch.node.fs.disk.available
              - elasticsearch.node.fs.disk.total
              - elasticsearch.node.operations.completed
              - elasticsearch.node.operations.time
              - elasticsearch.os.cpu.usage
              - jvm.gc.collections.count
              - jvm.memory.heap.max
              - jvm.memory.heap.used
              - jvm.threads.count

      transform/rename_node_name:
        metric_statements:
          - context: datapoint
            statements:
              - set(attributes["elasticsearch.node.name"], attributes["name"]) where attributes["name"] != nil
              - delete_key(attributes, "name") where attributes["elasticsearch.node.name"] != nil

      groupbyattrs/node:
        keys:
          - elasticsearch.node.name
          - elasticsearch.cluster.name

      cumulativetodelta: {}

      resource/cluster_name_override:
        attributes:
          - key: elasticsearch.cluster.name
            value: "<elasticsearch-cluster-name>"
            action: upsert

      transform/scope_override:
        metric_statements:
          - context: scope
            statements:
              - set(name, "github.com/open-telemetry/opentelemetry-collector-contrib/receiver/elasticsearchreceiver")
              - set(version, "")

      resourcedetection:
        detectors: [env]
        env:
          resource_attributes:
            host.name:
              enabled: true

      batch:
        timeout: 30s
        send_batch_size: 2048
        send_batch_max_size: 4096

      attributes/cardinality_reduction:
        actions:
          - key: process.pid
            action: delete
          - key: process.parent_pid
            action: delete

      transform/metadata_nullify:
        metric_statements:
          - context: metric
            statements:
              - set(description, "")
              - set(unit, "")

    exporters:
      otlphttp:
        endpoint: ${env:NEWRELIC_OTLP_ENDPOINT}
        headers:
          api-key: ${env:NEWRELIC_LICENSE_KEY}
        compression: gzip
        timeout: 30s
        retry_on_failure:
          enabled: true
          initial_interval: 5s
          max_interval: 30s
          max_elapsed_time: 300s

    service:
      pipelines:
        metrics/elasticsearch:
          receivers: [prometheus/elasticsearch]
          processors:
            - memory_limiter
            - metricstransform/prom_to_otel
            - filter/allowed_metrics
            - transform/rename_node_name
            - groupbyattrs/node
            - transform/scope_override
            - resourcedetection
            - resource/cluster_name_override
            - attributes/cardinality_reduction
            - cumulativetodelta
            - transform/metadata_nullify
            - batch
          exporters: [otlphttp]
```

Replace the following values in the configuration:

-   `<elasticsearch-cluster-name>`: Your Elasticsearch cluster name
-   `elasticsearch-exporter-prometheus-elasticsearch-exporter.monitoring.svc.cluster.local:9114`: Your exporter's Kubernetes service address

#### Deploy the collector [#deploy-collector-prom-k8s]

Deploy the collector using either the NRDOT or OTel Collector Contrib image. Update the `image` field below based on your choice:

-   **NRDOT:** `newrelic/nrdot-collector:latest`
-   **OTel Collector Contrib:** `otel/opentelemetry-collector-contrib:latest`

Save the following as `otel-collector-deployment.yaml` and apply with `kubectl apply -f otel-collector-deployment.yaml`:

```yaml
apiVersion: apps/v1
kind: Deployment
metadata:
  name: otel-collector-elasticsearch
  namespace: monitoring
  labels:
    app: otel-collector-elasticsearch
spec:
  replicas: 1
  selector:
    matchLabels:
      app: otel-collector-elasticsearch
  template:
    metadata:
      labels:
        app: otel-collector-elasticsearch
    spec:
      containers:
        - name: otel-collector
          image: otel/opentelemetry-collector-contrib:latest
          args:
            - "--config=/etc/otel/config.yaml"
          env:
            - name: NEWRELIC_LICENSE_KEY
              valueFrom:
                secretKeyRef:
                  name: newrelic-credentials
                  key: NEWRELIC_LICENSE_KEY
            - name: NEWRELIC_OTLP_ENDPOINT
              valueFrom:
                secretKeyRef:
                  name: newrelic-credentials
                  key: NEWRELIC_OTLP_ENDPOINT
          volumeMounts:
            - name: config
              mountPath: /etc/otel
          resources:
            requests:
              memory: "128Mi"
              cpu: "100m"
            limits:
              memory: "256Mi"
              cpu: "500m"
      volumes:
        - name: config
          configMap:
            name: otel-collector-prometheus-es
```

#### Verify the deployment [#verify-prom-k8s]

1.  Check the collector pod is running:

    ```bash
    kubectl get pods -n monitoring -l app=otel-collector-elasticsearch
    ```

2.  Check collector logs:

    ```bash
    kubectl logs -n monitoring -l app=otel-collector-elasticsearch -f
    ```

3.  Verify data in New Relic:

    ```sql
    FROM Metric SELECT count(*)
    WHERE metricName LIKE 'elasticsearch.%'
    AND elasticsearch.cluster.name = '<elasticsearch-cluster-name>'
    SINCE 10 minutes ago
    ```

> #### 💡 TIP
>
> **Correlate APM with Elasticsearch**: To connect your APM application and Elasticsearch cluster, include the resource attribute `es.cluster.name="your-cluster-name"` in your APM metrics. This enables cross-service visibility and faster troubleshooting within New Relic.

## Troubleshooting [#troubleshooting]

If you encounter issues during installation or don't see data in New Relic, see our comprehensive [troubleshooting guide](https://docs.newrelic.com/docs/opentelemetry/integrations/elasticsearch/troubleshooting) for step-by-step solutions to common problems.

For Kubernetes-specific issues like pod discovery, RBAC permissions, or network connectivity, refer to the [Kubernetes troubleshooting section](https://docs.newrelic.com/docs/opentelemetry/integrations/elasticsearch/troubleshooting#kubernetes-troubleshooting).