Monitorear Elasticsearch autohospedado con OpenTelemetry

Monitoree su clúster de Elasticsearch autohospedado instalando el OpenTelemetry Collector directamente en servidores o máquinas virtuales. New Relic ofrece opciones de implementación flexibles para adaptarse a la configuración de su infraestructura y a sus requisitos de monitoreo.

Puede elegir entre tres opciones de recolector:

NRDOT: Distribución de New Relic de OpenTelemetry
OTel Collector Contrib: OpenTelemetry Collector estándar con componentes aportados por la comunidad
Receptor de Prometheus: para entornos que ya ejecutan un exportador de Elasticsearch para Prometheus

Opciones de instalación

Seleccione la distribución del recopilador que se ajuste a sus necesidades:

Antes de que empieces

Antes de configurar el monitoreo de Elasticsearch con NRDOT, asegúrese de tener:

Recolector NRDOT instalado:

Debe tener el recolector NRDOT instalado y en ejecución como un servicio de systemd en su host.
Si no está instalado, siga la guía oficial de instalación de NRDOT para instalar el recolector usando el administrador de paquetes (DEB/RPM), que configura el servicio systemd automáticamente.
Privilegios de acceso requeridos:
Tu New Relic
Privilegios de root o sudo en la máquina host
Privilegios de administrador de clúster de Elasticsearch con privilegio de clúster monitor o manage (consulte la documentación de privilegios de seguridad de Elasticsearch para obtener detalles)
Requisitos del sistema:
Versión 7.16 o superior de Elasticsearch - Esta integración requiere un clúster moderno de Elasticsearch
Conectividad de red - HTTPS saliente (puerto 443) al endpoint de ingesta OTLPde New Relic
Valores de configuración listos:
Punto de conexión de Elasticsearch - La URL de su clúster de Elasticsearch (p. ej., http://localhost:9200)
Nombre del clúster - Un identificador único para su clúster en New Relic

Configurar el monitoreo de Elasticsearch

Una vez instalado el recolector NRDOT, reemplace el archivo de configuración predeterminado del recolector con la configuración de monitoreo de Elasticsearch. Esto habilitará la recopilación de métricas de Elasticsearch. Las métricas y los logs del host son opcionales y se pueden agregar por separado.

El archivo de configuración se encuentra en: /etc/nrdot-collector/config.yaml

Sugerencia

Respalde su configuración predeterminada: Antes de modificar el archivo de configuración, cree una copia de seguridad para conservar los ajustes predeterminados:

bash

$sudo cp /etc/nrdot-collector/config.yaml /etc/nrdot-collector/config.yaml.backup

Para configurar el colector:

Abra el archivo de configuración con un editor de texto con privilegios de root o sudo:
bash
```
$sudo nano /etc/nrdot-collector/config.yaml
```

Elimine todo el contenido existente y reemplácelo con la siguiente configuración para el monitoreo de Elasticsearch:

Importante

Reemplace el valor endpoint con el endpoint de su clúster de Elasticsearch y actualice elasticsearch.cluster.name en el bloque del procesador con un nombre único para identificar su clúster en New Relic.

receivers:
  elasticsearch:
    endpoint: "http://localhost:9200"
    collection_interval: 15s
    metrics:
      elasticsearch.os.cpu.usage:
        enabled: true
      elasticsearch.cluster.data_nodes:
        enabled: true
      elasticsearch.cluster.health:
        enabled: true
      elasticsearch.cluster.in_flight_fetch:
        enabled: true
      elasticsearch.cluster.nodes:
        enabled: true
      elasticsearch.cluster.pending_tasks:
        enabled: true
      elasticsearch.cluster.shards:
        enabled: true
      elasticsearch.cluster.state_update.time:
        enabled: true
      elasticsearch.index.documents:
        enabled: true
      elasticsearch.index.operations.merge.current:
        enabled: true
      elasticsearch.index.operations.time:
        enabled: true
      elasticsearch.node.cache.count:
        enabled: true
      elasticsearch.node.cache.evictions:
        enabled: true
      elasticsearch.node.cache.memory.usage:
        enabled: true
      elasticsearch.node.shards.size:
        enabled: true
      elasticsearch.node.cluster.io:
        enabled: true
      elasticsearch.node.documents:
        enabled: true
      elasticsearch.node.disk.io.read:
        enabled: true
      elasticsearch.node.disk.io.write:
        enabled: true
      elasticsearch.node.fs.disk.available:
        enabled: true
      elasticsearch.node.fs.disk.total:
        enabled: true
      elasticsearch.node.http.connections:
        enabled: true
      elasticsearch.node.ingest.documents.current:
        enabled: true
      elasticsearch.node.ingest.operations.failed:
        enabled: true
      elasticsearch.node.open_files:
        enabled: true
      elasticsearch.node.operations.completed:
        enabled: true
      elasticsearch.node.operations.current:
        enabled: true
      elasticsearch.node.operations.get.completed:
        enabled: true
      elasticsearch.node.operations.time:
        enabled: true
      elasticsearch.node.shards.reserved.size:
        enabled: true
      elasticsearch.index.shards.size:
        enabled: true
      elasticsearch.os.cpu.load_avg.1m:
        enabled: true
      elasticsearch.os.cpu.load_avg.5m:
        enabled: true
      elasticsearch.os.cpu.load_avg.15m:
        enabled: true
      elasticsearch.os.memory:
        enabled: true
      jvm.gc.collections.count:
        enabled: true
      jvm.gc.collections.elapsed:
        enabled: true
      jvm.memory.heap.max:
        enabled: true
      jvm.memory.heap.used:
        enabled: true
      jvm.memory.heap.utilization:
        enabled: true
      jvm.threads.count:
        enabled: true
      elasticsearch.index.segments.count:
        enabled: true
      elasticsearch.index.operations.completed:
        enabled: true
      elasticsearch.node.script.cache_evictions:
        enabled: false
      elasticsearch.node.cluster.connections:
        enabled: false
      elasticsearch.node.pipeline.ingest.documents.preprocessed:
        enabled: false
      elasticsearch.node.thread_pool.tasks.queued:
        enabled: false
      elasticsearch.cluster.published_states.full:
        enabled: false
      jvm.memory.pool.max:
        enabled: false
      elasticsearch.node.script.compilation_limit_triggered:
        enabled: false
      elasticsearch.node.shards.data_set.size:
        enabled: false
      elasticsearch.node.pipeline.ingest.documents.current:
        enabled: false
      elasticsearch.cluster.state_update.count:
        enabled: false
      elasticsearch.node.fs.disk.free:
        enabled: false
      jvm.memory.nonheap.used:
        enabled: false
      jvm.memory.pool.used:
        enabled: false
      elasticsearch.node.translog.size:
        enabled: false
      elasticsearch.node.thread_pool.threads:
        enabled: false
      elasticsearch.cluster.state_queue:
        enabled: false
      elasticsearch.node.translog.operations:
        enabled: false
      elasticsearch.memory.indexing_pressure:
        enabled: false
      elasticsearch.node.ingest.documents:
        enabled: false
      jvm.classes.loaded:
        enabled: false
      jvm.memory.heap.committed:
        enabled: false
      elasticsearch.breaker.memory.limit:
        enabled: false
      elasticsearch.indexing_pressure.memory.total.replica_rejections:
        enabled: false
      elasticsearch.breaker.memory.estimated:
        enabled: false
      elasticsearch.cluster.published_states.differences:
        enabled: false
      jvm.memory.nonheap.committed:
        enabled: false
      elasticsearch.node.translog.uncommitted.size:
        enabled: false
      elasticsearch.node.script.compilations:
        enabled: false
      elasticsearch.node.pipeline.ingest.operations.failed:
        enabled: false
      elasticsearch.indexing_pressure.memory.limit:
        enabled: false
      elasticsearch.breaker.tripped:
        enabled: false
      elasticsearch.indexing_pressure.memory.total.primary_rejections:
        enabled: false
      elasticsearch.node.thread_pool.tasks.finished:
        enabled: false
processors:
  memory_limiter:
    check_interval: 60s
    limit_mib: ${env:NEW_RELIC_MEMORY_LIMIT_MIB:-100}
  cumulativetodelta: {}
  attributes/cluster_state_aggregate:
    include:
      match_type: strict
      metric_names:
        - elasticsearch.cluster.state_update.time
    actions:
      - key: type
        action: delete
      - key: state
        action: delete
  filter/critical_operations:
    metrics:
      datapoint:
        # Filters to keep only: query, index, get, merge
        # Affects only 4 metrics: *.operations.completed and *.operations.time
        # All other metrics pass through unchanged
        - 'attributes["operation"] == "query" or attributes["operation"] == "index" or attributes["operation"] == "get" or attributes["operation"] == "merge" or attributes["operation"] == nil'
  resource/cluster_name_override:
    attributes:
      - key: elasticsearch.cluster.name
        value: "<elasticsearch-cluster-name>"
        action: upsert
  resourcedetection:
    detectors: [ system ]
    system:
      resource_attributes:
        host.name:
          enabled: true
        host.id:
          enabled: true
        os.type:
          enabled: true
  batch:
    timeout: 30s
    send_batch_size: 2048
    send_batch_max_size: 4096
  attributes/cardinality_reduction:
    actions:
      - key: process.pid
        action: delete
      - key: process.parent_pid
        action: delete
  transform/metadata_nullify:
    metric_statements:
      - context: metric
        statements:
          - set(description, "")
          - set(unit, "")
exporters:
  otlphttp:
    endpoint: ${env:NEWRELIC_OTLP_ENDPOINT}
    headers:
      api-key: ${env:NEWRELIC_LICENSE_KEY}
    compression: gzip
    timeout: 30s
    retry_on_failure:
      enabled: true
      initial_interval: 5s
      max_interval: 30s
      max_elapsed_time: 300s
service:
  pipelines:
    metrics/elasticsearch:
      receivers: [elasticsearch]
      processors: [memory_limiter, resourcedetection, resource/cluster_name_override, attributes/cardinality_reduction, filter/critical_operations, attributes/cluster_state_aggregate, cumulativetodelta, transform/metadata_nullify, batch]
      exporters: [otlphttp]

(Opcional) Para Elasticsearch seguro con autenticación y SSL, modifique la configuración del receptor:

receivers:
  elasticsearch:
    endpoint: "https://localhost:9200"
    username: "your_elasticsearch_username"
    password: "your_elasticsearch_password"
    tls:
      ca_file: "/etc/elasticsearch/certs/http_ca.crt"
      insecure_skip_verify: false
    collection_interval: 15s

(Opcional) Para recopilar métricas del host, agregue el receptor hostmetrics:

receivers:
  hostmetrics:
    collection_interval: 60s
    scrapers:
      cpu:
        metrics:
          system.cpu.utilization: {enabled: true}
          system.cpu.time: {enabled: true}
      load:
        metrics:
          system.cpu.load_average.1m: {enabled: true}
          system.cpu.load_average.5m: {enabled: true}
          system.cpu.load_average.15m: {enabled: true}
      memory:
        metrics:
          system.memory.usage: {enabled: true}
          system.memory.utilization: {enabled: true}
      disk:
        metrics:
          system.disk.io: {enabled: true}
          system.disk.operations: {enabled: true}
      filesystem:
        metrics:
          system.filesystem.usage: {enabled: true}
          system.filesystem.utilization: {enabled: true}
      network:
        metrics:
          system.network.io: {enabled: true}
          system.network.packets: {enabled: true}
      process:
        metrics:
          process.cpu.utilization:
            enabled: true

Y agregue a los pipelines de servicio:

service:
  pipelines:
    metrics/host:
      receivers: [hostmetrics]
      processors: [memory_limiter, resourcedetection, batch]
      exporters: [otlphttp]

(Opcional) Para recopilar logs de Elasticsearch, agregue el receptor filelog. Asegúrese de que el usuario que ejecuta el servicio del recolector (nrdot-collector) tenga acceso de lectura a sus archivos de log de Elasticsearch:

Si ejecuta Elasticsearch en Linux (Host):

receivers:
  filelog:
    include:
      - /var/log/elasticsearch/elasticsearch.log
      - /var/log/elasticsearch/*.log

Si ejecuta Elasticsearch en Docker:

receivers:
  filelog:
    include:
      - /var/lib/docker/containers/*/*.log
    operators:
      - type: move
        from: attributes.log
        to: body

Y agregue a los pipelines de servicio:

service:
  pipelines:
    logs:
      receivers: [filelog]
      processors: [resource/cluster_name_override]
      exporters: [otlphttp]

(Opcional) Para agregar etiquetas de metadatos personalizadas a sus métricas, utilice el procesador resource/static_override:

processors:
  resource/static_override:
    attributes:
      - key: env
        value: "production"
        action: upsert
service:
  pipelines:
    metrics/elasticsearch:
      receivers: [elasticsearch]
      processors: [memory_limiter, resourcedetection, resource/cluster_name_override, resource/static_override, attributes/cardinality_reduction, cumulativetodelta, transform/metadata_nullify, batch]
      exporters: [otlphttp]

Almacene el archivo de configuración.
Configura las variables de entorno:
Crea un directorio de anulación de systemd:
bash
```
$sudo mkdir -p /etc/systemd/system/nrdot-collector.service.d
```
Cree el archivo de configuración del entorno:
bash
```
$cat <<EOF | sudo tee /etc/systemd/system/nrdot-collector.service.d/environment.conf
$[Service]
$Environment="NEWRELIC_OTLP_ENDPOINT=https://otlp.nr-data.net:4318"
$Environment="NEWRELIC_LICENSE_KEY=YOUR_LICENSE_KEY_HERE"
$Environment="NEW_RELIC_MEMORY_LIMIT_MIB=100"
$EOF
```
Actualice la configuración con sus valores:
- Reemplace https://otlp.nr-data.net:4318 con el punto final de su región
- Reemplace YOUR_LICENSE_KEY_HERE con su clave de licencia de New Relic real
- Reemplace 100 con el límite de memoria deseado en MiB para el colector (predeterminado: 100 MiB). Ajuste según las necesidades de su entorno

Reinicie el colector NRDOT para aplicar los cambios:

bash

$sudo systemctl daemon-reload
$sudo systemctl restart nrdot-collector.service

Verificar la recopilación de datos

Verifique que el recolector NRDOT se esté ejecutando y recopilando datos sin errores:

Verifique el estado del servicio del colector:
bash
```
$sudo systemctl status nrdot-collector.service
```
Monitoree los logs del recopilador en busca de errores:
bash
```
$sudo journalctl -u nrdot-collector.service -f
```
Busque conexiones exitosas a Elasticsearch y New Relic. Si ve errores, consulte la guía de solución de problemas.

Antes de que empieces

Antes de configurar el OTel Collector Contrib, asegúrese de tener:

Privilegios de acceso requeridos:

Tu New Relic
Privilegios de root o sudo en la máquina host
Privilegios de administrador de clúster de Elasticsearch con privilegio de clúster monitor o manage (consulte la documentación de privilegios de seguridad de Elasticsearch para obtener detalles)
Requisitos del sistema:
Versión 7.16 o superior de Elasticsearch - Esta integración requiere un clúster moderno de Elasticsearch
Conectividad de red - HTTPS saliente (puerto 443) al endpoint de ingesta OTLPde New Relic
Valores de configuración listos:
Punto de conexión de Elasticsearch - La URL de su clúster de Elasticsearch (p. ej., http://localhost:9200)
Nombre del clúster - Un identificador único para su clúster en New Relic
Importante
Debe tener OpenTelemetry Collector Contrib instalado en su host antes de continuar. Recomendamos instalar mediante paquetes oficiales (.deb o .rpm) para garantizar que la unidad de servicio systemd se cree correctamente.

Configurar el monitoreo de Elasticsearch

Una vez que OTel Collector Contrib esté instalado, reemplace el archivo de configuración predeterminado del collector con la configuración de monitoreo de Elasticsearch. Esto habilitará la recopilación de métricas de Elasticsearch. Las métricas y los logs del host son opcionales y se pueden agregar por separado.

El archivo de configuración se encuentra en: /etc/otelcol-contrib/config.yaml

Sugerencia

Respalde su configuración predeterminada: Antes de modificar el archivo de configuración, cree una copia de seguridad para conservar los ajustes predeterminados:

bash

$sudo cp /etc/otelcol-contrib/config.yaml /etc/otelcol-contrib/config.yaml.backup

Para configurar el colector:

Abra el archivo de configuración con un editor de texto con privilegios de root o sudo:
bash
```
$sudo nano /etc/otelcol-contrib/config.yaml
```

Elimine todo el contenido existente y reemplácelo con la siguiente configuración para el monitoreo de Elasticsearch:

Importante

receivers:
  elasticsearch:
    endpoint: "http://localhost:9200"
    collection_interval: 15s
    metrics:
      elasticsearch.os.cpu.usage:
        enabled: true
      elasticsearch.cluster.data_nodes:
        enabled: true
      elasticsearch.cluster.health:
        enabled: true
      elasticsearch.cluster.in_flight_fetch:
        enabled: true
      elasticsearch.cluster.nodes:
        enabled: true
      elasticsearch.cluster.pending_tasks:
        enabled: true
      elasticsearch.cluster.shards:
        enabled: true
      elasticsearch.cluster.state_update.time:
        enabled: true
      elasticsearch.index.documents:
        enabled: true
      elasticsearch.index.operations.merge.current:
        enabled: true
      elasticsearch.index.operations.time:
        enabled: true
      elasticsearch.node.cache.count:
        enabled: true
      elasticsearch.node.cache.evictions:
        enabled: true
      elasticsearch.node.cache.memory.usage:
        enabled: true
      elasticsearch.node.shards.size:
        enabled: true
      elasticsearch.node.cluster.io:
        enabled: true
      elasticsearch.node.documents:
        enabled: true
      elasticsearch.node.disk.io.read:
        enabled: true
      elasticsearch.node.disk.io.write:
        enabled: true
      elasticsearch.node.fs.disk.available:
        enabled: true
      elasticsearch.node.fs.disk.total:
        enabled: true
      elasticsearch.node.http.connections:
        enabled: true
      elasticsearch.node.ingest.documents.current:
        enabled: true
      elasticsearch.node.ingest.operations.failed:
        enabled: true
      elasticsearch.node.open_files:
        enabled: true
      elasticsearch.node.operations.completed:
        enabled: true
      elasticsearch.node.operations.current:
        enabled: true
      elasticsearch.node.operations.get.completed:
        enabled: true
      elasticsearch.node.operations.time:
        enabled: true
      elasticsearch.node.shards.reserved.size:
        enabled: true
      elasticsearch.index.shards.size:
        enabled: true
      elasticsearch.os.cpu.load_avg.1m:
        enabled: true
      elasticsearch.os.cpu.load_avg.5m:
        enabled: true
      elasticsearch.os.cpu.load_avg.15m:
        enabled: true
      elasticsearch.os.memory:
        enabled: true
      jvm.gc.collections.count:
        enabled: true
      jvm.gc.collections.elapsed:
        enabled: true
      jvm.memory.heap.max:
        enabled: true
      jvm.memory.heap.used:
        enabled: true
      jvm.memory.heap.utilization:
        enabled: true
      jvm.threads.count:
        enabled: true
      elasticsearch.index.segments.count:
        enabled: true
      elasticsearch.index.operations.completed:
        enabled: true
      elasticsearch.node.script.cache_evictions:
        enabled: false
      elasticsearch.node.cluster.connections:
        enabled: false
      elasticsearch.node.pipeline.ingest.documents.preprocessed:
        enabled: false
      elasticsearch.node.thread_pool.tasks.queued:
        enabled: false
      elasticsearch.cluster.published_states.full:
        enabled: false
      jvm.memory.pool.max:
        enabled: false
      elasticsearch.node.script.compilation_limit_triggered:
        enabled: false
      elasticsearch.node.shards.data_set.size:
        enabled: false
      elasticsearch.node.pipeline.ingest.documents.current:
        enabled: false
      elasticsearch.cluster.state_update.count:
        enabled: false
      elasticsearch.node.fs.disk.free:
        enabled: false
      jvm.memory.nonheap.used:
        enabled: false
      jvm.memory.pool.used:
        enabled: false
      elasticsearch.node.translog.size:
        enabled: false
      elasticsearch.node.thread_pool.threads:
        enabled: false
      elasticsearch.cluster.state_queue:
        enabled: false
      elasticsearch.node.translog.operations:
        enabled: false
      elasticsearch.memory.indexing_pressure:
        enabled: false
      elasticsearch.node.ingest.documents:
        enabled: false
      jvm.classes.loaded:
        enabled: false
      jvm.memory.heap.committed:
        enabled: false
      elasticsearch.breaker.memory.limit:
        enabled: false
      elasticsearch.indexing_pressure.memory.total.replica_rejections:
        enabled: false
      elasticsearch.breaker.memory.estimated:
        enabled: false
      elasticsearch.cluster.published_states.differences:
        enabled: false
      jvm.memory.nonheap.committed:
        enabled: false
      elasticsearch.node.translog.uncommitted.size:
        enabled: false
      elasticsearch.node.script.compilations:
        enabled: false
      elasticsearch.node.pipeline.ingest.operations.failed:
        enabled: false
      elasticsearch.indexing_pressure.memory.limit:
        enabled: false
      elasticsearch.breaker.tripped:
        enabled: false
      elasticsearch.indexing_pressure.memory.total.primary_rejections:
        enabled: false
      elasticsearch.node.thread_pool.tasks.finished:
        enabled: false
processors:
  memory_limiter:
    check_interval: 60s
    limit_mib: ${env:NEW_RELIC_MEMORY_LIMIT_MIB:-100}
  cumulativetodelta: {}
  resource/cluster_name_override:
    attributes:
      - key: elasticsearch.cluster.name
        value: "<elasticsearch-cluster-name>"
        action: upsert
  resourcedetection:
    detectors: [ system ]
    system:
      resource_attributes:
        host.name:
          enabled: true
        host.id:
          enabled: true
        os.type:
          enabled: true
  batch:
    timeout: 10s
    send_batch_size: 1024
  attributes/cardinality_reduction:
    actions:
      - key: process.pid
        action: delete
      - key: process.parent_pid
        action: delete
  transform/metadata_nullify:
    metric_statements:
      - context: metric
        statements:
          - set(description, "")
          - set(unit, "")
exporters:
  otlphttp:
    endpoint: ${env:NEWRELIC_OTLP_ENDPOINT}
    headers:
      api-key: ${env:NEWRELIC_LICENSE_KEY}
service:
  pipelines:
    metrics/elasticsearch:
      receivers: [elasticsearch]
      processors: [memory_limiter, resourcedetection, resource/cluster_name_override, attributes/cardinality_reduction, cumulativetodelta, transform/metadata_nullify, batch]
      exporters: [otlphttp]

(Opcional) Para Elasticsearch seguro con autenticación y SSL, modifique la configuración del receptor:

receivers:
  elasticsearch:
    endpoint: "https://localhost:9200"
    username: "your_elasticsearch_username"
    password: "your_elasticsearch_password"
    tls:
      ca_file: "/etc/elasticsearch/certs/http_ca.crt"
      insecure_skip_verify: false
    collection_interval: 15s

(Opcional) Para recopilar métricas del host, agregue el receptor hostmetrics:

receivers:
  hostmetrics:
    collection_interval: 60s
    scrapers:
      cpu:
        metrics:
          system.cpu.utilization: {enabled: true}
          system.cpu.time: {enabled: true}
      load:
        metrics:
          system.cpu.load_average.1m: {enabled: true}
          system.cpu.load_average.5m: {enabled: true}
          system.cpu.load_average.15m: {enabled: true}
      memory:
        metrics:
          system.memory.usage: {enabled: true}
          system.memory.utilization: {enabled: true}
      disk:
        metrics:
          system.disk.io: {enabled: true}
          system.disk.operations: {enabled: true}
      filesystem:
        metrics:
          system.filesystem.usage: {enabled: true}
          system.filesystem.utilization: {enabled: true}
      network:
        metrics:
          system.network.io: {enabled: true}
          system.network.packets: {enabled: true}
      process:
        metrics:
          process.cpu.utilization:
            enabled: true

Y agregue a los pipelines de servicio:

service:
  pipelines:
    metrics/host:
      receivers: [hostmetrics]
      processors: [memory_limiter, resourcedetection, batch]
      exporters: [otlphttp]

(Opcional) Para recopilar logs de Elasticsearch, agregue el receptor filelog. Asegúrese de que el usuario que ejecuta el servicio del recopilador (otelcol-contrib) tenga acceso de lectura a sus archivos de log de Elasticsearch:

Si ejecuta Elasticsearch en Linux (Host):

receivers:
  filelog:
    include:
      - /var/log/elasticsearch/elasticsearch.log
      - /var/log/elasticsearch/*.log

Si ejecuta Elasticsearch en Docker:

receivers:
  filelog:
    include:
      - /var/lib/docker/containers/*/*.log
    operators:
      - type: move
        from: attributes.log
        to: body

Y agregue a los pipelines de servicio:

service:
  pipelines:
    logs:
      receivers: [filelog]
      processors: [resource/cluster_name_override]
      exporters: [otlphttp]

(Opcional) Para agregar etiquetas de metadatos personalizadas a sus métricas, utilice el procesador resource/static_override:

processors:
  resource/static_override:
    attributes:
      - key: env
        value: "production"
        action: upsert
service:
  pipelines:
    metrics/elasticsearch:
      receivers: [elasticsearch]
      processors: [memory_limiter, resourcedetection, resource/cluster_name_override, resource/static_override, attributes/cardinality_reduction, cumulativetodelta, transform/metadata_nullify, batch]
      exporters: [otlphttp]

Almacene el archivo de configuración.
Configura las variables de entorno:
Crea un directorio de anulación de systemd:
bash
```
$sudo mkdir -p /etc/systemd/system/otelcol-contrib.service.d
```
Cree el archivo de configuración del entorno:
bash
```
$cat <<EOF | sudo tee /etc/systemd/system/otelcol-contrib.service.d/environment.conf
$[Service]
$Environment="NEWRELIC_OTLP_ENDPOINT=https://otlp.nr-data.net:4318"
$Environment="NEWRELIC_LICENSE_KEY=YOUR_LICENSE_KEY_HERE"
$Environment="NEW_RELIC_MEMORY_LIMIT_MIB=100"
$EOF
```
Actualice la configuración con sus valores:
- Reemplace https://otlp.nr-data.net:4318 con el punto final de su región
- Reemplace YOUR_LICENSE_KEY_HERE con su clave de licencia de New Relic real
- Reemplace 100 con el límite de memoria deseado en MiB para el colector (predeterminado: 100 MiB). Ajuste según las necesidades de su entorno

Reinicie el OTel Collector Contrib para aplicar los cambios:

bash

$sudo systemctl daemon-reload
$sudo systemctl restart otelcol-contrib.service

Verificar la recopilación de datos

Verifique que el OTel Collector Contrib se esté ejecutando y recopilando datos sin errores:

Verifique el estado del servicio del colector:
bash
```
$sudo systemctl status otelcol-contrib.service
```
Monitoree los logs del recopilador en busca de errores:
bash
```
$sudo journalctl -u otelcol-contrib.service -f
```
Busque conexiones exitosas a Elasticsearch y New Relic. Si ve errores, consulte la guía de solución de problemas.

Use este enfoque si ya tiene un exportador de Elasticsearch de Prometheus ejecutándose en su entorno, o si está migrando desde un stack de monitoreo basado en Prometheus.

Sugerencia

Recomendado: si aún no tiene un exportador de Prometheus en ejecución, use las pestañas NRDOT Collector u OTel Collector Contrib en su lugar. Se conectan directamente a la API de Elasticsearch sin necesidad de un componente exportador adicional.

Instale y configure el exportador de Elasticsearch

El exportador de Elasticsearch para Prometheus expone las métricas de Elasticsearch en formato Prometheus en el puerto 9114.

Descarga la última versión:

bash

$wget https://github.com/prometheus-community/elasticsearch_exporter/releases/download/v1.8.0/elasticsearch_exporter-1.8.0.linux-amd64.tar.gz
$tar xzf elasticsearch_exporter-*.tar.gz
$cd elasticsearch_exporter-*

Inicie el exportador (apúntelo a su instancia de Elasticsearch):
bash
```
$./elasticsearch_exporter --es.uri=http://localhost:9200
```
Sugerencia
Para Elasticsearch seguro, use: --es.uri=https://username:password@localhost:9200 --es.ssl-skip-verify
Verifique que el exportador se esté ejecutando:
bash
```
$curl http://localhost:9114/metrics | grep elasticsearch_cluster_health
```
Debería ver métricas con formato de Prometheus como elasticsearch_cluster_health_status, elasticsearch_cluster_health_number_of_nodes, etc.

Configurar el recolector

Esta configuración funciona tanto con NRDOT como con OTel Collector Contrib. Colóquelo en la ubicación de configuración adecuada para su recolector:

NRDOT: /etc/nrdot-collector/config.yaml

OTel Collector Contrib: /etc/otelcol-contrib/config.yaml

Si aún no tiene un recolector instalado, siga primero los pasos de instalación en las pestañas NRDOT Collector u OTel Collector Contrib de arriba, y luego regrese aquí para aplicar esta configuración.

Esta configuración recopila métricas del exportador de Elasticsearch y las traduce a nombres de métricas compatibles con OpenTelemetry que alimentan el dashboard de Elasticsearch de New Relic.

receivers:
  prometheus/elasticsearch:
    config:
      scrape_configs:
        - job_name: 'elasticsearch'
          scrape_interval: 15s
          static_configs:
            - targets: ['localhost:9114']  # <-- elasticsearch_exporter address

processors:
  memory_limiter:
    check_interval: 60s
    limit_mib: ${env:NEW_RELIC_MEMORY_LIMIT_MIB:-100}

  metricstransform/prom_to_otel:
    transforms:
      - include: ^elasticsearch_cluster_health_number_of_data_nodes$
        match_type: regexp
        action: update
        new_name: elasticsearch.cluster.data_nodes
      - include: ^elasticsearch_cluster_health_status$
        match_type: regexp
        action: update
        new_name: elasticsearch.cluster.health
        operations:
          - action: update_label
            label: color
            new_label: status
      - include: ^elasticsearch_cluster_health_number_of_in_flight_fetch$
        match_type: regexp
        action: update
        new_name: elasticsearch.cluster.in_flight_fetch
      - include: ^elasticsearch_cluster_health_number_of_nodes$
        match_type: regexp
        action: update
        new_name: elasticsearch.cluster.nodes
      - include: ^elasticsearch_cluster_health_number_of_pending_tasks$
        match_type: regexp
        action: update
        new_name: elasticsearch.cluster.pending_tasks
      - include: ^elasticsearch_cluster_health_active_shards$
        match_type: regexp
        action: update
        new_name: elasticsearch.cluster.shards
        operations:
          - action: add_label
            new_label: state
            new_value: active
      - include: ^elasticsearch_cluster_health_active_primary_shards$
        match_type: regexp
        action: update
        new_name: elasticsearch.cluster.shards
        operations:
          - action: add_label
            new_label: state
            new_value: active_primary
      - include: ^elasticsearch_cluster_health_initializing_shards$
        match_type: regexp
        action: update
        new_name: elasticsearch.cluster.shards
        operations:
          - action: add_label
            new_label: state
            new_value: initializing
      - include: ^elasticsearch_cluster_health_relocating_shards$
        match_type: regexp
        action: update
        new_name: elasticsearch.cluster.shards
        operations:
          - action: add_label
            new_label: state
            new_value: relocating
      - include: ^elasticsearch_cluster_health_unassigned_shards$
        match_type: regexp
        action: update
        new_name: elasticsearch.cluster.shards
        operations:
          - action: add_label
            new_label: state
            new_value: unassigned
      - include: ^elasticsearch_indices_docs$
        match_type: regexp
        action: insert
        new_name: elasticsearch.node.documents
        operations:
          - action: add_label
            new_label: state
            new_value: active
      - include: ^elasticsearch_indices_docs$
        match_type: regexp
        action: insert
        new_name: elasticsearch.index.documents
        operations:
          - action: add_label
            new_label: state
            new_value: active
          - action: add_label
            new_label: aggregation
            new_value: primary_shards
          - action: add_label
            new_label: elasticsearch.index.name
            new_value: _all
      - include: ^elasticsearch_indices_docs_deleted$
        match_type: regexp
        action: insert
        new_name: elasticsearch.node.documents
        operations:
          - action: add_label
            new_label: state
            new_value: deleted
      - include: ^elasticsearch_indices_fielddata_evictions$
        match_type: regexp
        action: update
        new_name: elasticsearch.node.cache.evictions
        operations:
          - action: add_label
            new_label: cache_name
            new_value: fielddata
      - include: ^elasticsearch_indices_query_cache_evictions$
        match_type: regexp
        action: update
        new_name: elasticsearch.node.cache.evictions
        operations:
          - action: add_label
            new_label: cache_name
            new_value: query
      - include: ^elasticsearch_indices_fielddata_memory_size_bytes$
        match_type: regexp
        action: update
        new_name: elasticsearch.node.cache.memory.usage
        operations:
          - action: add_label
            new_label: cache_name
            new_value: fielddata
      - include: ^elasticsearch_indices_query_cache_memory_size_bytes$
        match_type: regexp
        action: update
        new_name: elasticsearch.node.cache.memory.usage
        operations:
          - action: add_label
            new_label: cache_name
            new_value: query
      - include: ^elasticsearch_transport_rx_size_bytes_total$
        match_type: regexp
        action: update
        new_name: elasticsearch.node.cluster.io
        operations:
          - action: add_label
            new_label: direction
            new_value: received
      - include: ^elasticsearch_transport_tx_size_bytes_total$
        match_type: regexp
        action: update
        new_name: elasticsearch.node.cluster.io
        operations:
          - action: add_label
            new_label: direction
            new_value: sent
      - include: ^elasticsearch_filesystem_data_available_bytes$
        match_type: regexp
        action: update
        new_name: elasticsearch.node.fs.disk.available
      - include: ^elasticsearch_filesystem_data_size_bytes$
        match_type: regexp
        action: update
        new_name: elasticsearch.node.fs.disk.total
      - include: ^elasticsearch_process_open_files_count$
        match_type: regexp
        action: update
        new_name: elasticsearch.node.open_files
      - include: ^elasticsearch_indices_indexing_index_total$
        match_type: regexp
        action: update
        new_name: elasticsearch.node.operations.completed
        operations:
          - action: add_label
            new_label: operation
            new_value: index
      - include: ^elasticsearch_indices_search_query_total$
        match_type: regexp
        action: update
        new_name: elasticsearch.node.operations.completed
        operations:
          - action: add_label
            new_label: operation
            new_value: query
      - include: ^elasticsearch_indices_get_total$
        match_type: regexp
        action: update
        new_name: elasticsearch.node.operations.completed
        operations:
          - action: add_label
            new_label: operation
            new_value: get
      - include: ^elasticsearch_indices_merges_total$
        match_type: regexp
        action: update
        new_name: elasticsearch.node.operations.completed
        operations:
          - action: add_label
            new_label: operation
            new_value: merge
      - include: ^elasticsearch_indices_indexing_index_time_seconds_total$
        match_type: regexp
        action: update
        new_name: elasticsearch.node.operations.time
        operations:
          - action: add_label
            new_label: operation
            new_value: index
          - action: experimental_scale_value
            experimental_scale: 1000
      - include: ^elasticsearch_indices_search_query_time_seconds$
        match_type: regexp
        action: update
        new_name: elasticsearch.node.operations.time
        operations:
          - action: add_label
            new_label: operation
            new_value: query
          - action: experimental_scale_value
            experimental_scale: 1000
      - include: ^elasticsearch_thread_pool_active_count$
        match_type: regexp
        action: insert
        new_name: elasticsearch.node.operations.current
        operations:
          - action: update_label
            label: type
            new_label: operation
      - include: ^elasticsearch_os_cpu_percent$
        match_type: regexp
        action: update
        new_name: elasticsearch.os.cpu.usage
      - include: ^elasticsearch_os_load1$
        match_type: regexp
        action: update
        new_name: elasticsearch.os.cpu.load_avg.1m
      - include: ^elasticsearch_os_load5$
        match_type: regexp
        action: update
        new_name: elasticsearch.os.cpu.load_avg.5m
      - include: ^elasticsearch_os_load15$
        match_type: regexp
        action: update
        new_name: elasticsearch.os.cpu.load_avg.15m
      - include: ^elasticsearch_jvm_gc_collection_seconds_count$
        match_type: regexp
        action: update
        new_name: jvm.gc.collections.count
      - include: ^elasticsearch_jvm_gc_collection_seconds_sum$
        match_type: regexp
        action: update
        new_name: jvm.gc.collections.elapsed
        operations:
          - action: experimental_scale_value
            experimental_scale: 1000
      - include: ^elasticsearch_jvm_memory_max_bytes$
        match_type: regexp
        action: update
        new_name: jvm.memory.heap.max
      - include: ^elasticsearch_jvm_memory_used_bytes$
        match_type: regexp
        action: update
        new_name: jvm.memory.heap.used
      - include: ^elasticsearch_thread_pool_threads_count$
        match_type: regexp
        action: update
        new_name: jvm.threads.count
        operations:
          - action: aggregate_labels
            label_set: [name, cluster]
            aggregation_type: sum
      - include: ^elasticsearch_indices_segments_count$
        match_type: regexp
        action: update
        new_name: elasticsearch.index.segments.count

  filter/allowed_metrics:
    metrics:
      include:
        match_type: strict
        metric_names:
          - elasticsearch.cluster.data_nodes
          - elasticsearch.cluster.health
          - elasticsearch.cluster.in_flight_fetch
          - elasticsearch.cluster.nodes
          - elasticsearch.cluster.pending_tasks
          - elasticsearch.cluster.shards
          - elasticsearch.index.documents
          - elasticsearch.index.segments.count
          - elasticsearch.node.cache.evictions
          - elasticsearch.node.cache.memory.usage
          - elasticsearch.node.cluster.io
          - elasticsearch.node.documents
          - elasticsearch.node.fs.disk.available
          - elasticsearch.node.fs.disk.total
          - elasticsearch.node.open_files
          - elasticsearch.node.operations.completed
          - elasticsearch.node.operations.current
          - elasticsearch.node.operations.time
          - elasticsearch.os.cpu.usage
          - elasticsearch.os.cpu.load_avg.1m
          - elasticsearch.os.cpu.load_avg.5m
          - elasticsearch.os.cpu.load_avg.15m
          - jvm.gc.collections.count
          - jvm.gc.collections.elapsed
          - jvm.memory.heap.max
          - jvm.memory.heap.used
          - jvm.threads.count

  transform/rename_node_name:
    metric_statements:
      - context: datapoint
        statements:
          - set(attributes["elasticsearch.node.name"], attributes["name"]) where attributes["name"] != nil
          - delete_key(attributes, "name") where attributes["elasticsearch.node.name"] != nil

  groupbyattrs/node:
    keys:
      - elasticsearch.node.name
      - elasticsearch.cluster.name

  cumulativetodelta: {}

  resource/cluster_name_override:
    attributes:
      - key: elasticsearch.cluster.name
        value: "<elasticsearch-cluster-name>"
        action: upsert

  transform/scope_override:
    metric_statements:
      - context: scope
        statements:
          - set(name, "github.com/open-telemetry/opentelemetry-collector-contrib/receiver/elasticsearchreceiver")
          - set(version, "")

  resourcedetection:
    detectors: [system]
    system:
      resource_attributes:
        host.name:
          enabled: true
        host.id:
          enabled: true
        os.type:
          enabled: true

  batch:
    timeout: 30s
    send_batch_size: 2048
    send_batch_max_size: 4096

  attributes/cardinality_reduction:
    actions:
      - key: process.pid
        action: delete
      - key: process.parent_pid
        action: delete

  transform/metadata_nullify:
    metric_statements:
      - context: metric
        statements:
          - set(description, "")
          - set(unit, "")

exporters:
  otlphttp:
    endpoint: ${env:NEWRELIC_OTLP_ENDPOINT}
    headers:
      api-key: ${env:NEWRELIC_LICENSE_KEY}
    compression: gzip
    timeout: 30s
    retry_on_failure:
      enabled: true
      initial_interval: 5s
      max_interval: 30s
      max_elapsed_time: 300s

service:
  pipelines:
    metrics/elasticsearch:
      receivers: [prometheus/elasticsearch]
      processors:
        - memory_limiter
        - metricstransform/prom_to_otel
        - filter/allowed_metrics
        - transform/rename_node_name
        - groupbyattrs/node
        - transform/scope_override
        - resourcedetection
        - resource/cluster_name_override
        - attributes/cardinality_reduction
        - cumulativetodelta
        - transform/metadata_nullify
        - batch
      exporters: [otlphttp]

Reemplace los siguientes valores en la configuración:

<elasticsearch-cluster-name>: el nombre de su clúster de Elasticsearch para su identificación en New Relic.
localhost:9114: la dirección de su elasticsearch_exporter si se ejecuta en un host o puerto diferente.

Configurar variables de entorno

Cree una anulación de systemd para inyectar las variables de entorno requeridas. Reemplace <collector-service> con el nombre de su servicio de recolector (nrdot-collector o otelcol-contrib):

bash

$sudo mkdir -p /etc/systemd/system/<collector-service>.service.d

Cree el archivo /etc/systemd/system/<collector-service>.service.d/environment.conf:

[Service]
Environment="NEWRELIC_OTLP_ENDPOINT=https://otlp.nr-data.net:4318"
Environment="NEWRELIC_LICENSE_KEY=YOUR_NEWRELIC_LICENSE_KEY"
Environment="NEW_RELIC_MEMORY_LIMIT_MIB=100"

Reemplace YOUR_NEWRELIC_LICENSE_KEY con su .

Sugerencia

Para las cuentas de la UE, use NEWRELIC_OTLP_ENDPOINT=https://otlp.eu01.nr-data.net:4318

Iniciar el colector

Reemplace <collector-service> con el nombre de su servicio de recolector (nrdot-collector o otelcol-contrib):

bash

$sudo systemctl daemon-reload
$sudo systemctl enable <collector-service>
$sudo systemctl restart <collector-service>

Verifique el estado del recolector:

bash

$sudo systemctl status <collector-service>
$sudo journalctl -u <collector-service> -f

Verifique los datos en New Relic

Después de unos minutos, verifica que los datos estén fluyendo hacia New Relic:

FROM Metric SELECT count(*)
WHERE metricName LIKE 'elasticsearch.%'
AND elasticsearch.cluster.name = '<elasticsearch-cluster-name>'
SINCE 10 minutes ago

Sugerencia

Correlacione APM con Elasticsearch: Para conectar su aplicación APM y el clúster de Elasticsearch, incluya el atributo de recurso es.cluster.name="your-cluster-name" en sus métricas de APM. Esto permite la visibilidad entre servicios y una solución de problemas más rápida dentro de New Relic.

Ver sus datos de Elasticsearch

Una vez que el recopilador se esté ejecutando y enviando datos, puede explorar sus métricas de Elasticsearch, crear consultas personalizadas y configurar dashboards de monitoreo en New Relic.

Para obtener información detallada sobre cómo acceder a sus datos, escribir consultas NRQL y configurar alertas, consulte Buscar y consultar datos de Elasticsearch.

Resolución de problemas

Si encuentra problemas durante la instalación o no ve datos en New Relic, consulte nuestra completa guía de solución de problemas para obtener soluciones paso a paso a los problemas comunes.

Te ofrecemos esta traducción automática para facilitar la lectura.

Monitorear Elasticsearch autohospedado con OpenTelemetry

Opciones de instalación

Antes de que empieces

Configurar el monitoreo de Elasticsearch

Sugerencia

Importante

Verificar la recopilación de datos

Antes de que empieces

Importante

Configurar el monitoreo de Elasticsearch

Sugerencia

Importante

Verificar la recopilación de datos

Sugerencia

Instale y configure el exportador de Elasticsearch

Sugerencia

Configurar el recolector

Configuración completa del receptor de Prometheus

Configurar variables de entorno

Sugerencia

Iniciar el colector

Verifique los datos en New Relic

Sugerencia

Ver sus datos de Elasticsearch

Resolución de problemas

Te ofrecemos esta traducción automática para facilitar la lectura.

Monitorear Elasticsearch autohospedado con OpenTelemetry

Opciones de instalación .css-21sua1{background:none;border:none;width:0;padding:0;}

Antes de que empieces

Configurar el monitoreo de Elasticsearch

Sugerencia

Importante

Verificar la recopilación de datos

Antes de que empieces

Importante

Configurar el monitoreo de Elasticsearch

Sugerencia

Importante

Verificar la recopilación de datos

Sugerencia

Instale y configure el exportador de Elasticsearch

Sugerencia

Configurar el recolector

Configurar variables de entorno

Sugerencia

Iniciar el colector

Verifique los datos en New Relic

Sugerencia

Ver sus datos de Elasticsearch

Resolución de problemas

Opciones de instalación