---
title: Google Cloud Managed Kafka monitoring integration
source: https://docs.newrelic.com/docs/infrastructure/google-cloud-platform-integrations/gcp-integrations-list/google-cloud-managed-kafka-monitoring-integration
---

[New Relic integrations](https://docs.newrelic.com/docs/infrastructure/introduction-infra-monitoring) with the [Google Cloud Platform (GCP)](https://cloud.google.com/) include one that reports [Google Cloud Managed Service for Apache Kafka](https://cloud.google.com/managed-service-for-apache-kafka) data to New Relic. This document explains how to activate the GCP Managed Kafka integration and describes the data it reports.

## Features

Managed Service for Apache Kafka is Google Cloud's fully managed, Apache Kafka–compatible streaming platform, including Kafka Connect. New Relic Managed Kafka integration collects throughput, lag, retention, and connector-health metrics across clusters, topics, partitions, consumer groups, and Kafka Connect clusters and connectors.

## Activate integration [#activate]

To enable the integration, follow standard procedures to [connect your GCP service to New Relic](https://docs.newrelic.com/docs/connect-google-cloud-platform-services-infrastructure):

-   [Connect with Workload Identity Federation](https://docs.newrelic.com/docs/connect-google-cloud-platform-services-infrastructure)

## Polling frequency [#polling]

New Relic integrations query your GCP services according to a polling interval that varies by integration. The polling frequency for Google Cloud Managed Kafka is 5 minutes. The resolution is 1 data point every minute.

> #### ⚠️ IMPORTANT
>
> Managed Service for Apache Kafka supports 1-minute polling intervals for WIF integrations in limited preview. To request access, contact [New Relic support](https://support.newrelic.com/).

## Workload Identity Federation [#wif]

### Find and use data [#find-data-wif]

After you enable the integration, your Managed Kafka resources appear as entities in the New Relic entity explorer. To see dashboards and manage services, go to [one.newrelic.com > All capabilities](https://one.newrelic.com/all-capabilities) > Infrastructure > GCP.

All Managed Kafka metrics available in GCP Cloud Monitoring are collected as dimensional metrics in the `Metric` event type. Additional metrics beyond this table are collected automatically. See [Google's Managed Kafka metrics documentation](https://cloud.google.com/monitoring/api/metrics_gcp_i_o#gcp-managedkafka) for the complete list.

#### Entities

**Managed Kafka entities**

| Entity          | Entity type                                  | Resource type                                             |
| --------------- | -------------------------------------------- | --------------------------------------------------------- |
| Cluster         | `KAFKACLUSTER`                               | `managedkafka.googleapis.com/Cluster`                     |
| Topic           | `KAFKATOPIC`                                 | `managedkafka.googleapis.com/Topic`                       |
| Topic Partition | `GCPMANAGEDKAFKATOPICPARTITION`              | `managedkafka.googleapis.com/TopicPartition`              |
| Consumer Group  | `GCPMANAGEDKAFKATOPICPARTITIONCONSUMERGROUP` | `managedkafka.googleapis.com/TopicPartitionConsumerGroup` |
| Connect Cluster | `GCPMANAGEDKAFKACONNECTCLUSTER`              | `managedkafka.googleapis.com/ConnectCluster`              |
| Connector       | `GCPMANAGEDKAFKACONNECTOR`                   | `managedkafka.googleapis.com/Connector`                   |

### Metric data [#metrics-wif]

#### Key metrics — Cluster

| Metric name                                   | Unit    | Description                                                          |
| --------------------------------------------- | ------- | -------------------------------------------------------------------- |
| `gcp.managedkafka.cluster.cpu.utilization`    | Percent | CPU utilization across broker nodes in the Managed Kafka cluster.    |
| `gcp.managedkafka.cluster.memory.utilization` | Percent | Memory utilization across broker nodes in the Managed Kafka cluster. |
| `gcp.managedkafka.cluster.storage.used_bytes` | Bytes   | Storage in use by the Managed Kafka cluster.                         |
| `gcp.managedkafka.cluster.broker_count`       | Count   | Number of brokers currently running in the Managed Kafka cluster.    |
| `gcp.managedkafka.cluster.topic_count`        | Count   | Number of topics on the Managed Kafka cluster.                       |

#### Key metrics — Topic

| Metric name                               | Unit  | Description                                                      |
| ----------------------------------------- | ----- | ---------------------------------------------------------------- |
| `gcp.managedkafka.topic.byte_in_count`    | Bytes | Bytes produced to the Managed Kafka topic.                       |
| `gcp.managedkafka.topic.byte_out_count`   | Bytes | Bytes consumed from the Managed Kafka topic.                     |
| `gcp.managedkafka.topic.message_in_count` | Count | Messages produced to the Managed Kafka topic.                    |
| `gcp.managedkafka.topic.partition_count`  | Count | Number of partitions in the Managed Kafka topic.                 |
| `gcp.managedkafka.topic.byte_size`        | Bytes | Total on-disk size of the Managed Kafka topic across partitions. |

#### Key metrics — Topic Partition

| Metric name                       | Unit  | Description                                                        |
| --------------------------------- | ----- | ------------------------------------------------------------------ |
| `gcp.managedkafka.byte_size`      | Bytes | On-disk size of the topic partition.                               |
| `gcp.managedkafka.last_offset`    | Count | Most recent offset produced to the topic partition.                |
| `gcp.managedkafka.first_offset`   | Count | Earliest available offset in the topic partition.                  |
| `gcp.managedkafka.log_end_offset` | Count | Log-end offset for the topic partition.                            |
| `gcp.managedkafka.consumer_lag`   | Count | Number of messages the partition is ahead of its slowest consumer. |

#### Key metrics — Consumer Group

| Metric name                                       | Unit  | Description                                                                              |
| ------------------------------------------------- | ----- | ---------------------------------------------------------------------------------------- |
| `gcp.managedkafka.offset_lag`                     | Count | Number of messages the consumer group is behind the latest offset, faceted by partition. |
| `gcp.managedkafka.committed_offset`               | Count | Last offset committed by the consumer group.                                             |
| `gcp.managedkafka.consumer_group.member_count`    | Count | Number of active members in the consumer group.                                          |
| `gcp.managedkafka.consumer_group.rebalance_count` | Count | Number of consumer-group rebalances observed.                                            |

#### Key metrics — Connect Cluster

| Metric name                                          | Unit         | Description                                                                   |
| ---------------------------------------------------- | ------------ | ----------------------------------------------------------------------------- |
| `gcp.managedkafka.cpu.worker_core_usage_time`        | Seconds      | CPU-seconds consumed by workers in the Kafka Connect cluster.                 |
| `gcp.managedkafka.memory.worker_usage`               | Bytes        | Memory currently in use by workers in the Kafka Connect cluster.              |
| `gcp.managedkafka.connector_request_rate`            | Count        | Number of connector API requests processed by the Kafka Connect cluster.      |
| `gcp.managedkafka.connect_cluster.worker_count`      | Count        | Number of workers currently running in the Kafka Connect cluster.             |
| `gcp.managedkafka.connect_cluster.request_latencies` | Milliseconds | Latency distribution for connector API requests on the Kafka Connect cluster. |

#### Key metrics — Connector

| Metric name                                               | Unit  | Description                                                       |
| --------------------------------------------------------- | ----- | ----------------------------------------------------------------- |
| `gcp.managedkafka.connector_tasks`                        | Count | Number of tasks running for the connector, faceted by task state. |
| `gcp.managedkafka.connector_total_error_record_count`     | Count | Number of records that produced errors on the connector.          |
| `gcp.managedkafka.connector_sink_active_records`          | Count | Active records buffered on sink connectors.                       |
| `gcp.managedkafka.connector_source_records_written_count` | Count | Number of records written by source connectors to Kafka.          |
| `gcp.managedkafka.connector_sink_records_read_count`      | Count | Number of records read by sink connectors from Kafka.             |

For the complete list of Managed Kafka metrics, see [Google's Managed Kafka metrics documentation](https://cloud.google.com/monitoring/api/metrics_gcp_i_o#gcp-managedkafka).