Metric names and labels

The reporter turns the names from Kafka’s two registries, neither of which Prometheus accepts as they are, into one lowercase, Prometheus-friendly form, Strimzi-style. Kafka’s tags, Yammer scopes and the broker’s identity become labels.

Flattening rules

Every name is lowercased, every character other than a letter or a digit becomes an underscore, consecutive underscores collapse into one, and leading and trailing underscores are removed. What goes into the name depends on the registry.

Kafka SPI metrics

An SPI metric is named {namespace}_{group}_{name}. The namespace comes from Kafka’s metrics context and depends on the process: kafka.server on a broker, kafka.producer or kafka.consumer in a client. On a broker, the socket-server-metrics group’s connection-count metric becomes:

kafka_server_socket_server_metrics_connection_count

If the context has no namespace, the prefix is left out: socket_server_metrics_connection_count. Kafka’s metric tags become labels of the series.

Yammer metrics

A Yammer metric is named {group}_{type}_{name}, without a namespace. The broker’s kafka.server group, ReplicaManager type and UnderReplicatedPartitions name become kafka_server_replicamanager_underreplicatedpartitions.

The Yammer scope becomes labels. Kafka encodes it as key.value.key.value:

• topic.<name> becomes a topic label.
• topic.<name>.partition.<n> becomes topic and partition labels.
• Any other scope is split pairwise into labels. A scope that does not split into pairs becomes a single scope label.

# Per-topic (scope="topic.<name>")
kafka_server_brokertopicmetrics_bytesinpersec_total{topic="my-topic"}

# Per-partition (scope="topic.<name>.partition.<n>")
kafka_cluster_partition_underreplicated{topic="my-topic", partition="0"}

Some metrics exist at both levels: kafka_server_replicamanager_underreplicatedpartitions is the broker-wide aggregate, without topic or partition labels, and kafka_cluster_partition_underreplicated{topic, partition} the per-partition gauge that shows which partition is degraded.

Timers and histograms become a _count counter plus one gauge series per quantile, 0.5, 0.95 and 0.99, told apart by a quantile label.

Counters and gauges

Kafka SPI metrics whose names end in -total are cumulative counters and are exported as monotonic sums; all other SPI metrics are gauges. Yammer counters and meters are exported as monotonic sums of their count, and Yammer gauges as gauges.

On the way into Prometheus, the collector’s Prometheus exporter adds a _total suffix to monotonic sums whose names do not end in _total already. The Yammer meter BytesInPerSec leaves the reporter as kafka_server_brokertopicmetrics_bytesinpersec and is queried as kafka_server_brokertopicmetrics_bytesinpersec_total, as in the quickstart dashboards.

Counters are cumulative since the reporter started and never reset while the JVM runs. Query them with rate() or increase(), never as raw values.

Labels on every series

Kafka gives a broker’s reporter a metrics context with the cluster ID and the node ID. The reporter attaches them to every series as OTLP resource attributes, and the collector’s Prometheus exporter turns them into labels when resource_to_telemetry_conversion is enabled. The dots in the attribute names become underscores:

kafka_controller_kafkacontroller_activecontrollercount{
  kafka_cluster_id="...", kafka_node_id="1"
} 1

• kafka_cluster_id: the cluster ID.
• kafka_node_id on KRaft brokers, or kafka_broker_id on ZooKeeper-mode brokers.
• service_instance_id: a UUID generated for each reporter instance, so two reporters in the same JVM stay distinguishable. A restarted reporter gets a new one. Override it with a stable value through otlp.metric.reporter.resource.attributes=service.instance.id=....

The namespace from the context is not a label; it is the prefix of SPI metric names. Attributes set in otlp.metric.reporter.resource.attributes, as key=value,key=value, are added to every series and win over Kafka’s context when a key appears in both. In a producer or consumer, Kafka’s context has no cluster or node ID, so client series do not carry those labels.

JVM runtime metrics carry the same resource labels as the Kafka metrics, so by (kafka_cluster_id, kafka_node_id) joins them with broker series.

JVM, self-monitoring and client metrics

These streams keep the names their sources give them:

• JVM runtime metrics use the names of OpenTelemetry’s runtime instrumentation, such as jvm_memory_used_bytes.
• Self-monitoring metrics start with monedula_reporter_; the self-monitoring reference lists them.
• KIP-714 client metrics keep the names the client pushes, such as org_apache_kafka_producer_request_latency_avg for the Java producer’s request latency in Prometheus. The reporter adds kafka_cluster_id, kafka_node_id and client_id by default; Collect Kafka client metrics with KIP-714 lists the optional labels. The client’s instrumentation scope arrives as otel_scope_name and otel_scope_version.

Sample names

Once the reporter runs on brokers and clients, these names, among many others, appear in Prometheus:

# Broker-side Kafka SPI
kafka_server_socket_server_metrics_connection_count

# Broker-side Yammer
kafka_controller_kafkacontroller_activecontrollercount
kafka_controller_kafkacontroller_offlinepartitionscount
kafka_server_replicamanager_underreplicatedpartitions
kafka_server_brokertopicmetrics_bytesinpersec_total
kafka_network_requestmetrics_requestspersec_total
kafka_cluster_partition_underreplicated

# Java producer with the reporter, Kafka SPI
kafka_producer_producer_metrics_record_send_rate

# JVM runtime (memory, GC, threads, classes, CPU)
jvm_thread_count
jvm_memory_used_bytes
jvm_gc_duration_seconds_sum
jvm_cpu_time_seconds_total

# Reporter self-monitoring
monedula_reporter_export_success_total
monedula_reporter_export_failure_total
monedula_reporter_export_duration_ms