// Module included in the following assemblies:
//
// metrics/assembly-metrics.adoc

[id='con-metrics-kafka-exporter-lag-{context}']

= Monitoring consumer lag with Kafka Exporter

[role="_abstract"]
{kafka-exporter-project} is an open source project to enhance monitoring of Apache Kafka brokers and clients.
You can configure the `Kafka` resource to xref:proc-metrics-kafka-deploy-options-{context}[deploy Kafka Exporter with your Kafka cluster].
Kafka Exporter extracts additional metrics data from Kafka brokers related to offsets, consumer groups, consumer lag, and topics.
The metrics data is used, for example, to help identify slow consumers.
Lag data is exposed as Prometheus metrics, which can then be presented in Grafana for analysis.

Kafka Exporter reads from the  `__consumer_offsets` topic, which stores information on committed offsets for consumer groups. 
For Kafka Exporter to be able to work properly, consumer groups needs to be in use. 

A Grafana dashboard for Kafka Exporter is one of a number of xref:ref-metrics-dashboards-{context}[example Grafana dashboards] provided by Strimzi.

IMPORTANT: Kafka Exporter provides only additional metrics related to consumer lag and consumer offsets.
For regular Kafka metrics, you have to configure the Prometheus metrics in xref:proc-metrics-kafka-deploy-options-{context}[Kafka brokers].

Consumer lag indicates the difference in the rate of production and consumption of messages.
Specifically, consumer lag for a given consumer group indicates the delay between the last message in the partition and the message being currently picked up by that consumer.

The lag reflects the position of the consumer offset in relation to the end of the partition log.

.Consumer lag between the producer and consumer offset

image:consumer-lag.png[Consumer lag]

This difference is sometimes referred to as the _delta_ between the producer offset and consumer offset: the read and write positions in the Kafka broker topic partitions.

Suppose a topic streams 100 messages a second. A lag of 1000 messages between the producer offset (the topic partition head) and the last offset the consumer has read means a 10-second delay.

[discrete]
== The importance of monitoring consumer lag

For applications that rely on the processing of (near) real-time data, it is critical to monitor consumer lag to check that it does not become too big.
The greater the lag becomes, the further the process moves from the real-time processing objective.

Consumer lag, for example, might be a result of consuming too much old data that has not been purged, or through unplanned shutdowns.

[discrete]
== Reducing consumer lag

Use the Grafana charts to analyze lag and to check if actions to reduce lag are having an impact on an affected consumer group.
If, for example, Kafka brokers are adjusted to reduce lag, the dashboard will show the  _Lag by consumer group_ chart going down and the _Messages consumed per minute_ chart going up.

Typical actions to reduce lag include:

* Scaling-up consumer groups by adding new consumers
* Increasing the retention time for a message to remain in a topic
* Adding more disk capacity to increase the message buffer

Actions to reduce consumer lag depend on the underlying infrastructure and the use cases Strimzi is supporting.
For instance, a lagging consumer is less likely to benefit from the broker being able to service a fetch request from its disk cache.
And in certain cases, it might be acceptable to automatically drop messages until a consumer has caught up.
