Skip to main content
Mapagam

Managing Consumer Groups

1. Understanding Group Coordinator

Concept Description Detail
Coordinator Broker managing a group's membership & offsets One per group
Selection Hash of group.id → __consumer_offsets partition Deterministic
Leader Member One consumer computes assignment Group leader

Example: Find coordinator

kafka-consumer-groups.sh --describe --group billing \
  --members --bootstrap-server localhost:9092

2. Understanding Rebalancing

Trigger Cause Effect
Member Join/Leave Scaling or crash Reassign partitions
Topic Change Partition added New assignment
Stop-the-world Eager protocol pauses all Use cooperative

Example: Rebalance listener

consumer.subscribe(List.of("orders"), new ConsumerRebalanceListener() {
    public void onPartitionsRevoked(Collection<TopicPartition> p) { commit(); }
    public void onPartitionsAssigned(Collection<TopicPartition> p) { /* init */ }
});

3. Listing Consumer Groups

Flag Purpose Detail
--list Show all group IDs One per line
--state Include group state Stable/Empty
--all-groups Apply to every group Bulk ops

Example: List groups

kafka-consumer-groups.sh --list --bootstrap-server localhost:9092

4. Describing Consumer Groups

Column Meaning Detail
CURRENT-OFFSET Last committed offset Per partition
LOG-END-OFFSET Latest produced offset Tip of log
LAG END minus CURRENT Backlog

Example: Describe a group

kafka-consumer-groups.sh --describe --group billing \
  --bootstrap-server localhost:9092

5. Viewing Consumer Lag

Metric Description Detail
Partition Lag END - committed per partition Granular
Total Lag Sum across partitions Overall health
records-lag-max JMX consumer metric Live monitoring

Example: Check lag

kafka-consumer-groups.sh --describe --group billing \
  --bootstrap-server localhost:9092 | awk '{print $1,$3,$6}'

6. Resetting Offsets

Option Target Detail
--to-earliest Oldest offset Full replay
--to-latest Newest offset Skip backlog
--shift-by Relative move +/- N
--dry-run Preview only Then --execute
Warning: Offset reset requires the group to be inactive (no live members) or it will fail.

Example: Reset to earliest

kafka-consumer-groups.sh --reset-offsets --group billing \
  --topic orders --to-earliest --execute \
  --bootstrap-server localhost:9092

7. Deleting Consumer Groups

Aspect Description Constraint
--delete Remove group + offsets Must be empty
Active Group Cannot delete with members Stop consumers
Effect Re-consume per offset reset State lost

Example: Delete a group

kafka-consumer-groups.sh --delete --group old-group \
  --bootstrap-server localhost:9092

8. Configuring Session Timeout

Config Scope Default
session.timeout.ms Consumer-side liveness 45000
group.min.session.
timeout.ms		 Broker lower bound 6000
group.max.session.
timeout.ms		 Broker upper bound 1800000

Example: Broker bounds

group.min.session.timeout.ms=6000
group.max.session.timeout.ms=300000

9. Setting Heartbeat Interval

Config Description Detail
heartbeat.interval.ms Frequency to coordinator 3000
Ratio 1/3 of session timeout Guideline
Missed Triggers eviction after session Rebalance

Example: Align heartbeat

props.put("session.timeout.ms", 30000);
props.put("heartbeat.interval.ms", 10000);

10. Configuring Max Poll Interval

Config Description Detail
max.poll.interval.ms Processing budget per batch 300000
Symptom Repeated rebalances if exceeded Tune work
Mitigation Lower max.poll.records Or offload

Example: Prevent eviction during heavy work

props.put("max.poll.interval.ms", 900000);
props.put("max.poll.records", 50);

Was this article helpful?

Related Cheat Sheets