Breaking News: Grepper is joining You.com. Read the official announcement!

kcat commands to list topics

Sumit Rawal answered on August 1, 2023 Popularity 1/10 Helpfulness 1/10

answer kcat commands to list topics

kcat commands to list topics

Comment

# List all available topics
kafkacat -b localhost -L | grep topic

# Count the number of packets in a topic, by streaming it:
kafkacat -b localhost -t ztf_20180519_programid2 -e -o beginning -f 'Topic %t [%p] at offset %o\n' | wc

# get list of topics & offsets. the output are tab-separated columns <topic_name> <partition_id> <offs_earliest> <offs_latest>
kt topic -partitions -filter 'ztf_.*' | jq -r '.name as $name | .partitions[] | [$name, .id, .oldest, .newest] | @tsv' | sort

# get the number of alerts in all topics (see [here](https://github.com/dirac-institute/zads-terraform/blob/master/provisioning/broker/config/zads-delete-expired-topics#L11) for explanation on what this does)
kt topic -partitions -filter 'ztf_.*' | jq -r '.name as $name | .partitions[] | [$name, .id, .oldest, .newest] | @tsv' | awk '{a[$1]+=$4-$3} END {for(i in a) print i"\t"a[i]}' | sort

# last message offsets, using JVM-based tools (slow)
kafka-run-class kafka.tools.GetOffsetShell --broker-list localhost:9092 --topic ztf_20180516_programid1 --time -1
# first message offsets, using JVM-based tools (slow)
kafka-run-class kafka.tools.GetOffsetShell --broker-list localhost:9092 --topic ztf_20180516_programid1 --time -2
# Subtract the results of last - first to get the number of messages in the topic.

Popularity 1/10 Helpfulness 1/10 Language whatever

Source: gist.github.com

Tags: list whatever

Link to this answer
Share Copy Link

Contributed on Aug 01 2023

Sumit Rawal

0 Answers Avg Quality 2/10

kcat commands to list topics

Contents

More Related Answers

kcat commands to list topics

Grepper

Documentation

Social

Legal

Contact

Oops, You will need to install Grepper and log-in to perform this action.