Originally, Kafka was developed at LinkedIn to provide a high performance messaging system to track user activity (page views, click tracking, modifications to profile, etc.) and system metrics in real-time.

Messaging: Kakfa can be used in scenarios where applications need to send out notifications. For instance, various applications can write messages to Kafka and a single application can then read the messages and take appropriate action (e.g. format the message a certain way, filter a message, batching messages in a single notification).

Metrics and logging: Kafka is a great tool for building metrics and logging data pipelines. Applications can publish metrics to Kafka topics which can then be consumed by monitoring and alerting systems. The pipelines can also be used for offline analysis using Hadoop. Similarly, logs can be published to Kafka topics which can then be routed to log search systems such as Elasticsearch or security analysis applications.

Commit log: Kafka is based on the concept of a commit log which opens up the possibility of using it for database changes. The stream of changes can be used to replicate database updates on a remote system.

Stream processing: The term “stream processing” generally refers to Hadoop’s map/reduce style of processing when applied to data in real-time. Kafka can be used by streaming frameworks to allow applications to operate on Kafka messages to perform actions such as counting metrics, partitioning messages for processing by other applications, combining messages, or applying transformations on them.

There are several use Cases of Kafka that show why we actually use Apache Kafka.MessagingFor a more traditional message broker, Kafka works well as a replacement. We can say Kafka has better throughput,built-in partitioning, replication, and fault-tolerance which makes it a good solution for large-scale messageprocessing applications.MetricsFor operational monitoring data, Kafka finds the good application. It includes aggregating statistics from distributedapplications to produce centralized feeds of operational data.Event SourcingSince it supports very large stored log data, that means Kafka is an excellent backend for applications of eventsourcing.