Breaking News: Grepper is joining You.com. Read the official announcement!
Check it out

Kafka Architecture

Pragya Keshap answered on February 21, 2023 Popularity 10/10 Helpfulness 3/10

Contents


More Related Answers

  • kafka create topic
  • Kafka-dependency
  • do you need java installed for kafka
  • kafka nodejs example
  • What is Kafka?
  • kafka create topic
  • Why Kafka is best for Data Streaming?
  • What happens if Kafka topic is full?
  • Kafka-dependency
  • APACHE KAFKA
  • create kafka topics
  • kafka 4 apis
  • Kafka
  • Kafka Brokers - Kafka Cluster
  • kafkacat consume
  • How Kafka works?
  • Kafka strems
  • What are some of the features of Kafka?
  • What are the major components of Kafka?
  • Kafka
  • Kafka-dependency
  • Why is Kafka technology significant to use?
  • What can you do with Kafka?
  • Features of Kafka Stream.
  • apache kafka benefits and use cases
  • Kafka Components
  • What exactly does Kafka do?
  • What is Kafka used for?
  • What is the best way to start the Kafka server?

  • Kafka Architecture

    1

    Topics

    A stream of messages that are a part of a specific category or feed name is referred to as a Kafka topic. In Kafka, data is stored in the form of topics. Producers write their data to topics, and consumers read the data from these topics.

    Brokers

    A Kafka cluster comprises one or more servers that are known as brokers. In Kafka, a broker works as a container that can hold multiple topics with different partitions. A unique integer ID is used to identify brokers in the Kafka cluster. Connection with any one of the kafka brokers in the cluster implies a connection with the whole cluster. If there is more than one broker in a cluster, the brokers need not contain the complete data associated with a particular topic.

    Consumers and Consumer Groups

    Consumers read data from the Kafka cluster. The data to be read by the consumers has to be pulled from the broker when the consumer is ready to receive the message. A consumer group in Kafka refers to a number of consumers that pull data from the same topic or same set of topics.

    Producers

    Producers in Kafka publish messages to one or more topics. They send data to the Kafka cluster. Whenever a Kafka producer publishes a message to Kafka, the broker receives the message and appends it to a particular partition. Producers are given a choice to publish messages to a partition of their choice.

    Partitions

    Topics in Kafka are divided into a configurable number of parts, which are known as partitions. Partitions allow several consumers to read data from a particular topic in parallel. Partitions are separated in order. The number of partitions is specified when configuring a topic, but this number can be changed later on. The partitions comprising a topic are distributed across servers in the Kafka cluster. Each server in the cluster handles the data and requests for its share of partitions. Messages are sent to the broker along with a key. The key can be used to determine which partition that particular message will go to. All messages which have the same key go to the same partition. If the key is not specified, then the partition will be decided in a round-robin fashion.

    Partition Offset

    Messages or records in Kafka are assigned to a partition. To specify the position of the records within the partition, each record is provided with an offset. A record can be uniquely identified within its partition using the offset value associated with it. A partition offset carries meaning only within that particular partition. Older records will have lower offset values since records are added to the ends of partitions.

    Replicas

    Replicas are like backups for partitions in Kafka. They are used to ensure that there is no data loss in the event of a failure or a planned shutdown. Partitions of a topic are published across multiple servers in a Kafka cluster. Copies of the partition are known as Replicas.

    Leader and Follower

    Every partition in Kafka will have one server that plays the role of a leader for that particular partition. The leader is responsible for performing all the read and write tasks for the partition. Each partition can have zero or more followers. The duty of the follower is to replicate the data of the leader. In the event of a failure in the leader for a particular partition, one of the follower nodes can take on the role of the leader. 

    https://www.projectpro.io/article/apache-kafka-architecture-/442

    Popularity 10/10 Helpfulness 3/10 Language whatever
    Source: Grepper
    Link to this answer
    Share Copy Link
    Contributed on Feb 21 2023
    Pragya Keshap
    0 Answers  Avg Quality 2/10


    X

    Continue with Google

    By continuing, I agree that I have read and agree to Greppers's Terms of Service and Privacy Policy.
    X
    Grepper Account Login Required

    Oops, You will need to install Grepper and log-in to perform this action.