Breaking News: Grepper is joining You.com. Read the official announcement!

Avro serialization example#

Sumit Rawal answered on June 1, 2023 Popularity 3/10 Helpfulness 1/10

answer Avro serialization example#

Avro serialization example#

Comment

The code widget below demonstrates a consumer reading Kafka messages using Avro serialization. We haven’t delved into the workings of Avro, but in general, we can create messages of type GenericRecord or generate Java classes from our schema using Avro tools. The code in the widget uses the GenericRecord approach which loses type-safety and requires casts when the message is read on the consumer’s end.

package io.datajek.kafka.consumer;

import org.apache.avro.generic.GenericRecord;
import org.apache.kafka.clients.consumer.ConsumerRecord;
import org.apache.kafka.clients.consumer.ConsumerRecords;
import org.apache.kafka.clients.consumer.KafkaConsumer;

import java.time.Duration;
import java.util.Properties;
import java.util.regex.Pattern;

public class ConsumerWithAvroSerializerExample {
    public static void main(String[] args) {

        // Set up properties
        Properties props = new Properties();
        props.put("bootstrap.servers", "localhost:9092");
        props.put("key.deserializer", "org.apache.kafka.common.serialization.StringDeserializer");
        props.put("value.deserializer", "org.apache.kafka.common.serialization.StringDeserializer");
        props.put("group.id", "DataJekConsumers");
        props.put("auto.offset.reset", "earliest");

        // We also pass-in the URL for the schema registry
        props.put("schema.registry.url", "http://localhost:8081");

        // Create a Kafka consumer and subscribe to topic
        KafkaConsumer<String, GenericRecord> consumer = new KafkaConsumer<>(props);
        consumer.subscribe(Pattern.compile("datajek-.*"));

        try {
            while (true) {
                Duration oneSecond = Duration.ofMillis(1000);

                // Since the Producer wrote GenericRecords in the topic, our consumer will
                // read GenericRecords from the topic.
                ConsumerRecords<String, GenericRecord> records = consumer.poll(oneSecond);

                for (ConsumerRecord<String, GenericRecord> record : records) {

                    // Read the fields of the GenericRecord and print them on the
                    // console.
                    System.out.println("Offset " + record.offset() + " " +
                            "Car Make :" + record.value().get("make") + " " +
                            "Car Model :" + record.value().get("model"));
                }

                // Commit the offset
                consumer.commitSync();
            }
        } finally {
            consumer.close();
        }
    }
}

Popularity 3/10 Helpfulness 1/10 Language java

Source: Grepper

Tags: avro java serialization

Link to this answer
Share Copy Link

Contributed on Jun 01 2023

Sumit Rawal

0 Answers Avg Quality 2/10

Avro serialization example#

Contents

More Related Answers

Avro serialization example#

Grepper

Documentation

Social

Legal

Contact

Oops, You will need to install Grepper and log-in to perform this action.