Depending on your environment settings, some DLQ message bodies will be in either Protobuf or JSON format:

If the format is JSON, you do not need to follow the remainder of this section and must not use protoc on your message output.

If the format is Protobuf, you will need to deserialise the message before it is human readable, instructions for which are below.

To deserialise an encoded protobuf message, you must have access to the corresponding .proto file that contains the message definition in question. Many of these .proto files can be found in the 'Downloading the proto files' part of the Streaming API documentation for the relevant API. Once you have this file you can convert the message into human readable JSON. If you do not have access to the corresponding .proto file, please contact Thought Machine support, following the process in your Service Procedure Manual.

To determine the correct .proto file for deserialising the body of a DLQ:

Go to the documentation for the specific DLQ in question.

Find the message type under the 'What message has been sent to the DLQ?' section.

Find the .proto file that contains this message type. There will be a Message line in the file, where is the message type from the previous step.

There are many ways that a .proto file can be used to read a message in protobuf format. If a lightweight approach is needed, Thought Machine recommends the use of Protoc.

Protoc can be run on any protobuf bytes to convert to JSON as follows:protoc --decode=

The package name is defined in the .proto file by the line: package ;

For example:

protoc --decode=test_package.TestMessage path_to_proto/test.proto

would deserialise protobuf bytes into JSON format for a protobuf message of type TestMessage where:

The TestMessage type is defined in 'test.proto'.

The 'test.proto' contains a package test_package; line, which defines the package name.

The 'test.proto' file is located in your /path_to_proto/ directory.

If using protoc in conjunction with Kafkacat, you should only read the message body from Kafka before passing it to protoc. For example: docker run --rm confluentinc/cp-kafkacat kafkacat -b -t -C -D “” | protoc --decode=test_package.TestMessage path_to_proto/test.proto.