Centralized log analysis has the following problem, solution, and solution requirements.

Problem

Traditionally, an application writes log events to log files that are stored in the local filesystem of the server that the application runs on. Given a system landscape based on a microservice architecture, that is, with a large number of deployed microservice instances on a large number of smaller servers, we can ask the following questions:

How do I get an overview of what is going on in the system landscape when each microservice instance writes to its own local log file?

How do I find out if any of the microservice instances get into trouble and start writing error messages to their log files?

If end users start to report problems, how can I find related log messages; that is, how can I identify which microservice instance is the root cause of the problem? The following diagram illustrates the problem:

Figure 1.12: Microservices write log files to their local file system

Solution

Add a new component that can manage centralized logging and is capable of the following:

Detecting new microservice instances and collecting log events from them

Interpreting and storing log events in a structured and searchable way in a central database

Providing APIs and graphical tools for querying and analyzing log events

Solution requirements

Some solution requirements are as follows:

Microservices stream log events to standard system output, stdout. This makes it easier for a log collector to find the log events compared to when log events are written to microservice-specific logfiles.

Microservices tag the log events with the correlation ID described in the next section regarding the Distributed tracing design pattern.

A canonical log format is defined, so that log collectors can transform log events collected from the microservices to a canonical log format before log events are stored in the central database. Storing log events in a canonical log format is required to be able to query and analyze the collected log events.