How to Redirect Docker Logs to a Single File

Sometimes, when troubleshooting or monitoring a Docker container, we need to see the application’s output streams. Containerized applications generate standard output (stdout) and standard error (stderr) like any other software. The Docker daemon merges these streams and directs them to one of several locations, depending on which logging driver is installed. The default driver makes the container output easy to access, and if we want to copy the information to another location, we can redirect docker logs to a file.

Let’s take a look at how we can manipulate logs generated by the default json-file log driver. This driver places the log output in a system directory in JSON formatted files but provides a command line tool for displaying log contents in its original format.

Docker whale leaping into a single file with Scalyr colors signifying a way to redirect docker logs to file

Viewing Container Logs

Let’s start with a simple container. We’ll create one that prints a message every second.

$ docker run --name test -d busybox sh -c "while true; do $(echo date); sleep 1; done"

This command is an example from the Docker website. It executes a minimal Docker container with a Bourne shell script that logs a timestamp once every second. We’ll use it for a couple of examples.

So the easiest way to view container logs is with the docker logs command. Start the container and then display the logs.

$ docker run --name test -d busybox sh -c "while true; do $(echo date); sleep 1; done"
13f622b1e214d7e84b6aa2093484f932ab55c14752459d3075b41b1a0c60cbaf
$ docker logs test
Thu Oct 18 01:55:27 UTC 2018
Thu Oct 18 01:55:28 UTC 2018
Thu Oct 18 01:55:29 UTC 2018
Thu Oct 18 01:55:30 UTC 2018

This command displays all of the output the container has generated so far.

If there’s too much output for a single screen, pipe it to a buffering command like less.

$ docker logs test | less
Thu Oct 18 01:55:27 UTC 2018
Thu Oct 18 01:55:28 UTC 2018
Thu Oct 18 01:55:29 UTC 2018
Thu Oct 18 01:55:30 UTC 2018
Thu Oct 18 01:55:31 UTC 2018
Thu Oct 18 01:55:32 UTC 2018
Thu Oct 18 01:55:33 UTC 2018
Thu Oct 18 01:55:34 UTC 2018
Thu Oct 18 01:55:35 UTC 2018
Thu Oct 18 01:55:36 UTC 2018
Thu Oct 18 01:55:37 UTC 2018
Thu Oct 18 01:55:38 UTC 2018
Thu Oct 18 01:55:39 UTC 2018
Thu Oct 18 01:55:40 UTC 2018
Thu Oct 18 01:55:41 UTC 2018
:

This will still send all of the output generated so far through less, so you’ll need to page through to get to the end.

But sometimes you want to follow the logs as the container is running. There’s a command line option with an appropriate name for that: run docker logs with –follow.

$ docker logs --follow test
Thu Oct 18 01:55:27 UTC 2018
Thu Oct 18 01:55:28 UTC 2018
Thu Oct 18 01:55:29 UTC 2018
Thu Oct 18 01:55:30 UTC 2018
Thu Oct 18 01:55:31 UTC 2018
Thu Oct 18 01:55:32 UTC 2018
Thu Oct 18 01:55:33 UTC 2018
Thu Oct 18 01:55:34 UTC 2018
Thu Oct 18 01:55:35 UTC 2018
Thu Oct 18 01:55:36 UTC 2018
Thu Oct 18 01:55:37 UTC 2018
Thu Oct 18 01:55:38 UTC 2018
Thu Oct 18 01:55:39 UTC 2018
Thu Oct 18 01:55:40 UTC 2018
Thu Oct 18 01:55:41 UTC 2018

After printing all of the container output so far, the command will continue to follow the logs. You can use the short form of the option -f, similar to the tail command, too.

The docker logs command also offers an array of convenient options, such as filtering output based on time and displaying timestamps. You can read more about them here. But what if you want to redirect the docker logs to a file for viewing offline?

Redirect Docker Logs to File

Since Docker merges stdout and stderr for us, we can treat the log output like any other shell stream. To redirect the current logs to a file, use a redirection operator.

$ docker logs test > output.log

To send the current logs and then any updates that follow, use –follow with the redirection operator.

$ docker logs -f test > output.log

This command saves the output for future reference, but what if you want to keep the logs and view them at the same time? Opening another shell and tailing the output file feels like a hack. There must be a better way.

So, let’s use tee to watch the output and save it to a file at the same time.

$ docker logs -f test |tee output.log
Thu Oct 18 02:08:52 UTC 2018
Thu Oct 18 02:08:53 UTC 2018
Thu Oct 18 02:08:54 UTC 2018
Thu Oct 18 02:08:55 UTC 2018
Thu Oct 18 02:08:56 UTC 2018
Thu Oct 18 02:08:57 UTC 2018
Thu Oct 18 02:08:58 UTC 2018
Thu Oct 18 02:08:59 UTC 2018
Thu Oct 18 02:09:00 UTC 2018
Thu Oct 18 02:09:01 UTC 2018
Thu Oct 18 02:09:02 UTC 2018
Thu Oct 18 02:09:03 UTC 2018

We still see the container output. Now, open another shell and check the output file.

$ tail -f output.log
Thu Oct 18 02:08:52 UTC 2018
Thu Oct 18 02:08:53 UTC 2018
Thu Oct 18 02:08:54 UTC 2018
Thu Oct 18 02:08:55 UTC 2018
Thu Oct 18 02:08:56 UTC 2018
Thu Oct 18 02:08:57 UTC 2018
Thu Oct 18 02:08:58 UTC 2018
Thu Oct 18 02:08:59 UTC 2018
Thu Oct 18 02:09:00 UTC 2018
Thu Oct 18 02:09:01 UTC 2018

The file is being updated too! We have the output on the terminal, and tee saves it to the file we specified at the same time.

Redirecting Stdout/Stderr in the Container

Of course, there’s another way to save your container logs to a file. Instead of sending output to stderr and stdout, redirect your application’s output to a file and map the file to permanent storage outside of the container.

When to eschew the standard output completely is a judgment call that you should make when you take into account how much output your application generates, how often you think you’ll need to refer to it, and what kind of logging infrastructure is available to you. If, for example, you need to store events for compliance purposes, a logging framework like Java’s Logback may be a better option than capturing stdout and stderr.

That said, let’s modify the previous example to save the output to a file. We’ll change the loop to print the date to a file in /tmp. We’ll also run the container with /tmp mapped to the same directory of our host. Note that Docker will not accept relative paths on the command line, so if you want to use a different directory, you’ll need to use the complete path.

$ docker run -v /tmp:/tmp --name test -d busybox sh -c "while true; do date > /tmp/output.log; sleep 1; done"

Take a look at /tmp/output.log on the host system.

$ tail -f /tmp/output.log
Tue Oct 17 22:03:45 UTC 2018
Tue Oct 17 22:03:46 UTC 2018
Tue Oct 17 22:03:47 UTC 2018
Tue Oct 17 22:03:48 UTC 2018
Tue Oct 17 22:03:49 UTC 2018
Tue Oct 17 22:03:50 UTC 2018

Now, take a look at docker logs.

$ docker logs test

There’s nothing there. Since we redirected the output to a file, there is nothing for Docker to capture. Of course, we would need to capture both stdout and stderr in a production application.

Use the Method That Works for You

We covered several different ways to capture docker logs for saving and analysis. The best method is the one that works for you. In a development environment, the docker logs command is a powerful tool that works well with other command line tools. You can use Docker’s built-in tools to view, filter, and copy logs to a file.

In a production environment, log management should be part of an enterprise strategy. Scalyr’s log aggregation tools help you aggregate, process, search, and analyze your logs. Contact us, or sign up for a free trial to see how our tools make it easier to find what you need in the vast ocean of log entries produced by your growing application infrastructure.

This post was written by Eric Goebelbecker. Eric has worked in the financial markets in New York City for 25 years, developing infrastructure for market data and financial information exchange (FIX) protocol networks. He loves to talk about what makes teams effective (or not so effective!)

Leave a Reply

Your email address will not be published. Required fields are marked *