Nginx access log to standard output (or journald)

· 561 words · 3 minute read

It is not as easy as using /dev/stdout. TLDR: Use access_log syslog:server=unix:/dev/log;. Even if you use /dev/stdout, Nginx is always going to open a new FD from /dev/stdout as a regular file / PTY. This is not supported in systemd.

So, one day you want to migrate from /var/log/nginx/access_log.* to a more standarized way of logging, say, systemd-journald. Then, you remember that systemd by default redirects standard output of services to the journal. And you tries to figure out how to get Nginx to log to stdout and stderr instead of the default log files.

The error log is fairly easy. Nginx docs on ngx_core_module#error_log says you can just put error_log stderr;, and it just works.

However, the access log is way more complicated. First, you don’t find anything on ngx_http_log_module#access_log mentioning stdout. Then, if you Google “nginx access_log stdout”, you get plenty of blogs and Stack Overflow answers saying: just put access_log /dev/stdout; and it works!

Sure this could work. But how? Under what conditions?

The fact is, most people willing to get Nginx to log to stdout are using Docker, which has a volatile /var/log/, and it makes sense to log to stdout. Moreover, inside Docker containers, /dev/stdout are just plain files or ptys that can be open(2)‘ed whenever we like.

This is not the case at all in systemd.

If you ever tried to put access_log /dev/stdout; in your config file, it will complain something like: Failed to open /dev/stdout: No such device or address.

This is because systemd services by default have their stdio connected to sockets, provided by systemd after fork(2). If you try ls /proc/<PID of service>/fd/1, you will mostly see something like: 1 -> 'socket:[5120309]'.

This socket, which differs from a regular file or a PTY, cannot be open(2)‘ed. It can only be written or closed.

Moreover, Nginx does not support writing to an open file descriptor for access_log. The error_log stderr; works because it specifically supports using the keyword stderr to represent file descriptor 2 and just write to that fd without opening anything. For access_log, there is no such option, and Nginx will always open your file, and that fails.

The reason of not supporting reusing an open FD 1 is due to performance concerns. From the Nginx mailinglist, Valentin V. Bartenev vbart@nginx.com wrote:

On Wednesday 17 February 2016 16:26:01 Aleksandar Lazic wrote:
> Hi.
> 
> how difficult is it to be able to add "access_log stdout;" to nginx, 
> similar like "error_log stderr;"?
> 
> I ask because in some PaaS environment is it difficult to setup a 
> dedicated user yust for nginx.
> 
> It fits also a little bit better to http://12factor.net/logs
> 
[..]

What's the problem with "access_log /dev/stdout"?

Please note that writing logs to stdout can be a bottleneck, or cause nginx
to stuck.  The "error_log stderr;" exists mostly for development purposes.

  wbr, Valentin V. Bartenev

Then how do we log to stdout? Strictly speaking, in a systemd environment with default unit configuration, you can’t, at least directly from Nginx itself (you can definitely spawn a pipe that connects some FIFO to FD 1, of course). Then, what if we just want to log to journald?

Fortunately, Nginx supports writing access log to syslog. Thus, we can just use

access_log syslog:server=unix:/dev/log;

to log to /dev/log, which is going to be redirected to the system journal.