Improve OS X Performance by Tweaking Apache Logging, And Options for Rotating Logs

Photo of Greg
Photo by ginnerobot - http://flic.kr/p/4TiJLN

Especially if you’re migrating from a WHM/cPanel server, you’ll likely find that the default configurations for Apache logging and error logging on OS X leave a lot to be desired. Here’s how to clean things up a bit, and here are some of the advantages and disadvantages of two different options for rotating your logs — newsyslog and piping to the rotatelogs utility.

Cleaning Up and Rotating Apache Logging

If you’ll be hosting multiple sites on a single server, or even if you host only one site but it receives anything more than a trickle of traffic, the default setup of a single, non-rotating log file is hardly suitable: it offers neither performance nor convenience. If you’d like to rotate your Apache log files while also improving performance of the server, you can leverage either newsyslog or piped logging, neither of which is set up to do anything with your logs by default. At the same time, you can also eliminate logging of “internal dummy connection” messages, which appear as a result of how Apache keeps connections open, and prepare your logging setup to skip logging in the case of various conditions which you set, such as visits occurring from a bot which you’ve separately categorised as ‘bad’.

I’ll come back to log rotation in a moment, but first to get started, if you take a look inside the .conf file for the default site, you’ll see something along the lines of the following within the VirtualHost at /Library/Server/Web/Config/apache2/sites/0000_any_80.conf:

	CustomLog "/var/log/apache2/access_log" combinedvhost
	ErrorLog "/var/log/apache2/error_log"

This configures the basic logging setup, which differs in only one way from the logging specified in the main httpd_server_app.conf; the latter is the same, except that it uses the ‘common’ log format instead of ‘combinedvhost’. Since the ‘common’ format neither tells you which vhost was being visited nor any referrer information, ‘combinedvhost’ is generally preferable.

Whether you’d like to use piped logging or newsyslog, each of which I describe in separate articles, you should be aware that Apache will pay attention to multiple distinct logging directives for each vhost — so if you just add new ways of logging, your server will wind up logging the same visits two or more times. Also, the type of logging specified in httpd_server_app.conf will only be used if no logging is specified for a vhost. This means that if you want to change logging globally, you need to remove or change what is already in httpd_server_app.conf before adding anything new, and if you want to change logging vhost by vhost, you need to remove whatever will already be placed in the vhost configuration file by the default template file for virtual hosts, which lives at .../apache2/sites_disabled/0000_default_default.conf, and which forms the basis for all new virtual hosts you create.

So, the first thing I’d suggest is tweaking the appropriate line in httpd_server_app.conf — perhaps commenting out the original and adding this instead:

	#CustomLog "/private/var/log/apache2/access_log" common
	CustomLog "/private/var/log/apache2/access_log" combinedvhost

Note that if you decide to make this edit, you’ll need to keep an eye for other differences in the default httpd_server_app.conf after software updates, in case something gets changed. Yes, Apple recommends that edits be made via include files, but this particular change is less suitable for an include file, since as I mentioned above, Apache will pay attention to multiple separate CustomLog directives, rather than simply overriding previous directives with later ones.

You can also prevent logging of “internal dummy connection” messages, which appear as a result of how Apache keeps connections open, by making the active CustomLog line look like this instead:

	CustomLog "/private/var/log/apache2/access_log" combinedvhost env=!donotlog

Then add the following to a separate file and load it via an ‘include’ in httpd.server_app.conf, using the same method I described in the separate article on essential performance tweaks (see “Essential Performance Tweaks for Your New OS X Server”):

SetEnvIf Remote_Addr "127\.0\.0\.1" donotlog
SetEnvIfNoCase User-Agent ".*internal dummy connection.*" donotlog

You can also take advantage of the ‘donotlog’ environment variable when constructing a global ban list, which I’ll cover later on in a separate article.

Finally, you can comment out the corresponding line from the default configuration file for virtual hosts, which lives at .../apache2/sites_disabled/0000_default_default.conf:

	#CustomLog "/var/log/apache2/access_log" combinedvhost

This will cause logging for each vhost to fall back to whatever is defined in the main httpd_server_app.conf file.

Options for Rotating Apache Logs on OS X Server

Now that you’ve cleaned up Apache logging a bit, one option for rotating your logs is to let Apache operate as normal, writing its log files itself, and then schedule a separate process (such as newsyslog) to come along and rotate those log files every so often. The other is to pipe logging output from Apache to a separate process (such as rotatelogs), and let that separate process handle both the writing of log information and the rotation of the logs. Each approach has advantages and disadvantages.

Newsyslog for Apache Log Rotation

Advantages of newsyslog for Apache log rotation include:

  • Minimal modification to the default configuration
  • Slightly better performance
  • Consistent naming for current log files
  • Easier to apply on a global basis

And disadvantages of newsyslog:

  • Less configurable
  • Harder to apply on a vhost-by-vhost basis
  • Graceful restart of Apache required to move logging to new file

Piping to Rotatelogs for Apache Log Rotation

Advantages of piping to rotatelogs include:

  • Huge range of configuration options
  • Easier to apply on a vhost-by-vhost basis
  • No graceful restart of Apache required to move logging to new file

And disadvantages of the rotatelogs approach:

  • Slightly lower performance compared to native Apache logging
  • Name of current log file changes with each rotation

(There’s also an important difference in how the two setups handle Apache errors which are not host-specific, such as logging startup and shutdown activity: in the case of native Apache logging, all errors go into one file, while in the case of piped logging defined at a vhost level, those errors go into the regular error_log file, while host-specific errors go into whatever file is being written via the pipe.)

You might wonder why I’ve listed native Apache logging as offering slightly better performance, given that just about everywhere I’ve read says that the big advantage of piped logging is that the pipe can introduce buffering. Well, as far as I can tell, this is just plain wrong: Apache does buffer its logging output, which is specifically why it is recommended that newsyslog not be set to compress rotated logs automatically (because it does not know for sure when Apache will finish writing to the old log). Maybe it was once true, with some past version of Apache, that logs were not buffered, but as far as I can tell, it has not been true for more than a decade. In addition, my own rudimentary tests on my own server give the edge — albeit only a very slight edge — to native logging as compared to piped. Obviously, your setup and traffic characteristics may well differ in important ways from my own, potentially yielding different performance comparisons — so when in doubt, test for yourself!

The bigger deal, in my view, is the graceful restart that is required each time your logs are rotated by newsyslog — a graceful restart that will, among other things, empty your APC cache. If you’re looking to rotate logs each day, this might be something you’d rather avoid. Is it a big enough deal to counterbalance its other advantages? As with so many things, the answer is that it depends, but I’ve described how to try out each option in separate articles.

All material on this site is carefully reviewed, but its accuracy cannot be guaranteed, and some suggestions offered here might just be silly ideas. For best results, please do your own checking and verifying. This specific article was last reviewed or updated by Greg on .

This site is provided for informational and entertainment purposes only. It is not intended to provide advice of any kind.