Improve OS X Server Performance With Piped Logging as a Web App

The second of two built-in options for rotating Apache access logs and error logs on OS X Server is piping to the rotatelogs utility. You can set it up on a host-by-host basis using a very simple web app that can be enabled with a single checkbox in Server.app.

If you’d like to rotate your Apache log files while also improving performance of the server, you can take advantage of piped logging, which I mentioned as one of two options in the earlier article about cleaning up Apache logging. (See “Improve OS X Performance by Tweaking Apache Logging, And Options for Rotating Logs”.)

As I described in that article, there is some disagreement about the relative merits of rotation via piped logging and rotation via newsyslog (personally, I prefer the latter), but rather than diving into an academic debate and which one should be the best solution, there’s nothing to stop you from giving each one a try, perhaps doing a bit of benchmarking, and selecting the one which best suits your own needs and the traffic profile of the sites you’re hosting.

Using rotatelogs, you can set up access and error logs that rotate every 7 days with something like the following:

	CustomLog "|/usr/sbin/rotatelogs /var/log/apache2/access_log-%Y-%m-%d 604800 -1" combinedvhost
	ErrorLog "|/usr/sbin/rotatelogs /var/log/apache2/errorlog-%Y-%m-%d 604800 -1"

If you’ve followed the suggestion of preventing logging of “internal dummy connection” messages, which I mentioned in that earlier article, you can tweak the first line slightly:

	CustomLog "|/usr/sbin/rotatelogs /var/log/apache2/access_log-%Y-%m-%d 604800 -1" combinedvhost env=!donotlog

See the Apache documentation on rotatelogs for more details, and the main Apache documentation on log files for all the yummy goodness you might want to include in your log files. The main Apache documentation also mentions split-logfile, which is one way of post-processing large log files to break them up by virtual host.

Due to trickiness in the way Server.app handles single and double quotation marks — and an apparent inability to process anything like the examples above without botching the quotation marks — when it reads from the default template file for virtual hosts, which lives at .../apache2/sites_disabled/0000_default_default.conf, and which forms the basis for all new virtual hosts you create, I would recommend against placing the log directives directly in the .conf template. Instead, you can turn it into a web app and use this method in some of your virtual hosts — say, just the busier ones where separate piped logs are more important — or you can insert it directly into the main httpd_server_app.conf via an include.

If you comment out the default logging method for both access and errors from 0000_default_default.conf, this will take effect for all new virtual hosts created after you make the change. (You’ll need to do this manually for hosts you’ve already created, as they won’t be regenerated from the default.) Why bother to remove it, if we’re setting a custom log format separately anyway? As I mentioned in the earlier article, the answer is simple: you can define multiple distinct logs for each virtual host, and if you add custom rotating logs to what is already there by default, you’ll wind up with two sets of logs for each visit.

The web app we can use to enable piped rotating logs on a host-by-host basis is made of two parts: a file of Apache directives which will earn itself an ‘include’ when the vhost .conf file is written by Server.app, and a .plist file which references it. The .conf part is easy: it’s exactly what’s written above, or your preferred variation on it. The .plist part is not much harder; here’s one way to do it:

<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd">
<plist version="1.0">
<dict>
	<key>name</key>
	<string>net.mulhauser.webapp.rotator</string>
	<key>displayName</key>
	<string>Rotating Access and Error Logs</string>
	<key>includeFiles</key>
	<array>
		<string>/Library/Server/Web/Config/apache2/extra/httpd_grm_rotator.conf</string>
	</array>
	<key>launchKeys</key>
	<array/>
	<key>proxies</key>
	<dict/>
	<key>installationIndicatorFilePath</key>
        <string>/Library/Server/Web/Config/apache2/extra/httpd_grm_rotator.conf</string>
	<key>requiredModuleNames</key>
	<array/>
	<key>requiredWebAppNames</key>
	<array/>
	<key>sslPolicy</key>
	<integer>0</integer>
</dict>
</plist>

And here’s an example of the full .conf part of the web app, with the internal dummy connection code included for reference:

#
# Apache Log File Rotator
# Activated and deactivated by net.mulhauser.webapp.rotator web app
#
# Note we could also use "50M" rather than "604800 -1" to rotate every 50 MB, but this also starts a new
# log file each day
#

# We will use this to ensure we don't log the internal dummy connections used to keep connections alive
SetEnvIf Remote_Addr "127\.0\.0\.1" donotlog
SetEnvIfNoCase User-Agent ".*internal dummy connection.*" donotlog

CustomLog "|/usr/sbin/rotatelogs /var/log/apache2/access_log-%Y-%m-%d 604800 -1" combinedvhost env=!donotlog
ErrorLog "|/usr/sbin/rotatelogs /var/log/apache2/errorlog-%Y-%m-%d 604800 -1"

Once the .plist and .conf are dropped into Apache’s /webapp and /extra directories respectively — just create the latter as root — and the web service is restarted, the web app appears as an option in all hosts, as shown here along with a couple of other custom web apps:

Note that ‘extra’ is one of the standard directories which Apache won’t be surprised to find lurking in the configuration directory tree. Although it might seem logical to add a different new directory to keep your own custom configurations separate from the Apple-installed code and separate from Apache standards, in my experience this gives Apache fits and so is best avoided.

What if something goes wrong?

Speaking of Apache having fits, if at any point you make a mistake while editing a site’s .conf file, you may find a .conf.syntaxError file in .../apache2/sites/ with the previous .conf file left untouched (you’ll be able to tell by the modification dates); if a similar error occurs while you’re setting up a web app, you may find that the whole web service stops responding, and Apache repeatedly attempts to restart every 10 seconds or so. If this happens, don’t panic! To fix the problem, first eradicate whatever change it was that produced the error — but that won’t be enough. Next, stop the web service:

Then restart Apache:

Then, finally, start the web service again:

(Remember the web service is not the only service which requires Apache; merely stopping the web service is no guarantee of stopping Apache.)

When you’re not dealing with errors or other problems and simply want to restart Apache, the graceful restart on its own will do fine.

In addition, if you want to rename, move, or otherwise tweak a web app after it’s already been installed, it’s important to deactivate it from the command line first:

You can manually start it up again with ‘start’ in place of ‘stop’.

• macOS
• page speed