Adjust the Apache Proxy in Server 5 for Higher Performance

Photo of Greg
Photo by stu00c - http://flic.kr/p/4PTZpe

Several small tweaks can improve the performance of the Apache proxy introduced in version 5 of the Server app for OS X.

Out of the box, the reverse proxy introduced in version 5 of the Server app for OS X does its job mostly fine, but a couple of small tweaks are worth making, and a couple more are essential if you’ll be serving any sites over HTTPS. The proxy settings live in a separate configuration directory adjacent to the main configuration directory for the main web service instance of Apache, and the following three files will be of particular interest:

/Library/Server/Web/Config/Proxy/apache_serviceproxy.conf
/Library/Server/Web/Config/Proxy/apache_serviceproxy_customsites.conf
/Library/Server/Web/Config/Proxy/servermgr_serviceproxy_customsites.plist

There’s also one other file, ServiceProxySettings.plist, which you may want to modify. For some reason, Server 5 shipped with the log level for the proxy set to ‘debug’, and if left at that level, your server will log a great deal of stuff you will likely never want to see, except in the context of, say, actual debugging. (Many years ago, there was an additional reason to leave logging at the ‘debug’ level: certain known bugs in the proxy disappeared when using ‘debug’ logging and reappeared when using less detailed levels such as ‘notice’.) You can change the LogLevel setting in both ServiceProxySettings.plist and apache_serviceproxy.conf to something like ‘notice’ if you’re not keen to suck up processing time and dilute important log information by recording such large volumes of activity.

To take effect, changes to proxy settings files need to be followed by a restart of the Apache instance which provides the proxy service:

sudo /Applications/Server.app/Contents/ServerRoot/usr/sbin/serviceproxyctl restart

Note that this is not the same as the web service Apache instance.

While we’re on the apache_serviceproxy.conf file, if you’re not using other Services like calendaring or contacts, you may choose to shut off the proxy on other than the standard HTTP and HTTPS ports:

listen 80
listen 443
#listen 8008
#listen 8800
#listen 8443
#listen 8843

See the documentation file included in the Apache config directory for more details on these ports:

/Library/Server/Web/Config/apache2/ReadMe.txt

And the Internet Explorer performance fix from “Improve OS X Server Performance for IE Browsers” should also be applied a little further on in apache_serviceproxy.conf, commenting out the outdated TLS handling for Internet Explorer:

#SetEnvIf User-Agent ".*MSIE.*" nokeepalive ssl-unclean-shutdown

Finally, in order to provide a home base for more extensive adjustments to the proxy, I think it’s worth adding a single include toward the top of the apache_serviceproxy.conf config, after the listen lines, which will reference an additional file of settings specifically for this part of the server that faces the outside world. Just as an example to update those from “Essential Performance Tweaks for Your New OS X Server” — and obviously depending on your server’s traffic levels, memory, and general performance capabilities — performance tweaks for the proxy might include something like the following:

KeepAlive On
MaxKeepAliveRequests 150
KeepAliveTimeout 7
StartServers 10
MinSpareServers 10
MaxSpareServers 20
ListenBackLog 512
MaxRequestsPerChild 1000
MaxRequestWorkers 256

ServerSignature Off
ServerTokens Prod

You might also choose to increase the ProxyTimeout if you ever need your server to do some heavy lifting with databases or other jobs that you don’t want to time out prematurely. A proxy performance tweaks file is also the place to configure OCSP stapling to speed up TLS connections, which I’ll cover in a separate article.

For now, at least, it appears to be safe to store this proxy performance include along with the other proxy configuration files by creating a new /extra directory within /Library/Server/Web/Config/Proxy, akin to the /extra directory in /Library/Server/Web/Config/apache2.

Once you’ve finished with various adjustments to the new proxy service, I think it’s worth keeping a close eye on the proxy error log for awhile, just in case any problems crop up. This job is made a great deal easier by changing the log level as I described above, since it cuts out so much noise and makes any problems easier more obvious. In monitoring the logs, I’ve noticed many bots making malformed requests, quite a few SSL downgrade failures since disabling RC4 ciphers as I’ll describe in a separate article, and dropped connections every now and again, which often look like this:

[proxy_http:error] [pid 12345] (20014)Internal error: [client 12.34.56.78:98765] AH01102: error reading status line from remote server 127.0.0.1:34580, referer: http://example.com

Searching on the error code AH01102 turns up several bug discussions and several potential fixes, but nothing I’ve encountered looks definitive enough to try out on a production server. Some discussions point the finger toward timeout or keepalive issues, as is also the case with errors like these (which appear to be related to a bug going back years):

[proxy_http:error] [pid 12345] (70007)The timeout specified has expired: [client 12.34.56.78:98765] AH01095: prefetch request body failed to 127.0.0.1:34580 (127.0.0.1) from 12.34.56.78 (), referer: http://example.com

Likewise for these, which also turn up in multiple Apache Bugzilla discussions:

[proxy_http:error] [pid 12345] (70008)Partial results are valid but processing is incomplete: [client 12.34.56.78:98765] AH01095: prefetch request body failed to 127.0.0.1:34580 (127.0.0.1) from 12.34.56.78 (), referer: http://example.com

Notably, most of the latter appear to occur during bot visits, as distinct from visits from real browsers.

Given that timeouts and keepalive issues keep coming up in discussions, and also given that tweaks using proxy-sendchunks also keep appearing, it may be that all these could be expunged with some relatively minor adjustments.

And then there’s also the very occasional batch of load balancer errors which occur just as the logs are being rotated, and which seem to suggest significant problems, yet everything does actually restart just fine, and the service appears to work normally:

[mpm_prefork:notice] [pid 12345] AH00171: Graceful restart requested, doing restart
[rewrite:crit] [pid 54321] (2)No such file or directory: AH00666: mod_rewrite: could not init rewrite_mapr_lock_acquire in child
[proxy:crit] [pid 54321] (2)No such file or directory: AH00920: Failed to reopen mutex balancer://com.apple.devicemgr.rails in child
[proxy_balancer:crit] [pid 54321] (2)No such file or directory: AH01206: Failed to init balancer balancer://com.apple.devicemgr.rails in child
[so:warn] [pid 12345] AH01574: module proxy_fcgi_module is already loaded, skipping
[so:warn] [pid 12345] AH01574: module status_module is already loaded, skipping

At first glance, these errors, and AH00666 in particular, seem to suggest that something funky is going on with permissions — yet because the errors occur so rarely, it seems unlikely that there is any obvious permission-related misconfiguration somewhere.

All in all, however, it’s been refreshing to see that relatively few errors have been cropping up in the proxy service, and the vast majority of errors which do occur are due either to bad bots or to deliberate restrictions on how the server handles HTTPS connections.

All material on this site is carefully reviewed, but its accuracy cannot be guaranteed, and some suggestions offered here might just be silly ideas. For best results, please do your own checking and verifying. This specific article was last reviewed or updated by Greg on .

This site is provided for informational and entertainment purposes only. It is not intended to provide advice of any kind.