A Better HAProxy Health Check For Dynamic Websites

Nobody wants their website to go down, or worse, for users to notice the site is down. Because of this most larger websites will run on multiple servers to provide some level of high availability. In a multi-server architecture there is typically a load balancer (or cluster of load balancers) to distribute the load among a pool of web servers. When a server goes down it’s taken out of the pool until it is once again ready to handle requests. HAProxy (a software load balancer) has the ability to perform this task by doing periodic health checks on all the servers in a cluster. The default settings, though, could give false positives in some cases, and thus create a bad user experience by allowing ill application servers to continue receiving requests.

When in HTTP mode HAProxy’s default health check is a simple OPTIONS request. This has the advantage of being a very lightweight request, and is easy to identify and filter from server logs. Consider this scenario though: HAProxy balances the load between several web servers running nginx and PHP-FastCGI. If nginx is up but PHP-FastCGI goes down, nginx will still properly handle the OPTIONS request from HAProxy, giving the impression that all is well. HAProxy continues sending requests to the ill server which in turn get a 504 Gateway Timeout (or similar) response. Not a very good situation.

A solution would be to use a deeper health check, one that goes beyond nginx to the PHP-FastCGI process. That way if PHP-FastCGI goes down, the whole server is presumed ‘down’.

backend appservers

 mode http

 option httpchk HEAD /health_check.php HTTP/1.1\r\nHost:\ example.com

 server web1 x.x.x.x:80 weight 5 check inter 2000

 server web2 x.x.x.x:80 weight 5 check inter 2000

 server web3 x.x.x.x:80 weight 5 check inter 2000

In the above example I’m using a custom health check request which will be processed by PHP-FastCGI. health_check.php is a lightweight script that contains simply <?php echo "I'm healthy"; ?>. I also added a host header so that the health check will be handled by a specific nginx virtual host. The nginx vhost config has this in it:

location = /health_check.php {

 access_log		off;

 fastcgi_pass	127.0.0.1:9000;

 fastcgi_index	index.php;

 include	/etc/nginx/fastcgi_params;

}

And there you have it–a better HAProxy health check for dynamic websites.

5 Comments to A Better HAProxy Health Check For Dynamic Websites

1. 

   
   

   Powered by Evernote Publisher