We consult a high traffic site which gets about a million hits a day. We use scalr.net to autoscale the site , launch more servers during peak traffic hours, create mysql/percona read replicas automatically as and when needed etc. We do use memcache which takes the load off the mysql database. However, to make this setup sto scale higher and to reduce the load on the web server, and inturns reduce the number of servers we deploy, we decided to put varnish cache between nginx load balancer and frontend apache server.
Here are the graphs which show significant reduction in the resources, load, memory and bandwisth utilization on the web servers.
In conclusion, varnish is a very effective server side caching technology and can reduce your infrastructure from many high configured web servers to a small number ofmoderately configured servers. Our next step to optimize this infrastructure would be moving away from resource intensive Apache web server to lightweight and high performance alternatives such as nginx or cherokee.
