July 2008 Archives

As I mentioned yesterday, the FreeBSD 7 Release doesn't support Broadcom 5722 network adapter which is combined in Dell R300, today, I installed 7.0 Stable 200807 amd64 on Dell R300, and the last stable version can identify the 5722 network adapter.

Tomorrow, this box as the load balance which is running Nginx will be put on rack for production use. Crossing my fingers.
I planed to install FreeBSD 7.0 Release on Dell R300, but unfortunately, 7.0 Release doesn't support the network adapter Broadcom 5722 which is combined in Dell R300.

Some of my friends said that 7.0 stable in snapshot can work with 5722, but I'm not sure if it's stable enough to support the production environment, so finally, I'll have to replace it with Dell 1950 which combines Broadcom 5708C :(

Web cache is very important to a high traffic site, by holding those pictures, css and js files those are not changed frequently, it can reduce pressure to the backend servers, it's very helpful to improve the site speed.

As you may know, Squid is widely used for web cache service, but we have another choice, that is Varnish. Here is the introduction about Varnish from it's offical site:

"Varnish is a state-of-the-art, high-performance HTTP accelerator. Varnish is targeted primarily at the FreeBSD 6/7 and Linux 2.6 platforms, and takes full advantage of the virtual memory system and advanced I/O features offered by these operating systems."

Some of my friends have already implemented Varnish as their web cache server in the production platform, and most of them replaced Squid with Varnish, they told me that Varnish has better performance than Squid in the high traffic site.

So I wanna install a web cache server for my site, although my site traffic is pretty low :)

Firstly, I'd like show you the basic workflow about how web cache works with web server.


 
varnish_workflow.jpg


Ok, let's start to install Varnish, the last stable version of Varnish is 1.1.2, you may download the source code from here. My current web server is running on FreeBSD, Apache, PHP and MySQL, and Varnish will be installed on RedHat Linux.

Installation
1. Createing group and user 'www' for Varnish
# groupadd www
# useradd www -g www

2. Compiling and installing Varnish
# cd varnish-1.1.2
# ./configure --prefix=/home/varnish
# make && make install

3. Creating cache file and log file folder
# mkdir -p /data02/cachefile
# chown -R www.www /data02/cachefile
# mkdir -p /data02/cachelog
# chown -R www.www /data02/cachelog

4. Varnish configuration file
Example from my site:

backend mywebserver {
       set backend.host = "YOUR_WEB_SERVER_IP";
       set backend.port = "80";
}

acl purge {
       "localhost";
       "127.0.0.1";
       "192.168.1.0"/24;
}

sub vcl_recv {
       if (req.request == "PURGE") {
               if (!client.ip ~ purge) {
                       error 405 "Not allowed.";
               }
               lookup;
       }

       if (req.http.host ~ "^www.isoracle.com") {
               set req.backend = mywebserver;
               if (req.request != "GET" && req.request != "HEAD") {
                       pipe;
               }
                 elseif(req.url ~ "\.(php|cgi|pl)($|\?)") {
                       pass;
                 }
               else {
                       lookup;
               }
       }
       else {
               error 404 "isoracle.com Cache Server";
               lookup;
       }
}

sub vcl_hit {
       if (req.request == "PURGE") {
               set obj.ttl = 0s;
               error 200 "Purged.";
       }
}

sub vcl_miss {
       if (req.request == "PURGE") {
               error 404 "Not in cache.";
       }
}

sub vcl_fetch {
       if (req.request == "GET" && req.url ~ "\.(txt|js)$") {
               set obj.ttl = 3600s;
       }
       else {
               set obj.ttl = 10d;
       }
}


For more detailed description about the Varnish configuration file, please refer to the offical document.

5. Starting varnishd
# /home/varnish/sbin/varnishd -n /data02/cachefile -f /home/varnish/vcl.conf -a 0.0.0.0:80 -s file,/data02/cachefile/cache.dat,500M -g www -u www -w 50,200,10 -T 127.0.0.1:1800 -p client_http11=on

For more options about starting up varnishd, please refer to "varnishd --help".

6. Starting varnishncsa
# /home/varnish/bin/varnishncsa -n /data02/cachefile -w /data02/cachelog/varnish.log &

7. Checking Varnish status
# /home/varnish/bin/varnishstat
This command will show you the current Varnish status, such as cache hints, cache misses etc, they're useful for health check.

Verification
We can use curl to check the HTTP header information:

# curl --head http://www.isoracle.com/
HTTP/1.1 200 OK
Server: isoracle/Stable 1 (Unix) PHP/5.2.0
Last-Modified: Sat, 12 Jul 2008 21:54:39 GMT
ETag: "96347-cc05-4879281f"
Content-Type: text/html
Content-Length: 52229
Date: Sun, 13 Jul 2008 01:36:12 GMT
X-Varnish: 1652882410 1652882077
Age: 8877
Via: 1.1 varnish
Connection: keep-alive

We can see that Varnish works now.

Finally, you need change your DNS record to point the web server IP to cache server IP to let the requests hint the cache server.