Dougal Campbell's geek ramblings

WordPress, web development, and world domination.

W3 Total Cache Plugin

There’s a new WordPress cache plugin in town, and it’s called W3 Total Cache. This plugin is one of the contestants in the 2009 Weblog Tools Collection Plugin Competition.

The W3 Total Cache plugin (W3TC for short) is an advanced cache plugin, and you can only have one of those on your site at once. So if you’re currently using WP Super Cache, you’ll have to disable that first before enabling W3TC. [side note: I tried to find a good link for information about the advanced-cache functionality, but I came up empty. we need a Codex page for that]

If you’ve used WP Super Cache before, you’re probably wondering what’s different about W3 Total Cache? Everything. WP Super Cache is a static disk cache. It creates files  on disk with a certain lifespan, and gets the web server to use those instead of firing off PHP to generate pages on-the-fly, thus saving CPU and time. So, on the bright side, it saves your server a lot of work. On the down side, you lose some of the dynamic nature of your site. The entire page will remain cached as-is until the cached version expires, or until it is forced to regenerate (because you update the page, a comment is added, etc).

W3 Total Cache works differently. It utilizes multiple techniques to improve performance, including: object and query caching; page caching; HTML, CSS, and javascript minification; gzip compression; CDN (Content Distribution Network) support; and browser caching via ETags. Currently, the caching is based on APC and/or memcache. I’ve suggested that they add XCache, as well. The closest other plugin is probably Batcache, which also uses a memcached backend for caching.

Page caching is what WP Super Cache does. Except that W3TC uses your in-memory cache (APC or memcache). Memory being faster than disk, you’ll typically see an improvement when serving pages cached in this way. This does mean, however, that you want to have a decent amount of memory reserved for your cache. The W3TC docs suggest dedicating at least 128M with APC. If your server is strapped for RAM, or you can’t set up a dedicated memcache server, this plugin might not be for you.

Object and query caching means that the results of MySQL database queries are saved in the memory cache for later use. For example, when someone views your page normally, WordPress (or any other dynamic web application) will fetch things like your category list, your blogroll, and other information from the database, build the HTML page for the browser, and then promptly discard all of the database results when it finishes. The next time someone views the page, it runs all those same database queries again, even if nothing has changed, wasting CPU cycles and time. W3 Total Cache will save the results of these queries and other objects that WordPress builds in the memory cache for re-use.

Minification is the technique of stripping out unneeded information from your HTML, CSS, and javascript. This can include extra whitespace and comments. Depending on how your original files are formatted, this can result in fairly significant savings (probably 10-20% in many cases). The smaller the files, the faster they can be delivered to the browser. This technique also includes combining multiple CSS or javascript files into a single request. Normally, each CSS and javascript file linked to from your theme requires your browser to make a separate network connection to fetch it (a relatively slow operation). By combining these, the plugin reduces the number of network connections needed.

Gzip compression is a binary compression like you see with ZIP files. This is a more advanced method of converting a large file into a smaller one. Again, meaning you can serve it up to the browser faster. Delivering content to the browser as fast as possible is important, because the browser speed perception of the user can often determine whether or not they stick around to read that content.

Support for CDNs (Content Distribution Networks) means that you can host static files like images, CSS, and javascript on a seperate server. Typically, CDNs are optimized for static content. Some CDNs are also geographically distributed, and automatically route the requests to the nearest/fastest location. Even if you don’t use an external CDN (like Akamai, Amazon S3, or Voxel), you can set up your own server optimized for static content, and use that.

ETag support is currently supported in WordPress for feeds (I added that to the core myself, way-back-when). This is a feature of the HTTP specs designed to let servers and browsers coordinate caching. Basically, the first time a browser requests a page from a server, the server will generate an ‘ETag’ header with a unique identifier based on the last time the page was modified. The browser can save a local copy of the page in its cache, along with the ETag. The next time the browser requests the same page, it can send the ETag back to the server. If the ETag on the server-side hasn’t changed (the page hasn’t been modified), then the server returns a “304 Not Modified” response, which tells the browser to just display its cached copy of the page. W3 Total Cache extends this support to other WordPress-generated pages, not just feeds.

I’ve got the plugin running on this server currently. The only problem I’ve run into was that with the HTML minification “line break removal” option turned on, it caused my reinvigorate.net analytics script to fail for some reason. I haven’t fully explored all the options yet, and I don’t have as much memory allocated to memcached as I’d like, but it seems to be working pretty well so far. I should be able to judge more after it’s been running for a few days.

You can enable various optimizations independently of each other, so there is a lot of flexibility for utilizing various caching techniques while still allowing some things to remain dynamic. I’m curious to hear from others who try this plugin out, and see what they think about the balance between performance improvements and dynamic elements.

About Dougal Campbell

Dougal is a web developer, and a "Developer Emeritus" for the WordPress platform. When he's not coding PHP, Perl, CSS, JavaScript, or whatnot, he spends time with his wife, three children, a dog, and a cat in their Atlanta area home.
This entry was posted in WordPress and tagged , , , , , , , , , , , , , , , . Bookmark the permalink.

67 Responses to W3 Total Cache Plugin

  1. Thanks for the write up, I’m going to try this out later (read “much later” due to lack of time) myself but I’d love to see benchmarks with details about how the benchmarks are done.

    I hope they add a filesystem cache too so people on shared hosting accounts get to use it.

    Oh, and your site may be minified but the plugin adds a great big chunk of html comment text to the end of each page!

    I really must get around to writing that “Hacking wp-super-cache” article..

    • pundit says:

      That huge html comment was the first thing I noticed (and removed). The second thing I realised was that I was on a shared host and didn’t have either memcached or APC installed (and haven’t succeeded in installing either; and even if I do, I’m certain I’ll violate some TOS or another in running the daemon).

      I wish they add filesystem-based caching too. I like many of the things it provides, like DB caching, minifying (and consolidating) JS and CSS files and intelligent compression. It’s like they took every section of Yahoo!’s YSlow and addressed it in one monster plugin. (Which is ironic given this site uses the W3TC plugin and YSlow gives it an ‘E’ grade.)

  2. Andy Skelton says:

    How come no mention of the original memcached page cache, batcache? That’s what WordPress.com uses. I wrote it to mitigate the resource spikes seen during Apple events on live.gizmodo.com. It looks like W3TC has more features but they are based on the same idea. You might also mention that while wp-super-cache is brilliant with single-server setups, it doesn’t work with multiple servers—that’s another reason we use batcache instead. Then again, we may consider switching to W3TC now.

  3. BGR says:

    What should one ask Bluehost to see if one’s account and their servers support W3TC?

    During a Special Election I have comments and polls that are being used actively and would want that to remain dynamic so looking forward to seeing how flexible it is.

    • Dougal says:

      You need to find out if you have access to APC (Alternative PHP Cache) or memcached. If you are on shared hosting, the chances might be slim, though.

  4. dimo says:

    So, as I got it – some of this caching php-plugins are needed for that wordpress plugin to work?
    My host doesn’t have and doesn’t want to install Memcached, Xcache, etc.
    Or that is an independent plugin as wp-super-cache is?

    • Dougal says:

      Yes, for W3 Total Cache to work for you, your web server must have APC or memcached installed. These are completely outside of WordPress. APC is an extension to PHP, and memcached is a stand-alone service that runs on a host server (could be running on the web server, or a completely different machine).

      In a shared hosting environment without good jailing (a way of isolating one user’s environment from another’s), there’s no good way to secure the information in the caching services. I run on a Slicehost VPS (virtual private server), where I am able to administer my server as I please and install whatever software I need. However, I realize that many users don’t have the incentive to pay extra for that type of setup.

  5. Blake Helms says:

    How well does this work with WP-Touch or some of the other mobile themes. I know that was a problem with WP-Supercache for a while

  6. Pingback: Twitter Trackbacks for W3 Total Cache Plugin [gunters.org] on Topsy.com

  7. Pat says:

    I think I’ll stick with what works. Donnacha. Please do not stop developing your plug in.

    -Pat

  8. @Donncha O Caoimh,
    I don’t take donations, but you can support me with a link, which removes the excerpt. :) People can of course hack the plugin, but it’s not necessary imho considering the value provided.
    Apache bench shows at least 50% performance improvement in the tests I did. However, it’s difficult to compare apples to apples because this plugin does a number of different things. At the end of the day, nothing beats memory caching, that’s why I started there, coupled with the fact that for sites with frequent comments or a lot of dynamic functionality, there’s very little performance penalty to keep those sites updating in near real-time, while the rest of site remains heavily cached (including feeds, CSS, JS, etc etc).
    @Dougal,
    This plugin is ready for clustered hosting environments and shared hosting environments as well. So whether you have multiple WP installations on a single server or you use WP MU, all of your objects can only be identified by the WP instance that created them. So if someone like mediatemple.net created a “memcached container” this plugin would already be “safe” to for all the users on the grid server.
    Lots of this stuff is covered in the extensive FAQ in the plugin.
    Anyway, thanks for the coverage Dougal and look for numerous bug fixes and features shortly. I should hit version 1.0 (which will include disk caching for shared hosting-based bloggers) next month. :)

  9. Ozh says:

    There’s a critical security hole in W3TC right now (that the author is aware of). People should wait for next version before installing it.

  10. Viper007Bond says:

    Before I ran Squid as a reverse proxy (forced 5 minute caches), I’d use WP-Super-Cache and then cache the caching folder to RAM. Worked quite well. Apache ultimately ended up being the weak point though (5000+ simultaneous users plus countless more scraping your feeds tends to do that) which is why we opted to switch to using Squid (it serves everything directly from RAM without hitting Apache).

    Still though, I may have to check this out for my other not-quite-so-popular sites. :)

  11. inkey says:

    Your plugin would not work with custom wp-content/ directory.

    If in wp-config.php defined custom wp-content directory path int can not install and work. It’s sad.

  12. Brent Ozar says:

    Can you elaborate on how W3TC handles S3/CloudFront as a CDN? I only see an option for FTP in the CDN screens, and S3/CloudFront aren’t directly accessible via FTP, are they?

  13. @Brent Ozar, s3/CloudFront currently aren’t supported nor can you access them via FTP, but will be supported by v1.0. My goal was to support common CDNs out of the gate and allow those who did not want to use a CDN, to use a subdomain on their own server to improve the progressive render performance of their theme via browser pipelining. Please stay tuned.

  14. @Ozh, appreciate your input on the issue and it has been rectified. A new release with numerous other changes will be available in a few minutes.

  15. Just a note for anyone curious about the effects of caching your WP site with W3TC or WP Super Cache:

    Yesterday, when Ozh mentioned a security problem in W3TC, I disabled it, and I decided to leave my site with no caching in place, just to watch how the system stats changed. It didn’t take long for my server to start having problems. My monitoring system started alerting me about increased swap usage and high load averages. After about 90 minutes of that, I enabled WP Super Cache. This quieted the alerts as things settled back down.

    When I saw that W3 Total Cache 0.75 was out with a fix for the security problem, I disabled WP Super Cache, and re-enabled W3 Total Cache.

    The main difference I can see, system-wise, is that the database caching done by W3 Total Cache is a big boon to my system. The number of database queries drops by around 50%. I also see a slightly lower number of interrupts and context switches. The CPU usage also appears to drop a little bit (probably mainly as a result of the reduced MySQL queries), but this 4-core server spends most of its time with about 96% idle CPU, anyways.

    The point here is that both WP Super Cache and W3 Total Cache can help your system handle traffic better. And that the in-memory database query caching seems to have an extra bonus performance impact (the good kind), even beyond what the built-in MySQL query cache already gives you (if you have it enabled).

    [Don't know why this posted anonymously before. It was supposed to be me. --D]

  16. Smart Boy says:

    Will be looking into this caching system shortly. Looks very promising. Like how it will allow for a more dynamic viewing experience.

  17. @Dougal, the thing that’s really nice to see is how minify when fully used with a subdomain for your CDN improves your progressive render. Don’t forget to enable HTTP compression on your subdomain btw.

  18. Pingback: Twitter Trackbacks for W3 Total Cache Plugin [gunters.org] on Topsy.com

  19. Smirk says:

    Is CloudFront support planned anytime soon?

  20. W3TC sounds awesome. Based on the comments, though, I still see some compatibility problems that some people are experiencing. But still, this looks promising.

  21. @Smirk, yes s3 and cloudfront will be supported by version 1.0.

  22. Hey all, version 0.7.51 will be available in a few minutes! :)

  23. Version 0.7.51 available now!

  24. Mimi says:

    Please excuse my ignorance… but I’m on a shared hosting, and installed this plugin. I’m getting a yellow message that says :Memcached nor APC appear to be installed correctly

    But later it says enabled, in green. Is this working properly? I have zero knowledge about this kind of stuff (or how to check if they are working properly). But I loved the idea of using it, because my hosting is bugging me about CPU usage – and Super Cache is not an option for me, as I have a lot of stuff that need to be dynamic.

    Thanks a lot for your help, and please excuse my English :)
    Mimi

    • @Mimi, if you’re running the latest version the notification about APC or Memcached support will only disappear after you click the link to close it. At the moment, W3 Total Cache does not support other caching methods, only memory-based. Disk caching is coming in version 0.8.

      Why is it that super cache is not an option for you?

      • Mimi says:

        Hi! I know I can hide it, I just wasn’t sure if it was working or not.

        Super Cache is not an option because I run Ozh “Who Sees Ads” plugin to serve different ads per country. Using Super Cache invalidates Who Sees Ads (as is dynamic content) – and W3 Total Cache would be the ideal solution.

        I really am hoping to use it as soon as it will be available for shared hosting users :)

  25. Version 0.7.52 just released!

    • Thanks. I have already discussed the plugin with the author. Evidently the issue that caused him to see different results than Dougal (and other site owners currently using the plugin) is and I quote:

      “I then ran a PHP script to see if the memcached server was accepting connections, and it said connection refused.”

      I’m waiting for him to find the time to resolve his memcached installation issue and retest and have subsequently added a “test memcached” button to each of the tabs (in v0.7.5.2) to make sure that similar issues are easy to troubleshoot.

      • Smirk says:

        Glad to hear! My blog’s redisign is almost over and I’m going to use W3 Total Cache and really looking forward for S3/Cloudfront integration cause I’m willing to use that Amazon budget CDN for images and scripts. And there’s actually no decent Cloudfront plugin out there.

  26. Hey all! Version 0.8 is finally here with some exciting improvements:

    * Added disk as method for page caching
    * Added support for mirror (origin pull) content delivery networks
    * Added options to specify minify group policies per template
    * Added options for toggling inline CSS and JS minification to improve minify reliability
    * Added option to update Media Library attachment hostnames (when migrating domains etc)
    * Added “Empty Cache” buttons to respective tabs
    * Added additional file download fallback methods for minify
    * Improved cookie handling
    * Improved header handling
    * Improved reliability of Media Library import
    * “Don’t cache pages for logged in users” is now the default page cache setting
    * Fixed minify bug with RSS feeds
    * Fixed minify bug with rewriting of url() URI in CSS
    * Addressed more page cache invalidity cases
    * Addressed rare occurrence of PHP fatal errors when saving post or comments
    * Addressed CSS bug on wp-login.php
    * Addressed rare MySQL error when uploading attachments to Media Library
    * Modified plugin file/directory structure
    * Confirmed compatibility with varnish and squid

    • Hi, just installed the plug on my site and it seems to brake the RSS feed… is this a known issue? i tried to disable just RSS caching but the problem remains as long as the plug is active.

      this is the message -

      XML Parsing Error: no element found Location: http://feeds.feedburner.com/ronenbekerman

      cheers

      • Nice site. I just followed that link and the normal feedburner page loaded with the latest post. Can you duplicate the error somehow?

        • Well, i did deactivate it.

          One more odd thing i noticed is that if i wasn’t logged in to the site the site wouldn’t load, just show a white page and that’s all. I installed the plug a few hours ago and did notice that page views stopped for that time.

          I hope this helps, i really don’t want to enable it again if it stops the site from working – I’ll setup a second wordpress install ans see if i can recreate this issue.

          • This is not a known issue. What other plugins are you running, did you disable them all and see if you could duplicate? I presume you’re running the latest wordpress? You can find me at wordpressexperts [at] w3-edge [dot] come

  27. Mel Garage organizat says:

    What an excellent information. I cant resist my self from appreciating you. When I read it first time I got little bit confused. But for the second time I completely understand the benefits of this W3 Total Cache Plugin. I will surely implement the tips practically later. Any how, thanks for such beautiful post.

  28. Vu Condo Hotels says:

    Nice post! I just followed this link.After reading this post I have installed plug on my site, it is very easy to install. I didn’t understand about W3TC. Can you elaborate on how W3TC handles S3/CloudFront as a CDN.Thanks for sharing this information.

    • Currently W3TC doesn’t support S3/Cloudfront as I underestimated it’s popularity (because I considered it an over-priced solution). Integration with Amazon Web Services will be available in v1.0.

  29. Vu Condo Hotels says:

    Nice post! I just followed this link.After reading this post I have installed plug on my site, it is very easy to install. I didn’t understand about W3TC. Can you elaborate on how W3TC handles S3/CloudFront as a CDN.Thanks for sharing this information. I got good knowledge. Keep posting.

  30. Trace says:

    Thanks for the feedback on this plugin. I noticed you are back to using WP Super Cache, any reason why?

  31. Pingback: W3 Total Cache Plugin?W3TC? – ????

  32. Pingback: W3 Total Cache Plugin for Wordpress Eats WP Super Cache’s Lunch! – Human3rror

  33. Moumita Paul says:

    Hi,
    Is this plugin compatible with php speedy?Because when I use it,it renders a blank page of my website.please help me.

  34. Glenn Kilpatrick says:

    Hi guys,

    I get this error

    Plugin could not be activated because it triggered a fatal error.

    /var/www/vhosts/whitbyseaanglers.co.uk/httpdocs/newsite/wp-content/w3tc/dbcache could not be created, please run following command:
    chmod 777 /var/www/vhosts/whitbyseaanglers.co.uk/httpdocs/newsite/wp-content/w3tc

    However I do not have a file called httpdocs/newsite/wp-content/w3tc so I am unable to chmod it.

  35. Pingback: Wordpress + W3 Total Cache + MaxCDN - WordPress Republik

  36. Pingback: Keeping Up With Google: Speed a Factor in Page Rank | DigitalAppleJuice

  37. Pingback: Keeping Up With Google: Speed a Factor in Page Rank « madbadcat.org

  38. Shaun Snapp says:

    W3 also triggered a fatal error in my 3.0 installation. This is the most troublesome plug in I have  ever installed. De-installation instructions from Frederick are not correct, and he has no documentation for where files should go, so I can not re-install from FTP. Plug in has consumed days of my time, and Frederick just does not acknowledge that his software affects other parts of WordPress. Frederick’s entire demeanor is a concern. He writes cryptic messages and then disappears when a problem pops up. Chipper demeanor disguises a lot of fact covering. His website only shows positive responses. I should know, mine, which were not inflammatory were censored. Its like working with SAP. 

    • Matt Barba says:

      You don’t get it. You don’t need FTP.

      If you checked the forums, you would find that Frederick actually has some of the best free support I’ve ever seen and the countless testimonials state that. If Frederick didn’t reply to you it’s likely because he’s not at his desk or is working for one of his paying customers.

      I’ve been watching this project for some time, and I know for a fact that there used to be hundreds of comments on his site because many people failed to read the note that says “ask questions on the forums.”  Obviously he’s removed support questions from there because that’s not sustainable for community.Why does he do this?  Clearly so we can all benefit from the support he freely gives in a place where the community gathers.  Entertainingly, you seem to think that Frederick is somehow doing us a disservice by encouraging people to post critical feedback and support issues in the most public place possible.  Rather then trying to “disguise” and “fact cover”, Frederick obviously is trying to build a top plugin.  If you’re dissatisfied with its current implementation then HELP HIM.  W3TC is in the top 10 highest rated plugins out of 10,000+ available, so obviously you’re on your own here. You’re definitely not going to get what you want by bad mouthing him, and your certainly not promoting an environment where the rest of us want to freely give away our code to help your blog.

      I don’t think you realize, but Frederick is actually an executive at Mashable.com and doesn’t have lots of spare time.BTW, this site runs W3TC.

  39. Jason Floyd says:

    Any idea how to deal with the w3-total-cache-config.php file when working on multiple servers?

    Obviously I could put this on an NFS share and link to it during my capistrano release, but it seems hacky. Too bad w3tc doesn’t store settings in the DB instead I guess.

    Thanks

  40. Leang says:

    Does is still work, nowadays? It’s been a while… I get frustrated sometimes

Leave a Reply

%d bloggers like this: