Dougal Campbell's geek ramblings

WordPress, web development, and world domination.

Server Reconfig

This blog will be moving to a new server very soon. If all goes well, you shouldn’t notice a thing. But just in case, I figured I’d give a warning, so that if you try to visit and you get an error, or the site won’t come up, you’ll know to just come back again later instead of thinking that my site was an early victim of the 2012 apocalypse or something.

I’m doing more than just moving onto a new server though. I’m moving to a new hosting provider, and I’m reconfiguring many aspects of how my services are set up. If you’re interested in the technical nitty-gritty, read on.


Some of you might recall that a little over a year ago, I moved from my original hosting onto Slicehost. I am now moving again, to a service called prgmr.com. I’ve had no problems with Slicehost, but the pricing for prgmr.com is lower, which made it possible for me to expand from 2 servers to 4 servers for less money.

Slicehost VPS hosting is pretty no-frills. You get a server with an initial OS installed on it, and a web-based management tool with some DNS tools and a Java applet console. With prgmr.com, it’s even more no-frills, because the console isn’t web-based. It’s SSH all the way, baby. It’s not for everybody, but it suits me fine. And even though I have more servers, it reduces my monthly hosting costs by about 40%. That said, I’m also reducing my virtual CPU power and disk space a little bit, and Slicehost has multiple facility locations, which might be factors for someone else shopping for hosting. For me, the ability to afford more servers was pretty high up.

So, that said, let’s look at what I’m changing in my setup, and why I felt it was important to move up to 4 servers.

All along, even before Slicehost, I have had trouble dealing with traffic spikes on my web server. I’ve used WP Super Cache and W3 Total Cache. I’ve used XCache for PHP code caching, and for my WordPress object cache. I’ve configured Apache to set caching headers for static files using mod_expires to reduce requests. But I still get load and memory spikes that grind my server into dust. That shouldn’t happen with only 200 visitors per hour (granted, a page view can generate about 50 HTTP requests). Obviously, I’ve configured my server poorly.

Like many large PHP applications, WordPress can be a bit hungry for memory. So when it fights against MySQL and an in-memory object cache for resources, things can start to get dicey. And of course, I’ve got a few other things on the server. When I get hit by a traffic spike (popular article, spammer runs, errant search spiders, etc), memory goes away fast. The machine starts to swap, things get slow, load average spikes as processes begin to wait for resources, and it all snowballs. I’ve got some homemade scripts that keep an eye on things and attempt to restart various services in order to force things back into line, but it’s a pretty heavy-handed way to deal with the problem.

In my new setup, here’s what I’m doing to fix it:

  • I’ll be running Apache, MySQL, and Memcached all on separate servers, instead of together on one host.
  • I’m switching from the Apache pre-forking model to the threaded worker model.
  • I’m switching from mod_php to FastCGI (mod_fcgid) and php-cgi.

There will probably be other tweaks, as well, but those are the biggies. I’m expecting this new setup to handle waaaay more requests than the old one. Oh, and I’m definitely open to any pointers from performance tuning gurus. Please share links and tips!

When the switch-over happens, there might be a period of transition while the DNS changes propagate. I don’t plan to post again until I’ve moved this blog fully to the new host. So you’ll know it’s happened when a new article appears here.

About Dougal Campbell

Dougal is a web developer, and a "Developer Emeritus" for the WordPress platform. When he's not coding PHP, Perl, CSS, JavaScript, or whatnot, he spends time with his wife, three children, a dog, and a cat in their Atlanta area home.
This entry was posted in WordPress and tagged , , , , , , , , , , , , , , . Bookmark the permalink.

19 Responses to Server Reconfig

  1. Steve says:

    What sized slice are you using?

    I have been having similar problems with the 512mb slice. mod_php uses up far to much memory.

    I am currently experimenting with Nginx and PHP-FPM (the php-fcgi successor) and hope to switch over to that.

    I think one major problem you have is the huge amount of HTTP requests needed per page view. You have a horde of js files that need to be requested. I suggest concat’ing them into a single JS file. The same with CSS files.

    Host the images, js and css on another server, I use Amazon S3 for this. mod_rewrite can easily do this without having to do any code changes.

    Chcek your logs to see how many 404 requests you get. 404′s are not cached with wp-super-cache and can really harm performance. You can decrease a performace hit by messing with your themes 404 page.

    Three VPS’s seems overkill!

    Good luck. I look forward to hearing how it all works out for you. I am constantly struggling with performance.

    • Dougal says:

      Yes, the numerous little js, css, and image files add up to a lot of requests. W3 Total Cache alleviates this some (you can configure it to combine js and css requests), so I’ll probably be trying it again in the new setup. I may also experiment with moving requests for static files to a different server, either one of my own, Amazon S3, or something similar.

      I don’t think that 3 servers is going to be overkill. Keep in mind that I run several other sites besides my blog on the server. Mostly some low-traffic WordPress sites, but some other projects, like Twitual, as well. Twitual is where a lot of the memcached usage will come in. On the other hand, when all is said and done, it might be that 1024MB is overkill for my Apache usage. That would be a nice problem to have, for a change.

  2. jcwinnie says:

    I too would be interested in knowing what the final config comes to be. Will you still tweak the MySQL server?

    • Dougal says:

      I’ll try to give more details later, once the move is complete. And yes, I’ll be tweaking the MySQL server. I’ve scaled back a lot of the buffers and caching right now, but on the new 512MB dedicated server, I was able to give it a lot more breathing room. Once I have real traffic on it, I’ll be able to tune it more.

      I don’t claim to be an expert MySQL DBA, but I own a copy of the High Performance MySQL book, and I’m pretty good with Google ;)

  3. Im not so sure about using yet another plugin on my installation of wpmu/bp but I would like to have a solid performance thats consistent.. Do you think that off loading mysql would give me a good gain. without needing any plugins..

    just to clarify my site is running fine.. I do have a lot of stuff on the main page. I’d rather do the offloading of mysql to another vps now before theres a problem.

  4. Ashfame says:

    Have you tried Nginx instead of Apache? Threaded model may seem to do the job but I am not sure if it will work well or better than Nginx. I am just not a geek into server administrations.

  5. I just wonder what’s wrong with your server that such a small amount of traffic will cause serious spikes. I have loads more traffic than you do, and never encounter such problems on my single dedicated server. I do not use server-side PHP caching utility, because it causes more problems than it solves with some apps. I do use WP Super Cache and performance is fine.

    Peace,
    Gene

  6. James D Kirk says:

    I’m in line with Steve and Ashfame, Dougal. Some time ago I got into Slice (as I recall, I think it was because of you that I did. Ironic!) and have since increased from the 256K to 512K, but not because I was having challenges, but anticipated business growth. It was during that upgrade that I opted to look into something that might be better, more efficient than the standard LAMP set up. While it took some quick and deep learning, I was able to nicely configure Nginx, MySQL, and FastCGI. I also switched over from the stand alone WordPress installations to using only WordPress Multi User. I think currently there are about 6 MU installs with I’m not even sure how many domain and subdomain sites being served.

    Don’t get me wrong, I still don’t have tons of traffic, but all is running on that one slice, along with a domain that I use to serve out all images and other static files (for my various personal sites on the slice.) The one thing I can say with assuredness is that immediately after I got this set up going, I noticed a difference; both in the effective time pages were delivered to the end user, and more importantly to me as an administrator, the amount of time it took for back end admin pages to refresh in the browser.

    Oh, I should add that I am also running WP Super Cache (with the htaccess files reworked for Nginx) and that helps tremendously for the front end of the site.

    Don’t get me wrong, I’d love to “need” to have 3 or 4 servers. Maybe someday!

    P.S. Argh. Can’t even type this early in the a.m. my site link was supposed to be BoldlyGoing.com. Go figure! JDK.

  7. Andrew says:

    Why dont use nginx+php_cgi option? I use this with xcache and my blog is superfast.

    Example: 14 queries. 0.136 seconds.

    P.S Your comment-plugin parse “<!–" construction. Not so good, yes?

  8. Just FYI: We get .325 sec performance on our main blog with over 32 WordPress plugins. I don’t think our audience would detect a tenth of a second or two either way. But I’d be curious to see how/if nginx integrates with cPanel. It’s not officially supported.

    Peace,
    Genr

  9. Why not use wp-minify to minify your css and js files? Then you can use WP Super Cache and get the benefit of static caching!

  10. Viper007Bond says:

    I’m surprised you need so many servers. One my WordPress installs was handling a few thousand simultaneous users on a single (but I guess beefy) server with only WP-Super-Cache installed. Static content (theme images, CSS) is served off a nginx static subdomain.

    We did end up having to install a Squid reverse proxy though as Apache was grinding to a halt (mainly from mod_rewrite doing things like redirects for our feeds and stuff like that). We opted for that over nginx as the site has no dynamic content (no comments or anything) so it’s not a problem if a URL is cached for 5 minutes.

    Regardless though, grats on the move! :D

  11. Rob Scott says:

    Hi,

    Just wondering why you need 4 servers to take 200 visits an hour? Is this just WordPress, or are there other applications on there too?

    I recommend Hyper Cache for traffic spikes too – we serve in excess of 30,000 hits a day on a site with 13,000 posts reasonably well on $300 a month server using wordpress and hyper cache pretty easily.

    The most important thing I have found is isolating and improving or removing plugins which eat resources – including WP Supercache which I found tended to leak memory inder heavy load but this was a few versions ago, so may no longer be relevant.

  12. Rob Scott says:

    Oh and using fastcgi for php will lower your memory footprint significantly – don’t use apache for this, and you’ll notice some kind of improvements (the learning curve is a bit of a bind mind you!).

  13. pshah says:

    I use nginx + phpcgi and it did the trick for me on my website: http://priteshjshah.com/
    It uses a lot less memory than apache. Not sure it would work for you :) but worth a try.

  14. angelina says:

    Thank’s for the advice!
    angelina

  15. Nerd says:

    Before you jump into splitting your site into 4 separate servers let me help you configure what you currently have. You only need one server. I’ll explain when we chat. Perhaps through Google Talk, or email, whichever you prefer. I have one blog that withstood 35 million hits in a month on one server.

    I use your object-cache file to store database objects in Xcache, and I really appreciate your work, so I don’t mind giving you a hand to leverage your own work to save you a lot of time, and a big server management headache.

  16. I moved to nginx over a year ago, and it’s absolutely without doubt the best move I ever made. Apache has become a total dog, and I’m continuously surprised that anyone continues to use it if they have a choice. Read it and weep:

    F UID PID PPID PRI NI VSZ RSS WCHAN STAT TTY TIME COMMAND
    0 101 2541 1 15 0 152780 5280 wait Ss ? 0:00 /usr/bin/php-cgi
    1 101 9656 2541 15 0 179032 30520 stext S ? 0:45 /usr/bin/php-cgi
    1 101 9818 2541 15 0 179460 30964 stext S ? 0:39 /usr/bin/php-cgi
    1 101 9990 2541 15 0 185612 36532 stext S ? 0:38 /usr/bin/php-cgi
    1 101 10255 2541 15 0 179528 31020 stext S ? 0:25 /usr/bin/php-cgi
    1 101 10968 2541 15 0 178884 30264 stext S ? 0:06 /usr/bin/php-cgi
    5 0 2565 1 18 0 59540 3500 rt_sig Ss ? 0:00 nginx: master process /usr/sbin/nginx -c /etc/nginx/nginx.conf
    5 101 2566 2565 15 0 59540 5264 stext S ? 0:38 nginx: worker process
    5 101 2567 2565 15 0 59540 5364 stext S ? 0:38 nginx: worker process
    5 101 2568 2565 15 0 59540 5440 stext S ? 0:44 nginx: worker process
    5 101 2570 2565 15 0 59540 5312 stext S ? 0:43 nginx: worker process

  17. pshah says:

    Not sure if it helps someone, but I recently signed up for a cheap VPS here http://fsckvps.com/
    They start at $9.95/month for 512MB, pretty darn cheap and I’ve read good reviews about them. I’m personally really happy with http://rackspacecloud.com and have my website hosted with them.

Leave a Reply

%d bloggers like this: