When Your Site Blows Up

February 2nd, 200725 Comments

Paris Hilton had a big week last week. So did my company’s site PopCrunch.

PopCrunch was averaging about 2000-3000 unique visitors per day. Then we hired a full time writer (a great move by the way and evidence that it works to invest a lot into a single site rather than many). Then came ParisExposed.

Traffic soared to more than 20,000 uniques per day last week. AdSense income was (and is) through the roof (it actually scared me because I thought there was some weird click fraud going on at first). The bad news? We had to move the site between three different servers in the course of 2 days. Each move we were told that the site was just pulling in too much traffic and was breaking the httpd process (even our MidPhase VPS broke).

We finally moved to a high-end cluster server with a company called Mosso that has been nothing short of phenomenal (I’m not being paid to tell you this, btw). Sure, it costs $100/month, but let me tell you that the peace of mind that I have with these guys is just priceless.

Anyway, at first I wasn’t sure what to think about paying $100 for a single site server. What if the site traffic died off? What if this is all because of a short term phenomenon?

Tough questions to deal with, but once a site blows up, you’ve really got to make it stay blown up. You’ve got to work your ass off to make sure it doesn’t fall back to a pre-blow-up state. You’ve got to take the momentum and run with it. Invest in the site’s success by leveraging the momentum and working it in your long term advantage. Diversify your content so that you’re not dependent on a single search term. Just get out there and start working and don’t look back for a few days. These opportunities only come once in a while. Make sure that instead of one-time blow-up, you turn it into long-term blow up. Make it last. But also make sure to invest in the infrastructure to maximize your success, even if it seems a little risky.

In the end, when you’re site blows up, you’ll probably get mucho exhausto, but, dude, it’s just plain exhilarating.

Related Articles:

Related Ads:

When Your Site Blows Up

Nate W. | February 2, 2007

Congratulations on the blow up! That is an excellent amount of traffic and I am sure Google rewarded you nicely! See…Google ain’t so bad! ;)

Tim Linden | February 2, 2007

$100 for a server?? I’m paying close to $400 for mine. Just think about how much more traffic and income you could have made if the site was 100% every time someone visited. That should be worth a measly $100 ;-)

Tim Linden | February 2, 2007

Ok so I spent the last hour or so reading about Mosso. The only thing for me is the cron jobs which I’m told they don’t support yet. However once they do I’m seriously considering it ;-)

Ryan | February 2, 2007

Tim, the other thing to consider is that there is still no good server side stat solution for this technology. You’ll need to use something like Google Analytics.

Ryan | February 2, 2007

NateW. I’ve never said Google is bad.

HART (1-800-HART) | February 3, 2007

Congrats! That’s been one of my biggest fears .. bandwidth - in case something does happen. If you don’t mind me asking .. just how much bandwidth is 20,000 uniques per day use?

Ryan | February 3, 2007

Hart, interestingly enough, the issue with the servers had nothing to do with bandwidth. If you notice, a lot of these hosting companies give you “unlimited bandwidth” or virtually unlimited bandwidth. In fact, PopCrunch had used less than 1% of its alloted monthly bandwidth.

Rather, the real issue that servers don’t publicize, is number of requests per second to the httpd process. So the only metric that really matters is the only metric that shared hosts don’t advertise or guarantee. At peak times of day, PopCrunch was receiving 20+ requests per second and this was putting a load on the servers that was either breaking the httpd process or bringing it to a crawl.

PopCrunch was effectively getting the traffic of a Digg without Digg. And as you probably know, “the Digg effect” is the affectionate term that Digg users have for the phenomenon of seeing a server crash simply in virtue of landing on the front page of Digg.

HART (1-800-HART) | February 3, 2007

That’s informative and good to know. Except a few of my larger sites (that will survive the loads) most of my other sites on my reseller account were created with just 0.5G bandwidth per month. I’ve figured, I’d just manually reallocate if I got caught from the spare or from others .. but, never thought of the actual request per second issue. We all should know that type of stuff about our servers.

Ryan | February 3, 2007

Hart,
If you need lots of cheap bandwidth (250GB/month for $6.95), I’d recommend using Colorteck. I’ve used them in the past and have had good success (much better than Dreamhost). Plus it’s hard to use all the bandwidth they provide.

http://colorteck.com/hosting-plans.html

HART (1-800-HART) | February 3, 2007

That’s not bad. Unfortunately, my reseller account has 200-G bandwidth combined limit for all my stuff. But, for starting out it’s great to be able to create almost unlimited accounts when you feel the urge. When I run into the bandwidth problem .. I’ll be moving over to Globat.com on a dedicated server (1000-G for $6.95). But, will still bookmark Colorteck :)

Lara | February 5, 2007

Haha, I had this problem when I ran a little graphic “dolls” website back when I was 12. [Talk about young entrepreneur ;) ] I simply couldn’t afford a dedicated server, so even after paying $30/month my site would go down for a month or my parents would kill me with those extra bandwidth fees.

Sadly… I don’t have that problem anymore. Haha, but keep up the good work! I’m a sucker for celeb blogs.

Tim Linden | February 13, 2007

I’ve been hearing reports that there was tons of downtime. Any input?

Ryan | February 13, 2007

Downtime with what? Mosso? I haven’t noticed any downtime.

On the other hand, if you mean that there was lots of downtime when PopCrunch initially blew up, then the answer is an emphatic yes.

S-W | February 14, 2007

Wow, kudos for you.

I have also moved to Mosso, and I’m getting my ready to launch. However, the Cron Job issue is a bit of a headache as Tim says.

My main motivations for going with Mosso is what you’ve said Ryan, to have a host that can handle spikes in the load without a major crash.

Question for you Ryan, if I may: with 20,000 uniques (20+ requests per second), did you have any problems with Mosso’s database?

Your sites are blogs, so I would think that the database load is more critical unless your pages are mostly cached instead of generated on each page request.

When more vistors post comments, do you see the DB could be a bigger issue in the future?

Anyway, great work and thanks for sharing– I have a new celebrity blog to add to my list :-)

Ryan | February 14, 2007

Honestly, I don’t know how they handle database load, nor how databases are stored across their cluster, but I do know that after moving to the Mosso server the site became stable even as traffic continued to climb. I’ve always been under the impression that database load issues piggyback on top of server resources (memory + CPU). So the various processes such as httpd, mysqld, clamd etc. all have the capacity to overwhelm server load.

BTW, traffic and income at PopCrunch have fallen off a bit (largely because the Paris Exposed story has fallen off a bit), but I’m still glad we made the move, as I fully expect for the site to continue to grow.

S-W | February 14, 2007

I suspect their database servers a separate set of clusters. However, I don’t think anyone knows if they can expand that cluster as easily as their web servers.

The other concern I had with most shared hosts (clustered or not) is that too much load would simply get you cut off.

But I read elsewhere than there are sites with millions of hits a month at Mosso, without any interference from Mosso. It seems like you went well above a million hits per month in that paris hilton popularity period, so Mosso did well to give you all the resources you needed.

These are the type of stories and feedback that makes me give Mosso a good chance even with some headaches like no cron jobs. Because in the end, it’s about having your hold up to traffic and always available.

Ryan | February 14, 2007

Here’s an easy solution for the cron job situation:

Setup a cron job on an ultra cheap hosting account ($4.99/month) with a PHP file containing something like the following code (say I wanted to run a script on PopCrunch):

< ?php
$urlGet="http://www.popcrunch.com/getredditdata.php";
$redditRaw=file_get_contents($urlGet);
?>

Ryan | February 14, 2007

Not sure if the above makes sense, but you basically cron a file on a different server that calls a file on the Mosso server.

Tim Linden | February 15, 2007

Didn’t think of it till now, but did a quick search and there are some sites that allow you to remote cron job. Basically what Ryan’s code does, but you don’t hvae to get the hosting account.

I’m assuming Popcrunch is wordpress, right? I’ve thought about trying it out to host my blogs. Not really neccessary though as my current server is using about 5%, but that is so it can handle the spikes.

NightShift58 | February 28, 2007

You don’t need to save providers to have access to CRON. There are a number of free or low-priced hosted CRON services available, for example:

http://www.hostedcron.com
http://www.webbasedcron.com
http://www.remote-cron.com

Tim Linden | February 28, 2007

So I tried Mosso for a week and am asking for a refund. I don’t know, maybe I joined at the worst week possible? Every time I tried to add a new site, it would get stuck in the process. IE it setup the site’s MySQL database and I imported it all, but it still didn’t setup the HTTP or FTP part. I also noticed it has a long load time. My pages were delayed 5 seconds, Pop Crunch loaded for me in 18 seconds.. On broadband.

Ryan | February 28, 2007

Tim,
Sorry to hear of your trouble. Admittedly, after the move, I had some trouble with adding other sites. That seems to have been resolved though and I’ve added three sites just today without problem.

Since moving to Mosso, I’ve set up tv.popcrunch.com and hiphop.popcrunch.com without any problem whatsoever.

PopCrunch is not loading in 18 seconds (unless there is a problem with one of the 3rd party javascripts running like blogads).

And the great news is that PopCrunch hasn’t crashed once since moving it over to Mosso, despite massive peaks of traffic (tonight around 8pm expect another MASSIVE peak because of the Antonella Barba controversy).

So while Mosso isn’t perfect, I do have to say that they’ve managed to keep PopCrunch up consistently and smoothly and that my profits are skyrocketing because of that (Two straight days where my AdSense revenue from just one site, on each day, covered the entire month’s server cost.)

Tim Linden | February 28, 2007

Hmm. Well I’ll wait and see what support says. If they convince me thinks will get better then maybe I’ll stay.

S-W | February 28, 2007

Tim, I am frustrated there as well. But I’m holding out hope for better times. Won’t go into the gory details, but I fear that my neurons will get burned out dealing with Mosso server quirks.

If you run a simple Wordpress installation, then no problems. Anything more sophisticated, you will have issues.

HART-Empire.com » MAN .. I suck at CSS *sniffles* (HART-EMPIRE NETWORK) | March 2, 2007

[…] I’ve created a base template file that contains only the network sites. I would like to create a way to GET THAT SCRIPT from some central location so I can update itself if there is a change. I have yet to figure it out, or get that working. I felt hopeful .. when I actually recalled Ryan of College-Startup casually suggesting an intriguing PHP statement in the comments that was very intriguing that I plan to investigate further. […]

Share your thoughts!!!

performancing hive

When Your Site Blows Up was written by Ryan on February 2nd, 2007 at 1:32 am and posted in Blogging

  • Our Sponsors

Debt Help UK
Are you in need of debt help in the UK? Well why don't you find out to regain control of your finances? Simply visit The Debt Clinic.

Loan
Need a Loan? At All About Loans our friendly and courteous service will ease the process for you.

best buy savings
Look for the best buy, also when comparing savings accounts.

Blog Stats