The Digg Effect Analyzed

What is Digg?

Digg is a user driven social content website. Everything on digg is submitted by the digg user community, and after that other digg users read submissions and digg what they like best. When a story receives enough diggs, it is promoted to the front page for the millions of digg visitors to see.

The obvious effect on a website after a front page digg article is the huge increase in visitors and traffic. All those digg readers can saturate the site completely, causing the site to buckle under the strain. The Digg Effect is the term given to this phenomenon.

What's your favorite file system on Linux?

Few days ago, I was looking at the results of a poll on this site. With measly 250 votes collected by then, I had a feeling that it's results just couldn't be trusted. I needed more votes! And then it strucked me, it would be great if all those digg readers spent just a few seconds making their vote. With such many votes the poll would become much more credible. Of course, with digg, the hard part is getting a story to the front page, but with a little promise to the readers that it wouldn't take them more than a second to vote (and see results immediately) it all worked out very well.

So, if by any chance you too wanted to know which file systems on Linux are the most popular, I think we now have a pretty good picture:

As I don't see a good reason to close the poll yet, you can still vote here, thank you. :)
I'll also take this opportunity to thank all the digg readers who participated!

The Digg Effect analyzed

What follows is a short analysis of the Digg Effect. I'm aware that there are already lots of articles like this, but because I enjoyed reading all of them, I suppose someone will find this one useful, too.

Let's start with the screenshot taken from the Google Analytics tool:

I don't think there's much to add after looking at the picture, it's quite obvious that after the story got promoted to the front page, almost 90% of visitors to the site were coming from digg.com. The heavy traffic lasted for about 5 hours. Looking at the logs, that period closely matches the time that the story spent on the front page. After that the number of visitors started decreasing.

Here's what venerable webalizer says for this month:

As many people have already observed, having a story published on digg.com quickly burns your bandwidth, and in comparison all your previous traffic looks like there was none. :) On the other hand, after about one day everything is back to normal.

Finally, here's a nice report, courtesy duggtrends.com, which shows the history of the submission (number of diggs through time):

So, it took something like 18 hours for the story to get enough diggs (around 50), before it got promoted to the front page. And the rest is history.

The Drupal

This site is based on the excellent Drupal CMS. Now, I have a confession to make. I actually haven't been prepared to be announced on a such high profile site like digg.com. Meaning, I didn't have caching enabled, I didn't have throttling mechanism configured, in fact, I haven't done anything to optimize this site for such a high traffic. Also, it is hosted on a shared hosting server, so at the first moment when I noticed large number of visitors, I was thinking "Oh, no, everything is going to collapse!".

But, after some investigation I noticed that not only everything was in order, but also the site was very responsive, like nothing was going on. Needless to say, I'm very impressed with all I've seen. I can only recommend Drupal to all of you that are still in search of your holy grail CMS, and not only because of Drupal's great performance.

For a patient reader, here's yet another graph, taken from the useful Drupal graphstat module:

And finally the last graph is from Alexa:

More information

Typically, when story hits the front page other bloggers find it interesting and cover it on their own pages. I managed to find one site, which is interesting because you can see how the poll proceeded (they took the snapshot of the results at the time they were posting their story):

http://alternaria.blogspot.com/2006/09/ext3-se-lleva-todos-los-votos.html

If you're interested in more information regarding the Digg Effect, there's a great article that explains some less known but completely valid facts about it:

Digg Effect: The Top 10 Things Webmasters Should Know.

And remember, file systems are mostly about reliability, but their performance is close second important parameter!

Have fun!

Comment viewing options

Select your preferred way to display the comments and click "Save settings" to activate your changes.

Hosting

Not to take any glory away from the Drupal coders, but you must have a really good shared hosting provider. They are obviously not doing any overselling. Mind sharing the name of your hosting?

What Drupal Modules Are You Using

You said you survived the Digg without a problem (or caching) - what modules do you have "on" and have you made any other changes to the initial install? That'd be nice to know too - thanks!

List of modules used on this site

Sure, no problem.

Speaking of built-in modules, everything is "on" except: contact, image_gallery, legacy, locale, throttle.

I also have the following additional modules installed and enabled: adsense, google_analytics, graphstat, gsitemap, image, img_assist, nodewords, robotstxt.

Hope it helps!

Encouraging

Well, it was encouraging to find this article. I'm using Drupal for a couple of projects and I am a bit worried about how they will perform once the traffic builds. You never mentioned who your hosting provider is. Would you mind sharing that info, if not publicly in a private email? Thanks!

Hosting provider

I could reveal hosting provider, but it wouldn't be helpful because we are speaking of a local provider in Croatia. So it is not one of the popular ones.

OTOH, it's true that the server wasn't overbooked at the time. Any hosting provider that is not overselling it's resouces should be fine.

These days I switched the caching on because on my tests it shows improvement in performance. We'll see how it works.

The Digg effect?

It's but a young upstart compared to a good slashdotting.