The Digg Effect Analyzed
What is Digg?
Digg is a user driven social content website. Everything on digg is submitted by the digg user community, and after that other digg users read submissions and digg what they like best. When a story receives enough diggs, it is promoted to the front page for the millions of digg visitors to see.
The obvious effect on a website after a front page digg article is the huge increase in visitors and traffic. All those digg readers can saturate the site completely, causing the site to buckle under the strain. The Digg Effect is the term given to this phenomenon.
What's your favorite file system on Linux?
Few days ago, I was looking at the results of a poll on this site. With measly 250 votes collected by then, I had a feeling that it's results just couldn't be trusted. I needed more votes! And then it strucked me, it would be great if all those digg readers spent just a few seconds making their vote. With such many votes the poll would become much more credible. Of course, with digg, the hard part is getting a story to the front page, but with a little promise to the readers that it wouldn't take them more than a second to vote (and see results immediately) it all worked out very well.
So, if by any chance you too wanted to know which file systems on Linux are the most popular, I think we now have a pretty good picture:
As I don't see a good reason to close the poll yet, you can still vote here, thank you. :)
I'll also take this opportunity to thank all the digg readers who participated!
The Digg Effect analyzed
What follows is a short analysis of the Digg Effect. I'm aware that there are already lots of articles like this, but because I enjoyed reading all of them, I suppose someone will find this one useful, too.
Let's start with the screenshot taken from the Google Analytics tool:
I don't think there's much to add after looking at the picture, it's quite obvious that after the story got promoted to the front page, almost 90% of visitors to the site were coming from digg.com. The heavy traffic lasted for about 5 hours. Looking at the logs, that period closely matches the time that the story spent on the front page. After that the number of visitors started decreasing.
Here's what venerable webalizer says for this month:
As many people have already observed, having a story published on digg.com quickly burns your bandwidth, and in comparison all your previous traffic looks like there was none. :) On the other hand, after about one day everything is back to normal.
Finally, here's a nice report, courtesy duggtrends.com, which shows the history of the submission (number of diggs through time):
So, it took something like 18 hours for the story to get enough diggs (around 50), before it got promoted to the front page. And the rest is history.
This site is based on the excellent Drupal CMS. Now, I have a confession to make. I actually haven't been prepared to be announced on a such high profile site like digg.com. Meaning, I didn't have caching enabled, I didn't have throttling mechanism configured, in fact, I haven't done anything to optimize this site for such a high traffic. Also, it is hosted on a shared hosting server, so at the first moment when I noticed large number of visitors, I was thinking "Oh, no, everything is going to collapse!".
But, after some investigation I noticed that not only everything was in order, but also the site was very responsive, like nothing was going on. Needless to say, I'm very impressed with all I've seen. I can only recommend Drupal to all of you that are still in search of your holy grail CMS, and not only because of Drupal's great performance.
For a patient reader, here's yet another graph, taken from the useful Drupal graphstat module:
And finally the last graph is from Alexa:
Typically, when story hits the front page other bloggers find it interesting and cover it on their own pages. I managed to find one site, which is interesting because you can see how the poll proceeded (they took the snapshot of the results at the time they were posting their story):
If you're interested in more information regarding the Digg Effect, there's a great article that explains some less known but completely valid facts about it:
And remember, file systems are mostly about reliability, but their performance is close second important parameter!