Home / Culture and Society / Science and Technology / Digg, Reddit, Netscape: The Wisdom of Crowds or Mob Rule?

Digg, Reddit, Netscape: The Wisdom of Crowds or Mob Rule?

Please Share...Print this pageTweet about this on TwitterShare on Facebook0Share on Google+0Pin on Pinterest0Share on Tumblr0Share on StumbleUpon0Share on Reddit0Email this to someone

"A person is smart. People are dumb, panicky, dangerous animals" – "K," Men In Black

If I'm smart, and you're like me, you're smart. We're both smart, and other people like us must also be smart. In fact, we're smarter than the self-anointed media gatekeepers that trumpet inanity while burying important news in the interest of ratings. What we need is to be the new gatekeepers, together. Working together, the smartest people will be highlighting the news, rather than the dumbest.

Or so the theory goes. In reality, I'm smart and you're smart, but some of you like pictures of tattoos and second-rate web comics and third-rate political candidates. Worse, some of you are conspiracy theorists, celebrity gossip hounds, or Mac users. Worst of all, some of you just don't vote like you should. This site sure isn't as good as it used to be, before all the newbies showed up.

There are problems with the current wave of user-driven sites, like Reddit, Digg, and Netscape, but are the problems inherent to the model, or can software tweaks fix them? Are there even really problems?

The Wisdom of Crowds
19th century scientist Francis Galton observed that a collection of individuals, acting independently, managed to achieve what even experts could not: averaged together, their answers were right, though no individual managed to get as close as the average did. In his case, the exercise was estimating the weight of a slaughtered ox, but in James Surowiecki's 2004 book The Wisdom of Crowds, he suggested that the same principle holds true in many cases. And it might, but achieving a technical result (the weight of the ox) is a different type of exercise than rating quality.

While weight-guessing carries no obvious penalty or reward for guessing too high or too low, humans rate the quality of things based on non-obvious factors, resulting in surprising patterns. Does a horseshoe curve mean that a book is worth reading, or not? With the same number of one-star ratings and five-star ratings, can we determine that it's a three-star book? Or, if we plot the rating over time, it could be that one-star ratings represent an attempt by the rater to lower the visible aggregate rating, and five-star ratings an attempt by the rater to raise the visible aggregate rating. Having that aggregate rating visible to all makes sense from an online bookseller's perspective, but it undermines one of the fundamental elements of Galton's observation. The individuals in this case are not acting independently.

On Reddit, up-votes and down-votes seem to carry equal weight, and no reason need be given for either. On Digg, the value of individual actions is closely guarded by secretive and mysterious people, but it seems that burying an article carries more weight than digging it, though one does need to supply a reason. On Netscape, I have not spent enough time to weight the relative merits of sinks and votes, though the influence of human editors there sidesteps some of this in any case. Of these three sites, only Reddit makes any attempt to truly harness the wisdom of crowds, hiding (though not well) the aggregate votes on links for the first hour after posting. With the total score unknown, people may act independently, but once that first hour has passed, other factors begin to intervene. In fact, hiding the aggregate number doesn't work as well as it might, both because the numbers are only a click away, and because the order of links is still based on that number. You're only acting independently if you're looking at a strictly time-ordered listing (which exists on Reddit, on Digg, and on Netscape) and ignoring the existing score (if possible).

How many people do that? Jason Calacanis, who ought to know as a former owner of Netscape, suggests an 80/19/1 rule, in which only 1% of site visitors are submitting new content, with an additional 19% voting. The majority of these people presumably vote only on what's already on the front page, serving to make popular things more popular but doing little to actually direct the content of the site. Again Reddit captures the efforts of these people the best by mixing new content in with very popular content, though still with built-in indicators of how other people rate the link.

The biggest problem seems to be visibility. When one submits a link to one of these sites, how many people will actually see it? If the first few people who see it don't value it, most people will never even see it, so the control of the site falls to those most able to spend the most time on it. Buried or down-voted links aren't seen, at least not by most people, while very popular links are seen by nearly everyone. Digg makes it easy to give a link a five-star rating, but more difficult to give a link a one-star rating, and so obviously links get many more five-star ratings than one-star ratings. Reddit makes it easy to do both, but links that are hit with one-star ratings right away just disappear before they're widely seen. Also, people use the one-star rating for multiple purposes, not just to rate the quality of a link, resulting in over-emphasis on downvotes.

Mob Rule
The definition of "mob" I'm using here can usually be summarized as "large group of people who disagree with me." Let's face it, that's the problem. There may be some people out there who are perfectly happy with every link they see on these sites! I doubt it, but it's possible. People with short-term memories like seeing links that were just there a month ago. Young people like seeing links that were popular three years ago. Some people love seeing undated bits of trivia, or the same collection of photos stolen from random websites, or still more evidence that fire can't melt steel except when it can. Different individuals in the collection have different priorities, or different agendas, and sometimes the same people may vote differently from day to day, depending on what they had for breakfast. One day a person may be voting based on what they think other people will like, while the next day she votes only on what she likes. One morning someone votes against links with typos in the title, or overly-long titles, regardless of the link itself, but then something happens, and a long title with a typo gets their vote. All of these things should average out, given enough input, but it doesn't.

In fact, we come back to the issue of the people with the most time shaping the content. Not to put too fine a point on it, the people with the most time are not often the people with the most experience or wisdom. Those people are, we hope, creating content more than voting on it! How is it that anyone thinks that social news sites will avoid the Peter Principle? In a hierarchy — and make no mistake, the 80/19/1 rule makes each of these sites a hierarchy — every employee tends to rise to his level of incompetence. That the employees are paid in egoboo rather than cash makes no difference. Over time, each of these sites faces increasing problems of incompetent individuals in its collection, and struggles with how to limit their effect.

How is that anyone thinks that social news sites will avoid the problems inherent to committees? When a sufficiently large group of individuals is added to a committee, the result more or less exactly fails to please anyone (with apologies to Douglas Adams).

On May 1, we saw what happens when a mob turns ugly, as Digg first attempted to comply with legal orders against them and then decided to bend their will to that of the mob, though the mob clearly lacked the ability to see the potential results of their actions — or care about them.

Harnessing the Crowd
Is there a way to harness the wisdom of crowds without falling into the trap of mob rule? A way to avoid the Peter Principle? A way to keep social news sites from turning into loosely-coupled committees? If so, the approach must lie away from the direction these sites are going. Can a site truly harness the wisdom of crowds and still be popular? Part of what makes these sites popular is what also contributes to mob rule: popularity.

The key to Galton's observation about crowds is that each person acted individually. Had there been a list of previous guesses available, most people would have clustered around existing guesses, even if those initial guesses were completely off-base. If the guesses of the experts had been broken out separately, people would have clustered even more heavily around those, though they turned out to be just as wrong as everybody else. While technical estimates are a different type of problem than ratings, some of the same principles apply.

Reportedly 95% of Kevin Rose's submissions reach the front page of Digg, and many of the top submitters are seen over and over and over again. Both are a demonstration of this type of clustering. If social news sites didn't list the submitter of a link, that would help cut down on clustering. Not listing the top submitters, as Digg has recently done, is a step in the right direction.

Presenting each voter with a random list of links, including some very popular links and some unpopular links, with no clues about their popularity, would result in voting more likely to harness wisdom, though each voter would have a less pleasant experience than if they were only perusing popular links. That's the trade-off: by decreasing the pleasure of the 19%, these sites can increase the pleasure of the 80%. The problem with this is that it punishes the people who contribute the most to the site in favor of the people who contribute the least. Of course, the site owners might consider the source of their revenue, as well. That seems to be what's driving Netscape more than Reddit or Digg.

Another thing that might help with the purity of the results is disallowing or not counting votes from other websites and the front page, but only counting votes from the "random list" pages. Again, this has negative effects in other ways, as content producers are less likely to feature links from their own websites back to social news sites if it won't directly benefit them.

A Fool's Errand?
In fact, there are good reasons for each of the choices each of these sites has made, though most of the choices actually disrupt the wisdom of crowds. If it isn't possible to build a big site that truly does harness the wisdom of crowds, if that isn't the highest priority — and it clearly isn't — then what are these sites, exactly, and how long are they sustainable? Can a site devoted to delivering quality results to the 80% attract enough of a 20% community to survive?

With the current crop of social news sites, the emphasis is on the "social." I look forward to the day when the emphasis shifts to the "news."

Powered by

About pwinn

  • Mike

    “though the mob clearly lacked the ability to see the potential results of their actions — or care about them.”

    Please explain this statement more thoroughly.

  • Mike, that’s simple: it’s much easier to suggest that someone else should put their livelihood on the line to stand up for a principle than it is to put your own livelihood on the line for that same principle.

    Digg stands a greater-than-zero chance of losing court battles over the AACS key, as they explained on their site, but a horde of people with no skin in the game decided to make a stand, most of them not even considering that it could literally mean the end of Digg. That these people were not acting completely rationally is as obvious as the fact that most of them won’t see it that way.

    I certainly hope that Digg doesn’t lose, but it was a stupid display of bullying by a mob that even makes that a possibility.

  • Ksero

    Very well written. I have only one minor issue with it: the recommended-filter at reddit will theoretically over time learn what stories you like and present them to you even if the rest of the users didn’t like them. Though admittedly it’s still far from perfect.

  • Great piece, Phillip! I like the fact that Netscape is the most transparant about the “up” and “down” votes that stories get. Each is displayed on every story, so in theory stories aren’t “penalized” by a down vote, though I’m sure enough down votes trigger some kind of review by one of their anchors.

    One of the most intriguing things about social news sites to me is that in the end they are made up of the people who frequent them. Thus Digg is and probably always will be tech-centric, Netscape has a broad American cross-section sort of feel, and Reddit is more of an eclectic hodge podge.

  • Ksero, the recommended filter at Reddit seems particularly awful, and has a really bad unintended side effect: people end up using their down-votes to train the filter, which causes them to go out of their way to downvote things they would otherwise ignore.

    Basically, the down-votes do two things: one for the user, and another for the site readership. That’s unfortunate, because people end up thinking about only one or the other.

  • Want to actually harness the crowd? Then use Subvert and Profit… we sell front page placement on Digg. See my URL.

  • Another benefit of anonymity, as I see it, is that it makes ventures like that advertised in comment #6 difficult or impossible to operate. If they can’t know whether or not you’ve voted on an article, they can’t really pay you, which means you won’t do it, which means they can’t sell it.

    And only allowing votes from semi-random pages of links may make it difficult for people to vote on specific stories anyway, which would be another nail in the coffin of these types of operations.

  • I would like to say: bravo! This is an excellent follow-up to the recent Digg controversy, and delivered with intelligence and even-handed appeal. My own article on the issue is certain more one-sided though I am pleased it provoked some thoughtful reactions. I largely agree with your suggestions, but must (unfortunately) point out that they may be naive in nature.

    As much as I would love to see anonymity (to avoid things like: Kevin Rose’s Diggs all making the front page) I just don’t see it happening. At the end of the day, these are all for-profit sites watching out for their bottom line. To my mind this model of social news can and will only change if and when these sites are run by and for the people, as opposed to being controlled by a few individuals in power.

    Sound familiar? Yes, it is the classic political dilemma of democracy! My best hope: people will begin to see these problems in a new light and start to form organizations around such realizations, which in turn means having joint control over sites. Too simplistic? Probably. A nice idea? I’d like to think so.

  • Yes, I’m sure that these suggestions won’t be taken. The methods that would *actually* harness the wisdom of crowds would be unpopular, so the sites will settle for *pretending* to harness the wisdom of crowds, while actually setting up cliques.

    I’m not naive; I recognize this; my point was that harnessing the crowd *is* possible, and these sites aren’t doing it. And no, they won’t.

  • Nate

    Well written. Considering the systems at the level of granularity of the individual may prove to be problematic however because it is very likely that users interact and form social groups formally or informally external of the system. These groups may be able to influence certain content, promoting it or demoting it within the system, without any external appearance of group behavior. Discerning this behavior may be difficult because many if not all, of these systems fail to provide means of internal communication between their members, leaving them forced to use email, IM, or some other form of communication exclusively, all mediums that are untraceable by the internal architects and the external community at large. (Not that I’m suggesting we the services implement them, but rather that it is a factor that plays into our inability to understand the social forces at work).

  • Nate, the real key would be to prevent people from picking and choosing what links they could vote on. Either present them randomly, demanding an action one at a time (StumbleUpon style, I believe), or present them in a random listing, but don’t let people decide to vote on a link their friend told them about.

    Of course, that would make the site much less pleasant to use, so it’ll never happen. In which case we ought to stop pretending that we’re trying to harness the wisdom of crowds and admit we’re just hoping the popular clique likes the same thigns we do.

  • Nathan

    I must interject here that I believe StumbeUpon has succeeded in at least minimalizing the “mob” effect… What pages it sends you are effected by your interests, rating behavior, and voting habits. To see what other people have said about it, you must load the reviews page, almost totally eliminating the mob mentality while still being very, VERY good at providing pages the user is actually interested in.

  • I must admit I haven’t spent very much time at StumbleUpon at all. I mostly spend my time on Digg and Reddit, with a bit of Netscape. I used to spend time on ShoutWire, too, but that was some time ago.

    I’ve used StumbleUpon for Videos, but that’s about it.

  • Ah, what I wouldn’t give to see some good old fashioned mob rule in this country! The great thing about a mob – regardless of its relative level of wisdom – is that it refuses the dictates imposed on the individuals it’s made up of, that are handed down by a small group of power-hungry corporate rulemakers.

    The question of crowd wisdom vs. mob rule is very relevant in terms of the political climate online. The internet is supposed to be bringing democracy back to the Presidential vote, as I discussed in a recent post on my blog. But what I’d like to see is some of the power on display during social events, like the whole May Day thing on Digg, turned into actually getting underdog Presidential Primary candidates the attention, and funding they deserve. When will the mob power we occasionally see on display online translate into real-life activism in this country?

  • MonkeyT

    Anonymity will always undermine credibility. I don’t mean to assert that no useful information can be learned from anonymous indiviudals, it’s that a crowd full of geniuses is of little use if it cannot be distinguished from a crowd full of idiots. How about this: Organize the mob into small groups, then make certain the participants in those groups don’t change quickly. Keep the same people with the same people. Smart groups will emerge, dumb groups will emerge, and the crowd, in its wisdom, will hopefully learn who to watch. An idiot can lower the IQ of a crowd, but they can’t make a smart person dumb.

  • Kenny

    On comment #2:

    For the first point, though it is easier to get someone else to stand up for a principle than oneself, normally few people would even do that. Few people who visit Digg would stand a chance trying to defend themselves in court, but by demanding a larger organization get involved something might actually get done. Additionally, just the fact that a large number of people showed their dislike of the current situation shows where popular opinion stands.

    Though most people on Digg were just following the crowd rather than putting some thought into what they were doing, some believed that the exposure would be worth the risk. If this goes to court, and Digg wins, it would likely have a significant effect on anti-copying laws and their enforcement. If Digg does end, there are a number of other sites its audience could go to, e.g. Reddit, Slashdot, etc.

  • I agree with your article in general. Very good points, and well written. The big problem with this idea of a ‘social network’ being more wise (or, probably better put, less corrupted) than some group or organization is fundamentally flawed. I didn’t realize what the actual ‘experiment’ was that was used as a basis… so your article really enlightened me there.

    However, I think it is flawed in basic principal… for a ‘social’ network to be ‘smart’, they would have to…. ‘think’. I would argue that this generally does not happen. While panels and organizations can certainly be corrupt, there is peer review, research, and usually thinking going on. Yes, this can still be corrupted or wrong, but I think is right more of the time than the ‘popularity contest’ that often takes place in ‘social’ networks.

    I often see this problem present itself in reference to government systems. The idea that democracy in its pure form would somehow be good, ignores basic human nature. 51% should never rule. This is way, at its best, the USA is not a Democracy, but a Democratic Republic with a Constitution. The Constitution sets fourth some basic principals that keep human nature in check, and ensure that 51% can’t overrule the basic principals…. such as ‘All are created equal.’

    ANY social network needs checks and balances to keep human nature in-line. Modern Internet ‘social networks’ are no exception. Organizations, groups, committee, etc are in the same boat. If the checks and balances are not strong enough, or fail, human nature will corrupt the results… no matter how big or small the group, or an individual.


  • Tinu

    This was not only an excellent article, but long overdue to be written. How wise is the crowd? Even if the crowd is wise, it is always wise to yield to crowd wisdom? Are these tools even measuring the crowd correctly? And you’re right, the way you describe how people use Digg is exactly how I do it when I find myself there, rather than following a DIgg button to the site.

    This makes me want to go watch Gladiator, and write down all the quotes about what Rome is. That’s often the ultimate question for me whenever I run something (like a forum) that’s subject to the whim of the crowd as a large part of its success. Is Rome the mob? Is the mob fickle or of a larger intelligence than an individual? Food for thought, thank you thank you thank you.

  • Doctor Allen

    Submitting content to Digg is easy; you just set up a Google Reader account with feeds from the most popular geek sites and check in on it every hour or so. When a new story hits one of those sites, you submit it to Digg.

    Digg could eliminate the 20% altogether by automating this process. Just take an algorithm-driven aggregator, ala Google News, and make that the upcoming stories section of a social news site.

    Problem solved. However…

    I would like to see a community website that awakens human consciousness. That makes us better people. Digg seems to do the exact opposite.

  • When the masses turn the group vote into a popularity contest, you get people going onto sites like digg and molding the environment in their own image. People do bury things for less than noble reasons, one of which might be to get their own articles to the top.

  • Many commenters above are missing the point suggested by Francis Galton’s observation. The point isn’t that we need to separate the experts from the morons, nor that we need to able to identify everyone. In fact, both of those statements are taking, I believe, the exact opposite of what they should from the experiments.

    The point was that while none of the non-experts in themselves knew what they were talking about, in aggregate they came up with the right answer, beating out the experts. I’ll say it again, because some people missed it in the article: non-experts — in aggregate — came up with a better answer than experts. Your wrong answer and my wrong answer and the wrongs answers from a bunch of other people average out to something really close to the right answer.

    Once you grasp the implications of that, you see why identity isn’t an issue, any more than expertise is. The problem with expertise and authority is that people with it tend to rely on it, and it turns out that experts and authorities makes mistakes and have bad days, too. The idea of a large group of people is that a sufficiently large swarm of people ends up getting the right answer. Not individually, but collectively.

    Digg, Reddit, Netscape, and the others, are all relying on experts, and think that because they let people *establish* their expertise on-site rather than carrying it over from the real world, this is somehow a brave new world. It isn’t. It reward those with the most free time, but fails to take advantage of the very issue that it claims to rely on — the wisdom of crowds.

  • Great food for thought!

    I see a possible solution as a “trust” system for individual voters. Something that could downgrade the impact of those who vote only to vote, vote far too often, or too consistently for one contributers content. Part of the problem would be self-correcting (as in eliminating whackos) if a person just looking to mash buttons did not see immediate results.