Wikipedia founder admits to serious quality problems
http://www.theregister.co.uk/2005/10/18/wikipedia_quality_problem/
By Andrew Orlowski in San Francisco (andrew.orlowski at theregister.co.uk)
Published Tuesday 18th October 2005 03:48 GMT
Encouraging signs from the Wikipedia project, where co-founder and überpedian Jimmy Wales has acknowledged there are real quality problems with the online work.
Criticism of the project from within the inner sanctum has been very rare so far, although fellow co-founder Larry Sanger, who is no longer associated with the project, pleaded with the management to improve its content by befriending, and not alienating (http://www.kuro5hin.org/story/2004/12/30/142458/25), established sources of expertise. (i.e., people who know what they're talking about.)
Meanwhile, criticism from outside the Wikipedia camp has been rebuffed with a ferocious blend of irrationality and vigor that's almost unprecedented in our experience: if you thought Apple, Amiga, Mozilla or OS/2 fans were er, ... passionate, you haven't met a wiki-fiddler. For them, it's a religious crusade.
In the inkies, Wikipedia has enjoyed a charmed life, with many of the feature articles about the five-year old project resembling advertisements. Emphasis is placed on the knowledgeable articles (by any yardstick, it's excellent for Klingon, BSD Unix, and Ayn Rand), the breadth of its entries (Klingon again), and process issues such as speed.
"We don't ever talk about absolute quality," boasted one of the project's prominent supporters, Clay Shirky, a faculty tutor at NYU. But it's increasingly difficult to avoid the issue any longer.
Especially since Wikipedia's material is replicated endlessly on the web: it's the first port of call for "sploggers" who create phoney sites, spam blogs, which created to promote their clients in Google.
Wales was responding to author Nicholas Carr, who in a dazzling post (http://www.roughtype.com/archives/2005/10/the_amorality_o.php) on the transcendent New Age "hive-mind" rhetoric that envelops the "Web 2.0" bubble, took time out to examine the quality of two entries picked at random: Bill Gates and Jane Fonda.
He wasn't impressed by what he saw.
"This is garbage, an incoherent hodge-podge of dubious factoids that adds up to something far less than the sum of its parts," he wrote.
Something that aspires to be a reference work ought to be judged by the quality of the worst entry, he said, in response to the clock-stopped, right-time defense of the project, not by the fact it's got some good articles.
"In theory, Wikipedia is a beautiful thing - it has to be a beautiful thing if the Web is leading us to a higher consciousness," writes Carr.
Only it isn't.
"An encyclopedia can't just have a small percentage of good entries and be considered a success. I would argue, in fact, that the overall quality of an encyclopedia is best judged by its weakest entries rather than its best. What's the worth of an unreliable reference work?"
Why, as an Emergent Phenomenon™ it provides a subject that can be used for countless hours of class study for people like Clay Shirky, of course. Good for him - but what about the rest of us?
--------------------------------------------------------------------------------
Uncountable
Surprisingly, Wales agreed that the entries weren't up to snuff.
"The two examples he puts forward are, quite frankly, a horrific embarassment. [sic] Bill Gates and Jane Fonda are nearly unreadable crap. Why? What can we do about it?" he asked (http://mail.wikipedia.org/pipermail/wikien-l/2005-October/030075.html).
Traditionally, Wikipedia supporters have responded to criticism in one of several ways. The commonest is: If you don't like an entry, you can fix it yourself. Which is rather like going to a restaurant for a date, being served terrible food, and then being told by the waiter where to find the kitchen. But you didn't come out to cook a meal - you could have done that at home! No matter, roll up your sleeves.
As a second line of defense, Wikipedians point to flaws in the existing dead tree encyclopedias, as if the handful of errors in Britannica cancels out the many errors, hopeless apologies for entries, and tortured prose, of Wikipedia itself.
Thirdly, and here you can see that the defense is beginning to run out of steam, one's attention is drawn to process issues: such as the speed with which errors are fixed, or the fact that looking up a Wikipedia is faster than using an alternative. This line of argument is even weaker than the first: it's like going to a restaurant for a date - and being pelted with rotten food, thrown at you at high velocity by the waiters.
But the issue of readability poses even greater challenges. Even when a Wikipedia entry is 100 per cent factually correct, and those facts have been carefully chosen, it all too often reads as if it has been translated from one language to another then into to a third, passing an illiterate translator at each stage. (Possibly if one of these languages was Klingon, the entry might survive the mauling, but that doesn't appear to be the case very often).
Here the problems begin, because readability is a quality that can't be generated by a machine, or judged by one. It's the kind of subjective valuation that the Wikipedians explicitly hate: subjectivity is scorned for failing the positivist's NPOV test.
As a delicious illustration, Wikipedia appears to have a quality problem with the word "quality" itself. While Merriam Webster online offers us eight (http://www.m-w.com/cgi-bin/dictionary?book=Dictionary&va=quality&x=0&y=0) major definitions, including "a) degree of excellence : GRADE ... b : superiority in kind", and the Cambridge Dictionary three (http://dictionary.cambridge.org/define.asp?key=64728&dict=CALD), of which two are "how good or bad something is and of a high standard" Wikipedia's sister project Wiktionary definition begins this. "1 - (uncountable) general good value"
Now is that General Good Value as in something plucked from a Wal-Mart sale? And "Uncountable"? Yes, indeed.
If this was a Marvel Comic, our superhero Objectivity would by now be ensared in the evil coils of Subjectivity. There appears to be no escape. Or is there?
Not good enough - so what do we wikkin' do?
Re-working Wikipedia so it presents the user with something minimally readable will be a mammoth task. Although the project has no shortage of volunteers, most add nothing: busying themselves with edits that simply add or takeaway a comma. These are housekeeping tasks that build up credits for the participants, so they can rise higher in the organization.
And Wikipedia's "cabal" has become notorious for deterring knowledgable and literate contributors. One who became weary of the in-fighting, Orthogonal, calls it Wikipedia's HUAC - the House of Unamerican Activities prominent in the McCarthy era for hunting down and imprisoning the ideologically-incorrect.
So right now, the project appears ill-equipped to respond to the new challenge. Its philosophical approach deters subjective judgements about quality, and its political mindset deters outside experts from helping.
This isn't promising.
One day Wikipedia may well be the most amazing reference work the world has ever seen, lauded for its quality. But to get from here to there it will need real experts and top quality writing - it won't get there by hoping that its whizzy technical processes remedy such deficiencies. In other words, it will resemble today's traditional encyclopedias far more than it does today.
For now we simply welcome the candour: at least Wikipedia is officially out of QD, or the "Quality Denial" stage.
Bootnote Of the many, many atrocious entries, we'd like to bring one more to the HUAC's attention, and it's our very favorite. As of the time of writing (http://en.wikipedia.org/w/index.php?title=Jeanette_Washington&diff=18654297&oldid=16179591), whoever wrote the entry for soul legend Baby Washington has no idea who she is, but makes a wild guess, then gives up completely with the less-than-helpful advice: "Many have written inacurate information about Washington. She IS NOT "BABY WASHINGTON" from James Brown." (sic).
Indeed. But note that this entry has been edited no less than seven times and can be found replicated at Biography.com, Answers.com, Reference.com, InfoMutt, The Free Dictionary and hundreds of other sites.
You've got to love the web. Just bask in that collective intelligence. ®
