Wikipedia and other online databases provide a soupy morass of information

A thirst for knowledge

Wikipedia and other online databases provide a soupy morass of information, but where can we find the variety of views that leads to wisdom

Thursday April 13, 2006
The Guardian

"Just who would want to vandalise an entry on cheese?" wonders Skip, a Wikipedia administrator. Watching the online encyclopaedia's raw submission queue in real time can be unnerving. The online reference site that anyone can edit is defaced 20 times a minute and cheese, it seems, is one of the most popular targets for creative embellishment.

In the administrator's console, another fresh article - Wikipedia has more than a million now - scrolls past: "James is my fren," it reads in its entirety.

Robert McHenry, a former editor-in-chief of Encyclopaedia Britannica, has described Wikipedia as "a game without consequences". BBC Radio 1's afternoon DJs recently took turns to deface each other's entries live on air. MPs have joined in, too. But as Skip begins to guide me through the arcane and often Kafkaesque bureaucracy of Wikipedia, vandalism starts to look like the least of its problems.

Skip isn't his real name or his Wikipedia identity. It's a pseudonym the 30-year-old Silicon Valley IT professional uses as he documents the inner machinations of the project, along with a dozen other Wikipedia administrators, on a site called WikiTruth (www.wikitruth.info).

Wikipedia, endlessly replicated on the web, is one example of a glut of hazy information, the consequences of which we have barely begun to explore, that the internet has made endlessly available. Is Wikipedia really the best the net can offer - and if it isn't, where should we be looking for the answers?

While plenty of people nurse resentments against Wikipedia, having failed to win a consensus for their views, Skip's colleagues at WikiTruth have a different motivation. Branding themselves the true keepers of the flame, they argue Wikipedia's wounds are self-inflicted and unnecessary.

When the business author Nicholas Carr identified last October a typically banal Wikipedia entry (http://tinyurl.com/8mr5x), he prompted a rare admission. Wikipedia's co-founder and site owner Jimmy Wales agreed, calling the examples Carr cited "horrific crap". Yet these articles were mature, Carr pointed out, and had been edited hundreds of times. Might the mass participation be hurting, not helping?

Gradual deterioration

This gradual deterioration afflicts any utopian online space, and Skip ruefully notes even the best Wikipedia work - its catalogue of featured articles of the week - degenerates once out of the spotlight.

That isn't true, of course, of printed work such as Britannica's entries. But the encyclopaedia company has been hit hard, first by the arrival of CD-Rom-based rivals such as Microsoft's Encarta in 1993, and then the net. In 1996 it laid off its door-to-door sales staff. In 1999 it launched a website. The rise of Wikipedia as an "online encyclopaedia" has added to the pressure.

Now, though, Britannica has been taking the offensive. The company strongly rebutted a study conducted by journalists at Nature magazine that compared Wikipedia favourably to Britannica, and which was accompanied by an editorial plea for the scientific community to contribute to the project. The study blind-tested extracts from each site with experts, and was widely reported as showing them to be of comparable quality. "It should have said 31% less reliable and worse written," McHenry says of the Nature study. Britannica, meanwhile, says the study was biased towards Wikipedia. "It's offensive to lump these gross offences against publishing with a typo in Britannica," says its executive editor Theodore Pappas.

Britannica said Nature cited passages not in the encyclopedia and criticised it for refusing to publish the referees' reports. Nature says it stands by its report and can't release the full reports for confidentiality reasons.

Nature's news editor Jim Giles denies the journal had identified itself closely in the Wikipedia camp. "Each has its merits," he says. "In our editorial, we simply argued that Wikipedia has potential and scientists can help realise that potential."

Britannica's president Jorge Cauz identifies a homogeneity online he finds unsettling. "Internet discourse has the ability to negate the diversity of voices, and no one can differentiate between truth and myth," he says.

"It's a hall of mirrors," agrees Michael Gorman, the Briton who is president of the American Library Association (ALA), "and it's very addictive."

But for participants, the appeal fades, notes Skip. Some of Wikipedia's most valued contributors have left in the past year, with two waves of departures in recent months, he says. Former administrators speak of burnout, brought on by bureaucratic warfare. Now Wikipedia faces a fork. If it tightens its open approach, it risks losing its most active participants, for whom Wikipedia is a utopian cause.

Away from the hurly-burly of Wikipedia, even current events can seem oddly remote and processed once they are viewed online. Google News, for example, employs computer algorithms similar to those used in spam filters to identify and present the news. In looking for similarities, the news is homogenised and breaking stories fail to rise to prominence.

For the veteran researcher Daniel Brandt, who taught CIA whistleblower Philip Agee how to use computers, much of what a human editor provides is lost. "What's gone is any sense of 'a scoop' or 'an important development' or 'new information that puts a new slant on an ongoing story'. There's no authority, no perspective and no sense of historical continuity. It's a dumbing-down process," says the Texas-based Brandt.

Google News had a serendipity now missing, mourns the veteran blogger Jorn Barger. When it appeared in 2002, "the top article might come from anywhere in the world or in small-town America but people complained, I guess, that unwelcome perspectives were getting too much prominence and Google tweaked the algorithm," Barger wrote last year.

How then are we coping with this glut of unreliable information? Some are doing better than others, suggests Will Davies, a senior fellow at the Institute of Public Policy Research.

For Davies, the accumulation of information is no substitute for critical thinking and the problem is it begins to provide its own self-justification.

"It's a false supposition we can endlessly delay having to interpret and judge things by stacking more and more bits of data in front of us," he says. "That data is a comfort blanket in a way - we all do this. People are becoming addicted to getting more information all the time. You can see it when they get out their BlackBerrys as soon as they've stepped off a plane."

For the former journalist and author Dan Gillmor, this aggregation of information technology enables is synonymous with wisdom. "My readers by definition know more than me," he said recently. "They have facts we don't know."

But is the widespread availability of technology generating such wisdom or even improving our learning? For the ALA's Gorman, who in the 1990s wrote for librarians an influential guide to evaluating technology trends, such claims are risible. "No one would tell you a student using Google today is producing work as good as they were 20 years ago using printed sources. Despite these amazing technical breakthroughs, these technologies haven't added to human wellbeing."

Davies agrees. "It hasn't made us addicted to education," he notes. Nor do the skills required to aggregate information quickly and multitask between information streams encourage understanding.

Byproducts of businesses

And while technology enthusiasts celebrate the destruction of old industries, Gorman warns technology has failed to create economic conditions to take their place. Quality information costs money to edit but the best online collections of data - in what is sometimes called the "deep web" - are byproducts of successful print businesses. Lose these, he suggests, and we're left with the banality of Google and Wikipedia. Davies is more optimistic. People will return to traditional publishers as they see the consequences of the wiki approach, he thinks, and there will be an audience for both.

But supposing these businesses sur-vive. Will the world be able to read them? Google's relationships with publishers are fraught - its Print project is the subject of lawsuits - but this dispute may be of less lasting significance than we think.

In the US in the 1980s, a movement was born to bring the best of these expensive information collections to the public, free at the point of delivery. This movement predated the public internet and may yet transform it beyond recognition. Libraries began to negotiate collectively for access to databases, which their copyright holders today would never let the public view through Google. The members of San Francisco's Public Library, for example, can access the full Encyclopaedia Britannica, Lexis Nexis and more than 70 databases from any browser, simply by entering their library card number.

It hasn't been easy, points out Susan Hildreth, the city's former chief librarian and now California State Librarian. The state's sheer size steers some database owners to making deals with smaller regional libraries. But it's not difficult to imagine churches or community groups taking advantage of such a model. It's been a success and real library usage has increased, she says.

For Davies, we can be proud we have made a success of the technology infrastructure - laying down the pipes - but we have neglected the social institutions necessary to make them work. Obstacles remain to bringing the successful collective licensing model to the UK. It costs money and Gorman notes funding for US libraries is higher than for Britain's impoverished public services. And the utopian dreams of what Carr calls the "cult of the amateur" die hard.

"It's hard to tell someone who's devoting 40 hours a week to Wikipedia that it's going to fail," says Skip. "But it will."

He returns to his console. Somewhere in cyberspace, a Wikipedia editor is correcting the encyclopaedia's article for cheese.

Andrew Orlowski is San Francisco bureau chief for The Register (www.theregister.co.uk)