The Future of Wikipedia (my take), part 1

The future of Wikipedia is a perennial topic of discussion among Wikipedians and Wikipedia critics. It's a topic I've been thinking about for a while (see my prognostications from early 2007). I apologize in advance for a long post.

It seems like Wikipedia and the Wikimedia Foundation are finally turning the corner in several long-anticipated respects. The two long-heralded software projects, Unified Login and Stable Versions, are functional and moving toward implementation. The professionalization of the Foundation is starting to pay dividends: in the last few days, Executive Director Sue Gardner announced a $3 million, 3-year grant from the Sloan Foundation, followed a few days later by a $500,000 grant from philanthropists Vinod and Neeru Khosla.

Financial stability, and even financial flexibility, may be on the horizon, and the harshest critiques that could potentially derail the Wikimedia Foundation and Wikipedia (in particular, those of Larry Sanger and the recent, ongoing accusations by Danny Wool and Kelly Martin) seem to have spent most of their energy without much effect. That's not to say that these critiques are entirely unfounded, but it's becoming clear that the worst of them are either in the past or not of project-killing significance. So it's a good time to reassess the big issues that will shape the project's future.

The title and main topic of the newest Wikipedia Weekly podcast is The Future of Wikipedia. The discussion (the "feeback" and "Wii moment" sections, from 21:47 to about 52:52) is primarily about the future growth of Wikipedia; Andrew Lih and Liam Wyatt disagreed in the last podcast about how big we can expect Wikipedia to be in the years to come.

Andrew forcefully states an idea that parallels my own thoughts on Wikipedia's future: to come anywhere close to "the sum of all human knowledge", the project needs a "Wii moment", a reformulation of what it means to contribute to Wikipedia (along the lines of what the Wii did for gaming) that opens things up to huge numbers of people who never would have participated so before. The podcast discusses some of the basic things that will make editing more accessible: what-you-see-is-what-you-get editing, and a gentler culture that is more appealing to people with little patience for revert wars and wikilawyering.

I have a more expansive vision of what Wikipedia and its sister projects ought to become. In this and some follow-up posts, I'll lay out some of my ideas for major changes.

One of the most promising avenues for expanding the scope of the Wiki(p/m)edia community is news. Right now, Wikipedia has a troubled relationship to the news. One recent example: Obama's race speech, "A More Perfect Union", was undergoing a deletion discussion from the evening of March 18 (the day the speech was made and the article was written) until yesterday. In the meantime, the article got 4000 hits the first day, and after the initial news burst has been holding steady around 1000 hits per day. For news topics, people want the kind of synthetic, continually updated neutral view that Wikipedia (at its best) provides. But neither mainstream media nor the new media of partisan blogs and social news sites provide this, Wikipedia avoids this except for "notable" stories, and Wikinews operates no differently from traditional news, calling a story "done" once it's published.

In my view, most of Wikinews ought to be merged with Wikipedia, leaving only Original Reporting for Wikinews. For big topics that have both ongoing news and a long, broad history, Wikipedia ought to have separate subpages for more detailed explanation of specific news events (a la Wikinews articles, but continually open to update). This will encourage the participation of the thousands of news junkies who, at present, are not particularly welcome on Wikipedia (and don't want to waste their time writing Wikinews articles no one will read, if they even know about Wikinews).

Wikimedia could do even more with news. News is the subject of continual, massive interest, and the there is a large--and mostly unmet--demand for internet discussion of news. Most internet news sources do not have even rudimentary forums for discussion, and even for the ones that do, much more discussion happens offsite than on. For example, the top link on social news site reddit right now is this article on Time's politics blog, which does not allow comments. The reddit discussion is 140 comments long and counting. Even traditional news sites that do allow comments rarely have anywhere near that level of participation, even for articles that are heavily discussed at Digg, Reddit, slashdot, and the blogosphere.

The top social news sites are only modestly popular, and there is still plenty of room for new players. If Wikimedia started a social news site, and melded it on to Wikipedia along with other features that give users more of an outlet for interaction that is not centered on article improvement, Wikipedia could probably go from the #9 site on the internet (down from #8 last year) to the #1 site. That's not an end it itself, but it would have a huge impact on content in terms of turning readers into discussants, and discussants into contributors. Every article and news story would have a sleek discussion thread (maybe dynamic ones based on users' Wikimedia social networks, or imported social network data from Facebook, MySpace, and the others).

I realize that bits and pieces of this are being done elsewhere (including Wikia, e.g., with their politics site), but Wikipedia has the userbase and reputation to actually make it work.

Along with social networking and free discussion, article policies would have to be liberalized; at the very least, the notability concept should be retired, although a more integrated system of sorting articles based on the level of reliable sourcing could be put in place instead, so that readers always have a clear idea of whether they are reading a biography of a significant figure based on the work of professional historians, or the biography of somebody's grandfather pieced together from newspaper clippings and family records.

In my next post, I'll discuss my ideas about Original Reporting for Wikinews; although my above proposals would gut the current core of Wikinews, I envision a future for Wikinews even brighter than Wikipedia's, based primarily on citizen journalism.

What's missing from the Democrats' health plans

Despite the importance of health care (and particularly health care costs) in this U.S. election cycle, neither of the Democrats include any mention of tort reform or health courts in their health plans (Obama's; Clinton's). This has traditionally been a partisan issue, with Republicans for reforms to limit malpractice awards and Democrats against.

If newsbatch.com is to be believed, Democrats don't support tort reform because trial lawyer associations are big donors. Opensecrets.org gives a good rundown: Clinton has received over $13 million from lawyers and law firms (her top sector); Obama has received over $11 million (his top sector); McCain has received over $3 million (his second top sector after retirees). It's not that surprising that lawyers are the biggest sector for campaign donations, considering that they are wealthy and have more direct interest in politics and laws than any comparable profession. But without breaking those totals down more, it's hard to say that is specifically the reason why Dems oppose tort reform (since surely many lawyers support it, in addition to many many doctors).

Clinton does not mention malpractice or tort reform in her health plan at all. Obama, supposedly free of lobbyist influence and mostly free of big money special interest influence, mentions malpractice obliquely: "Obama will also promote new models for addressing physician errors that improve patient safety, strengthen the doctor-patient relationship, and reduce the need for malpractice suits." (p. 8) McCain's outline of a health care plan includes tort reform as one of its planks.

Hopefully more will come of that once the nomination is settled; if Obama is the Democratic nominee, that may be likely. Short of a single-payer system that eliminates the huge overhead of commercial insurance or doctors factoring cost into treatment decisions in a more serious and systematic way (both long shots for the near future), tort reform and/or health courts are the only things likely to make a serious impact on reducing the cost of health services. (Inflated pharmaceutical costs are another matter, the dynamics of which I'm still trying to figure out.)

Wikipedia traffic for content linked on the Main Page

How much exposure comes with time on the main page of Wikipedia? The short answer: a featured article may get between 15 thousand and 100 thousand hits, depending on the mass appeal of the topic and the day of the week; a new article featured on Did you know? will bring anywhere from several hundred to several thousand hits, depending on the hook and time of day it's up; a thumbnail from In the news may get tens of thousands of hits per day, on par with a Featured Picture (which is below the fold for most visitors).

Quirky, unexpected articles do much better than articles on better known topics. Examples from February:

• Peru, which typically gets 4-7 thousand hits per day, only had 37 thousand on its day in the spotlight.
• "Through the Looking Glass", a Lost episode, got 100 thousand hits, compared to a normal day of about 1 thousand.
• Golden plates, which usually gets 3-5 hundred hits, got 90 thousand.
• Other Featured Articles at the high end of the hit spectrum: Knut the polar bear, Ronald Reagan
  
• Articles at the low end: Barn Swallow, Constitution of Belarus, Irish phonology, Thoughts on the Education of Daughters

Consider some main page items from February 21:

• The Featured Article, Bengali Language Movement: 14 thousand hits
• A lunar eclipse thumbnail from In the news: 23 thousand hits
• The article about that eclipse: 135 thousand hits (although possibly much of that traffic came from elsewhere; there's no baseline to judge)
• Other In the news articles, some blurbs up to 5 days old: ranging from 2.5 thousand to 45 thousand.
  
• Did you know? articles: mostly in the 500-1000 hit range (for ca. a six hour period), while the image for the top article had twice the hits of the article itself.
• On this day articles: thousands of hits each, some over 10 thousand
  
• The Featured Picture, a dragonfly macro: 17 thousands hits

The most interesting thing about these numbers, to me, is how disproportionally heavy the aggregate traffic is for In the news than for other sectors. Much of the time the main articles for the newest news entries--represented by a single line each on the Main Page--have similar hit counts to the Featured Article--which has a whole box in better screen real estate. The anniversaries also have a strong showing, especially considering their placement on the bottom right of the screen.

The next time the Main Page gets redesigned, Wikipedians might want to do some more detailed research on which elements are most popular and factor the results into the design plans.

History of science viewing stats on Wikipedia

For the first time, there are accurate hit counts for comparing arbitrary articles. User:Henrik has a hit counter utility for Wikipedia pages, with statistics going back to mid-December 2007. (Estimated hit counts were available for up to the top 1000 most popular pages through the currently-offline WikiCharts.)

In browsing hit counts for history of science-related articles, it quickly becomes apparent that biographies have a much larger readership than explicit history articles. The monthly hit counts for the histories of science, medicine and technology (13829; 16925; and 15442, respectively, for February) are in the same range as the daily hit counts for Albert Einstein (ranging from 8,000 to 18,000 in February). Newton and Darwin bring in about half what Einstein does, and many other important figures in the history of science are in the 1,000-2,000 per day range. Unsurprisingly, most scholarly jargon concepts (important as they may be) are not read much: less than 100 hits per day for things like "Medicalization" and "Commensurability (philosophy of science)", and narrower concepts (the ones that even have articles) may get less than 10 hits per day. "Paradigm shift", however, gets almost 1,000 hits per day, and Structure of Scientific Revolutions gets a couple hundred.

I'm disappointed with what I expected to be the "head" of the distribution, the main historical overview articles, but the level of activity towards the "long tail" is relatively impressive. See, for example, the following sequence for total hits in February:

1. Science - 108271
2. History of science - 13829
3. History of biology - 4677
5. History of molecular biology - 1994
6. Phage group - 191
7. Max Delbrück - 1501
8. Luria-Delbrück experiment - 1230

These are in order of scale of the topic (and represent a possible trail of clicks), but are obviously not in order of popularity (or historiographical significance). The biography and the still-pedagogically-relevant experiment stand out with high hit counts relative to the scale of the topic.

For historians who want to reach a broad audience through Wikipedia, putting historical context into biographies and topics of contemporary interest is probably more effective than writing concept-, artifact- or event-based historical articles.

Tomorrow, I'll look at what kind of hit count boost time on the Main Page brings, and how hit counts vary according to article quality for topics of similar significance.