Students, patients: Beware of data in Wikipedia

Since it was founded in 2001, Wikipedia has become one of the most popular websites on the Internet. In 2013, it ranked ninth among all other sites in the U.S., with more than 72.5 million unique visitors per month, according to Nielsen. Worldwide, Web traffic data firm Alexa Internet ranks it as the sixth most popular site.

The nature of the website's data, where users submit, edit, or delete data for each entry, have created headaches and challenges for teachers and medical professionals. Now a new study by researchers from several U.S. universities and graduate programs has sought to shed light on just how much of the information contained in health-related entries is inaccurate (Journal of the American Osteopathic Association, May 2014, Vol. 114:5, pp. 368-373). Their conclusions are relevant to everyone involved: patients, teachers, and healthcare providers.

"Most Wikipedia articles for the 10 costliest conditions in the United States contain errors compared with standard peer-reviewed sources," the authors wrote. "Healthcare professionals, trainees, and patients should use caution when using Wikipedia to answer questions about patient care."

Wikipedia's coverage of dentistry is thorough, with its own dedicated "portal" page that starts with an overview of profession at the top. From there, the site offers an ever-changing selection of related articles and finally, a breakdown of the different areas of dentistry, such as fields of practice, restorative dentistry, tooth anatomy, and pathology.

Searching for dental topics inevitably leads to prominent results that link back to Wikipedia. The top result of a Google search for "caries" is a link to the Wikipedia page detailing what they are and how they're treated, complete with animated gifs of how they penetrate into dentin. That page alone lists 112 references, many of which are peer-reviewed journals.

Wikipedia's crowd-sourcing format lends itself to inevitable accuracy issues with its content; nevertheless, previous research has found that anywhere from 47% to 70% of medical students and physicians have acknowledged using the website as a reference. Whatever the actual number, the current study authors strongly believe that the data are influencing decision-making.

They do note that Wikipedia is not the Wild West and has established processes for weeding out vandalism and unverifiable information; consequently, half of the corrections are posted within three minutes of being verified. And in 42% of cases, corrections are made immediately, according to the researchers. "In general, medically related articles on Wikipedia are accompanied by a sufficient amount of reputable citations," the authors said of previous research on the topic.

To find out more, they focused on the 10 most costly medical conditions in the U.S. during the year 2008 as ranked by public and private expenditure. They sourced their rankings data from the Agency for Healthcare Research and Quality, then selected and printed the 10 most relevant corresponding Wikipedia entries on these entries on April 25, 2012. Selecting a single date gave the researchers a snapshot of these entries, since they are subject to change and updates.

To blind the process, 10 reviewers who were internal medicine residents or rotating interns were randomly selected to examine two of the articles, to add redundancy. The reviewers had to identify and fact-check each assertion in the article against a peer-reviewed sourced published within the last five years. These sources were uncovered with UpToDate as a search engine; failing that, PubMed, Google Scholar, or a search engine of their choosing could be selected.

Heart disease, cancer, mental disorders, trauma-related disorders, osteoarthritis, chronic obstructive pulmonary disease/asthma, hypertension, diabetes, back problems, and hyperlipidemia made up the list of the most expensive medical conditions.

Trauma-related disorders had the only entry in which there was not a statistically significant discordance between Wikipedia and a peer-reviewed source (p < 0.05). The same was true for the articles examined by blinded reviewers. And in the articles about major depressive disorder, osteoarthritis, chronic obstructive pulmonary disease, and diabetes mellitus, the researchers found a statistically significant discordance between Wikipedia and peer-reviewed sources for dissimilar assertions.

"Because our standard was the peer-reviewed published literature, it can be argued that these assertions on Wikipedia represent factual errors," the researchers wrote. However, they also noted that "most of the dissimilar assertions found by the reviewers failed to demonstrate discordance." A difference in opinion between reviewers about what qualified as an assertion and what was, for example, common knowledge, could have impacted the data.

While acknowledging study limitations, the researchers were confident enough in their conclusions to issue a warning. "Our findings reinforce the idea that physicians and medical students who currently use Wikipedia as a medical reference should be discouraged from doing so because of the potential for errors," they wrote.