Large-scale analysis of reference quality in heterogeneous Wikipedia datasets

Abstract

This study investigates the reliability of Wikipedia as a global encyclopedia by analyzing its references and assessing cross-lingual patterns of reference quality. The research introduces the concepts of reference need (RN) and reference risk (RR), measuring the percentage of sentences missing citations and the proportion of non-authoritative references, respectively. Calculating the RN score reveals a 20% decline over the past decade, accompanied by efforts to maintain the RR score below 1%. To enhance reference quality, the study proposes the collaborative editing of articles by pairing novice and experienced editors, demonstrating a lasting advantage in identifying unreliable sources. Additionally, the research examines over 5 million Wikipedia articles, revealing cross-lingual discrepancies in the perennial sources list and the persistence of untrustworthy sources across different language editions. The case study on Chinese, Russian, and Swedish Wikipedias highlights cultural variations in reference reliability, posing challenges for coordinating global knowledge on source credibility. As Wikipedia serves as a benchmark for various web applications, these findings and recommendations hold broad implications for the integrity of online information. The study also discusses the potential adoption of Wiki-style user collaboration to eliminate unreliable content in other web services.