Wikipedia Asks AI Companies to Use Paid API, Stop Web Scraping

Wikipedia's Strategy for the Age of AI

The online encyclopedia, Wikipedia, has recently detailed a straightforward approach to maintain website viability amidst the growing influence of artificial intelligence, even as website traffic experiences a downturn.

The Wikimedia Foundation, the entity responsible for operating Wikipedia, issued a call to action in a blog post. They urged AI developers to utilize its content with consideration, emphasizing proper attribution and access through the Wikimedia Enterprise platform, a paid service.

The Wikimedia Enterprise Platform

This platform, offered on an opt-in, subscription basis, enables companies to leverage Wikipedia’s extensive content on a large scale without overburdening the organization’s servers.

Furthermore, the revenue generated from this paid service directly contributes to supporting the Wikimedia Foundation’s non-profit objectives.

Addressing AI Scraping and Traffic Decline

Although the Wikimedia Foundation refrained from explicitly mentioning legal repercussions for unauthorized content scraping, they recently identified instances of AI bots scraping the website while masquerading as legitimate users.

Following enhancements to its bot detection mechanisms, the organization determined that a significant surge in traffic during May and June was attributable to these AI bots attempting to circumvent identification protocols.

Concurrently, the foundation reported an 8% decrease in “human page views” compared to the previous year.

Guidelines for AI Developers

Wikipedia is now establishing clear guidelines for AI developers and providers. These guidelines stipulate that developers of generative AI models should provide appropriate attribution to acknowledge the contributions of the human editors whose content informs their outputs.

“Transparency regarding information sources is crucial for building trust in online content,” the blog post stated. “Platforms should clearly indicate the origin of information and encourage users to explore and contribute to those sources.”

The post further explained that a reduction in visits to Wikipedia could potentially diminish volunteer participation in content enrichment and decrease individual donations supporting the organization’s work.

AI Tools for Editors

Earlier this year, the Wikimedia Foundation unveiled its AI strategy specifically designed for editors.

This strategy focuses on leveraging AI to assist editors with repetitive tasks, automating translations, and providing other tools to enhance their workflow – rather than replacing them.

• Focus: Supporting human editors, not replacing them.
• Applications: Automation of translation and tedious tasks.