FacebookInstagramTwitterContact

 

Brunei Youth League U15           >>           Pantai Mentiri Amateur Open Golf Club Championship           >>           Reverse Heart Disease Through DIETARY CHOICES: Here’s Where To Start           >>           How Curcumin Could Revolutionize BOWEL CANCER PREVENTION           >>           Brunei Youth League U18           >>           Temporary Half Lane Closure           >>           Kate Hudson Shares Glimpse Into Family Vacation With Kids Bingham And Rani           >>           Missing           >>           Perbadanan TAIB receive Two Prestigious Awards           >>           Bay FC's Racheal Kundananji Says Zambia Are Going To Shake Up Women's AFCON Hierarchy           >>          

 

SHARE THIS ARTICLE




REACH US


GENERAL INQUIRY

[email protected]

 

ADVERTISING

[email protected]

 

PRESS RELEASE

[email protected]

 

HOTLINE

+673 222-0178 [Office Hour]

+673 223-6740 [Fax]

 



Upcoming Events





Prayer Times


The prayer times for Brunei-Muara and Temburong districts. For Tutong add 1 minute and for Belait add 3 minutes.


Imsak

: 04:34 AM

Subuh

: 04:44 AM

Syuruk

: 06:09 AM

Doha

: 06:33 AM

Zohor

: 12:22 PM

Asar

: 03:48 PM

Maghrib

: 06:34 PM

Isyak

: 07:49 PM

 



The Business Directory


 

 



Internet & Media


  Home > Internet & Media


Wikipedia Is Struggling With Voracious AI Bot Crawlers


Andriy Onufriyenko via Getty Images

 


 April 3rd, 2025  |  01:30 AM  |   322 views

ENGADGET

 

The Wikimedia Foundation is getting pummeled by crawlers, which could cause issues for actual readers.

 

Wikimedia has seen a 50 percent increase in bandwidth used for downloading multimedia content since January 2024, the foundation said in an update. But it's not because human readers have suddenly developed a voracious appetite for consuming Wikipedia articles and for watching videos or downloading files from Wikimedia Commons. No, the spike in usage came from AI crawlers, or automated programs scraping Wikimedia's openly licensed images, videos, articles and other files to train generative artificial intelligence models.

 

This sudden increase in traffic from bots could slow down access to Wikimedia's pages and assets, especially during high-interest events. When Jimmy Carter died in December, for instance, people's heightened interest in the video of his presidential debate with Ronald Reagan caused slow page load times for some users. Wikimedia is equipped to sustain traffic spikes from human readers during such events, and users watching Carter's video shouldn't have caused any issues. But "the amount of traffic generated by scraper bots is unprecedented and presents growing risks and costs," Wikimedia said.

 

The foundation explained that human readers tend to look up specific and often similar topics. For instance, a number of people look up the same thing when it's trending. Wikimedia creates a cache of a piece of content requested multiple times in the data center closest to the user, enabling it to serve up content faster. But articles and content that haven't been accessed in a while have to be served from the core data center, which consumes more resources and, hence, costs more money for Wikimedia. Since AI crawlers tend to bulk read pages, they access obscure pages that have to be served from the core data center.

 

Wikimedia said that upon a closer look, 65 percent of the resource-consuming traffic it gets is from bots. It's already causing constant disruption for its Site Reliability team, which has to block the crawlers all the time before they they significantly slow down page access to actual readers. Now, the real problem, as Wikimedia states, is that the "expansion happened largely without sufficient attribution, which is key to drive new users to participate in the movement." A foundation that relies on people's donations to continue running needs to attract new users and get them to care for its cause. "Our content is free, our infrastructure is not," the foundation said. Wikimedia is now looking to establish sustainable ways for developers and reusers to access its content in the upcoming fiscal year. It has to, because it sees no sign of AI-related traffic slowing down anytime soon.

 


 

Source:
courtesy of ENGADGET

by Mariella Moon

 

If you have any stories or news that you would like to share with the global online community, please feel free to share it with us by contacting us directly at [email protected]

 

Related News


Lahad Datu Murder: Remand Of 13 Students Extende

 2024-03-30 07:57:54

Russia Becomes First State To Recognise Afghanistan's Taliban Government

 2025-07-05 01:57:40

Trump Says US To Start Sending Out Tariff Letters

 2025-07-05 02:23:28