The Internet Archive’s Wayback Machine is one of the web’s most valuable resources, enabling us to access earlier versions of webpages and websites. It performs an invaluable role in preserving information that would otherwise be lost when websites go offline, as well as providing a practical tool to track updates made to a web page.
However, the organization says that it is now under severe threat thanks to media organizations blocking access to the archive’s web crawler – despite those same publishers relying on it for some of their own stories …
Wired reports that 23 major news sites are currently blocking the web crawler used by the Internet Archive to populate the Wayback Machine. One of these is USA Today.
USA Today published an excellent report that revealed how US Immigrations and Customs Enforcement delayed disclosing key information about the impacts of its detainment policies. The authors used the Internet Archive’s Wayback Machine to compile and analyze detention statistics from ICE and track how the agency had changed under the Trump administration […]
[Yet] USA Today Co., the publishing conglomerate formerly known as Gannet that runs both its namesake paper and over 200 additional media outlets, bars the Wayback Machine from archiving its work. “They’re able to pull together their story research because the Wayback Machine exists. At the same time, they’re blocking access,” Wayback Machine director Mark Graham says […]
According to analysis by the artificial-intelligence-detection startup Originality AI, 23 major news sites are currently blocking ia_archiverbot, the web crawler commonly used by the Internet Archive for the Wayback project. The social platform Reddit is too.
USA Today said that it had to take action to block scraping bots, and it is not specifically seeking to block the Internet Archive.
However, journalists say that the Wayback Machine is a vital tool that needs to be safeguarded.
A coalition collected more than 100 signatures from working journalists who recognize the tool’s value and presented a letter of support to the Internet Archive. Signatories range from television mainstay Rachel Maddow to independent reporters like Spitfire News’ Kat Tenbarge and User Mag’s Taylor Lorenz.
“In previous generations, journalists would turn to the physical archives of a local newspaper or of a local public library to access historical reporting and follow the threads of the present back into history,” the letter reads. “With many newspapers closed, and no clear path for local public libraries to preserve digital-only reporting, the work of safeguarding journalism’s record increasingly falls to the Internet Archive.”
I’ve signed, and other journalists can do so here.
- Official Apple Store on Amazon
- Apple’s iPhone cases: iPhone 17 | iPhone 17 Pro and Pro Max | iPhone Air
- Wireless CarPlay adapter (2026 update)
- AirTag holders and accessories
- Mac Pro-style Mac mini casing
- NordVPN – privacy-first VPN with no logs and independent audits to verify
FTC: We use income earning auto affiliate links. More.
Comments