The biggest names in AI are backing a new way to opt out of scraping
OpenAI, Google, Microsoft, and Anthropic say they will adopt a new standard that publishers can use to opt out of being scraped by AI, as reported earlier by Search Engine Roundtable . With this control, publishers can apply their AI training preferences across multiple AI companies, instead of setting them individually.
The shared "policy token" signal is proposed by Kevin Kelley, a technical lead for standards at Anthropic, as part of the Internet Engineering Task Force (IETF). It will live within a website's robots.txt file , which has long signaled which web crawlers can and can't scrape a site - whether for search indexing, archiving, or …
Read the full story at The Verge.
News is gathered automatically from public robotics & AI feeds on a schedule.
Related
Intel’s stock crashes, knocking $12B from government’s stake
Intel is being hurt by PC sales as the AI industry gobbles memory and other components causing PC prices to rise.
Anthropic expected to receive high ‘junk’ rating from credit agencies following IPO, report says
The new report says the AI lab will likely qualify for a high junk rating.
Supersonic airliner in doubt as company shifts to powering AI data centers
Boom’s airliner may be delayed, and its data center power business lost a customer.
Last-minute Halloween costumes for tech nerds
No Halloween costume? Just be a deceptively cute AI agent icon, a PHP vulnerability, or sleepy time Bryan Johnson.
How a Contractor’s Claude Agent Got Into the Pentagon
This week on “Uncanny Valley,” we discuss the contractor who plugged their AI agent into a Pentagon system, Anthropic’s IPO filing, and the volunteers gearing up to counter ICE at the midterms.
Cracks are emerging in the AI spending boom
Big Tech is pouring billions into AI infrastructure, but mounting debt, negative cash flow, and growing investor skepticism are raising questions about the sustainability of the AI spending boom.