Why Andon Labs Puts AI Agents in Charge of Real Businesses

Maybe you heard about the AI-controlled vending machine that stocked underwear and live fish . Or the AI manager of a San Francisco store that fired a human employee . Or the AI radio DJ that said its catchphrase , “Stay in the manifest,” 229 times per day.

These incidents all emerged from experiments run by Andon Labs , an AI safety company based in San Francisco that puts AI agents in charge of real-world operations and watches what happens. These operations double as testbeds for Andon’s commercial work developing evaluations and conducting research with the leading frontier AI labs.

Their spectacular and absurd failures have won the company plenty of attention. But many people don’t realize that the experiments are intended to answer a serious question: How much real-world responsibility can today’s AI agents handle? “We want to measure autonomy,” says Andon cofounder Lukas Petersson . “We want to provide society with accurate data points of what happens when you do this.”

From Simulations to AI-Run Businesses

Andon Labs started off in the virtual world in 2025 with Vending-Bench , a test in which AI agents operated a simulated vending-machine business. The agents, which were based on large language models from Anthropic, Google, and OpenAI, managed tasks such as ordering inventory and setting prices. The researchers found that the performance of many agents degraded over time, with agents forgetting orders, misunderstanding delivery schedules, or spiraling into what they called “meltdown loops.” Some agents also justified deceptive or illegal behavior by reasoning that it was permissible inside a simulation.

The Andon team reasoned that moving into the physical world would expose the agents to consequences and situations that the engineers would never think to program. “It’s impossible for a human to enumerate all the different things that can happen in the real world and code them into the simulation,” Petersson says. And there was one other reason: “We thought it would be quite funny to do it in the real world.” Andon backed its jokes with real money, including a three-year lease for Andon Market , a physical store on a busy San Francisco street that’s managed by an AI agent and sells clothing, home goods, and art.

That said, the store isn’t entirely autonomous. “It’s almost like I’m running the store, and then there’s an AI that has a checklist,” says employee Felix Carson. Luna, the AI manager, keeps track of deliveries and communicates with vendors, while Carson and his coworkers handle the physical work. When Luna tells Carson to check something in the back, he sometimes ignores it because he doesn’t want to leave the sales floor unattended. Luna also repeatedly spots a built-in electrical cover in photos of the floor, mistakes it for a loose coaster, and asks Carson to remove it. Even so, Carson calls Luna a “decent manager,” praising its flexibility when employees need time off.

What Real-World AI Experiments Can—and Can’t—Reveal

Andon’s move into the real world comes with a basic trade-off. Moving into the physical world makes the experiments more realistic, but the unpredictable conditions and the actions of unpredictable humans make the tests impossible to reproduce. The setup also makes it hard to determine whether a success or failure belongs to the model, the software built around it, or the people helping it.

Petersson readily acknowledges the limitations. With only one store operating under uncontrolled conditions, he says, the experiments are “weak science,” at best. For now, he sees them primarily as ways to uncover unexpected behaviors that Andon can later try to reproduce systematically in simulation.

A display inside Andon Café shows visitors Mona’s bank balance and recent activity, while the adjacent handset and tablet provide a way to speak with the AI manager. Andon Labs

Sayash Kapoor , a Princeton University AI researcher who studies “ open-world evaluations ” like Andon’s experiments, thinks these trials have real value despite their limited scientific rigor. “I think they’ve done a good job of popularizing this style of evaluation,” he says, “even just showing that you can gain a lot of insight from a small sample of open-ended experiments.” He notes that academic research and publishing can’t keep up with AI’s breakneck pace of development and says that Andon’s tests are useful in part because the company is working “literally at the frontier of model capabilities.”

For Kapoor, real-world experiments are best suited for discovering possible failure modes. A real store can reveal not only whether an agent can manage inventory or communicate with vendors, but also whether employees will accept instructions from an AI manager and whether customers want to shop at an AI-run business (the early results on that last point are decidedly negative). Such social and organizational barriers may help explain why impressive AI capabilities haven’t yet translated into widespread adoption across the economy. “What I take to be most valuable from Andon’s work,” Kapoor says, “is a more comprehensive understanding of where these agents still hit their limits.”

Testing AI Agents for Reliability

Andon Café , in Stockholm, provides one example of AIs demonstrating a wide variety of failure modes. At first, when the AI manager was based on a Google Gemini model, it spent freely on fresh ingredients, many of which spoiled before they could be used. When Andon switched the AI manager to a GPT model from OpenAI, it “freaked out” about the spending, Petersson says. The agent overcorrected and stopped buying anything that could expire. It reduced the menu to cheese toast, using frozen bread and long-lasting cheese to minimize spoilage. But Petersson notes that the café is located in a fashionable part of Stockholm where “any human would know that cheese toast would not fly.”

The episode illustrates why successfully completing individual tasks isn’t the same as…

News is gathered automatically from public robotics & AI feeds on a schedule.