6.3 Million Orders Vanished: How an AI Agent Reading an Old Wiki Broke Amazon’s Checkout

Six hours. That’s how long Amazon’s retail infrastructure was effectively dead on March 5th, locking customers out of checkout, account information, and product pricing across North America. The damage? A 99% drop in orders, roughly 6.3 million transactions that simply vanished into the digital void.

The culprit wasn’t a DDoS attack, a failed database migration, or a rogue intern with root access. According to internal documents viewed by the Financial Times and CNBC, the meltdown traces back to something far more embarrassing: an AI agent that hallucinated operational guidance from an outdated internal wiki, and an engineer who trusted it.

The Emergency Meeting and the Vanishing Document

Dave Treadwell, Amazon’s Senior Vice President of eCommerce Foundation, doesn’t usually mandate attendance at the weekly “This Week in Stores Tech” (TWiST) meeting. But on March 10th, he made it compulsory. The topic: a “deep dive” into why the company’s retail infrastructure had suffered four high-severity “Sev 1” incidents in a single week, incidents severe enough to take down critical systems.

The smoking gun was initially right there in the briefing documents. One internal memo identified “GenAI-assisted changes” as a contributing factor in a “trend of incidents” stretching back to Q3 2025. Then, according to the Financial Times, that reference was scrubbed before the meeting took place. Amazon later claimed only one incident involved AI tools and that “none of the incidents involved AI-written code”, attributing the March 5th meltdown to “an engineer following inaccurate advice that an agent inferred from an outdated internal wiki.”

The distinction between “AI-written code” and “AI-inferred advice” is the kind of corporate linguistic gymnastics that would make a PR team proud. The reality is simpler: Amazon’s own AI coding assistant, Q, suggested a course of action based on stale documentation, and the resulting deployment propagated with “high blast radius”, a term that appears repeatedly in internal communications to describe changes that ripple catastrophically through interconnected systems.

The Mechanics of a $200 Billion Oops

To understand how a wiki entry could tank a global e-commerce platform, you need to understand Amazon’s “control plane”, the software layer that orchestrates how data flows across its massive distributed infrastructure. When engineers make changes to this layer, they typically rely on documented procedures and automated safeguards.

The problem? Generative AI systems are probabilistic, not deterministic. Ask the same question twice, and you might get two different answers. That’s fine for drafting marketing copy, it’s lethal when you’re configuring load balancers or payment gateways.

On March 2nd, another incident tied to Amazon Q resulted in incorrect delivery times being displayed across marketplaces, causing 120,000 lost orders and 1.6 million website errors. Internal documents noted that “GenAI’s usage in control plane operations will accelerate exposure of sharp edges and places where guardrails do not exist.” Translation: we’re moving fast, breaking things, and discovering the safety rails are painted on.

The March 5th incident was worse. A production change deployed without “Modeled Change Management”, Amazon’s formal documentation and approval process, allowed a single authorized operator to execute a high-blast-radius configuration change with, as one document put it, “no automated pre-deployment validation” and “no guardrails.”

Controlled Friction: The Return of Human Gatekeepers

Amazon’s response reveals the uncomfortable truth about Amazon’s long-term automation strategy: when the robots start costing you millions per hour in lost revenue, you bring the humans back.

Treadwell announced a 90-day safety reset affecting approximately 335 “Tier-1” systems, services that directly impact consumers. Under the new “temporary safety practices”, engineers must now secure senior engineer sign-off for AI-generated code and use internal documenting tools that strictly adhere to central reliability engineering rules. The company is introducing what it calls “controlled friction” into the deployment pipeline, deliberate slowdowns requiring dual authorization and automated validation.

This is a remarkable reversal for a company that just laid off roughly 30,000 corporate workers between October 2025 and January 2026, with CEO Andy Jassy explicitly citing AI-driven “efficiency gains” as the rationale for needing fewer employees. The irony isn’t lost on observers: Amazon is spending $200 billion on AI infrastructure in 2026, more than any company on Earth, only to discover that the technology requires more human oversight, not less.

Elon Musk, never one to miss a chance to comment on AI safety, responded to reports of the meeting with a terse “Proceed with caution”, advice that apparently didn’t reach Amazon’s engineering teams in time.

Deterministic vs. Agentic: The Architecture Problem

The deeper issue here isn’t just that AI makes mistakes, it’s that Amazon tried to apply “agentic” AI (systems that make decisions and take actions autonomously) to infrastructure that requires deterministic reliability. When you’re handling 6.3 million orders, “mostly right” isn’t a viable engineering standard.

Amazon’s internal documents reveal the company is now scrambling to build “both deterministic and agentic safeguards”, acknowledging that the current generation of AI tools can’t be trusted to operate unsupervised in critical paths. The impact of AI on corporate management roles is becoming clear: someone needs to be accountable when the algorithm confidently recommends the wrong configuration, and that someone can’t be a Large Language Model trained on data that may or may not reflect current operational reality.

The risks of automating customer access are equally stark. When you automate the engineers who understand your systems, and then automate the systems themselves with tools that don’t understand context, you end up with a $1.5 trillion company brought to its knees by a hallucinated wiki citation.

The Poison Fountain Effect

Security researchers have long warned about “data poisoning”, adversarial attacks where bad actors feed false information into AI training data. But Amazon’s incident reveals a more mundane threat: organizational amnesia. Internal wikis are notoriously stale, procedures written for legacy systems persist years after those systems are deprecated. When AI agents treat this outdated documentation as ground truth, the result isn’t malicious sabotage, it’s institutional confusion at machine speed.

Anthropic’s research on small-sample poisoning demonstrates how easily model behavior can be skewed by contaminated training data. Amazon just discovered that you don’t need a sophisticated adversary to poison your AI, you just need an engineering culture that values velocity over verification.

What This Means for the Industry

Amazon isn’t alone in discovering that AI-assisted coding requires guardrails. Microsoft’s Satya Nadella recently admitted that AI writes up to 30% of Microsoft’s code, even as Windows 11 suffers from reliability issues that have forced the company into a “trust rebuilding” effort.

The uncomfortable reality is that generative AI excels at producing plausible answers, not necessarily correct ones. In a retail environment where a single configuration error can vaporize 6.3 million orders, “plausible” isn’t good enough. Amazon’s retreat to “controlled friction”, human-in-the-loop verification for AI-assisted changes, is an admission that the “move fast and break things” ethos, applied to AI-generated infrastructure changes, breaks things that cost millions of dollars per hour.

For engineering leaders, the lesson is clear: agentic AI belongs in sandboxed environments, not production control planes, until you’ve established deterministic safeguards that can verify every action. And maybe, just maybe, keep those senior engineers you were planning to replace with algorithms. You’ll need them to clean up when the wiki-reading robots get confused.