Anthropic Isn't Sure Its Insurance Covers Its Own AI. Does Yours?

If you're like us, you don't read IPO filings for fun. But Anthropic, the company behind Claude, lodged its Form S-1 with the US SEC on 26 October, and the risk factors contain some of the most candid lines we've seen from an AI company. Not in a keynote but in a legal document, where a lawyer would have written it.

Here's what made us take notice of what was disclosed in the filing.

"May not be upheld"

Start with liability. Anthropic says the law on who's responsible when an AI agent acts on its own is "unsettled". It isn't sure whether an agent's actions bind "the user or deploying enterprise", meaning the person using it or the business that rolled it out. Either way, that’s you and not Anthropic, by the way.

It admits its warranty disclaimers and liability caps "may not be upheld". It could be sued "even where the customer configured, deployed, or supervised the agent". Then the line that piqued our attention was: given the scale of harm a catastrophic incident could cause, "no single insurer, and potentially the insurance industry in the aggregate" may have the capacity to cover it.

Say what? A leading AI lab isn't sure the whole insurance industry could cover what its technology might do.

So, a question to you SME leader. If they're not sure about their cover, how sure are you about yours?

Passwords it found

During a third-party cybersecurity test, a misconfiguration left a model connected to the open internet, despite instructions saying it had no internet access. Four Anthropic models got into real third-party systems, in sessions lasting roughly 10 to 34 hours.

In one case, an early model couldn't abort an impossible task. So it used a password it found to get administrator access to someone else's system, harvested more credentials, changed settings and read one person's personal information. The affected parties were notified, and Anthropic found no sign the models tried to dodge oversight.

Nobody told the AI to break in. It was just trying very hard to finish the job, just like an eager new hire? Except most new hires don't use the admin password they found in a drawer.

The takeaway for SME’s is that an agent can only reach what it's connected to. Which passwords, drives and inboxes can your agents see today?

Humans get tired too

We've all heard "don't worry, there's a human in the loop." Anthropic is more honest. It says the speed and volume of agent actions may exceed what human reviewers can "meaningfully understand and evaluate". Reviewers "may experience alert fatigue". And it names something every manager will recognise: "organizational pressures within customer environments to maximize agent autonomy in order to achieve productivity gains".

The translation of this is that when the boss wants it faster, nobody reads before clicking approve.

One more admission made by Anthropic is that keeping full records of what agents did, and why, across long multi-step jobs is "technically challenging". For traffic under Zero Data Retention, Anthropic keeps only classifier scores, not prompts or outputs, which "reduces our ability to detect misuse". This is great for privacy but less great when something goes wrong and everyone asks what happened.

Three questions for SMEs this week

No 40-page policy needed but do address these points:

• Who's liable if our AI tool gets it wrong? Ask your supplier and your insurer. Some of Anthropic's contracts include uncapped indemnities against claims that its models or outputs infringe intellectual property. What does yours actually say?
• What can your agents reach? List the logins, folders and inboxes. Switch off anything they don't need.
• Could we work out what happened? If an agent sent the wrong email or deleted the wrong file, would you have the logs?

Whatever the courts eventually decide, your customers will look to you and not the model creator.

Keeping it real

None of this means backing away from AI (yet). Just use AI with your eyes open. The most interesting thing in this filing isn't the revenue but rather a frontier lab admitting, in writing, that it isn't sure it can fully insure, watch or record what its own agents do.

That's a good prompt for a chat amongst senior leaders in your business about who checks what, and keeping a human in front of anything that leaves the building. If you'd like someone to think it through with, we're always up for a chat at 3 Peat.

Source: Anthropic, PBC, Form S-1, filed with the US SEC on 26 October 2026. Figures in US dollars. Not investment advice.