Anthropic's IPO Filing Devotes Dozens of Pages to Risks, Including Catastrophic Ones

The Securities and Exchange Commission published Anthropic's S-1 form on Monday, a step ahead of the company's heavily anticipated initial public offering. Much of the early discussion has centered on whether Anthropic's fast-growing revenues can outpace its even higher costs.

The filing also strikes an optimistic tone about the technology. It says AI will rapidly transform every sector of the global economy in ways greater than the impact of industrialization, electricity, semiconductors, and the internet. Yet the company also spends a striking amount of space explaining how its business could fail.

The "Risk Factors" section runs 93 pages and more than 70,000 words, and it covers literally hundreds of possible problems that could hurt Anthropic in the coming years. Many of them are familiar to almost any company, including competition, costs, customer acquisition and potential regulation. Others point to weak spots specific to Anthropic, and to the hype-filled AI bubble more generally.

Ars Technica reviewed the section and picked out seven risks it found the most surprising, interesting, or thought-provoking. The outlet described its list as far from comprehensive, but said prospective Anthropic investors should keep those risks in mind.

The most serious category involves dangers that could "pose catastrophic or existential risks to humanity," in Anthropic's words. The company acknowledges that such concerns have been raised inside its own organization, as well as by other researchers, regulators, policymakers, and industry participants.

Anthropic says the specific worry does not apply to any of its current models. It concerns future systems that could "substantially automate AI research and development" without human help.

If those systems were "allowed to operate with a high degree of autonomy," the filing warns, they could advance so quickly that AI progress would outrun the company's ability to evaluate and safeguard new models.

The document therefore sets a bullish view of AI's economic impact beside a long inventory of what could go wrong. That contrast is likely to shape how investors read the filing as the offering approaches.