a16z · the podbrain notes ·
4 min read

Ben Thompson: Anthropic, the Pentagon, and the Limits of Private Power

Ben Thompson, founder of Stratechery, joins John Kukin and Jordi Hayes to discuss his essay "Anthropic and Alignment" following the Department of War's designation of Anthropic as a supply chain risk. Thompson argues this represents the first real-world manifestation of theoretical debates about AI governance and...

a16z a16z
Subscribe to Notes Upgrade
a16z episode thumbnail: Ben Thompson: Anthropic, the Pentagon, and the Limits of Private Power
a16z
Key Takeaways
  1. 01

    "If AI is as powerful as its builders claim, the people with guns are going to want to have a say" - Ben Thompson on government intervention in AI

  2. 02

    The Department of War designated Anthropic a supply chain risk after the company refused to remove safeguards against mass domestic surveillance and autonomous weapons

  3. 03

    "You may not be interested in politics, but politics has an interest in you" - Thompson's core thesis about AI companies facing inevitable government pressure

  4. 04

    Private AI development requires hundreds of billions in CapEx, making it economically impossible for governments to build competitive models independently

  5. 05

    Thompson argues that unilaterally imposing AI restrictions, even good ones, invites "very bad outcomes" for companies asserting such positions

  6. 06

    The Taiwan-China dynamic creates existential risks: "What happens if we get super powerful AI and China doesn't? They're going to bomb TSMC out of existence"

  7. 07

    Nuclear weapons regulation may not apply to AI since "dealing with weights and software is very different than dealing with fissionable material"

  8. 08

    Thompson warns against giving up democratic processes: choosing private executives over elected officials to make AI decisions abandons democratic governance

Get the latest ideas from a16z.

Plus the best new takeaways about artificial intelligence from other top podcasts — read in minutes, not hours.

or

By continuing, you agree to podbrain's Terms and Privacy Policy.

These notes may contain occasional inaccuracies. Learn how podbrain notes are made

Ben Thompson, founder of Stratechery, joins John Kukin and Jordi Hayes to discuss his essay "Anthropic and Alignment" following the Department of War's designation of Anthropic as a supply chain risk. Thompson argues this represents the first real-world manifestation of theoretical debates about AI governance and government control.

The conversation explores the fundamental tension between private AI companies and government authority, drawing parallels to nuclear weapons regulation while highlighting key differences. Thompson examines how AI's massive capital requirements create unique dynamics where private companies must serve broad markets to remain viable, yet face increasing government pressure as AI capabilities grow.

The discussion covers geopolitical implications, particularly around Taiwan and China, surveillance concerns, and the broader question of whether democratic processes can adequately govern transformative AI technology. Thompson emphasizes this is not a normative analysis but rather an examination of inevitable power dynamics as AI becomes more influential.

The Department of War Confrontation with Anthropic

The Department of War designated Anthropic a supply chain risk after the company refused to remove safeguards against mass domestic surveillance and autonomous weapons, marking the first major government-AI company confrontation.

Thompson notes information asymmetry may have played a role: "Department of War knows they're headed into a major, what is now looking like a drawn-out conflict" while Anthropic viewed the deadline as arbitrary.

The NSA's role as part of the Pentagon clarified why domestic surveillance became central to the dispute, with Thompson admitting "I was so confused why the Pentagon was so obsessed with domestic surveillance."

AI Economics vs Nuclear Weapons Regulation Model

The Making of the Atomic Bomb, which Dario Amodei reportedly admires, suggests parallels between nuclear and AI regulation, but Thompson identifies crucial differences in the economic models.

AI development requires "hundreds of billions approaching a trillion dollars a year in CapEx" that "is only sustainable and viable if you're selling to everyone," unlike government-controlled nuclear programs.

Thompson references Intel's Bob Noyce decision to design for consumer markets rather than government contracts, arguing this principle applies "on steroids with AI" due to massive upfront costs.

"Dealing with weights and software is very different than dealing with fissionable material" - making AI much harder to control through traditional regulatory approaches.

Geopolitical Stakes and the Taiwan Question

Thompson criticizes the lack of public discussion about Taiwan in AI policy debates, arguing "Taiwan is 70 miles off the coast of China" and creates dangerous dependencies.

"What happens if we get super powerful AI and China doesn't? They're going to bomb TSMC out of existence because suddenly that becomes optimal" - Thompson on escalation risks.

He advocates allowing Chinese companies to fab with TSMC and buy older-generation chips, arguing "it is a safer equilibrium to have China dependent on Taiwan than to try to cut them off."

Thompson warns that absolutist chip restrictions ignore massive trade-offs: "Everything is a trade-off" in AI geopolitics.

Democratic Governance vs Private Executive Power

Thompson identifies a fundamental choice: "I would rather have Dario Amodei making these decisions than whoever comes out of our screwed-up democratic process" represents "giving up on the democratic process."

He argues that without legal frameworks, "the only alternative is someone has to decide," leading to "a private executive making those decisions" about powerful AI systems.

"If AI is what it is, I think that's going to be intolerable to those with power to have a private executive making those decisions" - Thompson on inevitable government intervention.

His prescription involves working within democratic processes: Anthropic should "start a political action committee that lobbies for change" rather than unilateral resistance.

Surveillance Laws and Digital Age Loopholes

Thompson supports Anthropic's position on digital surveillance, noting "all these laws assume someone has to actually physically go somewhere and tap into a phone line."

"If you can do it with computers at scale, suddenly you had all these assumptions that limited what the government could do that magically disappear not because the law changed, but because we got computers."

He advocates for new surveillance laws rather than retrofitting old frameworks, similar to his antitrust analysis where "antitrust laws are fundamentally unsuited to dealing with aggregators."

Thompson warns AI will amplify existing surveillance loopholes "on steroids," making current legal frameworks even more inadequate.

Industry Dynamics and Competitive Positioning

OpenAI's approach involves agreeing "to have a jailbreak competition with the U.S. government" while maintaining some restrictions on digital surveillance capabilities.

Thompson notes Anthropic "has a local advantage in that most people in the industry are with them" but "a national PR problem" with the broader public.

Google's Project Maven history created precedent for employee resistance to military contracts, with Thompson noting this "naive view of the world that doesn't understand why militaries are important."

The fundamental tension persists: "Are you an American company subject to American law and even beyond law, just morally compelled to support the U.S. military or not?"

a16z
From a16z. Get a note like this from every new episode.
Subscribe to Notes Upgrade

These notes may contain occasional inaccuracies. Learn how podbrain notes are made

0 / 0
Link copied