Aaron Levy, CEO of Box, joins A16Z board partner Steve Sanofsky and A16Z general partner Martin Casado to discuss the enterprise implications of AI agents at scale.

The conversation explores the fundamental tension between Silicon Valley's optimistic timeline for AI adoption and the practical challenges enterprises face when deploying agents. They examine how software architecture must evolve when agents outnumber employees by orders of magnitude, the security and access control complexities this creates, and why existing enterprise systems like SAP aren't easily replaceable.

The discussion covers the economic implications of token-based compute budgets for engineering teams, the emergence of new business models enabled by agents, and why the diffusion of AI capabilities will take longer than many expect due to organizational and technical constraints.

Building Software for Agent-First Workflows

"If you have a hundred or a thousand times more agents than people, then your software has to be built for agents" - Aaron, describing Box's shift to prioritize agent interfaces alongside human ones

The emerging paradigm gives coding agents access to both SaaS tools and knowledge work workflows, creating a "superpower" where agents can code their way through any task

"Algorithmic thinking is really, really, really hard for the vast majority of people who have jobs" - Steve, noting that most people would fail at creating flowcharts for their work processes

The abstraction layer historically required "a highly skilled, very specific individual within an organization" to develop tools that others could use

Enterprise Security and Access Control Challenges

"You have all the liability of whatever they're doing" - Aaron on why agents can't be treated exactly like human employees despite having similar access

Agents pose unique security risks because "you can social engineer it 10 times easier than a human" and they can be prompt-injected to leak information

Box's CLI implementation with Claude creates coordination challenges when 5,000 employees have agents hitting shared systems "10,000 times an hour"

The current solution involves giving agents separate identities - "its own phone number," "its own Gmail account," and "its own credit card" - treating them as distinct entities

Why Legacy Systems Will Persist Despite AI

"It's just absurd to think you're going to vibe code your way to like SAP" - Steve, emphasizing the deep domain knowledge embedded in enterprise systems

Domain expertise in systems like SAP isn't "just represented in some well-orchestrated data layer" but exists "in the UI, in middle tiers, in just how you use it"

"The diffusion of AI capability is going to take longer than people in Silicon Valley realize" because enterprises face risks that startups don't have

Agents will eventually pressure companies to upgrade legacy systems, with agents saying "you need to finally rip out your legacy HR system, or I'm not going to be able to automate this workflow"

Agent Selection Criteria and System Architecture

"Agents are very, very good at picking the right back end for whatever they're doing" based on "cost parameters," "durability," and collective wisdom rather than interface polish

"People in the abstract say things like, now you're marketing to agents... I actually think that's almost exactly wrong" - Martin, arguing agents push for better systems, not better interfaces

The industry focus on agent-friendly interfaces misses that "agents are actually pretty smart at choosing the right technology" based on meaningful technical criteria

Historical precedent shows "layers never go away. They just get layered" because they encode "organizational boundaries" and "compatibility" requirements beyond just software logic

The Economics of Compute Budgets and Token Costs

"The engineering compute budget conversation is going to be the most wild one in the next couple years" as CFOs must allocate between 1% and potentially 100% of engineering expense to tokens

"Every single one of them has gone asymptotic in the last six months" - Steve on his 240 portfolio infrastructure companies seeing massive usage growth from increased software development

"The biggest problem right now is everybody is trying to figure out the economics when they're off by at least an order of magnitude on how big the opportunity is"

Engineers now face granular compute decisions: "Do you want that to be a long-running prompt? Do you want to parallelize that? What is your comfort level of wasted tokens?"

The token pricing model mirrors historical technology transitions where "IBM was selling more MIPS for fewer dollars every year" until algorithmic improvements made the constraint irrelevant