Relentless · the podbrain notes ·
4 min read

WTF is happening at xAI | Sulaiman Ghori

Sully Kongori is an engineer at XAI, one of the fastest-growing AI companies founded by Elon Musk in 2023. He joined the company when it had around 100 engineering staff and has worked across multiple projects including ASRock integrations, desktop suite development, iOS development, and currently Macro Hard product...

Relentless Relentless
Subscribe to Notes Upgrade
Relentless episode thumbnail: WTF is happening at xAI | Sulaiman Ghori
Relentless
Key Takeaways
  1. 01

    XAI operates at $2.5 million per commit to main repo, with engineers adding millions in value daily

  2. 02

    Tyler won a Cybertruck by getting a training run working on new GPUs in 24 hours

  3. 03

    XAI built Colossus data center in 122 days using temporary carnival permits to bypass regulations

  4. 04

    Human emulators run 1.5-8x faster than humans, opposite approach from other AI labs focusing on reasoning

  5. 05

    Tesla car computers could provide 1 million VMs for human emulator deployment at fraction of AWS cost

  6. 06

    Company operates with only three management layers: ICs, co-founders/managers, and Elon

  7. 07

    War room operations for Macro Hard have been running continuously for 4 months

  8. 08

    Model iterations happen daily or multiple times per day, including from pre-training

Get the latest ideas from Relentless.

Plus the best new takeaways from other top podcasts — read in minutes, not hours.

or

By continuing, you agree to podbrain's Terms and Privacy Policy.

These notes may contain occasional inaccuracies. Learn how podbrain notes are made

Sully Kongori is an engineer at XAI, one of the fastest-growing AI companies founded by Elon Musk in 2023. He joined the company when it had around 100 engineering staff and has worked across multiple projects including ASRock integrations, desktop suite development, iOS development, and currently Macro Hard product development.

The conversation covers XAI's unique culture of rapid iteration and minimal bureaucracy, where engineers can implement ideas the same day and show them directly to leadership. Kongori discusses the company's approach to building human emulators that operate faster than humans, their innovative infrastructure solutions including the record-breaking Colossus data center build, and their unconventional hiring and management practices.

Key topics include the technical challenges of scaling AI infrastructure, the decision to focus on speed over reasoning in their models, creative solutions like potentially using Tesla car computers as distributed computing resources, and the intense work culture that has enabled XAI to iterate at unprecedented speed in the AI industry.

The $2.5 Million Per Commit Engineering Culture

XAI operates at approximately $2.5 million per commit to the main repository, with individual engineers adding $12.5 million in value through five daily commits

The company maintains only three management layers: individual contributors, co-founders/managers, and Elon, enabling direct communication and rapid decision-making

"No one tells me no. If I have a good idea I can usually go and implement it that same day and show it to Elon or whoever and we got an answer" - Sully

Engineers can work across team boundaries freely, with fuzzy responsibilities allowing anyone to fix infrastructure issues and merge changes immediately

Record-Breaking Infrastructure and Creative Solutions

Tyler won a Cybertruck by successfully getting a training run operational on new GPU racks within 24 hours of Elon's challenge

The Colossus data center was built in 122 days using temporary carnival permits, the fastest way to bypass traditional permitting processes

XAI operates 80+ mobile generators and must coordinate with municipal power companies to seamlessly switch between grid and generator power without interrupting volatile training runs

Tesla car computers could provide 1 million VMs from 4 million North American Tesla vehicles, with 70-80% idle time for running human emulators at fraction of AWS costs

Human Emulator Strategy: Speed Over Reasoning

XAI chose to build human emulators running 1.5-8x faster than humans, opposite to other labs focusing on bigger models with more reasoning capability

"No one's going to wait around 10 minutes for the computer to do something that I could have done in five, but if it can be done in 10 seconds, well, I'd be happy to pay whatever amount of money for that" - Sully

Human emulators can perform any digital task requiring keyboard/mouse input and screen observation, with no software adoption required from customers

Internal testing revealed humans often omit 20+ steps when describing their workflows, requiring direct observation to capture unconscious autopilot behaviors

Unprecedented Model Iteration Speed

XAI produces new model iterations daily or multiple times per day, including from pre-training, which is "not something you ordinarily really see"

Hardware racks can be training-ready within hours of setup, sometimes the same day, compared to industry standard of days or weeks

The company runs 20+ different experiments simultaneously in parallel, enabled by their infrastructure buildout and small team size

War room operations for Macro Hard have been continuous for 4 months, with the team eventually outgrowing the original war room and moving to a cleared gym

Truth and Data Quality Challenges

Finding truthful data requires drilling down to fundamentals, similar to Connections by James Burke who connects seemingly unrelated concepts through physics and inventions

"The internet is not usually the ground truth for whatever thing it is" - the challenge is determining what constitutes fundamental truth in areas like physics or constitutional law

When Elon identifies Grok output errors on X, the team mobilizes immediately to fix issues within 12-24 hours with full postmortems

Evaluation construction is critical since "we're not measuring the fitness to any particular data set" but rather trying to measure against arbitrary truthful outputs

Unconventional Hiring and Team Structure

Everyone at XAI is an engineer, including the sales team: "The sales team is an engineer. Are all engineers. Everyone is an engineer"

The iOS team consisted of only 3 people during major feature rollouts, demonstrating the extreme talent density and individual leverage

Sully conducts 20+ interviews per week, using a proprietary computer vision problem that tests for simple solutions over complex ones

Hiring includes intentionally incorrect requirements to test if candidates will challenge assumptions and push back on impossible specifications

Relentless
From Relentless. Get a note like this from every new episode.
Subscribe to Notes Upgrade

These notes may contain occasional inaccuracies. Learn how podbrain notes are made

0 / 0
Link copied