Compute concentration, RL's reward signal failures, and the hard limits of vibe coding dominated the week's most substantive conversations. A $20B+ Cerebras-OpenAI hardware deal, Cosine's UK sovereign LLM mandate, and Anthropic's inference-driven cluster contracts all point to the same underlying dynamic: whoever controls compute controls the frontier. Meanwhile, practitioners from Wix to Cosine are surfacing the unglamorous ceilings — business logic complexity, code slop from correctness-only rewards, and the still-unsolved problem of AI agent memory — that separate the hype cycle from what's actually shipping.
The Cerebras-OpenAI deal is one of Silicon Valley's largest chip contracts ever
Andrew Feldman on Building a Chip 58x Larger Than Nvidia's
Cerebras has signed a deal to supply OpenAI with more than $20B in hardware over several years — one of the largest chip contracts in Silicon Valley history, and a direct signal that Nvidia alternatives are now operating at frontier scale.
Cerebras claims a 20x inference speed advantage over competitors — a concrete, falsifiable benchmark practitioners should stress-test
The Cerebras wafer-scale chip is 58x larger than any other chip on the market, eliminating inter-chip communication bottlenecks that plague GPU clusters
Feldman cautions that many headline AI deals "didn't have teeth" — even insiders can't always tell which announcements are real
Source episodes
Sourced from 80 episodes across 11 podcasts this week
"we'll be doing more than 20 billion dollars of hardware with them over the next several years."
Nvidia's balance sheet is a competitive weapon, not just a financial tool
Andrew Feldman on Building a Chip 58x Larger Than Nvidia's
Nvidia uses investments in NeoClouds and model builders to create soft lock-in that crowds out alternative chip suppliers. Feldman names this as a structural threat to the broader AI hardware ecosystem.
If Nvidia invests in a NeoCloud, that cloud is less likely to adopt non-Nvidia chips
Free GPU tokens to startups are, in Feldman's words, a "drug pusher" strategy — creating dependency before startups have the leverage to diversify
The tactic compounds: model builders receiving Nvidia investment face pressure not to use competing silicon
"I think these are drug pushers. Here, little girl, try a little bit. Just a little bit."
Hardware-software co-design is a genuine engineering discipline, not a coordination fix
Andrew Feldman on Building a Chip 58x Larger Than Nvidia's · Andrew Feldman: Why Hardware-Software Co-Design Is Never as Easy as It Looks
The collapse of the OS abstraction layer in AI has forced chip and software teams to design together — and the tradeoffs are painful, not just organizational. Companies that master co-design (Google with TPUs/Gemini, Cerebras with OpenAI) gain compounding advantages.
Every software simplification adds hardware complexity, rippling into schedules and resources
AI's scale and speed requirements have made the traditional separation of chip and software development untenable
Feldman: "The misconception is that it's easy and all you need to do is get in the room"
"Anything you do to make it easier to write the software makes it harder to do the hardware."
OpenAI is burying chat to bet on knowledge work
Codex is GONE
OpenAI merged ChatGPT and Codex and structurally de-emphasized general chat in the app UI — a concrete product signal that both OpenAI and Anthropic have concluded knowledge work, not conversational Q&A, is where AI delivers the most value and revenue.
Regular chat is now a small pop-up in the bottom right corner of the ChatGPT app — actively discouraged
The new default experience is ChatGPT Codex, foregrounding coding and document work
Anthropic is making the same strategic bet: AI is most valuable "when it's being used for real work"
Vibe coding hits a hard ceiling at business logic complexity
Wix Founder: Will Base44 Win the Vibe-Coding Wars? · "You're not gonna vibe code Shopify..." | Wix CEO
The Wix CEO's thesis: vibe coding tools are real, but they cannot replicate the layered business logic of mature SaaS platforms. An internal experiment makes this concrete.
A team of professional developers — including the original engineers who built the feature in Wix — couldn't replicate a hairdresser app in Base44 after two weeks
"You're not going to vibe code Shopify no matter how good you are — the business logic is too hard"
Atlassian is the SaaS company the CEO is most worried about: developer customers are the most likely to self-build with code
"Two weeks later they still didn't achieve that."
Base44's economics challenge the "build your own model" narrative
Wix Founder: Will Base44 Win the Vibe-Coding Wars?
For complex generative tasks, fine-tuning your own model yields only 1–30% cost savings over frontier models — far below the 14–16x figures sometimes cited. Quality still dominates the trade-off.
Base44 was a one-person company acquired for $80M; it has since roughly doubled revenue to ~$160M ARR
Wix's $2.8B market cap on $2.1B revenue and $400M free cash flow implies the market is assigning negative value to the core Wix business if Base44 alone would be worth ~$8B at peer multiples
The CEO has reversed his AGI timeline fears: "I think it's going to take much longer than I anticipated"
RL rewards correctness but silently reinforces code slop
Watching America Run Away With AI - Alistair Pullen (Cosine AI)
When the only RL reward signal is whether a unit test passes, every trajectory that reaches the right answer gets reinforced — including unsafe, bloated, or insane approaches. This is the structural root cause of AI-generated code quality problems.
A 256K-token rollout culminating in a 1 or 0 treats all tokens as equally responsible for the outcome — clearly wrong
Credit attribution to high-entropy decision points within trajectories is the next frontier for RL efficiency
Real user prompts are chaotic ("f you, it doesn't work, why the hell") — Anthropic's Claude Code trajectories give them access to this messy real-world distribution, a durable training data edge
"It could have come up with the most insane way of doing something... And all of those things get reinforced whether you like it or not when you give that reward based on purely correctness."
Memory in AI agents is still fundamentally a hack
Watching America Run Away With AI - Alistair Pullen (Cosine AI)
No current memory implementation — VectorDB, RAG, memory tools — clears the bar of "not a hack," according to Cosine's CEO. The real solution (continual learning baked into latent space) remains unsolved and is a minefield for reward hacking during RL.
Alistair reviewed Claude Code, ChatGPT, and Cosine's own implementation: none of them pass the test
Cosine's swarm used 253 sub-agents to build a mechanical watch compiler in one shot — a task frontier models can't do alone
Claude Sonnet is estimated at ~1–1.5T total parameters (MoE), 100B+ active — rare concrete numbers from a practitioner who reverse-engineered the architecture
Sovereign AI is a real business tailwind, not just geopolitics
Watching America Run Away With AI - Alistair Pullen (Cosine AI)
Cosine has won the UK government mandate to build Britain's first sovereign LLM, with compute allocated on the Isambard supercomputer — and US export controls have dramatically accelerated enterprise inbound interest.
Government compute allocation is more valuable than raising $50–100M in venture funding, because a large chunk of any raise would go straight to compute
Cosine's business model — licensing weights to customers who run their own inference — lets them skip the massive data center costs straining labs like Anthropic
Alistair argues Anthropic's recent cluster deals (e.g., Colossus) are driven by inference pressure, not training — a counterintuitive reframe of where the frontier scaling bottleneck actually sits
"Has it been a bit of a gift what's happened recently? Given our positioning, absolutely. Yes. Without a doubt."
Fine-tuning model weights is a destructive operation, not a surgical one
Karpathy: Context Windows are a cheap way to manipulate AI
Karpathy draws a sharp line between context window manipulation (cheap, reliable) and weight modification (risky, underdeveloped science). Practitioners who assume fine-tuning is safe and additive should reconsider.
Context windows "just work" and are the primary customization lever today
Touching weights risks degrading overall model intelligence — "fine-tuning without losing capabilities" is an open, unsolved problem
Continual learning and deep model adjustment remain a "developing science" with no reliable playbook
"It's a lot more tricky to touch the weights than just the context windows because you're actually fundamentally changing the full model and potentially its intelligence."
Hardware quality is the moat that AI won't dissolve
Boston Dynamics' Brian Ringley: Why AI Won't Commoditize Robots · Boston Dynamics' Brian Ringley on why great hardware isn't enough to build a humanoid robot
The conventional wisdom that a universal AI brain will commoditize robot hardware is wrong — the software explosion actually raises the stakes for physical quality. But even world-class hardware isn't sufficient on its own.
Ringley's thesis: you still need a body that can interact with the world, won't break down, and moves efficiently — regardless of how good the AI gets
The real humanoid challenges are reliability, serviceability, and cost — not flashy design or benchmark performance
Boston Dynamics believes it has the world's best hardware; the remaining gap is making it affordable and field-serviceable at scale
Key Takeaways
Compute is the real currency of AI strategy. The Cerebras-OpenAI $20B+ deal, Anthropic's inference-driven cluster contracts, and Cosine's government compute allocation all point to the same conclusion: whoever controls compute controls the roadmap.
Nvidia's moat is behavioral, not just technical. Investment-driven lock-in via NeoClouds and free GPU tokens is a deliberate strategy to crowd out alternatives before they reach scale.
Vibe coding is real but bounded. Business logic complexity is a hard ceiling — two weeks of professional engineering couldn't replicate a hairdresser app in Base44, and no one is vibe-coding Shopify.
RL's reward signal problem is producing code slop at scale. Correctness-only rewards reinforce any trajectory that passes a test, including unsafe and bloated ones; credit attribution within rollouts is the unsolved problem.
Fine-tuning model weights is riskier than most practitioners assume. Karpathy flags capability degradation as an open problem — context windows are the safe lever; weight modification is a developing science.
Sovereign AI is a genuine business category. Export controls and geopolitical fragmentation are creating real commercial tailwinds for non-US AI companies, with government compute allocation potentially more valuable than VC funding.