The Moving Bottleneck: Networking, Power, Memory, and the Race to Win AI

For all the competition among AI labs, their primary products are built in much the same way: a large language model is pre-trained on enormous datasets, post-trained, deployed, and refreshed every so often. In 2022, three researchers in Alberta, Canada published a contrarian roadmap for AI that rejects that entire premise.

In this episode of TechSurge, host David Goldman speaks with Patrick Pilarski, who co-wrote the Alberta Plan with Richard Sutton and Michael Bowling. Pilarski is a Canada CIFAR AI Chair, an Amii Fellow, and a professor in the Department of Medicine at the University of Alberta, where his lab builds prosthetic limbs that learn to anticipate what their users intend. In 2017, he was one of three Amii researchers who brought DeepMind’s first international research office to Edmonton, serving as its co-lead until 2023.

The conversation opens with the Alberta Plan’s core premise: an agent that constructs all of its knowledge from its own stream of experience, with no artificial separation between training and deployment, what Pilarski calls “temporal uniformity.” He contrasts this with how today’s major AI labs bolt reinforcement learning onto pre-trained language models during post-training, and addresses head-on why his co-author Rich Sutton has publicly argued that LLMs are a dead end.

Pilarski also breaks news on the podcast: he will soon join ExperienceFlow.AI, a San Francisco-based company with a research lab led scientifically by Richard Sutton, as Head of Edge Superintelligence, focused on bringing continual learning to computing environments where connectivity and compute are limited. He argues that the industry’s massive data center build-out does not lock AI into a single architectural path, and that existing hardware and software stacks have far more flexibility than commonly assumed.

The episode closes with a tour through Pilarski’s own research: how AI-powered prosthetic limbs learn to predict user intent in minutes rather than months, a landmark bone-anchored prosthetic surgery performed in Alberta, and the prospect of supernumerary limbs and wearable technology as a path toward genuine human augmentation. Pilarski also discusses a paper he co-authored arguing that AI agent safety concerns dominating recent headlines are, in fact, reinforcement learning problems the field has studied for decades.

Sign up for our newsletter at techsurgepodcast.com for updates on upcoming TechSurge Live Summits and future episodes.

Every gigawatt of new AI infrastructure now costs $50 to $60 billion to build, roughly $45 billion of which goes straight into GPUs. With more than 400 gigawatts of new compute needed by 2030, the industry faces a roughly $24 trillion capital bill. But according to this week's guest, the real threat to Western AI competitiveness isn't Nvidia, Google, or Broadcom out-maneuvering one another. It's China Inc., which is targeting a cost of under $10 billion per gigawatt, a five- to six-times cost advantage that could reshape who wins the AI infrastructure race.

In this episode of TechSurge, host David Goldman speaks with Raja Koduri, one of the most influential architects in the history of graphics computing. Raja twice led graphics at AMD, directed graphics architecture at Apple, and served as chief architect of Intel's Core and Visual Computing Group. His team helped bring high bandwidth memory to market for AMD's GPUs back in 2015, a technology that now underpins nearly every AI accelerator on the market.

The conversation centers on Raja's new startup, Oxmiq, which he describes as converting "electrons to tokens super efficiently." Raja and David dig into why the bottleneck in AI infrastructure has quietly shifted away from raw compute and toward memory hierarchy, how data moves between SRAM, HBM, and NAND, and between chips, now that models are too large to fit on a single die. Oxmiq's approach uses 3D-stacked, hybrid-bonded memory to unlock 10x the bandwidth of today's HBM and a 10x increase in token generation rate, even on older process nodes.

From there, the discussion turns to how AI coding agents are changing the economics of chip design. Raja argues that spec-writing, once the "boring" 90 percent of the job, is now the hardest and most valuable skill, while writing the code itself is increasingly something agents can handle. He points to OpenAI and Broadcom's Jalapeño chip as evidence that AI-assisted design can compress timelines that used to take multiple generations of custom silicon. Raja also reflects on lessons from building products alongside Steve Jobs at Apple and Lisa Su at AMD, and explains why he believes Intel's decision to kill 3D XPoint memory came at exactly the wrong moment.

The episode closes with Raja's outlook on where AI infrastructure goes next: a future of both massive "token factories" and smaller, personal "token banks," a shift he compares to the transition offices went through with server rooms, becoming invisible infrastructure that simply works. As he puts it, "the more boring you make it, the more it becomes fabulous."

Sign up for our newsletter at techsurgepodcast.com for updates on upcoming TechSurge Live Summits and future episodes.

Almost 2% of U.S. GDP will be spent on AI infrastructure this year, nearly double 2025's figure. But beneath those headline numbers, the composition of that spending has quietly flipped: for the first time, dollars spent on running models in production now outweigh dollars spent training them.

In this episode of TechSurge, host David Goldman speaks with Austin Lyons, a semiconductor analyst at Creative Strategies, co-host of the Semi Doped podcast, and author of the Chipstrat newsletter. Lyons previously worked as a hardware engineer at Intel and as a product manager on John Deere's autonomous tractor and Blue River Technology teams before turning to full-time chip industry analysis.

The conversation opens with why AI buyers have moved from assembling commoditized parts to buying entire pre-integrated systems, tracing how Nvidia's rack-scale approach, exemplified by its 72-GPU Grace Blackwell racks, made turnkey deployment the default, and why that raises the bar for any chip startup trying to compete. Lyons and Goldman then unpack how inference workloads have split into two distinct problems, prefill and decode, and how that split created an opening for SRAM-based challengers to outperform general-purpose GPUs on decode speed.

From there, the discussion turns to the rise of neoclouds, the GPU-rental companies that grew into public businesses worth well over $100 billion combined, and why so many traditional investors missed them. Lyons and Goldman work through the circular financing debate head-on: the mechanics of Nvidia's equity stakes, GPU-backed debt, and hyperscaler off-take agreements that critics compare to dot-com-era vendor financing, and the counterargument that demand is simply outrunning fixed supply.

The episode closes on Lyons's own framework for identifying the next trillion-dollar chip company, built on four conditions including the ability to run trillion-parameter models at rack scale, beat an incumbent on a key performance metric, and land a frontier anchor customer, along with a look at how AI-assisted chip design is lowering the barrier for more companies, from OpenAI to electric vehicle makers, to design their own custom silicon.

Sign up for our newsletter at techsurgepodcast.com for updates on upcoming TechSurge Live Summits and future episodes.