Every gigawatt of new AI infrastructure now costs $50 to $60 billion to build, roughly $45 billion of which goes straight into GPUs. With more than 400 gigawatts of new compute needed by 2030, the industry faces a roughly $24 trillion capital bill. But according to this week's guest, the real threat to Western AI competitiveness isn't Nvidia, Google, or Broadcom out-maneuvering one another. It's China Inc., which is targeting a cost of under $10 billion per gigawatt, a five- to six-times cost advantage that could reshape who wins the AI infrastructure race.

In this episode of TechSurge, host David Goldman speaks with Raja Koduri, one of the most influential architects in the history of graphics computing. Raja twice led graphics at AMD, directed graphics architecture at Apple, and served as chief architect of Intel's Core and Visual Computing Group. His team helped bring high bandwidth memory to market for AMD's GPUs back in 2015, a technology that now underpins nearly every AI accelerator on the market.

The conversation centers on Raja's new startup, Oxmiq, which he describes as converting "electrons to tokens super efficiently." Raja and David dig into why the bottleneck in AI infrastructure has quietly shifted away from raw compute and toward memory hierarchy, how data moves between SRAM, HBM, and NAND, and between chips, now that models are too large to fit on a single die. Oxmiq's approach uses 3D-stacked, hybrid-bonded memory to unlock 10x the bandwidth of today's HBM and a 10x increase in token generation rate, even on older process nodes.

From there, the discussion turns to how AI coding agents are changing the economics of chip design. Raja argues that spec-writing, once the "boring" 90 percent of the job, is now the hardest and most valuable skill, while writing the code itself is increasingly something agents can handle. He points to OpenAI and Broadcom's Jalapeño chip as evidence that AI-assisted design can compress timelines that used to take multiple generations of custom silicon. Raja also reflects on lessons from building products alongside Steve Jobs at Apple and Lisa Su at AMD, and explains why he believes Intel's decision to kill 3D XPoint memory came at exactly the wrong moment.

The episode closes with Raja's outlook on where AI infrastructure goes next: a future of both massive "token factories" and smaller, personal "token banks," a shift he compares to the transition offices went through with server rooms, becoming invisible infrastructure that simply works. As he puts it, "the more boring you make it, the more it becomes fabulous."

Sign up for our newsletter at techsurgepodcast.com for updates on upcoming TechSurge Live Summits and future episodes.