The Unassailable CUDA Moat That Makes Nvidia AI’s Unlikely Ruler

Bear with me for kicking things off with a well-worn trope: a piece of old Wall Street finance jargon that has recently seeped into Silicon Valley’s everyday vocabulary, because there is no way around talking about "moats" here.

Popularized decades ago by Warren Buffett to describe a company’s durable, defensible competitive edge, the term exploded into startup pitch decks after a leaked internal Google memo titled We Have No Moat, and Neither Does OpenAI made headlines, fretting that open-source AI would storm and loot Big Tech’s carefully fortified castle walls. Fast forward a few years, and those castle walls have barely been dented. Save for a short, sharp industry panic when DeepSeek first launched its competing open models, open-source AI tools have never meaningfully outperformed top closed proprietary systems. The twist? None of the big-name frontier labs — OpenAI, Anthropic, Google — actually hold a meaningful, sustainable competitive moat to speak of.

The only company that does have a real, unassailable moat right now is Nvidia. CEO Jensen Huang has openly called it his most precious "treasure". And no, it is not the cutting-edge silicon hardware you would expect from a chip manufacturer. It is a piece of software called CUDA. What sounds like a weird, FDA-banned lab chemical may very well be the only true uncrossable moat in the entire AI industry.

Short for Compute Unified Device Architecture, almost no one bothers to expand the clunky acronym, the same way no one spells out the full terms for "laser" or "scuba" — everyone just pronounces it KOO-duh. So what makes this all-important treasure so valuable? If you had to sum it up in one single word: parallelization.

Take a simple example: imagine you task a computer with filling out a full 9×9 multiplication table. On a single-core CPU, all 81 individual calculations run one after another, slowly and dutifully. But a GPU with 9 separate cores can split the work so each core handles its own full column, from 1×1 to 1×9 on the first core, 2×1 to 2×9 on the second, and so on, cutting total runtime by 9 times. Modern GPUs are even smarter: if programmed to recognize that multiplication is commutative, that 7×9 = 9×7, they can eliminate all duplicate work, cutting total operations from 81 to just 45 and nearly halving total runtime again. When a single large AI model training run can cost over $100 million, every tiny optimization adds up to massive savings.

Nvidia’s GPUs were originally built exclusively to render high-resolution graphics for video games. Back in the early 2000s, a Stanford PhD student and avid gamer named Ian Buck realized the GPU’s unique architecture could be repurposed for general high-performance computing outside gaming. He built an early programming language called Brook, got hired by Nvidia, and led CUDA’s full development alongside engineer John Nickolls. If AI eventually ushers in a permanent white-collar underclass or mass-deployed autonomous weapons, you can trace the entire chain of cause and effect back to some kid playing Doom who decided a demon’s scrotum absolutely needed to jiggle at a buttery-smooth 60 frames per second.

CUDA is not technically a standalone programming language — it is a full end-to-end "platform". I use that vague corporate buzzword for good reason: much like how The New York Times is technically a newspaper that also happens to run one of the world’s most popular mobile games, CUDA has evolved over decades into a sprawling, nested bundle of highly specialized software libraries built explicitly for AI workloads. Each tiny, hand-tuned library shaves nanoseconds off individual mathematical operations, and added together, they make Nvidia GPUs go brrr as the industry meme goes.

A modern high-end graphics card is far more than a circuit board crammed full of chips, memory and cooling fans. It is a carefully calibrated stack of cache hierarchies, specialized tensor cores, and dedicated streaming multiprocessors. To extend the metaphor: raw GPU hardware is like a professional commercial kitchen, where extra cores are just extra grilling stations. Even a kitchen with 30 separate grills will not run any faster than a 3-grill setup without a skilled head chef orchestrating tasks perfectly — that is the exact role CUDA plays for thousands of GPU cores. Those hyper-optimized CUDA libraries tuned for one specific matrix operation? They are the equivalent of absurdly specialized kitchen tools: a cherry pitter, a shrimp deveiner, fancy useless gimmicks for home cooks, but non-negotiable tools when you have 10,000 shrimp guts to yank out as fast as possible.

That brings us back to DeepSeek. Its team of elite engineers dug way below CUDA’s existing layers of abstraction to write code directly in PTX, the low-level assembly language built exclusively for Nvidia GPUs. Think of it as a garlic peeling workflow: unoptimized generic GPU code will fumble around picking at garlic skins with fingernails. Standard CUDA will give you the smart instruction to smash the full clove with the flat of a kitchen knife to pop the skin off in one go. Writing in PTX lets you dictate every single micro-instruction: lift the blade exactly 2.35 inches above the cutting board, hold it perfectly parallel to the clove’s equator, and strike down with the palm of your hand at exactly 36.2 newtons of force.

It quickly becomes obvious why CUDA is so invaluable to Nvidia, and so nearly impossible for any competitor to replicate. Tuning GPU performance at this level is an incredibly gnarly, complicated problem. You cannot just grab a new undergrad intern off the street, hand them a premium Claude Max subscription, and expect them to write optimized GPU kernel code. Working at this low level is slow, grinding, unglamorous work — unless you are one of DeepSeek’s elite, top 0.1% programmers.

A quick disclosure: for previous columns in this Machine Readable series, I already knew how to work with the programming languages I was breaking down for readers. That was not the case here. To keep up my usual standard of hands-on testing, I decided to spend an afternoon messing around with CUDA. It completely ruined my evening. A simple matrix multiplication that takes 3 lines of code in the popular PyTorch machine learning framework took me over 50 lines of messy, finicky code in CUDA. Squeezing every last drop of possible performance out of a GPU is an admirable, but incredibly tedious, task. Having dipped my toes in it, I can confirm the CUDA moat is very deep, very dark, and very forbidding.

CUDA’s dominance is not just built on a better set of tools, it is built on unbreakable ecosystem lock-in. Every major modern machine learning framework is built on top of CUDA, which only runs natively on Nvidia hardware. That means AMD’s chips routinely underperform Nvidia alternatives even when they technically have more cores and more VRAM printed on the spec sheet. Judging a chip’s real-world performance off its spec sheet alone is like judging a race car exclusively by its cylinder count — you never know how it will actually perform until you take it out on the track. A second disclosure: I originally planned to run side-by-side benchmarks of a top-tier Nvidia H100 and AMD MI300X for this piece, but there is no way I could expense both high-end chips without getting permanently blacklisted by Condé Nast’s finance team. You will have to take independent researchers’ word for it: even with better on-paper specs, AMD’s hardware still gets outperformed by Nvidia’s.

Nvidia’s secret edge in software? Unusual for a traditional chip company, it hires more software engineers than hardware engineers. If I was running AMD, I would copy that playbook immediately (though no one is asking for my two cents, obviously).

Every year a new group of hopeful startups and tech giants announce plans to drain Nvidia’s CUDA moat, and almost all of them end up drowning in it. OpenCL, the open standard backed by a coalition including Apple, AMD, and Qualcomm, was supposed to be the open, accessible Android to CUDA’s closed, polished iOS, but it never gained any meaningful industry traction. AMD’s official CUDA competitor ROCm is saddled with a name so terrible half the industry mispronounces it as rock cum — before AMD hires any more engineers, they should hire a new marketing team first. It is also plagued by constant bugs and crippling compatibility issues, to the point its public subreddit reads like an informal support group for frustrated users. We cannot forget Intel, either: it is already widely written off as a failing chipmaker, but its recent history proves it is also a failing software company. It launched its last-ditch multi-platform programming framework oneAPI in a bid for relevance, but as of 2026, CUDA still reigns completely unchallenged.

The only even remotely plausible challenger on the horizon is Modular, the startup led by legendary programming language designer Chris Lattner, the creator of Apple’s Swift and LLVM. But the entire industry’s open secret is that most high-profile AI researchers cannot write even a single clean line of C++ to save their lives. There are only a tiny handful of elite, experienced GPU kernel engineers on the planet, and the vast majority of them are already on Nvidia’s payroll. For years before AI became a trillion-dollar hype cycle, these engineers were quietly grinding away on CUDA, getting zero public fame or clout. Even state-of-the-art AI coding agents still struggle to write clean, optimized kernel code.

At the end of the day, Nvidia is far closer to Apple than it is to AMD or Intel. It is a world-class hardware company precisely because it is a software company first. Apple’s uncrossable moat against Android was never just the iPhone hardware — it was the full closed ecosystem: iOS, the App Store, and millions of developers who built exclusively for Apple devices. Sure, you can fold a Samsung Galaxy phone in half now, but do you actually want to use Samsung Pay? For the foreseeable future, the entire global AI industry will just have to accept Nvidia’s sky-high, often absurd price tags.

This is the first installment in a three-part Machine Readable series breaking down the little-known programming languages that power the entire modern AI ecosystem.