Why Making Chips Smaller Is a Never-Ending Game of Catch-Up

Why Making Chips Smaller Is a Never-Ending Game of Catch-Up: A Journey into 3D NAND, Moore’s Law, and Memory Madness

Alright, buckle up, dear readers, because today we're about to journey into the mesmerizing, head-scratching world of semiconductor memory technology. We’re diving headfirst into how the memory industry is dealing with Moore’s Law finally giving up on its trusty promise of getting better and cheaper every two years, and what happens when you shift from making 2D chips to stacking them like skyscrapers—also known as 3D NAND. Spoiler alert: it involves a lot of crying over depreciated factories, a touch of “string stacking” (it’s as weird as it sounds), and the world’s most expensive “pizza ovens.” So, let’s kick this off and peel back the silicon curtain!

1. Moore’s Law Has Left the Building

Gordon Moore once said that the number of transistors in an integrated circuit would double every two years, and for a while, that worked. Memory companies could just keep making things smaller, doubling the bits per wafer, and all was sunshine and reduced cash costs—until it wasn’t. Imagine you’re baking cookies, and every two years, your oven magically doubles its capacity. Amazing, right? More cookies for everyone! Now imagine your oven suddenly starts demanding double the kitchen space, twice the electricity, and an extra baker just to keep from catching on fire. Less amazing.

That’s where we’re at with Moore’s Law for memory. NOR memory, a type of flash that’s in a lot of older electronics, hit a wall at the 45-nanometer node. Like, literally hit a wall. The electrons wouldn’t cooperate anymore, and the cells collapsed in on themselves like a sad soufflé. Meanwhile, NAND figured out a workaround with something called Fowler-Nordheim tunneling, which sounds like a physics Ph.D.’s fever dream but essentially just shoves electrons through a wall instead of over it. NOR’s dream of scaling is long gone. NAND, however, wasn’t done yet—it had one more trick up its transistor.

2. 2D to 3D: From Maps to Skyscrapers

If you’re a semiconductor engineer and you’re told, “Hey, we’re running out of space,” you’ve got two options: either start a real estate war, or build vertically. In the early days of NAND flash, engineers focused on making everything as compact as possible—kind of like cramming all your furniture into one room. But at some point, if you want more space, you need to build up. Enter: 3D NAND, where the industry decided to create literal silicon skyscrapers.

The jump from 2D to 3D NAND is a bit like realizing your single-story house could be converted into a high-rise. Suddenly, you’re getting eleven terabytes of data per wafer instead of just five—because who wouldn’t love doubling their capacity? But here’s the catch: your quaint 2D house tools won’t cut it anymore. You’ll need to retrofit everything—the ovens, the wiring, the entire dang kitchen. Engineers went from drawing little floor plans to wrangling skyscraper elevators—both amazing for engineering, but awful for the wallet.

So, there we were, faced with an unprecedented slow transition—a five-year ordeal just to get to 3D NAND because, surprise surprise, when your equipment grows four times larger, you’ve got a problem fitting everything in your factory. Companies like Samsung and Micron realized they’d have to replace their entire setup, and that’s a bit like taking out your entire kitchen just to install a bigger oven. Good luck cooking anything during that time.

With limited space on 2D wafers, NAND flash memory transitioned to 3D, stacking layers to boost storage density. This innovation, similar to converting a single-story house into a skyscraper, requires extensive retrofitting of production equipment. For investors, 3D NAND represents both an increase in initial CapEx and long-term scalability benefits, positioning it as a core technology for future memory demands.

3. String Stacking: The 3D Skyscraper that Took the Elevator Twice

Okay, now imagine building a skyscraper… and then stacking another one on top of it. Sounds impressive, right? That’s string stacking. Samsung went ahead and built a skyscraper with 32 floors, then one with 64 floors. Meanwhile, Intel and Micron got a little “creative” and just took their 32-floor tower and stacked it again to make 64 layers. It’s like saying you built a new skyscraper but really just doubled up on the old one. Functionally, it still worked, but it took twice the effort, like running the elevator to the top floor twice because you forgot to press all the buttons.

It gets weirder when you realize that now the industry is trying to do 96 layers, and they’re just string stacking again! Imagine saying, “Yeah, we’re not building a true 96-layer skyscraper. We’re building a tower of towers.” It’s like Lego-ing your way to the sky, except every piece is worth millions of dollars and there’s a finance team constantly reminding you about the budget.

String stacking, where additional layers are stacked on top of existing 3D NAND structures, provides a cost-effective way to increase storage density. Companies like Samsung and Intel adopted string stacking to scale to 64 and even 96 layers, though the approach requires significant engineering effort. For investors, string stacking presents a cost-effective scaling method, reducing CapEx per unit of storage while increasing memory density.

4. The CapEx Problem: More Money, More Problems

Now, as if string stacking wasn’t enough, let’s throw money into the equation. You see, the industry doesn’t just love to innovate—it has to innovate because there’s always a competitor ready to jump into the pool. Micron was in a bind: every time they crunched the numbers on retrofitting for 3D NAND, the CFO probably took an extra couple of aspirin. But what do you do when Samsung makes the jump? You follow. It’s a business version of peer pressure on steroids. You can’t say, “Nah, I’m good,” because then you’d be out of the market faster than someone hitting a “game over” screen.

Capital expenditures (CapEx) for 3D NAND conversion? They’re wild. And not only are they wild, but they’re also slow. It used to be that new equipment meant some minor upgrades—now it’s a complete overhaul. And the cash cost? Sure, the cash cost per gigabyte goes down, but the upfront cost is so insane it’s like buying a Tesla to save on gas but needing a bank loan to cover the purchase price.

Transitioning to 3D NAND isn’t just a technical challenge; it’s a financial one, too. Companies like Micron face massive CapEx requirements for retrofitting factories to support new 3D architectures. While 3D NAND reduces costs per gigabyte in the long run, the upfront investment is substantial. For investors, CapEx decisions are critical in assessing a company’s financial health and ability to remain competitive in the memory industry.

DRAM Challenges: The New Recipe Dilemma

You’d think NAND was enough to keep semiconductor engineers busy. But nope, DRAM has its own set of scaling nightmares. Back in the day, you could just shrink everything by half, like slicing a cake into smaller, neater pieces. But now, engineers have realized that simply making DRAM smaller isn’t enough. They’re having to introduce entirely new materials, as if the old flour stopped working and they now need a whole new recipe to make the same cake.

Scaling DRAM has become a triple threat—longer timelines, higher costs, and way more complicated recipes. Picture a chef trying to make soufflé while also redesigning the oven to work on alien energy—that's DRAM scaling now.

As with NAND, DRAM scaling is hitting roadblocks that require new materials and techniques. In the past, DRAM scaling was straightforward; now, engineers are forced to experiment with new materials to maintain performance. This shift introduces longer development timelines and higher costs, adding pressure to companies already investing heavily in NAND. For investors, understanding these R&D costs is essential to anticipate the impact on profit margins and product cycles.

At the end of the day, the memory industry’s journey is a combination of brilliant breakthroughs, budget constraints, and doing everything because the competition is doing it too. Companies like Micron, Samsung, and SK Hynix are stuck in this dance—building bigger, building more, and doing whatever it takes to keep the memory industry not just alive, but thriving. Sure, Moore’s Law might not be the magical oven anymore, but that’s not stopping anyone from trying to make it work with whatever crazy tools they’ve got.

And while we as consumers keep wanting more data, more storage, and more speed, just remember the real heroes are those engineers trying to figure out how to turn a pizza oven into a silicon skyscraper factory—without burning the whole kitchen down.