Big Memory Shift Ahead

System architecture has been driven by the performance of memory. Processor designers would have liked all of the memory be fast SRAM, placed on-chip for maximum performance, but that was not an option. Memory had to be fabricated as separate chips and connected via a Printed Circuit Board (PCB). That limited the number of available I/O ports and the rate of data transfers.

Over time, space became available on-chip and small amounts of faster memory could be integrated. This faster buffer spawned the designs we have for cache and produced a significant reduction in the number of times a program had to go off-chip for its data. Memory consumes about half of the chip area today, but everything is about to change. Smaller geometries are stretching existing technologies in many ways and new memory technologies are preparing to replace the old.

A related story (Powerful Memories) explored some of the changes happening related to the need to reduce power consumption and introduced one new technology that could have a profound effect on memory architectures, namely 3D IC.

“Consider Micron’s Hybrid Memory Cube where DRAM slices are stacked,” says Arvind Shanmugavel, director of applications engineering at ANSYS/Apache. “They managed to reduce power by around 70% compared to a normal DDR approach.”

On-chip, little has changed with most of the memory being the 6T static RAM cell that has not seen significant change for decades. “Embedded DRAM or flash is more expensive,” explains Anand Iyer, director of product marketing for low power platform at Calypto, “because they require a different fabrication process that is not as well optimized for logic. So designers use them only when they are really needed.”

As a result of publishing a related story, LinkedIn users weighed in on some of the issues. Eric Bouche, a semiconductor consultant says “the SRAM already moved from 6T to 8T and prediction is that it will not be sufficient so the cell is getting larger with smaller nodes.”

Things Are About To Get Worse
“The DRAM needs a capacitor to go along with the 1 transistor bit-cell and that capacitor is not scaling,” says Prasad Saggurti, product marketing manager for embedded memory IP at Synopsys. “They used to use a trench capacitor and later a Metal-Insulator-Metal (MIM) or Metal-Oxide-Metal (MOM) capacitor. This made the bit-cell bigger and more complicated.”

The migration to finFETs will add another round of complications. “With SRAM you have a problem controlling Vt variability with 16/14 finFETs,” says Dave Lazovsky, CEO of Intermolecular. “Increased threshold voltages come at a cost of power consumption, so you do not get some of the benefits of a conventional shrink.

This affects more than just the memory cell itself. “Because of finFet process technology limitations, you are allowed to only use multiples of min size devices for sizing,” explains Hem Hingarh, vice president of engineering at Synapse Design. “This creates circuit optimization issue, particularly for sense amp designs because traditional transistor sizing does not work.”
With finFETS also come new failure mechanisms and this is making some people rethinking the amount of redundancy they will have to add into their designs. “If you look at the pareto failure charts from the foundries,” says Synopsys’ Saggurti, “you see that for smaller geometries, as you increase the amount of memory, you need to add additional amounts and types of redundancy. Then the intelligence of the memory repair solution becomes more important. Do you do row or column repair first? You cannot think of memory BiST as an afterthought.”

Deepak Sabharwal vice president of engineering for IPBU at eSilicon agrees. “A typical 1Mb memory block will have at least one repair element, which could be in the bit-line or word-line direction. The foundries provide guidelines about what kind of failures are expected more often. The most common is to have repair in the bit-line direction, which would be needed if you had a short in the bit-line or a fault in the sense amp. If we are talking about 1/2Gb memory blocks, then you will need additional repair capabilities. Alternatively, a user could opt to use ECC instead of repair. This gives them the advantage of dynamic correction and higher soft error rate tolerance.”

Sabharwal has an additional concern. “With the advanced nodes, bad bit-cells also come from aging. We continue to add more guard bands to get around some of these problems but they are getting exponentially bigger at each node.”
Saggurti does have one piece of good news for us. “The transition from planar to finFET has provided a one-time reduction in leakage. The increased gate capacitance in a finFET process has made dynamic power more important for the first time.”
At the same time, new architectures are emerging for non-volatile memories that allow them to depart from the limits of lithography. Instead of getting smaller, they are going vertical and within the next couple of years we can expect to see 3D NAND memories becoming available. This is an important advancement because flash technology does not scale well, with each smaller node delivering shorter lifetime.
New Memories
Almost every type of memory used today is going to be threatened over the next decade by new technologies. It starts with the spinning hard disk, which already is being threatened by NAND flash technology. “It is not just about performance advantages, but economics and the rate of innovations for driving down the cost per bit,” says Lazovsky.

But even NAND flash is under pressure from new memory types, most notably Resistive RAM (RRAM or ReRAM).  ReRAM works on the concept of the memristor that exhibits hysteresis. This allows a material to retain the last resistance it was set to after power is removed.

“The beauty of ReRAM is that the smaller you make it, the more physics works for you, the opposite of flash-based memories,” Lazovsky explains. “That is because ReRAM is based on resistive state change and as the devices get smaller, the ratio of Ion to Ioff or Ron to Roff increases, making multi-level cell (MLC) even more possible with advanced controller architectures.”

Vertical ReRAM architecture being developed by Samsung

Why are ReRAMs so important? “We are talking about pJ per bit,” says Lazovsky. “We are talking about 10 years of data retention at 100 degrees C. Endurance is hundreds of thousands of cycles versus NAND’s few thousand.”

And while we are on the subject of endurance, there is another memory technology on the horizon that could threaten DRAM. It is called  or MRAM. Consider the endurance requirements of a system. Today, they are established by the characteristics of DRAM. The retention in a DRAM cell is mS, with a large percentage of the operations being refresh cycles. If DRAM is replaced by non-volatile memory, and we don’t have to burn all of those cycles in refreshing, what is the true specification for endurance? According to EverSpin Technologies’ Web page “MRAM has infinite cycling endurance with no programming induced damage or wear-out.”

Inertial Delays
If these new memories really are as good as the claims, why are we not seeing them in production applications today? The answer appears to be inertia. “There is so much money going into DRAM and other memory technologies, how does a new technology and a startup company compete?” asks Eric Ruetz, director of engineering for mixed-signal at Synapse Design. “Going from node to node takes a lot of money.”

Lazovsky agrees. “NAND flash is a $30B industry that has tens of billions of dollars in capital infrastructure that would need to be retooled. The big four players represent 95% of the market and they have a lot of existing investment. The entire cost equation is CapEx, so they need to milk the tail of the revenues as long as they can.”

“It is not enough that it works,” adds Synopsys’ Saggurti. “It has to be replicable in new process technologies. So if a foundry can bring up a new process in 18 months, can these technologies also be available within that time frame? Otherwise high volume applications will not use it and they will be moving onto new nodes each couple of years.”

Adds , chief technology officer for Uniquify: “Technological and economic challenges still prevent this from becoming mainstream in the next four years.”

The promise is for a universal memory that has the properties of SRAM but which also is non-volatile. “This would be awesome, but mainstream proof is not there,” says Saggurti. “We need to see it in a mainstream process at a cutting edge node. Flash trails leading edge by a couple of nodes and if MRAM and ReRAM are still 4 or 5 nodes away, the high volume people are not looking at it, and thus nobody will look at it.”

New Applications
While most people are looking at cutting-edge applications, it appears that ReRAM may start to creep in via another door. Most consumer products get built in a 40nm or 55nm process technology today. This is the microcontroller market and this is the market that is likely to be fed by an emerging applications market — the Internet of Things (IoT).

The IoT market demands smaller horsepower compute engines, such as a microcontroller that has an integrated modem for connection to the Internet. Existing solutions have a small amount of embedded NVM, which is primarily NOR memory.

“The challenge with NOR is even more significant,” says Lazovsky. “To scale beyond 40nm is extremely challenging because of its architecture. These are even more amplified than those being experienced with NAND in the areas of endurance. Another problem is that NOR uses a high voltage, typically at least 10 volts, and that translates to power consumption.”

If we throw in another consideration — ReRAM can be built using standard CMOS process — then it starts to look very enticing. This reduces the number of mask steps by about eight, compared to NOR Flash, bringing down both the cost and power consumption. What about performance? “Switching speed has been shown to be down to 10nS,” says Lazovsky, “and add in endurance and data retention advantages.”

“Once it approaches the performance of embedded RAM,” says Uniquify’s Iyer, “more systems will be designed to ‘execute in place’ out of the NVRAM in the package. True shutdown will be fast.

Once it starts, we can expect to see a landslide. “ReRAM will encroach on NAND, then it will encroach on DRAM for certain applications where power consumption and cost per bit is important.” Lazovsky sees nothing to stop the train. “Consider enterprise storage applications that are using banks of DRAM chips that are being constantly refreshed. RRAM will provide a 10X reduction in cost per bit compared to DRAM and then the reduction in power will be over an order of magnitude.”

With those kinds of advantages, the question may not be “if” the change happens, but “when.” And when it does, we will be able to rethink many aspects of the way in which we design systems. The day of the big memory change is coming.