Bespoke Silicon Rattles Chip Design Ecosystem

Bespoke silicon developers are shaking up relationships, priorities, and methodologies across the semiconductor industry, creating demand for skills that cross traditional boundaries, and driving new business models that leverage these enormous investments.

Bespoke silicon designers today are a rare breed, capable of understanding the unique requirements of a specific domain, as well as a growing array of issues that can crop up in multiple stages of the design flow. At the most basic level, they need to understand what works best in hardware, what works best in software, and how to tightly integrate both to optimize performance and power.

“The software architect or the system architect needs to know what it takes to do silicon,” said Mo Faisal, CEO of Movellus. “Otherwise, they will be tossing out algorithms that are not practical for implementation in actual silicon. Software architects really need to be comfortable with what happens in hardware.”

Software allows bugs to be worked out in the field, and it provides a way to minimize obsolescence. But it also has to be written in a way that doesn’t impact performance. Kam Kittrell, vice president of product management in the Digital & Signoff group at Cadence, noted that Jim Keller — who has a long history of developing chip architectures at companies like Apple, AMD, and Tesla — “told his team that whatever chip they’re making, make sure it is programmable, and that they can write software for it. It seems like something they would know to do, but apparently, it’s not that easy.”

Software is one critical piece. Another one is advanced packaging, which is essential in high-performance chips with some type of AI, because not everything will fit on a single reticle-size chip.

“We have long worked with the traditional chip architect, but now we’re working with the system architects,” said Marc Swinnen, director of product marketing at Ansys. “This is a skill set that goes beyond the single chip. They now have to worry about issues with the PCB, package, thermal, and mechanical. These are all things the chip designer never had to worry about, but now does. You put in two different chips at different temperatures, you get thermal differences, and they warp.”

At the same time, there are organizational challenges that come with this. “The chip was done by one person, the thermal was another. Packaging was over in Israel, the thermal is in Bangalore, and now you need a team that has all these capabilities assembled together. A lot of companies are just not set up to do that. The designer rarely talks to the packaging designer, or it’s way after the fact. With thermal, if you take two chips that are going to be hot in their activity and you put them right next to each other on the interposer, that’s never going to work. It’s dead before we even start the design. If you only figure that out at the packaging stage, it’s too late to do anything, and you can’t ECO your way out of that. Organizationally, people are challenged to solve these problems because they’re not set up to do it.”

The good news is that there is plenty of room for improvement, and chip architects are enjoying an unprecedented level of freedom. The bad news is they have so many options that it is impacting design schedules.

“System engineers can now design their own bespoke silicon because they understand their system problem. They’ve got the real-world datasets. They’ve got the AI. It’s a glorious situation,” said Kevin McDermott, vice president of marketing at Imperas. “But please invite the verification team to the table. I might have another generation, and I’d like an option or a mode here. I’d like to flip this over. I would like to have multiple choices and options all the way through that. The verification person will sit down and tell you to validate and verify the core. But for every option, you’ve doubled the work, because you now need to test with and without. We all love the idea of the freedom and innovation — as long as we have some standards here to tape out on a reasonable timeframe.”

Getting these customized chips to market is non-trivial, but there are signs this may become easier. Ankur Gupta, vice president and general manager of Siemens EDA’s Tessent Division, pointed to Google’s design as an example of how to do this correctly. “What does the designer look like? What’s so different? Google did hardware with the Tensor software framework, so they were thinking about full design and verification. Now, the skill set is at least not just from a design and verification engineer perspective, but also for a software person who is not co-designing, but at least thinking through and co-architecting the software stack. Then, what did Google do? They used the TPU in their data centers for a year, and then opened it up to the industry. By the time they opened it up to the industry, it was a full stack, fully verifiable. Have we seen that elsewhere? Maybe there’s more to come, and that’s where the next five years of users are going to go.”

More iterations
Still, even the most advanced companies are learning as they go. In Google’s case, there were multiple iterations, noted Rupert Baines, chief marketing officer at Codasip. “It was something like a version 3 because there were architectural problems, and verification documents. It was not first time right.”

The big question is what else can be learned along the way, and how learnings in one area apply in another. “When you listen to the same team from Google talk about the journey, they looked at SPEC, which is Moore’s Law of compute performance. Then they looked at an exponential of YouTube, and ML, and all of the Android devices out there, and that gap was excessive. That’s how they came up with the TPU. These same minds are now talking about compute and networking. They’ve shifted the focus onto networking.”

What comes out of that remains to be seen. But the economics of Google developing a chip versus a much smaller company are very different. Codasip’s Baines said it’s important to consider how many mistakes can afford to be budgeted.

Faisal agreed. “When it comes to 3D, everyone’s heading in that direction,” he said. “Now, PCB designers either need to become chip designers, or the chip designers need to become package designers. They need to take steps in that direction, and it’s happening. We’ve got some pretty awesome team members doing that.”

The economics of bespoke silicon
With the growth bespoke silicon, much is changing.

“The original idea is that bespoke is tailored,” said Ansys’ Swinnen. “However, we’re seeing the blending or blurring of that, like in the TPU. When does bespoke also become something you can buy on the commercial market? Cisco recently came out with their networking chip. They finally built their own networking chip, but they’re also selling it on the market. So how bespoke is bespoke? If you can buy it, it’s a standard product.”

Imperas’ McDermott said this is a common problem. “Back in the day when I started doing ASICs, you would do a custom device because it was really good for a particular market segment, and the system engineer, the end customer that was driving it, was happy to have their own design. They were prohibited from selling it to other users because it was the same application. In fact, you might call them competitors. But the way they structured the deal was, ‘Yes, I’ll partially fund the NRE. You will have exclusive rights for a year. And then we will market this as a standard chip. And there’ll be some features that you will have access to because you have the complete software visibility, and it’s designed to fit your particular design style, and it’s really bespoke for you. But to offset the economic costs, we will sell it to the broader market, which might be your competitors, and they’ll get it a year later.’”

The lines between commercial and private are blurring in other ways, too. Cadence’s Kittrell noted that companies like Cisco are now selling their technology to different companies, and at different levels in the food chain. “Before they were selling the boxes to people making data centers, and now people are going to go with cloud more often than not,” he said. “So rather than build their own data center, especially if it’s a pure software application, they need a way to create and sell technology to these cloud providers and entice them to buy this, because that’s where the money’s shifting.”

The impact is being felt in all parts of the semiconductor ecosystem. “System designers are creating bespoke silicon to augment what may be a generalized solution, giving them differentiation in what used to be a competitive market driven by software differentiation,” Faisal said. “Bespoke silicon doesn’t necessarily mean that you have to make the entire chip by yourself. Google did a great job where they had bespoke silicon in a Samsung chassis, which led to their Pixel product. On top of that, if you look at NVIDIA, AMD, or Intel, they’re offering a chassis system for people to put in bespoke parts and customize their silicon package into a full solution. It may be chiplets. It may just be a chassis system that use for IP plug and play.

Others agree. “One chipmaker is making three chips with dark silicon, using the same die with different bits fused out,” said Codasip’s Baines. “One is for Private Customer A, one for Private Customer B, the third one for the open market, and Customer A and B both have a bespoke product. Their differentiated bit is 10%, then something is fused, and you’ve got some extra gates in there that you can’t use, but it makes sense.”

And some companies are opening up their bespoke architectures to drive innovation around those architectures. “Amazon’s Graviton 3 is now opened up as a 64-bit Arm architecture, and they’re saying it can achieve from 20% to 40% better total cost of ownership,” said Gupta. “If you look under the cover, what they’ve done is, they’re looking at your rack saying, ‘I’ve got the AWS Graviton 3 connected through a data processing unit (DPU).’ They’re looking at the von Neumann architecture issue, the bottleneck in logic and memory, and saying, ‘If I also design a special data processing unit instead of a standard network interface controller, then I can move data through so much faster.’ They’re offering it up to everybody. That’s a great example of software lagging hardware. Now, all of us in EDA have to port our software onto Arm 64 bit, and there is a lot of dependency on Intel x86. We’re going through that journey now, and it’s going to be a several-years’ journey.”

Bespoke silicon in the ecosystem
How the industry will change to embrace what bespoke silicon developers need, while still supporting all of the general-purpose development, isn’t entirely clear.

“If you can afford to make your own silicon and make it run faster than anything else out there, you’ve got to wonder at some point how much of a general market is going to be left for others,” Swinnen said. “These bespoke silicon companies will be prominent and will be driving the industry for a while. But there’s always going to be the general-purpose microcontrollers and sensors, and general-purpose chips. There’s still going to be that market, but the money and attention are going to the big guys who are driving not just the software and the hardware, but all of these 3D-IC architectures. There’s a lot of technology that you could build, but how are you actually going to build it? It’s Intel and AMD and Google that are driving this, and driving foundries to centralize around particular implementations.”

These leading-edge designs still need third-party IP and tools, as well. From an IP perspective, Faisel noted that because some bespoke silicon developers lack the deep expertise to create their own clock networks, having IP platforms that offer customizable tools, based on power, performance, area, and synchronization needs, becomes more and more important. “It all comes down to how to make it easier for customers to integrate, deploy, and meet the architectural specification by creating these intuitive platforms, so at the end of the day, they have the tools to start to offset some of the skills that they lack.”

The lack of skills and the complexity of these designs has been driving up professional services revenues. “Accenture, which is a professional services company, famous for accounting, has just bought XtremeEDA,” said Baines. “They’ve moved from accounting into ASIC design. Also, HCL Technologies, the Indian professional services company, famous for software and business processes, has a huge silicon team, weighted toward the back end, but all the way through. Front-end architecture is very closely aligned to the customer and the system. You want your own architect, but RTL implementation, place-and-route, and production, operations, foundry liaison, things like that, those are perfect things to outsource. That’s why people like Accenture and HCL are moving into that world.”

Gupta agreed. “Accenture is talking about software-defined hardware. Because of their entry, they’re also trying to build awareness of FPGAs, where, what you can do in software can be done more efficiently in hardware. So they’re talking about software-defined hardware being an example of that. It’s visionary, and they’re bringing in the sustainability component. They’re saying, ‘Don’t reinvent the wheel, consider sustainability and efficiency.’ What you can do in hardware is ultimately more efficient in terms of power consumption and the like.”

Conclusion
Bespoke silicon is having its day. How long that will last is anyone’s guess. There are already signs that companies are looking to add more scalability and reach for their design efforts. But at the same time, new opportunities are opening up around that, particularly on the integration and reliability side.

“We are all now talking about systems of systems,” said Gupta. “As we look to these companies, we’re seeing they have a system of systems mindset. They’re not just talking chips. They’re talking chips in a software stack, plus the end system. We are also now asking whether their job ends when they have built the chip. We don’t think so. We think there is a lifecycle management. We truly see this coming where people are designing bespoke silicon, but then they’re not just going to keep inventing more and more silicon. They’re going to try and get more out of that silicon. The way you can do that is in-life monitoring, in-life optimization. That optimization is going to take the software stack to be built for that silicon, for that hardware. In data centers, you need to prevent silent data corruption. And in automotive, it’s not just enough to detect a cybersecurity threat. You need to prevent that threat from happening.”

Related Reading
Bespoke Silicon Redefines Custom ASICs
A fundamental shift in the economics of processing and new use cases are making ASICs cool again.
Strengthening The Global Semi Supply Chain
Robust partnerships and multi-party innovation will be essential to manage complexity and cost.
Designing For Thermal
Solutions are needed early as thermal becomes a systems issue.