AI’s Value In Chip Design Depends On Data Availability

Experts at the Table: Semiconductor Engineering sat down to discuss the advantages and challenges in using AI in designing chips, with Chuck Alpert, Cadence Fellow; Sathish Balasubramanian, head of product marketing and senior director for custom IC at Siemens EDA; Anand Thiruvengadam, senior director and head of AI product management at Synopsys; Sailesh Kumar, CEO of Baya Systems; Mehir Arora, head of engineering at ChipAgents; and Daren McClearnon, product manager at Keysight. What follows are excerpts of that conversation. To view part one, click here. Part two is here.

L-R: Keysight’s McLearnon; Synopsys’ Thiruvengadam; ChipAgents’ Arora; Cadence’s Alpert; Siemens’ Balasubramanian; Baya Systems’ Kumar.

SE: Are chips developed with AI better than chips developed without AI?

Balasubramanian: It depends on what you’re defining as better. If I can tape out the same quality of a chip in 50% less time than without AI, then yes. In the latest IBS study, they measured man-hours with and without AI, and it’s almost a 50% productivity gain. That’s time to market. And you can do a lot more optimization. We’re looking at a lot of different ways to get to the right answer in a shorter amount of time without sacrificing core metrics like accuracy and verifiability. So definitely it’s better, and we are seeing that right now.

Alpert: If you’re just trying to get a chip out the door, it won’t be better. But you’re saving resources and money. And then what happens is design teams can say, ‘Look, we’ve got the same resources and the same people, but now we can explore a much wider solution space. We can do optimization AI and explore a much bigger solution space and find a better result.’ That’s a different story. But it really depends on how the design team wants to use the resources. So if they get the productivity gains, they’re going to go for more PPA.

Kumar: I have first-hand experience trying to use AI for interconnect design. Today, all the interconnects are built by humans, and because we’re an interconnect company, we have been actively looking at how we can apply AI to automate some of that design process. The problem is the lack of data, which makes it impossible to train a model that can do a better job than a human for any design. So far, we have not been able to get data and train a model that can do any task in five minutes. That lack of data is a critical element, and to address that we probably have to create synthetic data and learn from it. But that requires heavy investment. The autonomous car industry has multi-billions of dollars, so they can afford to invest in that. But if it costs you $500 million, can you afford it? That’s the challenge we are facing. The CAD industry overall is not that big. It’s less than $10 billion. So is there enough TAM to put the immense amount of investment required to get there? That is the challenge. It will happen. The cost will come down and it will scale. But the question is when.

Thiruvengadam: Many of our customers are public companies, and they have seen tangible, meaningful improvements with AI. Just take the optimization layer, for example. I’ve seen so many testimonials of customers showing significant, meaningful impact. For a given set of constraints, you can extract more PPA from a chip design. And for verification, they’re seeing a 10% or 20% coverage boost, which is a big deal. A 5.5% F_max boost is a big deal. And a 20% power reduction is a big deal. These are all tangible gains you can get from AI. For analog, it’s the same thing. And for test pattern count reduction, it’s a big deal for tester time. So you can see meaningful gains from AI that are actually realized by customers today, given the constraints, and it’s been there for several years.

SE: You’re working at the leading edge, though. Your customers are big companies that are building these chips. What’s the impact on smaller companies? Are they just going to roll out something that’s potentially inferior and say, ‘Okay, this is good enough and it’s cheaper?'”

Alpert: There’s a lot of customization. You don’t just build a general-purpose CPU. You can build custom parts because of AI. But you don’t have to build something different. It doesn’t have to be better, and it doesn’t have to necessarily be the best performing. But if it solves a problem that nobody else is solving better, with less overhead, that’s what you want.

Kumar: It also depends on what you call AI. Because AI is sexy now, people are calling scripts they wrote 10 years ago AI. But are you training a real neural network based on real data?

Thiruvengadam: Most of the generative applications are primarily based on RAG (retrieval-augmented generation), at least on the basic application. So you’re essentially leveraging one of those big commercial LLMs to do the job for you. But then, if you think about the gains, you’re really talking about whether you can get a static knowledge retrieval task. You get a meaningful improvement with that. You ask a question and you get an answer back. Still, that’s just static knowledge. When you get into the generative stuff, you can get an RTL snippet or a code generated that is 70% to 80% there already. That is a significant step forward. Going back to the productivity play, we don’t need to get 100%. Even 70% within a matter of minutes is a huge improvement in productivity.

Arora: I’d like to say two things on this. One comes back to the issue of data. The research has been done, and it turns out that by and large, over the past roughly 10 years since we’ve had AlexNet, there have been more improvements to the bottom-line performance of neural network and deep learning systems from algorithmic improvements than on data. That’s one thing. So data scarcity is a real, serious issue. We’ve effectively tapped the entire internet for the terabyte-scale data. We’re out of it. We’ve used all of it. Today, the emphasis is on data-efficient training, and there have been huge leaps and bounds made in data-efficient training, especially when we start to talk about new post-training paradigms like reinforcement learning. These are dramatically more data-efficient. So while it’s true that we need lots of investment in data, and that data will need to be curated, we’re able to do a lot more today than we were four years ago with far less data. The second thing I’d like to say is that while there are many applications today that are using retrieval-augmented generation methods by and large, I don’t believe RAG will pan out. In fact, I believed it was always a Band-Aid solution that has failed when applied to enterprise systems. This is because the information that you have within one enterprise is actually not indexed well enough to perform retrieval. You’re bottlenecked at the retrieval level, not at the generation level. In reality, the improvement comes from the training of the models, from the agentic search methods that you use, and from the tools that you provide the systems. That’s where we actually see the improvements in benchmarks and in products.

Alpert: But no one is going to give you their customer data, so that’s never going to work.

Arora: You don’t need to do any of that with customer data. I’d like to clarify. Enterprise data is not organized well enough for retrieval to work, and that by and large has been proven. RAG is kind of a graveyard of startups that have attempted to scale this and make it work well.

Balasubramanian: I agree with Anand that RAG has certain good qualities of retrieving and helping in terms of assistance. Where RAG doesn’t really help you is with pushing the barrier of innovation and creating something new, and that’s what we do most of the time. When you’re doing iterative designs, it’s fine. You can do the next version of a design and put it into your new tech. Anyone can do it. That’s not a problem. So RAG is mainly focused on that. Even within RAG, we have seen questions about how to divide the data. How do you do your documentation? How do you write? How do you organize your index, your data? There is a lot of innovation that’s happening. We have found out that we can build our own secret sauce for how we can make it more efficient. Even with data chunking, you can do off-the-shelf data chunking for many applications, but for EDA it’s different. It depends on how you label it.

Alpert: I don’t disagree. You can fine-tune a model. The problem is that every company keeps its data locked down, and there’s no sharing. Are you saying that an enterprise company that will not share their data is going to have to train their own model?

Arora: Absolutely not. But RAG has been a Band-Aid solution to the problem. How do we do knowledge injection into these models? It has failed for rather boring reasons, which is that when you chunk up your data, you restrict the amount of context that is available to them. When you use embedding-based systems, those embeddings are lossy in their form, and it’s hard to acquire that information. Instead, you have to go the agentic route. We have to use the agentic route in order to inject data into these systems, to get them to be compatible, and to learn how to use your existing EDA flows, how to learn from your documentation the structure, for example, of your SoC test benches, or your SoC designs, your whole repositories, and so forth. How do you funnel that information up into an agentic context and then also build indices over that data? It can’t come from just pure retrieval, because you end up being bottlenecked on retrieval.

Thiruvengadam: We’re not saying that RAG is the be-all, end-all.

Alpert: But saying that it’s failed is a little dramatic.

Arora: I would actually go far enough to say that it has failed.

SE: Let’s shift direction. As we get into heterogeneous integration and multi-vendor chiplets, these things evolve over time. What you start with might not be what you end up with. And you have digital twins on the outside looking at this stuff. So who owns that data?

Alpert: It’s whoever is building it.

Kumar: In the chip design world, the representation language hasn’t changed for the last 30 years or more. It can be many languages, but the gold standard is the ultimate source of what you can have. You can write models and different abstractions, but the source of truth is what you’re shipping.

SE: But you’re also extracting data as it runs, right?

Kumar: Yes. You can take Verilog through the whole CAD flow — simulation first, then verification. You synthesize it, do power modeling, and it’s a well-defined process. There are four or five steps in the CAD flow. Three of the companies in this discussion are the leaders in each of those steps. But even if I have access to every possible Verilog file ever written in the world, which is a lot of data, I would struggle to use that data in a very effective way to build a powerful AI system.

Balasubramanian: That’s a holy grail. The biggest challenge is data, and we have been doing a lot of research on how to use synthetic data on a physical system. There are a lot of gotchas. But if we can get access to the data, we’re definitely going to be training a model.

Kumar: A co-pilot today is amazing. Practically every engineer uses a co-pilot to automate Verilog writing. But can you replace an architect who can envision and design a chip using that data?

Balsubramanian: If you look at the generation portion of GenAI, most of the startups are on the front-end RTL generation side. The reason is that the amount of open-source data on the RTL side, or the Verilog side, is a lot greater than anything downstream on the analog side. Once you have data, of course, you’re going to create your own foundation model. You can fine-tune and fine-tune, configure it — there are a lot of different ways to make it happen. The beauty of this is that it can make the architect more productive. They can create a wire-frame model prototype from 10 different prototypes and figure out which one is going to work.

Alpert: I don’t have to have one RTL. If you can auto-generate, you can say, ‘Let’s try this one and this one.’ You can configure your systems in different ways and take them through the flow, and see which one really works in a way that you couldn’t without AI. That’s going to be really differentiating.

Balsubramanian: People today are doing multiple LLMs for the same problem, figuring out which one is better, giving any one input for one LLM to another one, and making it much better. So it’s a constant refinement of doing that and coming up with much more optimized code, right? We are not there yet, but we’ll get there, for sure. The question is when.

McClearnon: There are edges of the market, like mechanical and electromagnetic and thermal, that are less language-driven. There, the data scarcity is even more acute and more proprietary, because you’re tied to a foundry and some internal process. The best data is your data.  So there’s an economy of scale of data. How can you aggregate the training sets, either at the university level or government consortium level, or inside a larger enterprise? There’s a wealth of haves and haves-nots when it comes to data.