Artificial Intelligence Wonderland

Silicon Catalyst held its Sixth Annual Semiconductor Forum in Menlo Park on the SRI campus on November 9^th. Richard Curtin, Managing Partner for Si Catalyst, opened the event with a reference to Arthur C. Clarke’s “2001: A Space Odyssey” and noted how remarkable it was that a novel written back in 1968 was able to foretell the direction of the computer industry over 50 years into the future.

The publicity poster for the event is shown below. In the spirit of the evening’s topic, the image was created using Alice from Alice in Wonderland as a robot going down the rabbit hole in the style of Dali as input to an AI image generator. Mr. Curtin read notes from Dr. Lance Bell, Si Catalyst Partner that described the process of using these generators and noted that the more descriptive the input, the better the images that were produced. What the AI generator created in moments it was estimated would’ve taken his team days to create.

Fig. 1: Si Catalyst AI Wonderland publicity poster.

Some questions raised, though, were: At what point does convenience unsuspectingly eclipse truth and what are the ramifications of AI as a source of artificial information, not artificial intelligence, creating a false paradigm upon which future iterations find their foundation?

Be where? When deploying AI, is it bringing us where we asked it to go or somewhere else? We must always beware and be aware of this. A few lessons learned from the image generation exercise: What is your dataset? How was your AI trained? What is the level of bias? Dr. Bell also listed three more things: pay attention, pay attention, pay attention.

This all led into the evening’s panel discussion. The panel was moderated by David French, CEO of SigmaSense and a Silicon Catalyst Board Member. The distinguished panelists were Deirdre Hanford, Chief Security Officer, Corp Staff, Synopsys; CHIPS Act Department of Commerce Industrial Advisory Committee. Moshe Gavrielov, former CEO of Xilinx; Board member of TSMC and NXP. Ivo Bolsens, Senior Vice President, Head Corporate Research and Advanced Development, AMD.

David French joked that all of the questions weren’t provided by ChatGPT, although some were, and that most of the questions were submitted in advance by the audience. One of the most commonly posed questions was about AI as it pertains to the advancement of the semiconductor industry. The first question to Deirdre Hanford was: what is the status on the use of AI in chip design and where is it going?

Deirdre introduced the audience to the term BLUF, which stands for Bottom Line Up Front. Her BLUF is: There’s a lot going on in AI in design. It’s improving outcomes. It’s improving productivity and, even though it’s early rounds, there’s a big opportunity for the workforce as well. Deirdre mentioned that two years ago at HotChips 33, Synopsys CEO Aart De Geus gave a keynote address¹ about wrapping tools with an AI harness, using reinforcement learning, to help drive them. She likened this to the early days of logic synthesis where an engineer could start a run and leave for lunch and come back to better results. It was a way to reduce the “grinding” and be more productive. Likewise, setting up an AI harness allows an engineer to set up runs where the tools grind through alternatives to produce better power, performance and area (PPA). This has been expanded to verification, test and analog design migration, enabling engineers to be super productive.

She also described an autonomous driving analogy for simulations. In autonomous driving, going from stage 2 to 3 is where functionality is shifting from the driver to the AI. That’s about where we’re at with verification, where AI can remove a lot of the “toil.” Deirdre then asked the audience if anyone had read NVIDIA’s ChipNeMo paper². She recognized Dr. Raul Camposano, Silicon Catalyst Partner, affirming that he had read it, with an “of course” comment. (Full disclosure, I had the great privilege of reporting to Deirdre and Raul in my first post-graduate industry position in Silicon Valley.) Deirdre quickly summarized some of the results in the paper and while the results aren’t all at 100% yet, they’re already at a level that is useful and can increase productivity. In some cases, they can get you closer to your desired destination, but more work is needed to get all the way there. She concluded her answer by saying that AI can free us from having to toil and give us more time to do the cool stuff.

Ivo talked about the early days of computer aided design (CAD) and the progression to electronic design automation (EDA). Now we’re looking at electronic design creation, starting from a higher-level specification to take on a larger portion of the design process. Ivo likened it to using an airplane to get you closer to your destination quicker and then still needing to drive the rest of the way by car. The plane doesn’t get you all the way there, but it certainly shortens the time necessary.

Moshe mentioned the use of AI from a TSMC perspective and said there are two areas that have been publicly discussed. The first is the creation of standard cell libraries where the process has become so complex that it’s not possible for humans to understand all the design rules. A few examples are fed in and then the entire cell library can be built through the tools. It’s very complex with a lot of corner cases, and the tool generates higher quality and density libraries. The second is the use of moving analog IP to newer technology nodes. Moshe said it fits Ivo’s analogy of using an airplane in that it gets you closer to your goal, but still needs some human intervention for cleanup to reach the final product. Moshe senses that we are approaching a threshold and that once we cross it, it will be like a dam bursting with new capabilities.

David French then asked the panel about power and climate change.

Ivo said that we’re at the beginning of a revolution in semiconductor industry. We can build much more efficient architectures than we have today. Compute platforms are pretty inefficient, only exploiting 10-20% of compute capabilities. It’s also not only compute, but data too. Moving data to the compute uses a lot of energy. We’ll start looking at more ways to bring compute to the data and have compute in memory.

We’ll have to look at solutions in a holistic way and look at using resources at the edge and in the cloud. This can avoid transferring data and provide better privacy, for example moving weights to the data center. We’ll see more compute on networking, computing data on the move, compute in memory, in combination with traditional computing.

Moshe said that power is the big issue. We’re still in early and sloppy innings. We’ll need to find ways to apply ingenuity. Taiwan is limited by power and water, as is Arizona. There are opportunities to address challenges. This is all spells opportunity.

Deirdre referenced David’s earlier comment about the brain using only 20W while we’re increasingly using more and more power in data centers. The best power optimization is done architecturally, and we’ll need to get away from these first implementations. She noted that Pete Bannon, VP Tesla, mentioned at a TSMC presentation that some of their GPUs were sitting unused because they were unable to get data to them quickly enough and that must be inefficient. There’s a big obligation to address power, and it’s an industry responsibility.

David French asked the panel about disruption in the industry.

Moshe observed that when performing processor design back in the 70’s, designers weren’t referred to as tall thin engineers. They typically did architecture, micro architecture, algorithms, logic design, circuit design and sometimes even layout. Very broad in their ability to do things, but not as deep. Moshe said that he was an expert in the use of multi-dimensional Karnaugh maps and was a colleague of Zvi Kohavi, and then came Synopsys. It put him out of business and had to go into management. He said that this overall was a 30-year transition, and AI will do this in a third to a fifth of the time. AI will unfold at a much faster rate and be more impactful. An inhibitor will be an availability of data, but he expects that there will be a big donation and it will be fascinating to see how it unfolds.

Deirdre supported Chris Malachowsky at SUN. Engineers around him were concerned about losing their jobs. People are now working on much larger designs than the 40k-gates gate array in 1.5um LSI Logic that Chris and his team at SUN were working on. Deirdre said that we’re going to just keep automating away as an industry, we’re going to keep innovating as an industry, and the design community are going to do breakout things. She thinks it will accelerate and that while there will be a discontinuity, there’s still going to be that need to have to work to get all the way to the destination.

Ivo said that this event is a little like déjà vu for him. 40 years ago, he was on a panel about AI and was working on his PhD at the time. The compute power wasn’t there then, but now we have the compute power. Initially PCs were trying to get enough CPU power for Office, then they opened up. The same happened with mobile platforms. People were happy to be able to just make phone calls, and then when more compute power arrived a whole new set of applications and capabilities opened up. Ivo thinks that the same is going to happen with AI. Today most people are talking about transformers, natural language processing, big data, etc. Once enough compute is available to push it to the edge, it will really take off.

David asked the panel about how it will impact relationships between companies and universities and large companies and small.

Deirdre said that she is worried about the “haves” versus the “have nots.” There’s a crisis in universities’ ability to get large compute resources. We need to look at creating national resources to help “crazy” research continue at universities.

Moshe said that startups in semiconductors are not easy to do. Access to resources could be an issue, but creativity and ingenuity could help work around it.

Ivo said that open source is a given and allows people to leverage work. It’s important for large companies to feed the ecosystem for students and universities. It’s also useful to build a cloud structure and allow academics to take over the structure and allow students to be creative. We need to encourage universities to collaborate and look at ways to improve hardware.

Deirdre noted that inside of Synopsys, every group needs to consider how AI can disrupt their ability to protect their enterprise. The role of work is going to change, and it’s important to get ahead of it. For example, if marketing can create more collateral with ChatGPT, then maybe redeploy some of those people to more productive areas.

David asked if the panel had any advice for startups CEOs.

Moshe suggested getting more of the right people for advisors. It is important to generate a network of advisors, and maybe AI can help accelerate that. He also recommended watching successful CEOs and being open to getting input from people who have been there before.

Ivo believes that all chips will have AI capabilities in them in 5 years, so there are tremendous opportunities. AI goes through four generations of algorithms in the design cycle time of a single chip. Improving design cycles could help.

David remarked that there are still a lot of opportunities for startups.

It’s interesting to see the references to previous advancements in automation, and while there has been some angst, historically the increased productivity has led to more creativity and opportunities for new markets and jobs. A video recording³ of the event is also available online.

References

[1] HC33-K1: Does Artificial Intelligence Require Artificial Architects?

[2] ChipNeMo: Domain-Adapted LLMs for Chip Design

[3] Si Catalyst Fall 2023 Forum (Video Recording)