AI Market Ramps Everywhere

Artificial Intelligence (AI) has inspired the general populace, but its rapid rise over the past few years has given many people pause. From realistic concerns about robots taking over jobs to sci-fi scares about robots more intelligent than humans building ever smarter robots themselves, AI inspires plenty of angst.

Within the technology industry, we have a better understanding about the potential for the technology, but the ways in which it will develop are less clear. Semiconductor Engineering asked the community to assess the status of AI and machine learning (ML) and if they thought the technology was being overhyped.

“What makes AI so interesting is that it’s a global phenomenon with universities, established companies, start-ups and even countries all trying to move the game forward as fast as possible,” says Andrew Grant, senior business development director for Vision & AI at Imagination Technologies. “The Fourth Industrial Revolution is perhaps the first where people can see change happening on an almost daily basis.”

We are still in the early days of this. “In the technology adoption cycle, this technology has moved past the tech enthusiasts and visionaries that define the early market,” says Markus Levy, head of AI at NXP. “We are now standing at the edge of the chasm, which we are successfully crossing to reach the mainstream market. The good news is we know what it takes to cross this chasm and there are hundreds of companies around the world, including tech bellwether companies, working hard to make that possible. We believe that within the next couple of years this revolutionary technology will have made substantial foray into the mainstream market. Even though we know that this technology is real and not a passing attempt to grow a market, people will continue to use and misuse the buzzwords until they clearly understand the real meanings.”

It is the creation of those buzzwords that may separate the technical realities from mainstream’s currently perceptions. “ML is just pattern matching at its core, and often the two words are interchanged to sensationalize ongoing research and industry press releases,” says Sharad Singh, digital marketing engineer for Allied Analytics. “AI is definitely overhyped in the media as the next technological breakthrough that has profound life-changing applications, and institutions are cashing in on the hype to promote themselves.”

Some of the changes seen by the mass market may not be life-changing. “It might be overhyped today,” says Benjamin Prautsch, group manager for advanced mixed-signal automation at Fraunhofer EAS. “However, I believe that AI will be a core element in almost every future system. AI won’t be visible—just like the transistor. It’s effect, however, will be. AI will not only add new function to devices, it will also improve the electronic design and design automation, and many other fields.”

That is already happening. “AI is broadly deployed today, in many ways you many not notice, such as smart unlock features on your smartphone, using your face or fingerprint, predictive text in your emails and instant messages, and efficient energy-management monitoring,” says Steve Roddy, vice president of special projects in the Machine Learning Group at Arm. “However, some AI applications are overhyped, such as self-driving autonomous cars or companion robots replacing human interaction. The technology just isn’t sufficiently advanced for these kinds of things to be routinely and consistently deployed.”

Raymond Nijssen, vice president and chief technologist for Achronix, agrees. “The implications will be much broader than anyone can imagine. We do hear some wild claims, and some of them definitely are overhyped. But it will find its way into our lives and other areas of technology in ways that have not yet been foreseen. There will be a lot of development, but we will encounter some glass ceilings where we had high expectations that will not become reality. That will have a lot to do with where AI is just not intelligent enough.”

The term AI itself is problematic. “It is all about context and whose expectations are considered,” says David White, senior group director for R&D in the Custom IC & PCB Group at Cadence. “I believe there are extremes on both sides of the debate. I don’t believe we are anywhere near true machine intelligence that threatens our safety, and I don’t believe that AI and deep learning are pure hype with no redeeming engineering value. My expectations are that AI and deep learning would provide value in real-world systems for specific tasks, and in that context, I believe we are on track.”

And context is important. “Zachary Lipton, an assistant professor at Carnegie Mellon University, states that the AI hype is blinding people to its limitations and is dangerous in the long run,” says Allied Analytics’ Singh. “He argues that the current state of AI is poorly understood by the public, as the latter often associates AI with self-aware robots taking over humanity. In reality, machines still have a long way to go before being able to develop human-like intelligence. Legendary physicist Stephen Hawking and Tesla founder Elon Musk have both publicly spoken about the dangers of AI, while Microsoft co-founder Bill Gates believes there’s reason to be cautious.”

What complicates the picture is the rate of change. “It’s only a few years since Geoffrey Hinton’s team at the University of Toronto made breakthroughs in CNNs,” points out Grant. “Since then Google, Facebook and others have made many of their own developments available to the wider audience of data scientists, software developers and hardware teams.”

Understanding the roots of the technology can help. “If you look at AI, the best way to think about it today is a super-universal curve fitting function,” explains Achronix’s Nijssen. “Anything that fits that mold can make a lot of progress beyond what we see now. But there are other forms of intelligence that are not an extrapolation of patterns or images or sequences of events that have been seen before where actual interpretation and deeper understanding is necessary. Today, that is not part of what is being considered.”

The area covered by curve fitting is large. “We still haven’t cataloged all the ways and places where it can be used,” says Peter Glaskowsky, computer system architect at Esperanto Technologies. “Almost anywhere that decisions depend on recognizing repeated patterns, AI will play a role.”

Many of these will continue to involve humans. “There are so many areas that will benefit from the combination of person, machine and AI,” says Imagination’s Grant. “With that combination we can begin to tackle problems that would otherwise elude us. In health care, security and economics, for example, the opportunities are literally endless.”

Taking the human out of the loop is where problems may start. “During this process, it will be important to understand AI’s decision-making so the quality of decisions can be measured,” warns Fraunhofer’s Prautsch. “If the decision, however, gets too much attention over the process of decision-making, then hidden dangers could arise.”

And there will be failures. “There are opportunities within the market for one actor, or one group of actors, to do something that is sub-optimal around AI,” says Marc Naddell, vice president of marketing for Gyrfalcon Technologies. “If they over promote the capability of the solution, that could result in disappointment. That can be used as evidence that AI does not really live up to the billing.”

NXP’s Levy tackles this problem. “Every technology has the hype cycle with troughs of disillusionment. We view ML and AI as a natural progression of technological advances that has characterized human evolution for millennia. Look at it this way—humans have become the most successful species because we figured out how to transfer our acquired knowledge, problem-solving skills, and decision-making techniques to our progeny, not through genes, but extra-somatically. We have been doing the same thing to our machines by making them more efficient, smarter, and now the natural progression is to enable them to think. So unlike other technologies, AI & ML are not over-hyped or short-lived. They are fundamental to human nature.”

What of machines creating better machines? At present, the furthest we have gone is the employ these techniques to create better silicon. “There is now unprecedented interest and investment in applying ML to chip design,” says Jeff Dyck, director of engineering/R&D at Mentor, a Siemens Business. “This has led to a new generation of ML practitioners in EDA, many of which have a solid academic knowledge of ML. They are now developing promising results in controlled environments. However, we are still learning from the school of ML hard knocks about the challenges of bringing ML methods from the lab to production. Perhaps we are at the early stage of a golden age of ML for chip design, but we need to see the promising techniques in the lab successfully move to production for the value to be realized.”

Accelerating development
ML and AI run on very sub-optimal hardware today. “We will see AI processing move from CPUs and GPUs to dedicated AI accelerator chips,” says Glaskowsky. “Because these new devices are designed specifically for machine-learning algorithms, they will deliver better performance at lower prices, and they’ll be much more energy-efficient on the same tasks—typically 10 times better than GPUs and 100 times better than CPUs.”

And we are beginning to see custom silicon being used. “There are dozens of companies bringing AI chips to market in 2019 and 2020,” says Geoff Tate, CEO of Flex Logix. “Many will miss the mark, but some of them will deliver the goods enabling rapid growth of edge AI. The long-term winners in AI chips will be those who can keep up with the rapid pace of change as neural networks improve.”

According to a recent report by Allied Market Research, the global deep learning chip market is projected to reach $29.4 billion by 2025, growing at a CAGR of 39.9 % from 2018 to 2025.

Xilinx has jumped into this market in a big way. “They have invested billions in their Everest platform, expected to tape out by 2018 on 7nm technology,” says Sergio Marchese, technical marketing manager for OneSpin Solutions. “Flexible and powerful hardware platforms supporting heterogeneous computing are crucial to accelerate the development and deployment of machine learning and AI-based applications.”

We have to look at all metrics. “At some point, it is not just about performance,” warns Naddell. “It is about cost of ownership and that includes energy use.”

Achieving that will require a range of devices. “They will cover a wide range of cost and power points,” says Glaskowsky. “There will be AI chips (and IP blocks for SoC designs) that cost less than a dollar. Big standalone chips may cost over a thousand dollars, but will outperform a box full of GPUs costing far more. Most of the world’s AI processing will shift from legacy platforms to optimized solutions as quickly as the new silicon can be manufactured.”

Some of those devices are already in consumer devices. “Neural network accelerators will become ubiquitous, in every device in our environment—indeed we could call it ambient AI,” says Grant. “As the ability to process complex neural networks increases and the price per device falls, we will see this everywhere, from urban infrastructure to provide advanced services such as traffic and building management and security, to monitoring the elderly in care homes.”

There is a lot of work ahead. “The first generation of solutions is not very efficient,” says Nijssen. “Both training and inferencing are done in a very brute-force fashion. GPUs are useful, but they are simple-minded and they don’t allow for things that deviate from just pumping through a lot of MAC functions. There are many techniques that people have not had a chance to try out yet because the field is moving so quickly. Once the dust settles and the way that people do training becomes more uniform, and the algorithms do not change on a daily basis, you will see people pushing down the power consumption curve.”

“In the hardware space, it’s critical to have flexible, scalable and energy-efficient hardware that spans all performance points, from CPUs to GPUs and NPUs,” says Arm’s Roddy. “The market is expanding and will continue to ramp up. AI is here to stay.”

Coming in January: AI and ML still face many challenges. Politics, ethics, legal issues, trust are one group of issues that have to be dealt with. There are also many remaining technical problems that must be overcome before it can be taken to the next level. Read more, in Pushing AI Into The Mainstream.