Growing use cases include life science AI, reducing memory and I/O bottlenecks, data prepping, wireless networking, and as insurance for evolving protocols.
FPGAs are finding new applications in the age of artificial intelligence, high-speed wireless communications, medical and life science technology, and in complex chip architectures where they can improve the flow of data.
Field-programmable gate arrays (FPGAs) enable designers to reprogram or reconfigure digital logic after the chips have been deployed, which is essential in the AI world, where algorithms change far faster than chip architectures. Low upfront costs also allow FPGAs to be used as a prototyping tool for fixed-function chips such as ASICs, or as a temporary solution until a workload is settled. But due to their higher manufacturing costs, FPGAs are often found in low-volume, high-performance specialty applications such as fighter jets or measuring equipment in labs.
Nevertheless, the market for FPGAs is expanding. New application areas include:
The main consumers of high-end FPGAs in high volume are telecommunications companies (wireless and wireline), data centers, networking, and the military, aerospace, and government sectors. “Volumes in these applications can range from the thousands to greater than hundreds of thousands of units per year,” said Venkat Yadavalli, head of the Business Management Group at Altera. “On the applications slide, more features integrated into FPGAs, like embedded Arm cores and AI Tensor Blocks, have increased the usage of FPGAs in a broader range of embedded and edge AI applications.”
One evolving area is robotics, which require deterministic latency and real-time decision-making on the edge using heterogeneous data from cameras and sensors. “It could be voice, video, whatever — things that you need sensor fusion,” said Yadavalli.
Another big use case is medical imaging technologies, everything from when you go to the optometrist and they scan your retina, to MRI type systems. “You’re taking raw analog data, and you’re needing to filter it, process it, use it to reconstruct images, do a lot of matrix math,” said Rob Bauer, senior manager of the product marketing, adaptive, and embedded group, at AMD. “FPGAs are good at that. AI engines are very good at matrix math, so they’re ideal for those types of volumetric imaging applications.”
In today’s fast-changing, highly connected, and AI-enabled landscape, there’s no single answer about whether designers should use FPGAs, ASICs, or another type of IC.
“You don’t switch between one solution or another,” said Andy Nightingale, vice president of product management and marketing at Arteris. “What you have is a mix or a blend. Interconnects facilitate and support a blend of FPGAs for, say, cybersecurity or I/O bottleneck alleviation. FPGAs enable prototyping, but it may be that customers leave certain FPGAs in the system where they might say, ‘We want custom hardware here in the SoC, but we’ll leave some FPGA technology in there, because the benefits outweigh the turnaround time of doing some custom hardware only to find a spec or a standard has changed.’ If the cybersecurity standards change, they have to re-implement and readdress that, whereas an FPGA may have content that you could reprogram. Even though there’s a slightly higher overhead in terms of the power consumption, it may not be on the critical path, so you can work around that.”
When weighing up options, it is helpful to consider the cost per logic element being used in the FPGA. “There is a little bit of architecture that you need to get the cost benefit when you compare it to an ASIC,” said Yadavalli. “Compared to an ASIC or ASSP, you will always be slightly disadvantaged because they serve a fixed function that has been designed and optimized to the nth degree to only implement what you need, rather than you creating all the programmability. In that case, you will have some overhead in terms of doing either Ethernet or PCIe. You could switch all these things. You need to have all that circuitry, and there is a penalty in terms of that. It all comes down to the ROI, and whether the market is settled to the point that you need an ASIC.”
It’s about making design trade-offs to solve a specific problem. “What you’ll see if you look at the consumers of FPGA is they’re trying to do ever more complex things,” said Bauer. “With physical AI, you see people trying to integrate AI into these edge systems, where they’re bringing in data from sensors processing it right there, at super low latency. The data rates are increasing, the security requirements are increasing. It’s an optimization problem.”
Evolving algorithms
AI/ML models are a moving target, and while FPGAs can be reprogrammed to match them, it is a cycle of choosing the best chip as the situation evolves.
“If the workloads are going to change often, then I want general-purpose compute,” said Mo Faisal, CEO at Movellus. “I can optimize more and push that to the limit. Eventually you realize, ‘This highly flexible general-purpose compute is no longer valuable. I need to go customize. We’re in a cycle where my problem statement is changing so much that I am probably gaining more by going to reprogrammable compute.’ Eventually, we’ll run out of steam there and go back to what happened with ASICs and FPGAs in the past. Obviously, you can develop specific algorithms that are specialized for FPGAs. If you want to talk about scales in a particle accelerator, for example, they use 10,000 FPGAs to process the data that comes out at a certain rate. But if you want to now do that at scale, there’s only one CERN and one Fermilab. What if you want to have 10,000? You need to reconsider FPGA.”
Others agree it’s a case-by-case decision. “Whether to use programmable FPGAs is dependent on what kind of workload you’re looking at, and it’s the same in software,” said Kexun Zhang, head of research at ChipAgents. “People in life sciences are designing different model architectures. I see domains and fields where the exploration for model architectures is far from being converged, which is why people still need compilers and still need hardware to work at a reasonable speed for their bespoke model architectures. These are the domains where FPGAs will shine.”
For consumer and other mass markets, the cost and efficiency structures don’t make sense. “In a data center there may be benefits in the prototyping space, or for very specific AI changes,” said Nandan Nayampally, chief commercial officer at Baya Systems. “If you’re doing high-frequency trading, for example, and your algorithms change on a regular basis, you need to optimize for it. If I don’t have either the volume or the cost structure to support ASIC, then I might stick to FPGA. A lot of the new AI architectures are using programmable components and programmable engines, on the same die or multiple dies, and that gives you the ability to be more efficient and yet flexible.”
On an FPGA, as many hard-wired, parallel compute units can be built as the resources allow, depending on the size of the FPGA. “Compared to general-purpose GPU cores, FPGAs can be tailored to your application,” said Nightingale. “You can customize the parallelism as well.”
Virtual FPGA machines for complex algorithms, verification
Cloud-based FPGAs also can be used to offload compute-intensive workloads from the data center, for example, by using an Amazon Web Services EC2 virtual server. “You can go to AWS and commission a virtual machine,” said Russell Klein, program director at Siemens EDA. “It’s a physical machine that has what they call the F2 Instance. It has a PCIe card in it that has eight Xilinx FPGAs, and these all can be programmed over the PCIe bus with the main processor. I’ve been working with them to take our high-level synthesis tool to be able to program these F1 Instances, and say, ‘Here’s a function, let’s go push it onto the FPGA fabric across the PCIe bus.’ We just got this running where I can take a function, compile it through our high-level synthesis tool, and then take it through the Xilinx tools and have that interact with software running on the processor.”
Further, the F2 Instance is being used to offload very complex algorithms. “They’re primarily seeing this in the life sciences — people who are doing DNA analysis or chemical reaction analysis are looking at really complicated 3D math, and those people are programming the FPGAs and having those interact with the processors running on the host,” said Klein. “They can connect all of those through the PCIe connection and speed up those very complex algorithms. We’re going to see a lot in the AI space here because it goes faster and uses less energy. It’s the obvious next step in creating faster, more efficient inferencing and training environments. The capability is there. We need to get the industry to start taking advantage of it.”
Another use case for the F2 Instance is lower-cost hardware verification. “When SiFive was a startup, they needed to boot operating systems on their RISC-V designs before they were able to create the silicon,” said Klein. “They wanted to do that as a verification step. Instead of buying an FPGA prototyping system, they rented FPGA boards on AWS at $6 an hour. They could fit one of their CPU instances in one chip, and then they could run a multi-CPU, multi-core design by programming into multiple FPGAs and then multiple boards. They spun up their own low-budget emulator by taking advantage of these data center CPUs. AWS is seeing a lot of interest, and they have tooling around this to be able to run logic simulation with the FPGA fabric modeling part of the system. They can run all of that together, and it goes a lot faster. It’s like a hybrid FPGA prototyping system.”
Fig. 1. FPGAs have long been used as a prototyping tool for ASICs, using tools such as this proFPGA desktop prototyping system. Source: Siemens EDA
The entire card is connected to one processor on the server itself. “They have multiple of these boards plugged into a server, and each one of them goes into a different PCIe slot, so you could have multiple programs connected to multiple boards,” Klein explained. “You can spin that up very quickly and tear it back down. These are FPGA cards in the data center, accelerating hardware verification, accelerating biosciences and AI. This is a viable technique for addressing the compute challenges that we have today.”
Meanwhile, FPGAs are helping enable shift left design. “If you look at the explosion of new chip development driven by AI, and the cost of advanced nodes like 2nm, it’s more important than ever to get it right,” said AMD’s Bauer. “You can have a software engineering team using these emulation prototyping platforms to develop the software for their eventual ASIC well ahead of tape out.”
Reducing bottlenecks, data prepping
Another way FPGAs can save power and boost performance in the data center is by helping reduce memory and I/O bottlenecks in SoCs and chiplets through optimizing data movement.
Altera recently selected Arteris to help with this. “FPGAs can be placed directly into the data path to manage streaming of data, and they can minimize buffering, optimize throughput,” said Nightingale. “By managing the data in line, you can get the FPGA to pre-process the data that’s coming through, and that alleviates the most significant constraints in AI system performance by giving the CPU or the GPU or other processing units less work to do. Therefore, the memory and I/O bottlenecks that appear are lessened.”
It’s similar to inline processing, where FPGAs are put into the data stream itself. “Because of the large capacities that these things come in these days, you can do more processing on the same data as it goes through,” Nightingale said. “We see it all the way through, hand-in-hand, working with FPGA technology in the movement of data.”
In the data center FPGAs are being used as smart network cards with local memory. “You get these really large mesh topologies, and you have to be able to quickly move data from one point to another,” said Bauer. “With FPGAs, the reconfigurable nature of them, along with the super high speed connectivity and the memory and the latency, make it really good to put next to these AI compute elements, whether it’s a GPU or an ASIC. Then the customer can define exactly where to move the data. We have large memory right there next to the compute. The proximity of memory and compute is really, really important in AI applications. That’s high end.”
At the other end of the spectrum, there are really small FPGAs used in the GPU cluster, or server boards, to do the board orchestration, management, and power supply sequencing. “We call that server IO type use case,” said Bauer. “There’s FPGAs on the servers, controlling the boards and making sure everything’s powered correctly. Then there’s FPGAs moving the data to memory and between the different compute elements.”
Another new role for FPGAs is AI plumbing that needs to be done with infrastructure data coming into the GPUs or CPUs. “Data comes in, and there is a lot of programmability of these data packets in terms of how you want to manage it,” said Yadavalli. “You need a smarter way of doing network interface cards, smart NIC functions, or some kind of data plane management that needs to happen before that data can jump into a CPU or a GPU. FPGAs are playing a role in bringing that data in and prepping it in a way that it can be consumed by all the big GPUs and CPUs sitting in the back.”
Fig. 2: Accelerating AI Infrastructure with FPGAs. Source: Altera
Data prepping and cleaning is another emerging area. An AI application is only as good as the data. “You may have a state-of-the-art model or LLM, but if the data itself is heavily noisy, whether it’s business data or any commercial data, that is a garbage-in, garbage-out situation,” said Yadavalli. “To do the data prepping, where you get all this information coming in, you can automatically look at how to make it into standard information. You put in PPT, you put in voice commands, you put in text, and so on. On the other end, the LLM can only consume certain raw data that needs to be set up in a way that it can give you the best information. FPGAs happen to be a perfect match there, where they’re doing spatial manipulation of this diverse data that is coming in.”
Wireless communications infrastructure
Communication protocols such as 5G, 6G, Open RAN, and baseband applications are a strong FPGA market, and Altera and BigCat recently partnered to scale FPGA-based radio access network technology. “As wireless standards evolve, the first four- to five-year deployment cycles are always with FPGAs,” said Altera’s Yadavalli. “The standards are not settling down to the last degree, so the big gear providers, like the Ericksons, the Nokias, the Samsungs, and everybody across the world are not able to build ASICs in time. You need to prefetch this entire spec two years before so that you can get going and build an ASIC out there. Once the spec finalizes, you want these chips to be available, so that you can transition the networks into the next latest-and-greatest.”
AI algorithms also are changing quickly in the networking space, opening opportunities for FPGAs. “In 6G there is a lot of network-related AI processing that is being discussed, but nothing has been standardized, and there’s a lot of trepidation about when 6G gets deployed,” said Yadavalli. “A lot of people will either be behind, ahead, or misguided in certain directions. They need flexibility in terms of AI compute. Then there are smaller to bigger computers. Some of them can really push it into the baseband, where there is big cloud set up there. But there’s also a lot being deployed on the radio side, as well as a baseband box sitting underneath the big antenna. Those need to have FPGAs as a sidecar.”
For 5G, AMD has seen an uptick in its adaptive SoCs being used for beamforming applications. “That was a big part of the standard rolling out,” said Bauer.
Horizontal product differentiation
Vertically integrated companies are in a position to bake a particular accelerator or functionality into an ASIC, but much of the industry is horizontally integrated. A company that builds a range of different embedded processors will likely not build an accelerator function that’s fixed, because it would reduce their total market opportunity.
“These companies can put embedded FPGA fabric on their device or enable their SoCs to interface with external FPGAs,” said Siemens’ Klein. “We’re seeing a lot of these manufacturers starting to do that. It can be a way to deliver customized hardware functions that would be used by their system. Even though it’s going to be much bigger and more power-hungry than an ASIC-level implementation, it’s going to be much faster and more power-efficient than what’s going on in software.”
For example, if a company wants a wearable battery to go for a week without recharging, they’ve got to start looking at moving things off that processor. “If they’re going to be horizontally integrated, and they’re not building their own SoCs, they’re going to want programmable logic to move things off,” Klein explained. “Whether that programmable logic is embedded in the SoC they buy, or it’s a discrete component that sits off on the side, the FPGA is going to enable them to take advantage of that higher performance, lower energy.”
Embedded FPGAs also offer the ability to protect certain IP through sparsity and obfuscation. “Maybe there’s a software developer or software program that has their own secret algorithm,” said Andy Jaros, vice president of IP sales at QuickLogic. “They want to have a hardware accelerator with it. They develop it. They can put it in the eFPGA on the ASIC and not share it with anybody else.”
Emerging security threats
FPGAs can be reprogrammed to keep up with ever-changing regulations and growing threats, such as the specter of quantum hacking, there are concerns.
“We see uptake of FPGAs to ensure the authenticity of the code running on these systems, and ensure it’s secure from tampering,” said Bauer. “Also, for inline encryption of the data, some customers have their own proprietary techniques that the programmable logic is perfect for. Other customers are happy with standard approaches, which we’ve integrated. We have hard crypto blocks to do the inline end to end encryption.”
Other use cases include wireless or wired connectivity, and firewalls. “People are using FPGAs to do AI-based packet inspection to look for threats,” said Bauer. “You need the high speed connectivity, then you also need to implement the model right there in the fabric.”
There are drawbacks, however. “Standalone and embedded FPGAs offer tremendous flexibility, but that benefit can introduce security risks if the confidentiality and integrity of the bitstream configuration data isn’t properly managed, especially in shared environments like data centers,” said Scott Best, senior technical director of silicon IP at Rambus. “Similar to the risks of datacenter ASICs and SoCs running semi-independent virtual machines for different users, data center FPGAs require robust configuration and access controls to prevent unauthorized reprogramming or side-channel vulnerabilities when the programmable fabric is shared between users.”
Many FPGAs now have built-in encryption systems. “The protection of bit streams, which represent the programming of the FPGA, have gotten better. But they’re still a favorite target of attackers to try to be able to claim that you’ve been able to get into something, modify the program in a bit, stream on an FPGA, and get that to run,” said Dana Neustadter, senior director of product management at Synopsys, noting that hacking an FPGA is similar to things like secure boot. “In the processor world, we use secure boot to make sure that the correct program is running on a processor. In the FPGA world, that same task is done by the encryption and an authentication engine that verifies the programming of the FPGA.”
Conclusion
Low upfront costs mean FPGAs will keep their role as a prototyping tool, but in today’s high-tech landscape, they’re also finding a growing list of new roles to perform. Designers may determine the best solution is to use a mix of fixed-function and programmable devices in order to keep up with AI and ward off hackers.
Related Reading
Complex Mix Of Processors At The Edge
Diversity of compute elements proliferates for inference, but the mix varies by application.
For Chip Developers, HW/SW Co-Design Key To Data Center Efficiency
Every aspect of data center energy use must be optimized to reduce power consumption and enable more sustainability, from chips to transformers and edge compute.
 |
 |
 |
 |
 |
 |
 |
 |
 |
 |
Leave a Reply