Building Your Own NoC And The Hazards Of (Not) Changing

There is a perennial challenge that all R&D organizations face – how much of what we develop is essential to our competitive advantage and how much can be acquired at lower cost and risk rather than built from scratch? It’s easy to believe in the heat of battle that everything we are doing must be crucial. But the world continues to change around us. What was optional yesterday may be essential today and what was essential yesterday may be optional today. It is important to keep your eye on the prize, which is to get products to market profitably and quickly with the highest possible quality and value.

Figure 1: Using commercial NoC interconnect IP provides much greater financial benefits than the cost required to license and use it.

This is especially true for make-versus-buy decisions. You make decisions and you work with those decisions, then three years later markets have changed and new requirements have risen to the top of your list of priorities. You need to have everyone working on these new high priorities; whatever slipped down the list must be outsourced. It is crucial to develop a flexible infrastructure and methodologies to respond to ever changing market requirements.

On-chip networking approaches

We still see this in network-on-chip (NoC) interconnect IP. On-chip fabrics used to be developed entirely in-house until companies such as Arm introduced commercial crossbar IP (NIC-400), partially displacing some of those in-house efforts. These crossbars worked well in smaller designs but became too big, too slow and too power hungry in large SoCs with about 10-15 or more masters and slaves. Latency and die area exploded when designers tried to daisy chain crossbars together to accommodate the needs of larger, more complex SoCs. True network-on-chip architectures started to become more popular for their much better Quality of Service (QoS) as well as configurable bandwidth, optimized latency, lower power consumption, higher frequencies and layout efficiency within these larger SoCs.

Arteris IP introduced a commercial NoC interconnect early on called FlexNoC, which has become an industry standard, but some companies still decided to build their own in-house on-chip interconnects. Back then, that was an understandable decision. NoC technology was new and the relatively small in-house teams were closely coupled to the needs of the larger SoC team. However, over time, specializing on NoC interconnect needs inevitably resulted in the development of commercial solutions that performed much better than in-house developed IP. The key to this is the rate of learning: As a commercial IP provider, Arteris IP works with many different customers on large number of designs per year which results in a very fast and robust learning cycle. An in-house interconnect design team might spin a new fabric once or a few times a year, so their rate of learning is much less. And if their interconnect instance wasn’t perfect (and they usually weren’t), the fabric team would hand-tweak and tinker as needed to meet the goals of those few SoCs. As a result, senior managers got called to the hot seat and needed to be able to answer what happens when:

• The company tries to make more complex SoCs, or more chips per year?
• The company enters new markets requiring novel SoC architectures?
• Internal interconnect design team members change jobs or companies, resulting in loss of expertise?
• Budgets are not sufficient to stay on the leading edge of NoC interconnect design?

Scalability problems

The in-house interconnect development approach started to crack at the seams as designs got bigger, iterations in implementation became more frequent and the number of designs per year grew. Then it became obvious that what the internal interconnect group offered was not a robust NoC interconnect IP product but rather a software-enabled service. Scaling as a service – to handle increased demand – they had to add more and more engineering staff, pare down the interconnect (and therefore the chip) requirements, or extend the interconnect schedules. This led to death spirals where SoC projects are simultaneously late, don’t meet requirements, and are higher cost than the competition. Poor interconnect became a negative rather than a positive competitive advantage.

We worked with a major customer who found themselves in this trap. They would ship a spreadsheet, describing the interconnect they wanted, to their internal fabric team in another location. They’d get the generated interconnect RTL back months later. So much for supporting quick turn-around and a high number of SoC projects! They brought us in for a discussion and we were able to demonstrate that our IP products could not only provide interconnects in a tiny fraction of that time, but also that our NoC IP was higher quality – better QoS, lower congestion, you name it. Not surprising. We have specialized for years in NoC interconnect IP, they hadn’t.

Proving the ROI case

Figure 2: Using commercial NoC interconnect IP also allows users to reduce schedule time and project risk. An interactive financial model provides a means for each design team to calculate their own results based on their own experiences and assumptions.

This time around they wanted to make sure they could really justify a decision to switch because they were going to have to live with this for a while. So, they worked with us on a spreadsheet to compare their current internal solution with our solution – on the money they would save, the time they would save, and how much they would reduce risk. They could dial in their own numbers for SoC volume, price and cost per chip, how much time they would spend on each phase of NoC design, and then expected die area saving, staffing costs, and so on.

Figure 3: Schedule risk is the biggest killer of semiconductor profitability. Being even a few months late allows competitors to grab the most profitable opportunities.

Given that input, the spreadsheet calculated and graphed the monetary value of using our NoC IP, the time they would save out of the design cycle, and the impact on revenue from being later to market (the figures in this article).

That customer used the spreadsheet as a part of their own internal justification – and they chose to standardize on Arteris IP for their on-chip interconnects. We’ve found that the spreadsheet has also been quite helpful in justifying decisions at other companies. I should add first that the technical decision is usually quite easy for us. In 3 out of 4 cases, our reputation answers the technical part of the evaluation. In other cases, we perform an evaluation (usually on the customer’s current SoC project!) and easily prove that our results are technically superior. I don’t think we’ve ever found a counterexample. The real problem is in getting over the emotional investment in the in-house interconnect – “we’ve got this big team, we’ve got differentiation, we’ve got sunk cost, we don’t have to build that many designs anyway…” In some very small enterprises, we understand that cash constraints may justify their argument. In larger operations, the spreadsheet is helpful in encouraging a more realistic discussion on tradeoffs.

One more thought: The world continues to evolve around innovation in SoC architectures – technically in need to support safety, fail-operational behavior, AI accelerators and cache coherence between accelerators and clusters. And the NoC interconnect is the logical and physical instantiation of these SoC architectures. Both technical and business considerations add urgency for each of us to focus on what we do best and to use commercial solutions elsewhere. This is clearly where the world’s great chip development teams are headed. SoC architecture innovation based on proven, world-class NoC interconnect technology is the safer way to get innovative SoCs to market.

You can download our spreadsheet here.