Scaling AI Chip Design With NoC Soft Tiling

Tiling is about repeating modular units within the same chip to enhance scalability and efficiency; chiplets involve combining different silicon pieces to achieve a more diverse and powerful system within a single package.

Network-on-chip (NoC) soft tiling is complimentary but distinct from chiplets described above as it repeats modular units inside a NoC design. Soft tiling within a NoC offers scalability and reusability, making it ideal for automotive, communications, enterprise computing and consumer electronics system-on-chip (SoC) designs.

One example of soft tiling is in advanced driver-assistance systems (ADAS) in automotive applications, where real-time data is managed for AI-driven object detection and decision-making. In another example from the data center vertical, soft tiling facilitates the scaling of AI accelerators, enabling systems to handle data-intensive tasks such as large language model (LLM) training and generative AI.

As the demand for AI applications expands across multiple industries, semiconductor designers are rethinking traditional chip architecture. Requirements for higher computational power, energy efficiency, area optimization and faster deployment have made it clear that conventional design methods are no longer sufficient to meet the complexities of AI workloads. Soft tiling, an innovative solution in NoC designs, addresses two main challenges in system-on-chip (SoC) development. First, it allows for scalable, modular designs where each soft tile can be duplicated across the SoC without redesign. Second, it offers significant benefits for derivative designs, enabling the reuse of pre-verified tiles, which reduces design time, complexity and area constraints.

Performance, area, and power

NoC soft tiling in AI design allows architects to define and replicate one processing element across the SoC, eliminating the need to design each tile separately. This enables scalability without manually configuring each network interface and processing element. For example, a 4×4 soft tile array can easily scale to an 8×8 array, supporting AI-centric workloads like neural processing units (NPUs) and LLMs. Moreover, the flexibility of NoC soft tiling allows SoC designers to optimize for specific requirements, depending on the application.

The mesh network topology used with NoC soft tiling ensures efficient communication between tiles, avoiding bottlenecks and allowing for parallel processing across the chip. This combination of modularity, scalability and area efficiency simplifies initial SoC design while maintaining high performance.

AI workloads are power-intensive, so managing energy consumption efficiently is essential. NoC soft tiling addresses this challenge by enabling dynamic frequency scaling (DFS), which allows individual tiles to adjust power usage based on real-time demand. When certain tiles are not in use, they can be turned off, conserving energy and reducing overall energy consumption by up to 20%, as illustrated in figure 1.

Fig. 1: Efficient power management using NoC soft tiling. (Source: Arteris)

NoC soft tiling’s capability to manage power domains and area resources independently is especially beneficial in fluctuating AI workloads. Adjusting each tile’s power state and area usage as needed enhances performance while minimizing energy consumed.

This energy efficiency and area optimization make NoC soft tiling ideal for applications like edge AI devices, where energy management and space constraints are crucial. By supporting dynamic resource management, NoC soft tiling provides scalable performance across power- and area-sensitive applications, meeting the diverse demands of AI workloads across industries.

Dynamic reuse and faster integration

A key benefit of NoC soft tiling is its ability to support derivative design, allowing pre-verified soft tiles to be reused across multiple SoCs. This pre-configuration reduces integration time, cutting SoC development by up to 50%, as shown in figure 2. Designers can reuse tiles that have already been verified, ensuring that derivative designs can be rapidly developed and brought to market. This reuse also simplifies the verification process, as pre-tested tiles reduce the need for extensive revalidation, speeding up development cycles and enhancing reliability.

Fig. 2: Faster design integration by reusing pre-verified soft tiles. (Source: Arteris)

NoC soft tiling also helps address memory bottlenecks. In many AI systems, adding processing elements can lead to memory limitations that affect performance. Soft tiling enables specific component upgrades, like the memory subsystem, without requiring a complete SoC redesign.

These capabilities are particularly important in AI chip design, where innovation cycles are becoming faster and more frequent. By leveraging pre-verified tiles, designers can introduce new product generations quickly while maintaining the same high standards required when designing AI chips.

Versatility across AI workloads

As AI workloads continue to grow in scale and complexity, NoC soft tiling offers a flexible, scalable and power-efficient solution for complex SoC and derivative designs. By simplifying integration, optimizing performance and enabling the dynamic reuse of pre-verified tiles, soft tiling is set to shape the future of AI-driven semiconductor innovation. With mesh topology supporting seamless communication, dynamic power and area management, soft tiling will be essential in enabling the next wave of AI chip design.

NoC soft tiling features, supported with mesh interconnect topologies, are available within FlexNoC and Ncore interconnect IP from Arteris, providing a flexible and scalable foundation for AI-centric SoC designs. These NoC products are designed to handle the increasing complexity of AI applications by enabling seamless communication between processing elements, ensuring high-bandwidth data flows while minimizing latency. FlexNoC and Ncore dynamically adjust clock and power domains to conserve energy while maintaining performance. This flexibility, scalability, energy efficiency and area optimization combination make Arteris NoC IP an essential technology for next-generation AI systems.