Efficiency Defines The Future Of Data Movement

For decades, chip performance was measured by how much raw compute could be packed onto a die. However, that equation has changed. Moving data across a system-on-chip (SoC) now consumes more energy than the computations it performs. Efficient data movement has become a significant challenge for next-generation SoC designs. AI workloads are multiplying, hyperscale data centers are approaching power limits, and chiplet adoption is reshaping integration strategies.

This is especially evident in the data-hungry demands of AI. Generative models involve moving billions, sometimes trillions, of model parameters between high-bandwidth memory (HBM) and processing elements. Every byte that moves consumes energy, and the cumulative effect is staggering. Without significant efficiency gains, the power required for advanced AI systems could multiply several times over current levels, straining global energy resources. This reality reframes performance. It is no longer about achieving more speed at any cost but about operating within finite power budgets.

Fig. 1: Efficient data movement requires silicon-proven NoC IP. (Source: Arteris)

Lessons from mobile power management

Demand for efficiency isn’t new. In the early days of mobile phones, engineers faced severe power and thermal limits yet needed to maintain instant responsiveness. Arteris played a pioneering role in this era. The company’s network-on-chip (NoC) technology enabled early smartphone SoC developers to balance performance and efficiency across on-chip communication and power domains. These advances proved that intelligent data movement could deliver both performance and endurance within tight energy constraints.

Those same principles now apply to AI-driven designs, where energy efficiency is a key factor in scalability. Modern workloads require continuous data exchange among tightly integrated compute arrays, creating power data movement bottlenecks and escalating power density far beyond that of mobile devices. Techniques first proven in mobile SoCs, such as subsystem shutdown, clock gating, and fast wake-up, provide a proven foundation for managing these challenges. The ability to dynamically isolate and reactivate NoC regions allows teams to control power and thermal behavior while maintaining predictable latency and throughput.

Building on that foundation, the expertise Arteris developed in mobile phone SoCs now extends into AI and multi-die architectures, where engineers face similar trade-offs between performance, thermal limits, and power delivery. What once ensured day-long battery life in smartphones now enables scalable performance in data center and edge AI platforms. Across these domains, control of data movement remains the key to achieving higher performance with lower energy consumption.

Safety, security, and system constraints

Efficiency challenges extend far beyond power management, shaping how design teams manage reliability at the system level. As SoCs increasingly power critical infrastructure and autonomous systems, safety and security now rank alongside bandwidth as primary design priorities. Ensuring functional safety demands architectures that remain fail-safe under fault conditions, supported by error detection, isolation, and diagnosability features defined in standards such as ISO 26262.

At the same time, rising security requirements necessitate design measures that protect against both accidental faults and deliberate attacks. These include isolation domains, on-chip firewalls, and validation strategies that ensure integrity across data paths. Modern NoCs must also integrate rate adapters, domain crossings, and other functional blocks that maintain predictable performance while separating power and communication domains.

Each of these safeguards affects how data moves through the chip. These mechanisms add verification overhead, influence timing, and consume power, yet have become essential to protecting both functionality and safety. What were once optional design measures are now fundamental requirements for integration, especially as SoCs evolve beyond single-die architectures into complex multi-die and chiplet-based systems.

Fig. 2: Performance analysis is critical for design success. (Source: Arteris)

Multi-die scaling and hierarchies

Multi-die architecture and advanced packaging enable teams to scale beyond the limits of monolithic reticles. However, this flexibility introduces layers of data movement that must work seamlessly together. In a recent Semiconductor Engineering podcast interview with Ed Sperling, Arteris CEO Charlie Janac explained that chiplet-based systems create multiple communication hierarchies. One is within the die, another across dies, and soon there will be vertical layers in 3D ICs. Each layer must balance coherency and non-coherency while managing bandwidth and latency differences, all while ensuring predictable performance and secure isolation.

Efficient data movement across these layers has become a central challenge. Each additional interface or die-to-die transfer consumes power, adds latency, and increases verification complexity. To maintain performance, the NoC must coordinate communication across physical boundaries, reducing overhead while sustaining throughput and predictability.

Additionally, industry standards such as UCIe and CXL are becoming key enablers of this evolution to multi-die integration, introducing common approaches for die-to-die communication and coherency that are still being refined across the industry. UCIe already supports 3D packaging, though real-world interoperability across vendors and process nodes is still maturing. Meanwhile, recent CXL updates enhance memory expansion and accelerator coherency, while also introducing security and management enhancements.

Automation accelerates integration

Building the next generation of efficient SoCs depends as much on advancing the development process as on improving technologies. As systems grow in scale and interconnect complexity, turning architectural concepts into reliable, manufacturable silicon requires both new methodologies and experienced talent. Yet the pool of specialists in system IP remains limited. With few universities offering dedicated instruction in NoC or SoC integration, most expertise develops through on-the-job training. This reliance on a small group of experts increases project risk, especially when early decisions carry downstream consequences. Closing the skills gap and reducing manual effort have become critical, making automation an indispensable part of SoC development.

Arteris addresses this need with innovative methodologies that automate interconnect generation and SoC integration. With FlexGen, design teams can automatically generate and refine NoC configurations that meet power, performance, and area goals while adhering to physical design constraints. This approach replaces repetitive setup work with intelligent iteration, producing higher-quality results in less time. By turning interconnect generation into a guided process rather than a manual task, engineers can focus on system-level advancements.

Magillem integration automation extends this capability beyond the NoC to the full SoC assembly. By maintaining a single source of truth, defined through standards such as IP-XACT and SystemRDL, it keeps RTL, firmware, and documentation synchronized throughout the development process. Together, these tools establish a foundation where design intent flows cleanly from specification to implementation, reducing risk and making complex SoCs easier to build, verify, and scale.

Efficiency as a catalyst for innovation

Efficiency continues to redefine how data moves through complex systems. By combining smart NoC interconnects, standards, and automation, engineers can integrate AI-driven and multi-die architectures predictably while meeting targets for power, safety, and time to market.

As automation and physical awareness reshape integration, design and optimization are becoming more closely aligned within the same flow. Each iteration of SoCs will depend on smarter orchestration of data movement, with configurability and adaptability built in from the start. Energy efficiency has become the discipline that will determine both the method and the pace of the next generation of systems.