Innovations In High-Frequency Electromagnetic Simulation

High-frequency electromagnetic simulation has evolved from “wow, now I can see how electromagnetic fields behave” to needing to know how various EM fields interact in large, complex systems. During that time, I have been an R&D engineer and I have managed a team implementing various solver technologies. We’ve been presented with plenty of challenges as electronics have continued to proliferate, growing increasingly complex.

To meet the needs of the market, you need accurate answers faster. One significant bottleneck has been the time it takes to get an initial finite element (FEM) mesh for large system designs. Recently, we introduced HFSS Mesh Fusion, which allows a large system to be analyzed by meshing parts of the design independently. This makes the meshing process faster and more robust, allowing it to obtain a mesh when previous methods would fail.

Building upon electromagnetic simulation innovation

Mesh Fusion is the latest in a series of EM simulation innovations. Before my time at Ansys — in fact, before Ansoft was acquired by Ansys — critical features such as physics-based adaptive meshing [1], spurious-free vector basis functions [2], and the transfinite element method [3] were in the first version of HFSS released in 1989. Zoltan Cendes, founder of Ansoft (acquired by Ansys in 2008), was the force behind those important early features. He was at the forefront of defining spurious-free vector basis functions, foundational to high frequency FEM, as described in his paper “New vector finite elements for three-dimensional magnetic field computation.” Prior to this, it was impossible to provide robust and accurate answers for electromagnetic analysis using FEM.

Once that possibility existed, the next challenge was efficiency. For example, in 2007, we adopted hierarchical vector basis functions defined to work well for the iterative solver and enabled the use of different polynomial orders throughout the computational domain, called mixed order basis functions. Mixed order provides a way to model the fields more efficiently by using low order approximations where fields are more constant and higher order in regions of more complex field patterns. The order distribution is determined automatically in conjunction with the physics-based mesh adaption.

Fig. 1: A coax to waveguide microwave transition.

With automatic mesh adaption, you can launch a simulation without dealing with the mesh. HFSS automatically refines the mesh based on physics. The adapted mesh is obtained by first solving the fields on an initial coarse mesh. Error indicators are then computed and used to determine where to refine the mesh as well as to adjust the order of the basis functions. After a refined mesh is obtained, HFSS solves again and inspects the convergence criteria, typically by checking the change in S-parameters between consecutive meshes. This process continues until a final converged mesh is obtained that satisfies the convergence criteria. Adaptive meshing is not easy to get right, but we have fine-tuning it for over 20 years to ensure accuracy.

Another major innovation in EM simulation is the transfinite element method, which was introduced in the first release of HFSS. It provides an accurate way to inject and absorb waveguide and/or transmission line modes into the computational domain via the ports. Other methods have been proposed for modeling ports relying on a perfectly matched layer (PML) backing or a different modal method. The PML backing is less accurate, and it introduces many more unknowns, rendering it less efficient. The alternate modal method is more computationally expensive due to a fully dense matrix block related to the port unknowns akin to an integral equation on the port surface. The transfinite element method is the most accurate and computationally efficient technique where modes are used to represent basis functions on the port, leading to minimal overhead for extracting S-parameters. It was key to a more recent advancement where HFSS solves only for the S-parameters when the fields are not required. In combination with the transfinite element method, this has proven to be very efficient. It leads to significant memory savings, which enables many more frequencies (often 3x) to be solved in parallel for distributed frequency sweeps.

Software and hardware advance in lockstep

As the above innovations show, electromagnetic simulation software is constantly advancing, ensuring it is relevant for the large-scale simulation challenges of today. Taking advantage of high-performance computing (HPC) capabilities are key to that innovation. For example, we developed a shared memory multi-threaded direct solver in 1999 that sped up the simulation significantly. The next significant solver advancement was the introduction of the first iterative solver in 2007. The iterative solver is great for complex designs with RAM limitations. In 2009, the domain decomposition method, DDM [4] enabled the HFSS solver to be used across multiple compute nodes to access both more cores for speed and more memory for capacity. The global mesh is first partitioned, then the FEM is applied to each partition. Finally, a global iteration process obtains fully coupled and accurate results.

Due to the iterative nature of DDM, it has found the most success in large antenna systems where the convergence tends to be fast and number of excitations relatively small. Since introducing DDM, we have continued to add functionality to antenna system analysis with the introduction of a surface integral equation (IE) solver in 2010, finite element-boundary integral (FEBI) solver in 2011, and a finite array DDM and hybrid FEM-IE region solver in 2012. The SBR+ (Shooting and Bouncing Ray) solver was introduced in 2016 and is very efficient at solving huge antenna systems mounted on various platforms, including automotive radars in large dynamic traffic scenes at 77 GHz. We introduced a general 3D Component Array solver in 2019, which relies on similar technology to Mesh Fusion to enable fast and accurate analysis of large antenna arrays. Typically, there are relatively few 3D components needed to define an array, making this method very efficient. All these additions can help you tackle various large scale antenna systems as well as radar cross-section (RCS) analysis. Many of these solver enhancements are also directly applicable to EMI/EMC analysis related to radiated emissions.

Fig. 2: DDM of Helicopter launching from a ship’s stern and emerging from cutplane mesh.

The solver enhancements discussed so far are related to time-harmonic simulation (frequency domain), but if you’re interested in transient phenomena such as electrostatic discharge, lightning strike, and accurate time-domain reflectometry, you need a time-domain solver, such as the discontinuous Galerkin time domain (DGTD) solver hybridized with an implicit solver that we added to HFSS in 2010 or the optimized standalone implicit solver we introduced in 2012. The implicit solver is best for electrically small designs and/or designs with high geometric complexity while the DGTD solver excels at electrically large designs with modest geometric complexity. DGTD also has a dedicated GPU implementation providing typically 2x-4x speed up compared with an 8 core CPU.

Tackle integrated chip package and board complexities

For the past decade, we have been focused on optimizing the simulation performance of HFSS for large-scale electronics systems such as IC packages and boards. Because of the complex nature of these designs, we typically rely on our direct matrix solver that has been optimized and enhanced since its introduction. For example, early enhancements included the ability to distribute the direct solver across compute nodes for enhanced performance and capacity, which removed the need for a single, expensive machine with a large RAM footprint. We also introduced an ECAD geometry-aware meshing algorithm that is specialized for layered structures typically found in packages and PCBs.

In 2015, we introduced an auto HPC framework where the solver automatically determines the number and type of distributed solver tasks to use based on a list of compute nodes and their hardware resources. Users no longer need to struggle with the complex task of optimizing these settings to get a fast, successful solution. For example, to speed up a frequency sweep, you want to increase the number of frequencies solved in parallel, but this can lead to exceeding the memory capacity of the machine. With auto HPC, the solver determines the memory requirements for a solution, then optimizes number of tasks to execute.

Fig. 3: HFSS Mesh Fusion simulation of PCB with connectors and flex cables.

As GPUs became standard, the EM simulation software industry took the opportunity to use it parallel processing capabilities. At Ansys, we added direct solver support for GPUs in 2016, which can provide a performance boost for large designs, especially. This was further optimized in 2018, yielding up to a 2x speedup compared with an 8-core CPU. The following years saw several enhancements to the distributed direct matrix solver, including improved performance when solving for large numbers of excitations. We built on that with the ability to distribute both the matrix assembly and field post processing across compute nodes, so that all critical steps in the FEM solve process can be distributed and parallelized.

Fig. 4: Exponential innovation.

After all of these innovations and more, with the introduction of HFSS Mesh Fusion, we are now in an excellent position to tackle our customers’ largest and most complex designs. HFSS Mesh Fusion is designed to tackle large systems, so you no longer need to make compromises to obtain reliable simulation results. All these advancements together with advances in HPC and Cloud computing enable a new frontier in high frequency electromagnetic simulation.

Fig. 5: Improvements over three decades.

References:

[1] Automatic Adaptive Meshing: Delivers accurate, efficient and reliable solutions to Maxwell’s Equations

Z. J. Cendes and D.N. Shenton, “Adaptive mesh refinement in the finite element computation of magnetic field”, IEEE Trans. Magn., vol. MAG-21, pp. 1811-1816, Sept. 1985

[2] Spurious Free Vector Basis Functions: Enables reliable FEM solutions of Maxwell’s Equations

M. L. Barton, Z. J. Cendes, “New vector finite elements for three-dimensional magnetic field computation”, J. Appl. Phys., vol. 61, no. 8, pp. 3919-3921, 1987

[3] Transfinite Element Method: Enables highly accurate and efficient extraction of SYZ network parameters

Z. J. Cendes and J. F. Lee, “The transfinite element method for modelling MMIC devices”, IEEE Trans. on Microwave Theory and Techniques, vol. 36, no. 12, pp. 1639-1649, December 1988

[4] Domain Decomposition Method:  Enables distributed memory computing, key for many advanced HFSS solver features

M. N. Vouvakis, Z. J. Cendes, and Jin-Fa Lee, “A FEM domain decomposition method for photonic and electromagnetic band gap structures”, IEEE Trans. Antennas Propag., vol. 54, no. 2, pp. 721-733, February 2006