A look at what followed when a Russian company realized its approach to design data management and version control would not scale.
Authors: Andrey Medushevskiy, CAD/PDK Manager; Konstantin Bragin, ASIC Verification Lead; Michail Kakoulin, Head IC Design Centre
Milandr
Moscow, Russia
Milandr is a twenty-year-old product company based in Moscow, Russia, that builds high-reliability integrated circuit (IC) products for the aerospace, avionics, automotive, and consumer markets. Their product lines include microcontrollers and microprocessors on the digital IC side and SRAM memories, wire interface transceivers and RFICs on the analog/mixed-signal side. Milandr works with multiple FABs, include X-FAB and TSMC, to manufacture their chips and has worked in multiple geometry nodes from 1 micron down to 65 nanometer nodes.
IC design teams are under constant pressure to reduce cycle time and with each new chip they are looking for innovative ways to do this. Improving designer productivity is clearly one way to achieve that. While this article has been written with an analog/mixed-signal engineer in mind, these approaches are also applicable to other IC design engineers.
Milandr uses Cadence’s Virtuoso® Design Environment to develop their custom analog and mixed-signal ICs. Each Milandr engineer had been using his or her own libraries and setup files, and keeping track of multiple versions of layouts, schematics, test benches, and documents in untracked project workspaces.
To reliably use each other’s blocks, engineers would have to create custom scripts and communicate extensively with each other to understand which updates were made and when. More importantly, operational complexity was so high that new users would take a long time to come up to speed when trying to locate the data they needed. (See Figure 1.) Problems included use of wrong configurations, mismatched schematics and their layouts, and verification teams testing outdated versions.
With 150+ engineers across multiple design centers in Russia, the company realized that this approach to design data management and version control would not scale.
Figure 1: Complexity of project areas and workspaces
As team sizes grew, the engineers assigned to the specific projects were no longer co-located, which stressed the need to communicate even further through meetings and unproductive management overhead. Management had to rely on engineers to provide detailed status on data and the level of detail varied with the engineers. Some had no details at all and this was a problem. When engineers tried what-if scenarios and accidentally forgot to archive a working version, they then had to recreate the working version from memory.
After evaluating a number of tools, the team at Milandr chose SOS, a hardware configuration management (HCM) tool from ClioSoft. The addition of ClioSoft’s SOS has boosted Milandr’s design productivity by offloading manual data management from their designers. The HCM methodology also eliminated design data errors and synchronization issues between Milandr’s projects and design centers.
Overview of the design environment
Milandr has several design centers; the ones at headquarters in Zelenograd, Moscow, and a second in Nizhny Novgorod host the analog/mixed-signal chips design teams.
Working in the Cadence Virtuoso design environment, Milandr engineers have produced several successive versions of blocks that are reused over many design iterations. Typical mixed-signal IC design projects at Milandr last between six months and two years. Teams usually are broken up by expertise into digital, analog/mixed-signal and layout. A typical project may include about two to six analog engineers, about two to three digital engineers and about three to four layout engineers.
While Milandr does have a large number of engineers, each engineer is sometimes required to work on multiple design projects simultaneously. Prior to introducing SOS, each engineer’s workspace was a jumbled mess of links, versions, and myriad naming conventions. In case of an engineer’s absence — due to illness, vacation or exit — it was very difficult for another engineer to step in and pick up where the original engineer had left off.
Engineers are hired at multiple locations (either because of proximity with customers, availability of skill set, or cost of labor), so multi-site development is a necessary component of working in Milandr. Engineers from the different sites have to work together on the same project, making fast and accurate synchronization of the data essential. During the initial configuration of the ClioSoft system, Milandr CAD managers created a central file repository for all design data. From this centralized repository, project managers could always figure out the state of each revision of the design and track the complete history of these components. Taking advantage of SOS features, communication among engineers was seamlessly maintained through the use of triggers, like email notifications, on important events like check-ins, etc.
The experience level of the engineering community varies a lot, even within a project. Some engineers have less than six months of experience, while others may two to ten years of experience using the Cadence tools. Design experts may have 25 to 40 years of analog/ mixed-signal design experience, but may not have deep familiarity with either the Cadence tools or low level Unix (or operating system or scripting) expertise.
Engineers coming fresh out of school may be familiar with Cadence tools, but not have the expertise to create scripts to eliminate manual repetitive tasks. SOS’s integrated GUI within the Cadence tool framework means that engineers don’t have to learn a lot of new or extraneous commands or scripts to get their job done, which has made even the most junior engineers very productive.
Overview of hardware configuration methodology
ClioSoft’s HCM design methodology consists of four primary parts:
The HCM methodology provides many benefits to the design team. Chief among these is protecting users from accidentally losing valuable data and eliminating the need to manually keep track of thousands of details about the design, both of which have an immediate impact on productivity.
The SOS HCM also provides the ability to easily configure different versions of the design for verification. As blocks are updated, SOS allows the verification team to quickly create the needed data set, including verification test benches. The team can mix and match different configurations of the design and perform their tests with the confidence that they are using the appropriate data.
The HCM methodology also encourages more experimentation in block design. Because of the ease with which a designer can configure a design, block variants can be quickly tested and the best version selected. This approach leads to higher quality in the final design, according to Michail Kakoulin: “Our designers feel a little more free to play around to achieve the best results. You can experiment with subtle changes when you have a tool like SOS at your disposal. If a problem pops up, we can always get back to a previous version of the design.”
Adopting ClioSoft and HCM
After initial consultation with ClioSoft, the Milandr team set up the SOS system and created a central filer system to manage the design data repository. The remote office was connected to the system.
The remote design center was configured with a cache server that is automatically synchronized by SOS. This keeps network bandwidth requirements low, as a change in one block requires only the small subset of the design or library to be synchronized.
“Although the move to the central filer initially caused a hit in update time, the overall system works better for our team,” said Andrey Medushevskiy. “Designers can go to any Milandr center and have access to their latest work.”
As configured for Milandr, SOS provides each user with a private work area used for individual development. From within the work areas they link to smart caches. This has significantly reduced the time that it would otherwise take to open, read, and save files. SOS also can synchronize with all of the other work that is proceeding in parallel. The changes from one user in one design center are available to all other users, both locally and in the remote center. “SOS is invaluable even if you don’t have a remote office. If you do have a remote, it’s mandatory,” said Konstantin Bragin.
Once a designer is satisfied with the changes, he can check in a new version of the block and that data becomes available to the rest of the team. The data being managed also includes test benches, which often can be as significant as the IP being designed. By placing test benches under revision control, the design team always knows which versions of the IP go with which versions of the test bench. Design documentation derived from these test benches also is managed under SOS.
The Milandr CAD team chose to automate and standardize each designer’s work area, thereby increasing their ability to quickly deploy or replace new resources on a project.
Tool flow automation
The HCM methodology also allows Milandr’s CAD team to automate and perform error-free repetitive tasks. Some of these capabilities include:
New and updated user workspaces: The CAD team was able to develop and deploy scripts that would automate easy and quick access to the different project data and thereby make new users more productive quicker than before. You can see in Figure 2 and 3 below how much cleaner the work areas look compared to those in Figure 1.
Figure 2: Cleaner setup allows new user to bootstrap quickly
Management could also secure access to certain project data by use of these scripts. By ensuring the controlled use of cds.lib (managed by SOS), a Milandr-controlled common cdsinit (Cadence configuration file) template file and the automatic setup of relevant UNIX environment variables, the teams also could ensure that all engineers were working off the correct and approved project libraries in addition to the verified and tested EDA Vendor software tool versions.
Figure 3: Use of scripts created a much cleaner work area for engineers
Automated communication: Utilizing the “Exec Before” and “Exec After” checkin capabilities of the SOS system, the CAD team was able to automatically trigger a set of email notifications to all the relevant team members. Management also was able to keep track of the state of the project seamlessly. This had a desired side effect that all the team members were informed of the changes that had taken place and ensured that the team members were using the correct cells from the correct libraries, leading to a reduction in wasted verification time and simulator license usage.
Use of Tags and Snapshots: The CAD team was able to use the capabilities of SOS, including Tags and Snapshots, to mark milestones and event timelines in the project, as well as to identify rollback and recovery points to quickly recover from any unforeseen problems that occurred.
Visual Design Diff (VDD): Milandr engineers heavily use the Visual Design Diff (VDD) tool, which has capabilities that help the team to quickly identify the magnitude of changes by automatically highlighting the differences between two versions of a schematic or layout. Presented directly in the Virtuoso editor windows, VDD makes changes visually obvious and helps quickly verify what changed between any two revisions.
Figure 4: Controlled access to project data ensures all members are in sync.
Impact on productivity and design quality
One of the biggest impacts on productivity with the new HCM approach has been the interaction between designers. Once all the project data was centralized and saved on the server, engineers no longer needed to manage the data they stored on the network disks. Also every engineer knew where to find the data he needed to complete his tasks.
Communication among the project members improved significantly because of the email notifications that were automatically triggered when important events occurred (check-in, check-out, update, snapshots, tags, etc.) Furthermore, when data accidentally was written over or deleted, teams could automatically roll back to a known stable state, something they could not do before.
Management too has seen benefits. The enforcement of “comments” at check-in allows them to understand what changes were made, who made them, when they were made and why. This has increased discipline and accountability in the engineering teams, which has led to fewer bugs and faster time to market.
Next Steps
As Milandr looks ahead to the next set of innovations, they plan to take advantage of additional features that SOS provides. While they currently only use the main branch of the history tree, they are investigating use of Branching and Merging as a way to allow teams to be insulated from changes that are happening elsewhere in the company until such time that they are ready to absorb changes. This allows fine-grain control of the environment, the project libraries, PDKs and other related files.
Summary
Milandr has implemented an HCM methodology based on ClioSoft’s SOS platform. This methodology has led to improved design quality and increased designer productivity by allowing the design team to focus on the design and spend less time communicating repetitive information. Designers can be assured that they are using the correct versions of projects and work areas.
Tool flow automation helps to ensure quality by automating away many small tool tasks and by managing the production of data to make it ready for the next design flow step. In addition, productivity is increased by the HCM methodology’s focus on visualizing and tracking changes before they create problems downstream.
Milandr strongly believes that deployment of the SOS tool and methodology has been one of the critical success factors that has allowed them to scale their design activities to over 150+ engineers across multiple design centers in Russia.
 |
 |
 |
 |
 |
 |
 |
 |
 |  |
Leave a Reply