Using Agile Methods For Hardware

What used to work in design and verification no longer applies. An Agile development approach is one place to start.

popularity

agile methodology for software is getting a much closer look by hardware teams these days, because what used to work in SoC design and verification isn’t working nearly as well with rising complexity.

Development processes need to be constantly evolved to determine how to be more productive, deliver higher quality, cut costs in development, and how to get it right the first time. Proponents of ‘agile,’ a development approach that originated in software, purport that some of its key concepts are the way forward.

Harry Foster, chief verification scientist at Mentor Graphics believes in the fundamentals of an agile development approach, and said it helps to look at it from a historical perspective. “If you go back to the ’70s and ’80s, there were a lot of large projects, particularly a lot of mil/aero projects like satellites going up and blowing up. The government got tired of that and decided maybe they should look into this. There were a lot of really heavyweight software development models that were put in place, the classic one being the waterfall, ‘V’ development model. These were sequential development models and typically followed the classic flow of requirements, architectural design, coding, testing and maintenance. The origin of these models came out of manufacturing construction but the key difference is in that — and this is where the software community started running into problems — typically the requirements are pretty rigid. They are pretty well understood.”

Software developers complained that the requirements were not always clear enough. “A lot of the software community was realizing that they were delivering software to the customer that didn’t satisfy what the customer wanted,” said Foster. “The problem with these sequential processes is that they are very rigid, heavyweight processes so the software community realized that they couldn’t afford it — they’d deliver software to the customer and had to go back and iterate through a heavy process that takes too long, and they weren’t satisfying the customer. The motivation was to move to something that was lightweight that allowed the requirements to evolve at the same time as the development. That makes a lot of sense for the problem they were trying to solve.”

The question we have to ask today, he said, is whether the semiconductor design industry has the same types of problems, and whether it is possible to apply those kinds of development methods to hardware. “No, not from a purist point of view. That said, we do have the same type of problems in hardware and there is a lot we can learn and leverage. There is a lot that is being used today. In the agile development process, you continually create the product and deliver and go through this iteration loop. We can’t afford to do that in hardware. You could argue that although you can’t tape out a chip, we could deliver a model. At the same time, there are differences in hardware and software: Today in terms of SoC development, a lot of the SoC is made up of IPs, and these IPs are hard and fixed and rigid, and the requirements are well understood. If you were to iterate like that, you wouldn’t have time with the models to iterate like they do with software development.”

Hardware vs. software
Anupam Bakshi, CEO of Agnisys, believes we have come to such a level of abstraction that developing hardware is now an exercise in software development. “The Agile Manifesto as it applies to software development may not be fully applicable. For example, the first value, ‘individuals and interactions over processes and tools,’ should not be taken literally. Unlike software development that needs just a compiler, hardware development does rely on a multitude of EDA tools.”

However, he pointed out, “The 4th value of the Agile Methodology states, ‘responding to change over following a plan,’ which may seem not to be applicable to hardware development since ASICs are cast in stone and can’t be changed frequently. However, designers can make the ASIC configurable by relying heavily on configuration registers that will make the ASIC immune to changes and highly adaptable.”

An agile (with a lowercase ‘a’) development approach for hardware is useful and highly productive as long as the emphasis is on frequent production of working products, continuous communication with customers/among team members, and flexibility under changing requirements, Bakshi added.

At the same time, Drew Wingard, CTO of Sonics pointed out that in thinking what hardware teams can take from agile and apply to their designs, “we work our way back from one critical difference between chip design and software, which is we have this thing called the mask set. The concept of ‘agile mask set’ doesn’t really work.”

But it does work in other areas, he said. One of the concepts in agile is the idea of the sprint, which is identifying a set of key goals and trying to get to those goals as quickly as possible—and trying to learn about a set of things in that process. “That’s different from the traditional model of chip development activity because the agile goals tend not to be anywhere near as complete. The idea is you’re trying to do and explore at the same time so it’s very different from the waterfall approach that is much more traditionally practiced. The waterfall approach has a lot of merit, but it has a lot of history in it that comes from the way we think about the layers of a chip design. At some point we end up with a mask database, typically in GDS II, and before that we have the preliminary version of that database, which has to go through electrical rules checking, and design rule checking, and layout vs. schematic, and equivalence checking. Then you work your way back up from that, and you’ve got the whole place and route. Above that you’ve got the logic synthesis and test insertion. Backing up from that you’ve got an integration level where you are putting your components together, and backing up from that you’ve got the logic design, and the design of the mixed signal components, and all those things. Backing up from that finally you’ve got this architecture, these ideas, these algorithms that you’re going to try to put into hardware.”

The waterfall methodology can be characterized by doing all of those tasks as discrete steps, and Wingard noted there is a perception that the cost of doing the next step with incomplete information from the prior step is so high that it’s not worth doing, which is completely orthogonal to what agile teaches in software.

“In software, agile teaches do a little and go as deep as you need to go, but try to get this one part done and hopefully focus on that one part that maybe you’re the most worried about, or on that one part that you think might have the biggest implications on everything else,” he said. “And yes, you are only working on a part of the function out of the total functionality you’re going to build, so you’re going to have to focus a lot on things like interfaces, you’re going to have to focus a lot on building up along with it the test suites that help you prove to yourself that even though you’ve only implemented part of the function, you know that function is correct. In fact, in a number of the agile methods, they actually write the test first, and then they try to implement the software function that goes inside the test harness. Instead of writing a specification, they make the test the specification. There are some really interesting aspects of that that have some direct analogies in hardware design.”

Some agile adoption
While adoption is slow, there is growing evidence that hardware engineering teams are adopting certain agile ideas.

Wingard recalled that when engineering teams discussed internal milestones in a chip design a decade ago, there was a first compile of the top level. Then he started hearing about compile zero. “This is the object at the top level that exists before the first compile. What does that mean? That means that they are knowingly working with incomplete information, not just unfinished, but knowingly incomplete because they want to try to do a first floorplan of the chip. They want to do some other kinds of design planning. They want to begin to test the infrastructure they’re using for assembling the chip. They want to look at some things around clock partitioning or power partitioning or DFT long before their blocks are even 50% through the design process. To me, that’s an agile or shift left approach, but still it’s very heavy.”

Additional tooling to help make it easier for designers to deal with incomplete information is rather ad hoc, at this point, but is sorely needed.

Another area that could use a lot more help is in the area of portable testbenches, he said, so that for teams wishing to adopt agile methods, getting away from handwritten specification in Word, and trying to use the test harness as the specification, “I’d better be able to easily, automatically port that test harness into different environments.”

Another area where agile concepts are being applied today is between emulation and FPGA prototyping, observed Frank Schirrmeister, senior group director for, product management in the System and Verification Group at Cadence. “We have the same compiler and you can, in a very agile fashion, move forward and use a different capability and develop when your RTL is less mature. You still use emulation. Then once your RTL is more mature and you need really high speed for software development, you switch to verification.”

Defining Agile
Neil Johnson, principal consultant with XtremeEDA, asserted that agile is a big word and can include a lot of different things. “The thing that people most commonly associate with agile development is continuous deployment or incremental development. There is the assumption that if you’re building software, you’ve got an incremental development process, where at the end of each increment you actually end up deploying product. And if it’s deployed to the Web, or deployed to wherever, there is the idea that in software incrementalism is easy, and that’s why it can be developed incrementally and deployed incrementally. For hardware developers that’s been a stumbling block, not so much for FPGAs, but obviously for ASICs the concept of incrementalism doesn’t quite fit as nicely.”

And when people talk about agile development, he said he likes to rein things in a little bit and suggest there is more to agile development than just the idea of incremental development and incremental deployment. “There are specific technical practices like test-driven development (TDD), which is very applicable. When you look at technical practices individually, TDD is one that is very applicable. The idea of building cross-functional teams — that could be even more valuable for an SoC team than a software team. You can’t argue that synchronization between team members is easier or more valuable in software development. I would say it is more critical in hardware development. The idea of building a shared workspace is not something that hardware developers typically associate with agile development. But simple concepts like I’m sitting right beside the person that I work with and I ask them questions directly instead of shipping emails or filing bug reports, or just the means of communication within an agile team, is different. It’s not something people necessarily associate first with agile development.”

If agile development is looked at as an entire thing, an entire method that ultimately ends in incremental deployment, chances are it’s going to be a hard sell. “But if we break it down, if we look at the idea or look at the ideal of incremental deployment, see how we can mangle that to fit the technology but then also look at the other practices that are for some teams just as valuable or more valuable than incremental deployment because not all agile software teams deploy incrementally but many of them have a shared workspace, many of them have embraced the idea of zero defects,” said Johnson. “They have embraced practices like test-driven development, pair-programming, those types of things. It’s much easier to look at agile development one technique at a time, then maybe put them all together into our own methodology as opposed to assume someone else’s methodology.”

So what is holding agile back? Hardware engineers are being very careful or very deliberate in their decision making, he continued. “We don’t want to take on new ideas because we want to have the track record. But if we take a look at the track record we have now, I’m a verification engineer, and if I look at the constrained random verification track record or UVM track record, I look at that with a considerable amount of skepticism. Really, what we like to do is stick to the things that we’re used to. For example, we’ve come to believe that the ideals of constrained random verification are true, but the amount of frustration, the amount of schedule overruns, defect rates, all of that stuff has shown us that instead of being critical of new techniques because they have no track record.”



  • Lars Asplund

    Figuring out what works for you and your team is something that all teams (HW and SW) have to do. Often it’s not an either/or question. I don’t use pair programming all the time but ask for help when working on a piece of tricky code. Developers are expected to unit test their own code but if they write the test cases first or not (use TDD) boils down to personal preferences. Individuals and interactions over processes and tools.

    Still, tools are important and a vital component is the unit testing framework to enable short code/test cycles and continuous delivery. There are open source frameworks for almost every SW language you can think of ( https://en.wikipedia.org/wiki/List_of_unit_testing_frameworks ) but until recently there were none for HDLs. This has changed and today you have VUnit, http://vunit.github.io, and SVUnit , http://www.agilesoc.com/open-source-projects/svunit, to support VHDL and SystemVerilog. So nothing is stopping you from adopting more Agile ways of working, especially not when working with FPGA development.

    Lars Asplund, VUnit user and developer.

    • Kev

      Might work better with a merged design flow –

      http://parallel.cc

      RTL is passed its sell-by date, and SV & VHDL still don’t work properly for HW design (IMO).

      • Lars Asplund

        The key is the short code/test cycle which starts with the developer. To make that happen unit testing is done in the same language as the production code. Doing it in another language tend to create “special skills” performed by other people in other teams and the short code/test cycle is lost. This is why the Wikipedia list is so long. At least one framework for every language. If ParCC is the future than unit testing will have to be done in (ParC)C.

        • Kev

          ParC is a minimally extended C++, so it gets to reuse all the C++ tools and methodology (as C++ was built on C).

  • Kev

    A quick read of the Agile Manifesto –

    http://www.agilemanifesto.org/

    – kinda tells you where the mismatch between IC design and software lies –

    ICs need good specs and documentation.
    IC design is heavily dependent on processes and tools.

    Could it be done lots faster, certainly, but “agile” isn’t going to work unless the whole eco-system is “agile”.

    • nosnhojn

      the irony here is that before the manifesto, software also “needed” good specs and was heavily dependent on process and tools. then people figured out that specs/tools weren’t as necessary as communication and working software.

      • Kev

        Software generally has more flexible requirements than hardware, and in the open-source space you get the code so documentation is less important there. Getting a new SW release is often a matter of hours vs the months/years for a new/respun IC.

        • nosnhojn

          again, hardware requirements are firm right up until we’re forced into changing/dropping them. months/years used to be normal for new software releases until it became hours/days.

          I’m not suggesting we can be agile in exactly the same way sw teams can, but let’s acknowledge that hardware process is the way it is because we think/make it that way. it could just as easily be something else.

          • Kev

            If one of the fabs decided to do its own CAD flow in the Cloud, I think we could get there, but the current structure of the industry doesn’t lend itself to being agile.

    • Karl Stevens

      .Kev:”
      IC design is heavily dependent on processes and tools.” The physical design? yes. The functional design is where
      there is plenty of room for improvement. In my many years
      we did logic design. which is and was an iterative process that evolved from gross requirements, a block diagram with possibly a flow chart(diagram). I really wished for a way to define modules and to tie them together functionally to exercise the total machine even when only the raw data and expected result was available.
      Now that I am retired I found it. And as pointed out in the article there is a lot of IP to integrate. OOP lets me define classes at any level of detail and the compiler does the grunt work of building and connecting objects that can be a mix of HW and SW. Then the debugger has break points, single step, and shows variable values in a watch window or pop up text block.
      The only problem is it is a free Microsoft product that does not run on Linux with command line and TCL script.
      Did you notice the same few show up to discuss Agile, but that HW designers stay away in droves?

  • Karl

    Bakshi said “However, designers can make the ASIC configurable by relying heavily on configuration registers that will make the ASIC immune to changes and highly adaptable.”
    There is a tried and true methodology that worked for several generations of IBM mainframes and can be implemented much like an FPGA. That is microcode and the speed of embedded memory blocks that can be accessed simultaneously can create a flexible high performance chip.
    So instead of using an FPGA to emulate the HDL, the logic can actually be loaded and run on the FPGA.
    The end result will be a general purpose high speed chip. High level languages have assignment, loops, and conditional statements that can be parsed and easily made into microcode.
    Flexibility is the answer to change so when do we start?
    PS No, I do not expect any takers.