RTL Done And Other Bogus Development Milestones

My definition of progress has changed over the years. I don’t think about it much anymore but it was obvious in a talk I gave a few weeks ago to a diverse group of hardware developers. Part of that talk centered around how we define progress in design and verification. This is a normal thing for me; I was speaking to slides I’d used several times before and the message was no different than it’s been in the past… with one notable exception.

I like to use a pretty strict definition of progress. Progress to me has become design features verified and the metric I use is passing directed tests and/or functional coverage. The reason I use passing tests and functional as metrics are because they have a direct correlation with design quality and design completeness. For example, a passing test tells me (a) that some part of the design is implemented and (b) that part of the design is correct.

On the surface, this strict definition of progress probably seems pretty normal… until I itemized a list of tasks that wouldn’t count as progress.

Documentation was first. Documentation is necessary, don’t get me wrong. You can’t just wing it when it comes to building a massive SoC. But because we can’t confirm a correlation between documentation and quality or completeness of the corresponding design it can’t count as progress. Look at it another way, you can’t tape-out a design spec and validate it works. Then again, after design documentation is done it tends to go obsolete with the first dozen key strokes anyway so even if we could tape-out a spec, it’d be a bad idea.

RTL done – the point at which the design is coded and ready for testing – is another example. RTL done is a milestone that implies notable progress. It’s a celebration milestone, one that history has repeatedly shown to be entirely bogus. Why you ask? Because the celebration comes to end when the design bursts into flames on the 7^th clock cycle of the first test. I’ll admit to a tenuous correlation between RTL done and design completeness so it almost fits the progress criteria, but poor initial code quality is easily the disqualifying factor. If our Gantt chart schedules were accurate, the “RTL done” label would say “RTL written (but don’t get your hopes up because we’ve got months of debug ahead of us before it even remotely qualifies as shippable)”.

Testbench done is a verification engineer’s version of a bogus celebration milestone for all the same reasons, and more. Take the RTL written milestone and multiply it by constrained-random; that’s where we verification engineers are. We show up on the Gantt chart as “testbench written (but it doesn’t work and we don’t know why because it’s difficult to know what it’s even doing)”.

For me, the misguided definitions of progress we share in hardware development come down to us conflating effort expended and progress, especially early in the development cycle. While effort can be expended in any direction, progress suggests effort expended in the right direction. Which takes us back to passing tests and functional coverage as progress metrics. A passing test confirms our effort is taking us in the right direction.

How do we change? My recommendation is to take the project Gantt chart and remove everthing that doesn’t count as progress. When you discover the enormous and unacceptable gap between kick-off and your first batch of passing test results, devote time to closing that gap. Split the product into small sets of features, document and implement feature sets incrementally – likewise for the testbench – then write and run tests as early as possible to validate your progress as you go.

That and remember the design is the ultimate deliverable. For all the other items we have on a front-end development todo list, never forget that design completeness and quality are the only things that matter.