In my first post I described the methodology we use to build software. Now I'd like to explain how we ensure that the software built is of extremely high quality.

Defining quality is a good starting point. If you are considering buying a car you might focus on the fit and finish - do the doors sound like bank vaults, as some luxury models apparently are designed to? Or usability: Can you figure out the controls, and do they work consistently? Finally you might consider metrics regarding failure rates, residual value, etc.

Software is honestly not that different. In my view, high quality software installs quickly and easily without needing to refer to the documentation. Basic operations are intuitive, and the system feels coherent and consistent; moreover, it provides visible feedback on what it is doing - especially if things are not going according to plan. Once deployed, good quality software operates continuously, without degradation or unexpected failure. It warns you when problems are likely to occur and recovers gracefully after a failure. If it plays the sound of a bank vault door closing, so much the better...

At Attivio we strive to achieve all of these qualities, and many more. Ensuring quality is always our #1 consideration. Simply put, we'd rather explain why something is going to take longer than explain why it doesn't work. Our approach is, as with assembly, very deliberate:

• We practice test-driven development. No implementation is deemed complete until all the associated unit tests run correctly. All releases must achieve at least 80% test coverage and 100% compliance on various automated evaluations like CheckStyle and FindBugs. We conduct open code reviews of new modules at the end of each sprint, during the reflection period. There are no exceptions to this policy, and part of R&D's compensation is tied to meeting or exceeding all such requirements.

• We allocate time for resolving issues, even low and medium priority ones. No issue, no matter how small, is allowed to remain deferred for too long, even if it is a small item.

• Our QA team reviews and expands unit tests as part of the ongoing development process. They make sure that corner cases, edge cases, etc, are covered appropriately. Test automation is also software development - our QA team members are all capable Java programmers.

• QA is also responsible for reviewing any and all issues reported that were not caught by a unit test. In these cases the goal is to determine why the test didn't catch it, and then modify all related unit tests to cover it. (I recall reading an article about a high-performance auto manufacturer having a special team to investigate unusual accidents and mechanical failures back in the 80s - but can't find the reference now - This e-mail address is being protected from spambots. You need JavaScript enabled to view it if you have seen it.)

• We embrace automation: for GUI testing, system benchmarking and all other necessarily repeated tasks. Each build is benchmarked automatically across a range of configurations. We use some terrific tools like Selenium, JMeter and Siege to make this happen.

• All new hires are asked to write a detailed review of AIE after their one month anniversary. These reviews are analyzed and fed back into the product plan.

As of this writing, we have more than 3,000 unit tests covering 81.2% of AIE 1.2. At Attivio, quality is job 1 to 3,000 and counting! We invite you to judge for yourself. This e-mail address is being protected from spambots. You need JavaScript enabled to view it or visit our website for more information.

Sid Probstein, CTO

TrackBack URI for this entry

Comments (0)