In my previous post, I talked about the role that people play in automating development testing. However this isn’t the only critical factor for success in a software development business: the processes you use are just as important.
When we assessed our testing processes two years ago, it was clear that they were not working and would definitely not scale for building more features, faster. While our developers regularly built and delivered automated tests for the front-end and analysis components of the Coverity platform, this wasn’t the case for our Coverity Connect ‘console’ or, in fact, our end-to-end QA process. We were running too many manual tests and lacked meaningful metrics to tell us if we were on track for quality at each release milestone. We concluded that the problem was due in part to the disconnect between the methodology of the development organization and the methodology of the QA team.
The way to eliminate this disconnect was to automate more of the testing and interleave it with the development process. We tackled this in two phases. Our first focus was to increase the level of automation (automation is the pre-requisite that makes interleaving test and feature development useful). Over the course of 18 months we made significant progress in terms of automating the tests for Coverity Connect, as well as for our end-to-end QA process. During the next phase, we began concentrating on interleaving the development of automated tests in both areas with the actual feature development.
The end result of some of these “process” changes: 97% of our end-to-end tests are now automated, including the tests for license validation and web services. (The remaining tests are the ones that are hard to automate and require some form of manual inspection, e.g. visualizing the correct layout of a browser page.) We can now complete a full round of end-to-end tests in approximately two days, with three engineers. This has enabled us to broaden the scope of our testing and helped reduce the time to harden the latest maintenance release from two weeks to three days!
If you’d like to hear additional details about our process changes – and the results we achieved – you can click on my video interview: Transforming your processes to transform your testing.
The post Transforming your processes to transform your testing appeared first on Software Testing Blog.