Duration: 35:28

622 views

5

DevOps puts a huge emphasis on test quality: while it promises to drastically reduce delivery cycles, hasty deployment entails the hazard of propagating a regression bug into production due to lack of sufficient testing. Statement coverage is the most popular metric to determine the adequacy of a test suite. Yet, this metric is known to be a poor indicator of a test suite's capacity to detect bugs.

This talk will introduce and illustrate the benefits of another metric for test suite quality: the mutation score. The intuition of this score is simple: to determine if a test suite can detect bugs in a program, simply inject bugs in this program and check if the tests can detect them. In this talk, I'll cover
- the key principles of mutation analysis.
- the Pit tool to perform mutation analysis on Java program to assess the quality of JUnit test suites.
- the type of faults that Pit can inject and feedback it can provide about test suites
- the Descartes tool, an extension of Pit to scale mutation analysis to large Java projects
- concrete examples of weaknesses in the test suite that are missed by statement coverage and revealed by mutation analysis

Speaker(s):
Benoit Baudry (KTH Royal Institute of Technology)