A Case Study on Automated Source-Code-Based Testing Methods

Date14 May 2013 - 17 May 2013
EventDASIA 2013
LocationPorto, Portugal
We present results of a case study on a test generation approach called Flow-optimized Automated Source-code-based unit Testing (FAST) which generates test stimuli from information available in the source code, in particular taken from the detailed software interfaces. This allows automation of a significant part of testing, ranging from the test stimuli generation to the generation of the test report. A huge number of stimuli can be generated exploring the behaviour of the software under test under nominal and non- nominal conditions. Symptoms like timeouts, unexpected termination, run-time exceptions, out-of-range conditions and missing coverage are applied for defect detection. The goal of this study was to evaluate the FAST process in context of a real spacecraft flight software application and to get a feedback on its scalability regarding larger applications, its sensitivity on detecting defects in the code, the achievable test coverage, its compliance with software standards and potential limitations. We also consider the impact of coding style on suitability for automated testing. The results confirm that the approach (1) provides acceptable code coverage results without requiring manual intervention for test preparation and execution, (2) raises the probability of activation of exotic fault conditions, (3) may provide hints on locations in the code where robustness needs to be verified, and (4) identifies defects not found before by static analysis and intensive testing