Specification-based regression testing measurement with risk analysis

Abstract

Regression testing is essential to ensure software quality. A test team applies a regression test suite to ensure that new or modified features do not regress (make worse) existing features. Code-based regression test selection is good for unit testing, but it has a scalability problem. When the size of the object under test grows, it becomes hard to manage all relevant Risk Exposure information and to create corresponding traceability matrices for validation and coverage assessment. We propose a scalable alternative, namely a specification-based method for regression test selection. There are three major parts of our work: First, we propose and justify a new, specification-based regression test strategy built on two complementary test coverage goals, namely coverage of modified features, and "affected entities" (Targeted Coverage), and additional, complementary coverage of functional areas of risk (Safety Coverage). Secondly, we provide systematic methods for selecting these two types of regression test cases. Finally, to evaluate our approach, we use data from a real software product to do case studies. (Abstract shortened by UMI.)