Scalable Metamorphic Testing Approach for Web System Security

Abstract

Security testing aims at verifying that the software meets its security properties. In modern Web systems, however, this often entails the verification of the outputs generated when exercising the system with a very large set of inputs. Full automation is thus required to lower costs and increase the effectiveness of security testing. Unfortunately, to achieve such automation, in addition to strategies for automatically deriving test inputs, we need to address the oracle problem, which refers to the challenge, given an input for a system, of distinguishing correct from incorrect behaviour (e.g., the response to be received after a specific HTTP GET request). Moreover, another important objective in security testing is to discover vulnerabilities within a reasonable timeframe. This thesis addresses these challenges by first introducing a metamorphic testing approach (MST-wi) that integrates test input generation strategies inspired by mutational fuzzing and alleviates the oracle problem in security testing. It enables engineers to specify metamorphic relations (MRs) that capture many security properties of Web systems. To facilitate the specification of such MRs, we provide a domain-specific language supported by an Eclipse editor. MST-wi automatically collects the input data and transforms the MRs into executable Java code to automatically perform security testing. It automatically tests Web systems to detect vulnerabilities based on the MRs and collected data. However, metamorphic relations are typically executed on a large set of inputs, which is time-consuming and thus makes metamorphic testing impractical for large systems. We tackle this challenge by proposing an Automated Input set Minimization (AIM) approach that selects inputs to reduce testing costs while preserving vulnerability detection capabilities. AIM includes a clustering-based, black-box approach to identify similar inputs based on their security properties. It also relies on a novel genetic algorithm to efficiently select diverse inputs while minimizing their total cost. Further, it contains a problem-reduction component to reduce the search space and speed up the minimization process.