An integrated quality assurance framework for enterprise performance management systems

Abstract

Increasingly, large organizations, such as governments and hospitals, are deploying complex on-line enterprise performance management systems, which support business performance management by analyzing the data collected from their operational systems into data warehouses. Those systems are often run on a network of centralized servers and are accessed by thousands of geographically dispersed users via browser-based inter-faces. To be successful, an enterprise performance management system must be scalable and it must be reliable to provide critical business services 24 hours per day x 7 days per week. The system also must be accepted by users, which means it will be really used so that the value of the system to the business can be realized. Data in the performance management system can be sensitive; hence, the system must protect privacy when delivering data and reports to users. This thesis defines an integrated quality assurance framework which addresses scalability, reliability, usage and privacy issues for enterprise performance management systems. Compared to black box testing, the integrated quality assurance framework has several advantages. It uses basic technologies (metadata modeling, reporting, portal, etc.) of an enterprise performance management system to provide deep analysis on system logs to measure quality both during development of the system and when it is in production. This thesis performs a case study using our approach by simulating the enterprise performance management system for a major teaching hospital in our lab environment and integrating our quality assurance framework. Often the data in the production environment is highly sensitive and its access strictly regulated by privacy law. To accurately assess the quality of the performance management system in a test environment, this thesis presents a strategy for generating test data with similar characteristics to the real sensitive data, and we have a Java-based tool implemented and applied in our research.