Information and Communications Technologies (ICT) in Program Evaluation Practice: Exploring New Pathways to Evaluation Use

Abstract

Despite the growing importance of information and communication technologies (ICTs), their use in the field of program evaluation has been, to date, understudied. Using a mixed-method approach and a conceptual framework informed by social interdependence theory, this thesis sought to contribute to this area of research by exploring how evaluation practitioners engage with ICTs, and if and how this influences evaluation use. The comprised two parts are in a cascade design: a qualitative study relied on interviews with practitioners, followed by a quantitative study utilizing a survey instrument to obtain findings from a larger cohort of practitioner respondents. The findings from both studies revealed that ICTs’ value is greater for the use of evaluative results (findings use) than for benefits arising from evaluation processes (process use), and that these tools support evaluation activities by aiding background and foreground processes. When used in the background, ICTs can help evaluators by offering flexibility, improving speed, and efficiency. In contrast, ICTs used for foreground processes help foster communication with and engagement of stakeholders. Together, this makes the use of ICTs a valuable extension to existing evaluator skillsets, especially when the tools are mindfully incorporated to fit a project’s contextual complexity. The findings also point to a number of barriers – at both the individual and organizational levels - that inhibit further integration of technology into evaluation practice. A conceptual framework developed as a result of the findings visually depicts this process, and it represents a significant contribution of the thesis. Given the growing proliferation of ICTs in all aspects of life, further insight through systematic inquiry is much needed to keep abreast of these rapidly changing trends. The provision of such has implications not only for evaluation practitioners, client organizations and professional evaluation associations, but for the ongoing development of empirical inquiry in the area.