Understanding the dimensions of organizational evaluation capacity

Abstract

The question of evaluation capacity in governmental organizations has been studied empirically in only a limited fashion in recent years. The actual dimensions of evaluation capacity, or what evaluation capacity might look like in an organization, have not previously been identified systematically in the literature. This study sought to identify the key dimensions of evaluation capacity in Canadian federal government organizations as well as the manageable steps required to move from low to high capacity for each of these. The data collection approach was articulated around three phases. The first phase sought to identify the key dimensions of evaluation capacity in Canadian federal government organizations and characterize minimal and exemplary performance on each dimension through the development of a framework. This was achieved through a literature review and semi-structured key informant interviews. The second phase focused on refining and clarifying these dimensions by using key informant interviews. The third and final phase sought to validate the revised framework. This was achieved through its practical application in four federal government organizations and semi-structured interviews with those involved in applying the framework. The study concluded that evaluation capacity in Canadian federal government departments and agencies can be described through six main dimensions, each one broken down into further sub-dimensions. The capacity of these organizations on each of these dimensions and sub-dimensions can be assessed using four levels: low, developing, intermediate, and exemplary. The study found that organizations vary in terms of their capacity from one dimension to the next, and indeed, from one sub-dimension to the next. This study makes an important contribution to the body of knowledge on evaluation capacity; although much has been published in the evaluation literature on evaluation capacity building, the actual characteristics and attributes of evaluation capacity itself have not before been defined and described based on empirical data. Continuing research may focus on expanding the scope of the framework, while the main practical implication of this study is the use of the framework as an instrument of evaluation capacity for government organizations.