Practising environmental policy evaluation under co-existing evaluation imaginaries

This article presents a case study in which we show how different understandings ofwhat environmental policy evaluation ‘is’ and ‘should do influence the evaluation process as they are mobilized interchangeably. We hypothesize that practitioners deal with these inconsistencies by decoupling approaches, intentions and outcomes from each other.

In this way, innovation occurs in parts of the evaluation process while other parts follow a more traditional approach.

The claim in this article is that the organizational context is more important, in terms of explaining the selection and combination of evaluation approaches in practice, than the literature so far has acknowledged. We therefore aim to empirically explore how different evaluation approaches are selected and combined under the influence of co-existing, but contradictory societal views on what environmental policy evaluation ‘is’ and ‘should do’. How in their everyday work do evaluation practitioners address the multitude of evaluation approaches, given diverse societal expectations of evaluation? A case study of evaluators involved in a prominent Dutch environmental policy evaluation study in the PBL Netherlands Environmental Assessment Agency, illustrates how practitioners select and combine different approaches by drawing on two different evaluation imaginaries, which, as we demonstrate, creates both tensions and opportunities. This case reflects a more general tendency in environmental policy evaluation to draw on a tradition of technical-rational approaches to assess policy performance, while calls for novel approaches to assess complex policy processes are slowly gaining ground.

The ad hoc and patchwork evaluation style of our case illustrates how inconsistencies experienced due to the co-existence of the evaluation imaginaries were accommodated: a decoupling among intentions, approaches and outcomes allowed innovation to occur locally, while at the same time conforming to traditional values. Illustrative hereof in our case are the emergence of the ‘facts that matter’ approach or the inclination of users indicating the added value of the ‘Balance study’ in learning terms, while they principally refer to modernist evaluation outcomes. What does this example of co-production in evaluation praxis has to offer for policy evaluation practitioners? We showed how practitioners, and their peers, consciously or unconsciously, draw upon diverse societal views on what evaluation ‘is’ and ‘should be’. Such views may not be coherent, consistent, or even articulated. Awareness and articulation of societal expectations is indispensable considering the complex, multi-actor character of present-day governance processes to which policy evaluations have to accommodate and contribute.