An Introduction to Evaluation
Chris Fox
Robert Grimm
Rute Caldeira
SAGE Publications Ltd
1 Oliver’s Yard
55 City Road
London EC1Y 1SP
SAGE Publications Inc.
2455 Teller Road
Thousand Oaks, California 91320
SAGE Publications India Pvt Ltd
B 1/I 1 Mohan Cooperative Industrial Area
Mathura Road
New Delhi 110 044
SAGE Publications Asia-Pacific Pte Ltd
3 Church Street
#10-04 Samsung Hub
Singapore 049483
© Chris Fox, Robert Grimm and Rute Caldeira 2017
First published 2017
Apart from any fair dealing for the purposes of research or private study, or criticism or review, as permitted under the Copyright, Designs and Patents Act, 1988, this publication may be reproduced, stored or transmitted in any form, or by any means, only with the prior permission in writing of the publishers, or in the case of reprographic reproduction, in accordance with the terms of licences issued by the Copyright Licensing Agency. Enquiries concerning reproduction outside those terms should be sent to the publishers.
Library of Congress Control Number: 2016935864
British Library Cataloguing in Publication data
A catalogue record for this book is available from the British Library
ISBN 978-1-4739-0286-2
ISBN 978-1-4739-0287-9 (pbk)
Editor: Jai Seaman
Assistant editor: Alysha Owen
Production editor: Ian Antcliff
Copyeditor: Audrey Scriven
Marketing manager: Sally Ransom
Cover design: Shaun Mercier
Typeset by: C&M Digitals (P) Ltd, Chennai, India
Printed and bound by CPI Group (UK) Ltd, Croydon, CR0 4YY
About the Authors
Chris Foxis Professor of Evaluation and Policy Analysis at Manchester Metropolitan University, where he is Director of the Policy Evaluation and Research Unit (www.mmuperu.co.uk). PERU is a multi-disciplinary team of evaluators, researchers, analysts, and economists and undertakes evaluation and applied research in the UK and Europe. Chris’s work cuts across a number of policy areas. He has led numerous evaluations in the criminal justice system and currently is leading large, long-term research and evaluation programmes on reducing re-offending, and on prisoner education. Chris is also interested in social policy innovation and is Director of a European Commission-funded project ‘Innovative Social Investment: Strengthening communities in Europe’ that is studying innovative approaches to welfare reform in ten European countries.Robert Grimmis Associate Director at Ipsos and leads the Ipsos Public Affairs, Political and Social Research Team in Germany. In this capacity, Robert is responsible for the design and management of major social research projects for academic institutions, NGOs and Private Sector Clients. His work includes developing research instruments, sampling strategies, designing indicators and measurements, setting up baselines measures and continuous performance tracking. Robert is also leading on the development of Ipsos’ political polling in Germany. Before joining Ipsos, Robert worked as an academic at Manchester Metropolitan University. Robert’s research interests included social innovation and social investment and he has been involved in many European Commission funded research projects. Robert taught evaluation and social research methods to postgraduate students and continues to be a fellow at Manchester Metropolitan University.Rute Caldeirais a social scientist with a doctorate in sociology. She has 15 years of work experience, mostly in the NGO sector; the last eight years were dedicated to setting up planning, performance and monitoring systems both in complex not-for-profit and in community-based organisations with a view to enhance their ability to enable social impact. Rute currently heads Transparency International Monitoring, Evaluation and Learning Unit, which she set up in 2011. She brings in extensive knowledge of methods and methodological approaches, of evaluations and assessments, and of organisational capacity to absorb learning from impact studies and evaluations.
Introduction
Who should read this book?
This book has been written with two main audiences in mind.
The first group is students, in particular postgraduates doing taught courses in research methods and both undergraduates and postgraduates planning research projects, perhaps for a final year dissertation or a PhD.
The second group is people working in governmental or non-governmental organisations who are planning either to undertake an evaluation or commission an evaluation.
Why evaluation is important
Evaluation is a fascinating interdisciplinary research practice with a wide spectrum of applications. Evaluators continuously encounter new challenges; they work in changing environments, with a variety of people and organisations where they have to think themselves into new social settings. No single evaluation is like another and evaluators need to pragmatically develop innovative and robust research methods.
At a time of increasing calls for transparency and accountability, evaluation has moved centre stage in public policy, organisational planning and management. Government departments are under pressure to deliver social policy programmes that are effective and value for money. Chelimsky (2006) suggests that evaluation is an intrinsic part of democratic government for four reasons:
It reports information about government performance that the public needs to know. It adds new data to the existing stock of knowledge required for government action. It develops an analytical capability within agencies that moves them away from territoriality and toward a culture of learning. And, more generally, its spirit of scepticism and willingness to embrace dissent help to keep government honest. (Chelimsky 2006: 33)
Similarly, non-governmental organisations from Amnesty International to the Red Cross are accountable towards their funders and have to prove that they use donations in an appropriate way. Private sector organisations too are increasingly evaluating internal processes and operations. The evaluator has an important role in assessing and judging programmes, polices and interventions, and evaluation reports often determine the future of interventions.
Evaluation combines managing the expectations of different stakeholders – those who are affected by an intervention, those who help to deliver the intervention and those who have been funding it – with understanding the social complexities and remaining objective and neutral. While handling various stakeholders is challenging, understanding diverse points of views is also insightful and rewarding. Evaluation research