Evaluation and Research:
Coffey, a Tetra Tech Company, provides high quality and
independent evaluation, monitoring and
impact assessment services, covering
the full spectrum of research approaches,
methodologies and tools.
Coffey's team is one of Europe’s principal providers of evaluation and research services. We provide high quality, independent evaluations and impact assessments of public policies and programmes, and have developed a strong focus on the evaluation of information and communication activities. We are an industry leader known for rigorous and thorough evaluation that combines a deep understanding of social research methods with analytical excellence.
Through more than 20 years of experience in the European context, Coffey has a deep understanding of the multi-faceted EU policy environment. We have undertaken more than 250 engagements for the European Commission and have worked across 26 Directorates-General.
Our growing team’s diverse academic and professional backgrounds, as well as our knowledge of the majority of EU languages, enables us to work in a wide range of sectors. Our work covers multiple policy areas, including information and communication, foreign policy, customs, education and culture, fisheries, health, consumers, research and innovation, employment and social affairs.
We have expertise in both qualitative and quantitative approaches and methods including: benchmarking, case studies contribution analysis, theory-based approaches, cost-effectiveness and cost-benefit analysis, logic models, rapid evidence assessments, theory of change, and value for money analysis.
We have extensive experience in the following areas:
- Impact assessment and ex ante evaluation
- Interim and ex post evaluation
- Feasibility studies
- Rapid assessments
- Monitoring assignments and indicator development
Policy and Programme Evaluation
Policy and Programme Evaluation
Through our vast experience in evaluation and impact assessment we have gained a deep understanding of our clients’ areas of policy focus. This places us in a unique position to supply high-quality service. We take pride in designing methodologically robust and technically appropriate research that delivers what our clients need. In particular, we understand that our studies need to provide a sufficient level of technical rigour and focus to be useful to policy practitioners.
Mid-term evaluation of the Partnership Instrument
Created in 2014, the Partnership Instrument responds to the EU’s need to engage internationally across a broad spectrum of policy areas. It reflects the renewed emphasis on the foreign policy capability of the EU, supporting its bilateral, regional and multilateral agenda. Coffey is undertaking the mid-term evaluation of the Partnership Instrument, which will contribute to the overall evaluation of all External Financing Instruments of the EU.
The draft final report was published by the European Commission in the framework of an open public consultation on the topic. The evaluation is now finalised and feeds into the Commission Staff Working Document in preparation and the upcoming negotiations between the Commission and the European Parliament on the future of the EU’s toolbox.
Ex post evaluation on Union financial measures for the implementation of the Common Fisheries Policy and in the area of the Law of the Sea 2007-2013
Financial measures in the areas of control and enforcement, basic and additional data collection, scientific advice, and governance and international relations supported the implementation of the Common Fisheries Policy and the Law of the Sea between 2007 and 2013. Building on a theory-based approach, Coffey contributed to the ex post evaluation of these instruments using a broad range of quantitative and qualitative research tools. The results of the evaluation fed into the DG’s policy cycle and helped it to adapt the financial instruments to the needs observed and objectives stated.
Since 2012, under a framework contract with DG MARE, Coffey has been carrying out retrospective and prospective evaluations on the Common Fisheries Policy - evaluations of financial and regulatory frameworks or particular policy instruments, covering their technical, environmental, economic and social dimensions.
Final Evaluation of the Customs 2013 Programme
In 2014, Coffey carried out the final evaluation of the Customs 2013 programme, an EU expenditure programme that supported national customs administrations through collaboration fora and common IT systems. The evaluation covered the entirety of the programme, which ran from 2008 to 2013. To measure the programme’s impact and to assess it more broadly, the evaluation employed a two-part approach: a comprehensive assessment of the entire programme and a contribution analysis.
(Note: Coffey was trading as The Evaluation Partnership at the time of this report.)
Evaluation of the Implementation of the Strategy for Europe on Nutrition, Overweight and Obesity-related health issues
Overweight and obesity are epidemic - the latest figures show that over half of Europeans are overweight or obese. In response to this problem, in 2007 the European Union created the Strategy for Europe on Nutrition, Overweight and Obesity Related Health Issues. In 2012 the EC commissioned Coffey to carry out an independent external evaluation of the implementation of the strategy.
The aim of the evaluation was to assess both the implementation process (that is, if, how and how far relevant initiatives have been developed by the European Commission and other relevant actors) and its impact (to what extent the relevant initiatives and the strategy as a whole have produced results in line with their respective objectives).
(Note: Coffey was trading as The Evaluation Partnership.)
Evaluation of the European Commission’s Traineeship Scheme
Client: DG Education and Culture (EAC)
The European Commission’s Traineeship Scheme (commonly referred to as the Commission ‘stage’) has known a growing success since its launch in 1960. Each year, about 1,400 university graduates from all over the world spend five months in the services of the European Commission and of various EU agencies and bodies.
The specific objective of the evaluation was to provide an external, independent interim assessment of the Traineeship Programme covering the period 2007-2013. The evaluation followed an approach comprised of both traditional theory-based and counterfactual analysis. This served to update previous evaluation exercises and add to the existing body of research by providing hard data on the impact of the Traineeship Programme. By triangulating the evidence collected for the theory-based and impact parts of the evaluation, we were able to ascertain not only what difference the Traineeship Programme has made to former trainees in concrete terms, but also which parts of the programme have worked best, and why.
Evaluation of Communication Activities
Evaluation of Communication Activities
Communicating about the European Union to its citizens and other stakeholders poses significant challenges. This includes frequent misinformation about the EU from various sources as well as disinterested, apathetic or even hostile audiences. Despite, or perhaps due to these difficulties, the need to communicate about the EU remains as important as ever.
We are one of Europe’s leading providers of ex-ante, interim and ex-post evaluations of communication activities, tools and campaigns.
Communications Review of the EEA and Norway Grants 2009 - 2014
Coffey has been requested to assess the progress of implementation of communications strategies and plans across all 16 beneficiary countries of the EEA and Norway Grants. The study was carried out from October 2015 to July 2016.
Communicating about the existence, partnership opportunities and achievements is key to transparency, accountability and a successful implementation of the grants provided by Iceland, Liechtenstein and Norway. Conveying the objectives and ensuring visibility for the donors is an important responsibility for National Focal Points, Programme Operators and Project Promoters.
Coffey identified communications results and best practices. We also provided valuable recommendations for improved communications efforts and visibility of the donors and how the legal requirements on communications could be improved to ensure effective communications strategies and plans.
Evaluation of DG Trade’s Civil Society Dialogue
Client: DG Trade (TRADE)
Coffey was chosen to carry out an evaluation of the DG Trade’s Civil Society Dialogue meetings in Brussels. During the evaluation, we assessed the value and effectiveness of the Directorate’s Civil Society Dialogue process. The process is designed to provide a structured approach for the European Commission to discuss trade issues with civil society organisations and groups relevant to the policy discussions. These groups include trade associations, trade unions, labour groups and social and development NGOs.
The evaluation methodology included an assessment of Civil Society Dialogue meetings, online surveys of civil society organisations and DG Trade managers, in-depth qualitative interviews, reviews of similar processes elsewhere in the European Commission, and benchmarking of external organisations (the World Trade Organisation, United States Trade Representative, the Danish Government’s MindLab and the UK Cabinet Office).
The results of the evaluation helped the Directorate-General to renew its approach to discussion and debate with civil society and to ensure that consultation processes, such as the ones used during the Transatlantic Trade and Investment Partnership negotiations, continue to be useful and relevant.
Evaluation of information and communication activities towards the EU Member States in the area of EU Enlargement
The European Commission is committed to providing factual information on past and future enlargements and to cooperating with strategic partners and multipliers to communicate with the public on enlargement.
The 2015 evaluation, led by Coffey, was intended to provide findings and recommendations to help DG NEAR to improve the planning and implementation of future information and communication activities. The evaluation methodology focussed on analysis of existing evidence of communication performance against objectives set, as described in terms of references and communication contractors’ proposals and final reports. It also gathered primary research data through interviews with partners, observation and participation in press trips in Albania and Kosovo, five visibility events in Brussels, Edinburgh, London, Odense and Stuttgart, and 8 focus groups with members of public in four Member States (France, the Netherlands, Finland, and Austria).
Evaluation of the European Commission corporate communication campaign
Client: DG Communication (COMM)
In 2015, Coffey led the evaluation of the European Commission’s pilot corporate communication campaign. The pilot was intended to test a new way of communicating - for the first time the European Commission attempted to show how the EU makes a difference to people’s lives. The campaign used TV, online and print ads, which highlighted a range of projects and people benefiting from EU support and had the message “The EU working for you.” The evaluation ran alongside the campaign and fed back into the campaign’s on-going management. By running a pilot, the European Commission took the important step of first testing a new concept and then using this evaluation and other research to better understand if and how the new approach could be taken forward.
Monitoring, reporting and evaluation are critical features of decision-making processes. Through its Better Regulation agenda, the European Commission has committed to design, deliver and support the implementation of interventions of the highest possible quality. Compiling a robust information or evidence base is an essential component of better policy making. Lesson learning is instrumental to improving EU action, and it serves transparency and accountability objectives too. Coffey combines a unique set of skills to provide support to its clients in establishing comprehensive and reliable monitoring & evaluation systems.
Technical Assistance for the Partnership Instrument Monitoring and Reporting System (2017-2020)
While it conducted the mid-term evaluation of the Partnership Instrument (PI), Coffey identified a number of challenges with regard to monitoring and reporting. The challenges ranged from the introduction of a monitoring system ex post to the lack of a central repository of action documents and the high risk of inconsistent interpretation of the indicators applied.
As a follow-on, Coffey has been contracted to provide the FPI with assistance in the setting up, implementation and further improvement of an effective monitoring and reporting system. This builds on the team’s unique set of skills:
- In-depth understanding of the PI and its new monitoring and reporting system
- Experience in carrying out evaluations in the fields of external relations and development cooperation, evaluating change and contribution
- Experience in carrying out capacity-building projects
- Extensive experience in monitoring the quality of evidence being gathered, and
- Expertise in evaluation & monitoring methods and techniques
This technical assistance project notably involves the drafting of a set of user-friendly monitoring and reporting tools and templates, the provision of technical support through an online helpdesk, as well as the delivery of a series of comprehensive trainings to the range of the PI stakeholders (e.g. project managers, implementing partners) in Brussels and in EU Delegations to third countries (Brazil, Canada, China, India, Indonesia, Japan, Korea, Mexico, Russia, Thailand, USA).
Coffey designs, manages and delivers complex multi-country research projects in a variety of sectors.
We are continuously working to improve our sector and methodological expertise in order to apply sound approaches in complex environments. This on-going learning allows us to stay at the cutting edge of research and to adapt academic research to the needs of our clients. It also enables us to present frequently at conferences.
Research Executive Agency client satisfaction study and recommendations for stakeholder communication
Study on the Fiscalis 2020 and Customs 2020 Performance Measurement Framework for Directorate-General for Taxation and Customs Union
(Note: Coffey was trading as The Evaluation Partnership at the time of this report.)
“Moving away from death penalty in South East Asia” – Baseline Study for the Foreign Policy Instrument Service
Meet a few members of our team…
Melanie Kitchener – Principal Consultant
Melanie leads Coffey’s work on communication evaluations and has managed or contributed to about 50 evaluation projects. Melanie’s expertise lies in the design and implementation of research methods to evaluate the effectiveness and efficiency of communication activities intended to raise awareness, inform, generate debate or support behaviour change. Melanie has an MBA from Henley Business School and post-graduate and UK Chartered Institute qualifications in marketing. She is a native English speaker and knows Dutch, French and German. She enjoys swimming, reading and summer, and has two primary school age children.
Andrea Kobilsky – Senior Consultant
Andrea has lead or contributed substantially to over 30 evaluation assignments for the European Commission. She is an expert in development and application of evaluation methodologies to assess the impact of programmes and activities. She has also delivered training sessions and presented on conferences on various evaluation-related topics. Andrea holds a masters’ degree in public opinion analysis and has published several scholarly articles on media and political processes in Latin America. Andrea is bilingual in English and Spanish. She plays field hockey and loves classic screwball comedies.
Macarena Davies – Senior Consultant
Macarena has led or contributed to more than 20 evaluation assignments for the European Commission, UK government and international development organisations. She has extensive experience designing and applying evaluation methodologies for the ex-ante, interim and ex-post assessment of public policies and programmes. Macarena holds a BA degree in political science and government from Torcuato Di Tella University and is currently doing a MA in public policy and public administration from the University of San Andres. Macarena is bilingual in English and Spanish. She is an amateur photographer and loves writing her blog.
Andrew Hetherington - Senior Consultant
Andrew is a skilled evaluator, who specialises in designing and executing research methodologies. He has experience in managing complex, multi-country evaluations and studies, coordinating fieldwork and directing subcontractors, both in the EU and developing countries. Andrew is a native English speaker, is fluent in French and has a proficient level of German. He is an accomplished long-distance runner.
Dr Karolina Wrona - Senior Consultant
Having worked on close to a dozen projects for the European Commission, Karolina combines her experience in evaluating communication strategies and actions with in-depth knowledge of European regional development issues. Prior to joining Coffey, Karolina worked in academia, and before that in the West Midlands Councils (UK) and for the Polish Ministry of Regional Development. She speaks Polish, English and intermediate German. She enjoys coming up with new, sometimes admittedly quite odd, baking recipes.
Emma Godfrey – Senior Consultant
Emma is an experienced evaluator and has also worked as a policy researcher and editor. In the course of her career she has developed skills ranging from primary data collection and validation to quantitative and qualitative research methods. A capable project manager, she has wide-ranging experience working in multiple policy areas, including health, communication, taxation and customs, the labour market and social protection systems. Emma has a double Masters in Media and Journalism and a BSc in Philosophy, Politics and Economics. She speaks English and French. She loves maple syrup, tents in muddy fields and being on her bike.
Dr Anne-Claire Marangoni - Senior Consultant
Anne-Claire joined Coffey in early 2015, bringing expertise in institutional coordination within the European Commission and in the EU’s external policies. Before joining Coffey, she worked in academia as a lecturer, trainer and consultant. At the College of Europe, she trained high level officials for the rotating presidency of the EU (with a focus on foreign policy). In addition to her native French, Anne-Claire is fluent in English. She enjoys fireworks and the great British tradition of afternoon teas.
Robert Uden – Consultant
Robert is an experienced social researcher and evaluator. Since joining the Evaluation and Research Practice in 2014, he has been involved in complex programme evaluations for a range of international clients including the European Commission and the UK Department for International Development. With a background in economics, media studies and environmental sciences he brings a range of data collection and analysis skills to his work. He strongly believes in communicating clearly and making findings accessible. Robert wants to live in a world where technical reports are written in plain English and pie charts are forbidden. Recognising the limits in this vision, Robert keeps busy trying to change his corner of the world one Excel graph at the time. An outdoor enthusiast, he loves working with his hands and knows how to build a log cabin. He also finds writing about himself in third person similarly delightful.
Irina Jefferies – Consultant
Irina is a skilled researcher and evaluator with broad experience conducting desk-based research, stakeholder interviews, surveys and focus groups. She is also experienced in managing sub-contractors and delivering presentations to various audiences. Before joining Coffey, Irina worked in the legal services sector and gained significant experience in EU law. She is bilingual in Bulgarian and French and speaks fluent English. Irina is a skilled cook and enjoys skiing.
David Hoffmann – Junior Consultant
David joined the Evaluation & Research practice in late 2016 and has participated in impact assessments and program evaluations for five different Directorate Generals. Prior to joining Coffey, David led research projects and conducted programme evaluations in the international trade and development field. Most recently, his work cover project monitoring, organisational development and communications for two social enterprises in London and Manila. David holds an MSc in Urban Economic Development from University College London, and a BA in Philosophy, Politics and Economics from the University of York. He is fluent in German, Spanish, English and French. As a street photographer and music enthusiast, he enjoys exploring new cities and discovering their live music venues.
ul. Krucza 16/22
+48 (0) 22 578 2280
40 Bernard Street
WC1N 1LE, London
+44 (0) 20 7837 2881