Shanghai International Program for
Development Evaluation Training Program
April 9-17, 2007 Shanghai, China
Day One: Monday, April 9th
8:00 – 8:50 Registration – U-Shape Room IV, 3rd Floor, Lecture Building III
8:50 – 9:20 Opening Remark
Mr. l, International Department, Ministry of Finance, P.R. China
Chair: Mr. Li Kouqing, Deputy Secretary General, Asia-Pacific Finance and Development Center
9:20– 9:40 Group Photo
9:40– 10:00 Tea/Coffee Break
10:00– 10:20 Program Overview and Course Objectives
Mr. Ray C. Rist, Senior Evaluation Officer, World Bank (retired) and Ms. Linda Morra-Imas, Chief Evaluation Officer, International Finance Corporation (IFC) provide an overview of the program, highlighting the key topics to be covered and outlining the learning objectives for the five-day course.
10:20– 10:50 nce, China
10:50– 11:10 Development and Challenges in Evaluation of IFIs' Operations in China
Mr. Zou Ciyong, Director, International Department, Ministry of Fina
11:10– 11:50 Progress of Result Management in developing Asian countries and ADB
Mr. Li Guanghui, Advisor, SPD, Asian Development Bank
11:50 – 1:30 Lunch
1:30 – 3:15 Module 1: Introduction to Development Evaluation
Ms. Linda Morra-Imas, Chief Evaluation Officer, IFC leads this introductory session on the changing development context, the different approaches to evaluation, and the OECD/DAC principles for evaluation.
3:15 – 3:30 Tea/Coffee Break
3:30 – 5:30 Module 2: Planning an Evaluation
Mr. Ray Rist, Senior Evaluation Officer, World Bank provides an overview of the evaluation planning process. Issues addressed during the session include how to work with others in designing and conducting a development evaluation, developing terms of reference, delineating roles and responsibilities, and meeting with clients and other stakeholders.
5:30 ‑ 6:30 Small Group Work – Set-Up and Reading
This interactive session is conducted by Ray Rist and Linda Morra-Imas. The delegates are requested to review and discuss an example of a mid-term review report. The delegates are divided into small group break out sessions. After this initial group assignment, each small group will use course materials and knowledge acquired during the seminar, to work on specific evaluation tasks and develop an impact evaluation for the program. The group work is designed for participants to apply the knowledge acquired during the course to real-life situations. The group work is also intended to facilitate information exchange and networking among participating delegates.
Day Two: Tuesday, April 10th
8:30 – 10:45 Module 3: Front-End Planning
Linda Morra-Imas reviews the relationship between program stages and the broad evaluation question, the policy context, finding existing information about the program, program theory and logic models, and ends with a discussion of the evaluation design process and the design matrix.
______________________________________________________________________________
10:45 – 11:00 Tea/Coffee Break
11:00 – 12:00 Small Group Work – Identifying Stakeholders and Developing Terms of reference
______________________________________________________________________________
12:00 – 2:00 Lunch
2:00 – 3:45 Module 4: Evaluation Questions and Design Matrix
Linda Morra-Imas covers formulation of evaluation questions and concept mapping. She offers guidance on how to formulate descriptive, normative, impact, and cause-and-effect questions. The design matrix is offered as a useful tool in designing the evaluation.
______________________________________________________________________________________
3:45 – 4:00 Tea/Coffee Break
______________________________________________________________________________
4:00 – 5:30 Small Group Work – Developing the Program Theory of Change
6:30 – 9:30 Small Group Work – Outward Bound Training
______________________________________________________________________________________
Day Three: Wednesday, April 11th
______________________________________________________________________________________
8:30 – 10:30 Module 5: Results-Based Monitoring and Evaluation: Overview
Ray Rist conducts the RBM session containing four sub-components. The RBM session is based on ten steps that describe how performance-based monitoring and evaluation systems are built. The first introductory RBM segment helps participants plan, design, and implement a performance-based monitoring and evaluation system. The ten RBM steps begin with conducting a readiness assessment, move on to goal setting and indicator selection, establishing baseline data, and setting targets to ensure the use and sustainability of M&E systems. Throughout the RBM sessions, participants discuss the tasks needed to complete each of the ten steps, and discuss tools available to help along the way.
______________________________________________________________________________
10:30 – 10:45 Tea/Coffee Break
10:45 ‑ 12:00 Module 5 (Cont.): Results-Based Monitoring and Evaluation: Conducting a Readiness Assessment, Agreeing on Performance Outcomes, and Selecting Key Indicators
As a continuation of the previous session, Ray Rist assists participants with steps one to three on developing a results-based monitoring and evaluation system. Participants acquire skills on conducting a readiness assessment for M&E in their various countries, how to agree on performance outcomes to monitor and evaluate MDG targets, PRSP and HIPC initiatives, and other programs they manage. They also learn how to select key indicators to monitor and evaluate desired outcomes in their respective countries.
12:00 – 2:00 Lunch
2:00 – 3:45 Module 5 (cont.): Results-Based Monitoring and Evaluation: Baseline Indicators, Setting Realistic Targets, and Building a Monitoring System
Building on topics covered in the morning sessions, Ray Rist provides instructions on steps four, five, and six on collecting baseline indicators, setting realistic targets, and building a monitoring and evaluation system. Participants are provided with guidelines on identifying expected or desired level of project, program, or policy performance, factors to consider when selecting indicator targets, and fundamental principles in building a monitoring and evaluation system.
3:45 – 4:00 Tea/Coffee Break
______________________________________________________________________________________
4:00 – 5:30 Module 5 (cont.): Results-Based Monitoring and Evaluation: Reporting and Using Findings, Role of Evaluations, and Sustaining M&E within an Organization
This conclusive RBM session delivered by Ray Rist covers RBM steps seven through ten and ends with illustrations on how to report evaluation data and findings, use of explanatory notes and reporting negative results. This session also addresses evaluation utilization, information sharing strategies, and discusses evaluation demand, structure, credibility, accountability, and evaluation capacity as critical components for sustaining M&E in a development organization.
________________________________________________________________________
Day Four: Thursday, April 12th
______________________________________________________________________________________
8:30 – 10:30 Module 6: Data Collection Methods: Overview
Linda Morra-Imas presents this session designed to provide an overview of data collection methods, key issues pertaining to data validity, reliability, and precision. Participants are also introduced to key issues on conducting in-person interviews, use of observation, focus group interviews, administered questionnaires, diaries, self-report checklists, and use of expert judgments.
10:30 - 10:45 Tea/Coffee Break
10:30 – 12:00 Module 7: Evaluation Design Options
Ray Rist builds on the earlier session on evaluation questions. The evaluation design options session provides guidelines on developing effective evaluation designs based on the type(s) of evaluation questions. Evaluation designs discussed during this session include experimental, non-experimental and quasi-experimental designs, correlation studies, use of control variables, natural variation, causal tracing strategies, and the case study approach.
12:00 – 2:00 Lunch
2:00 – 2:45 Module 8: Sampling
Linda Morra-Imas covers the different types of samples, the concepts of confidence and precision, and how to determine the size of the sample needed. He shows how sampling is important even in a highly qualitative study with a one week field visit.
______________________________________________________________________________
2:45 – 3:45 Module 9: Data Analysis and Interpretation
Linda Morra-Imas discusses formulating a data collection strategy and analyzing qualitative and quantitative data. She presents basic descriptive statistics and measures of relationship.
3:45 – 4:00 Tea / Coffee Break
4:00 – 5:30 Small Group Work – Putting the Matrix Together
7:00 – 8:30 Individual Study: Small Group Work – Putting the Matrix Together
Day Five: Friday , April 13th
8:30 – 10:15 Module 10: Presenting Results
Linda Morra-Imas assists participants with developing communication strategies for their evaluations, and on use of a feedback process to communicate results with stakeholders. Participants are also offered general guidelines on how to organize their evaluation reports and how to deliver presentations on completed evaluations.
10:15 – 10:30 Tea/Coffee Break
10:30 – 12:00 Final Examination
12:00 – 1:30 Lunch
______________________________________________________________________________
1:30 – 2:30 Global Agenda of Managing for Development Results
Bruce Purdue, Co-Chair, OECD-DAC Joint Venture on MfDR; Chair, MDB Working Group on MfDR; Head, Results Management Unit, Asian Development Bank
2:30 – 2:40 Tea/Coffee Break
2:40 – 4:10 Presentations of Group Design Matrix
4:10 – 4:20 Tea/Coffee Break
4:20 – 5:50 Presentations of Group Design Matrix (Cont.)
7:30 – 8:30 Building Evaluation Capacity—Evaluation of Asian Development Bank Experience
Mr. Keith Leonard, Director of OED1, Asian Development Bank
Ms. Xue Yan, ADB TA expert
______________________________________________________________________________
Day Six Saturday, April 14th
8:30-10:00 M&E: National level case analysis
National Auditing Office, Mr. Liu Wei
10:00-10:15 Tea/Coffee Break
10:15-12:00 Country-led Joint Evaluation—Dutch ORET/MILIEV Programme in China
Chen Zhaoying, Executive Director, National Center for Science and Technology Evaluation (NCSTE)
12:00-1:30 Lunch
1:30-2:50 M&E: Sector Level Case Analysis
Mr. Keith Leonard, Director of OED1, Asian Development Bank
Ms. Xue Yan, ADB TA expert
2:50-3:00 Tea/Coffee Break
3:00-4:20 Project Performance Management System
Mr. Keith Leonard, Director of OED1, Asian Development Bank
Ms. Xue Yan, ADB TA expert
Day Seven (Sunday, April 15th) and Day Eight (Monday, April 16th)
Field Study
Two days of field trips will be scheduled to an urban development program of World Bank in Zhejiang province that has undergone evaluations. The evaluators will explain their design and strategies of data collection and analysis. They will also discuss the findings and implications of their evaluation findings.
Day Nine Tuesday
8:30-10:00 Video Conference: Discussion of Zhejiang Multicities Development Project
Roy Gilbert, World Bank
10:00-10:15 Tea/Coffee Break
10:15-11:00 Closing Ceremony
12:00-2:00 Farewell Lunch |
