This is the summary of conversation between Zarah and Ricardo as part of the pre-workshop interaction. Read her responses and views on issues related to evaluation of ICTs.
What constitutes credible evaluation and acceptable evidence in your context?
Evaluation should be scientific and unbiased. Examples of evidence in our case will be: community owned ICT enterprises, ICT tool most used in the community, people (male/female) using the ICTs (for what purposes, what info and what do they do with it), number of people with ICT enterprises, and policy for universal access in Cambodia.
How do you currently do M&E? What methods, approaches, etc. are you required / do you choose to use?
The project started with a delay, so we have not done an evaluation yet. For now we are monitoring using reports from our pilot coordinators. We are planning to combine OM with SPEAK (more details below). We also conduct project reviews every three months or so and we used SWOT analysis to evaluate our progress. The review is mainly discussion on the progress based on the activity plan, the challenges or problems encountered and how were these resolved.
What are the strengths and weaknesses of the way you currently do M&E?
The language barrier: the evaluation terms are difficult. We are just starting.
How familiar you are with the following evaluation approaches?
GEM (Gender Evaluation Methodology)
I have attended a workshop on GEM.
What is your level of understanding and experience conducting gender analysis?
Developed a monitoring plan. Challenge is how to integrate with OM+Speak.
OM (Outcome Mapping)
I have attended workshops and I use parts of it in my work.
If you are interested in using parts of OM, do you want to use it for ongoing monitoring during the project or for summative evaluation?
Had an OM workshop with Fred Carden. Used OM to define vision, mission, boundary partners, progress markers , etc. Will combine OM with SPEAK Self Evaluation System (“Strategic Planning, Evaluation and Knowledge). SPEAK is a computer software developed by Seán Ó Siochrú of Nexus Research, Ireland. Will experiment combining OM with computer assisted SPEAK – less paper work by doing it with software.
How would you differentiate between outputs and outcomes?
Outputs are observable achievable products that can be directly linked to a project. Outcomes are also results, but they may not be directly caused by the project.
MSC (Most Significant Change)
Have heard of it.
What successes and/or issues have you had in using qualitative approaches to monitoring and evaluation?
Sounds interesting from the qualitative side.
What other approaches do you use?
See answers above.
How much attention do you want us to give to LogFrames?
Provide reference materials. I already have some knowledge on it.
How much attention do you want us to give to ICT indicators?
Provide brief review and reference materials. Would be more interesting especially as we are an ICT project.
Preference for the reference materials to take home:
Both hard copies and electronic. Hard copies mailed later will be welcome.
Provide your three main expectations for this workshop.
- To learn from other projects using OM and OM combined with other methods; what are the challenges and how they resolved these.
- To learn more trick and approaches, practical tips that can use, to teach colleagues what I learned.
- Gather relevant reference materials.
Would you be willing and able to prepare a poster to visualize and summarize your project?
Powerpoint preferred. Will consider the poster after seeing the guidelines.