This is the summary of conversation between Judith and Ricardo as part of the pre-workshop interaction. Read her responses and views on issues related to evaluation of ICTs.
What constitutes credible evaluation and acceptable evidence in your context?
Credible evaluation is based on empirical data. Acceptable evidence needs to be practical, robust, credible (scientific method).
How do you currently do M&E? What methods, approaches, etc. are you required / do you choose to use?
Mainly conventional methods: survey-based, with statistical and econometric analyses, complemented with interviews and secondary data to compare with other research.
We are about to complete a 7-country survey on cell phone use. I suspect that the findings will likely tell us more about the perceptions by users and less about the actual impacts or results. For that reason we are considering adding some focus group work to gain a deeper understanding, though the challenge is to generalize the findings. MSC may constitute a useful approach.
What are the strengths and weaknesses of the way you currently do M&E?
Every approach has its limitations, each one can only approximate an understanding of reality (philosophy of science). On a practical level regarding surveys, a challenge is to come up with the right questions, and a limitation is that they are costly.
On indicators/databases, a limitation is the lack of comparable numbers (for instance the OECD database includes Mexico but no other Latin American countries); while the ITU are official statistics that are often inaccurate or manipulated.
How familiar you are with the following evaluation approaches?
GEM (Gender Evaluation Methodology)
What experience have you had in gender evaluation and gender training?
OM (Outcome Mapping)
How would you differentiate between outputs and outcomes?
When you evaluate, you are interested in outcomes; that is the result/impact of certain variables. Outcome is more related to a process than an output?
MSC (Most Significant Change)
What other approaches do you use?
None in particular.
How much attention do you want us to give to LogFrames?
Provide brief review – LF is our starting point.
How much attention do you want us to give to ICT indicators?
Provide reference materials.
Preference for the reference materials to take home:
Electronic copies only.
Provide your three main expectations for this workshop.
- To Become familiar with new methods
- Share ICT indicators and sources
- Learn about others’ challenges and how they are dealing with them