This is the summary of conversation between Chris and Ricardo as part of the pre-workshop interaction. The responses depict views on issues related to evaluation of ICTs.
What constitutes credible evaluation and acceptable evidence in your context?
Some background; the project has had lots of exposure to Michael Quinn Patton’s work, lots of influence based on his presentation at the African Evaluation Association Conference in Niamey, Niger conference. This shaped our thinking as a research institute, we are keen on UFE to make sure research results are used.
A priority is creating a dialogue between research and policy. We “…are not there yet” but we are learning. For instance what format to present the research to make it of interest to regulators and policy makers.
Evidence of what? OM has influenced us, so we don’t talk about impact, but about outcomes. Behavioural changes are the focus, understanding that development is long-term, and that contributions are more relevant than seeking to find short term impact.
What are the strengths and weaknesses of the way you currently do M&E?
Weakness: a lack of capacity within the organization. We are a strange breed in that we do applied research, and evaluation is a new skill area. This is a “retreat” for me.
We are still very young in relating with policy makers.
We are very poor at documenting what they do and reflecting on it. As a learning organization this is quite worrying, hence the need to first understand M&E better.
How familiar you are with the following evaluation approaches?
GEM (Gender Evaluation Methodology)
Not familiar at all.
What experience have you had in gender evaluation and gender training?
OM (Outcome Mapping)
I am familiar with it, I use it or parts of it. I have attended a workshop.
If you are interested in using parts of OM, do you want to use it for ongoing monitoring during the project or for summative evaluation?
Both. We produced a SchoolNet Namibia summative report, as an example.
How would you differentiate between outputs and outcomes?
Outputs are products and services, and outcomes are the benefits.
MSC (Most Significant Change)
Have heard about it.
What other approaches do you use?
Have been exposed to results based management and liked it. Want to combine with OM in a telehealth project that is just starting. Have felt that OM helped in understanding the development process, but he still finds that for dissemination an reporting of research results, it is hard to only focus on behaviour change. I have engineering background and still like the hard stuff. UFE helps, yet some users will also like the hard stuff (eg number of referral cases, cost benefit).
How much attention do you want us to give to LogFrames?
Provide brief review. It is always important to understand, btu a brief review will be enough.
How much attention do you want us to give to ICT indicators?
Provide reference materials. I use them a lot for comparative research using WB numbers.
Preference for the reference materials to take home:
Electronic copies only.
Provide your three main expectations for this workshop.
- To network with other members from around the world
- To learn what others have done
- To get feeding back what we have done
Would you be willing and able to prepare a poster to visualize and summarize your project?
Yes. This is important to get to know who the others are.