Made in Africa evaluation (MAE) is often critiqued for not having implementation methods, approaches and tools. Much discussion has been focused on building Indigenous Knowledge Systems (IKS) to inform the implementation of MAE. What possibilities does IKS have for developing types of data collection and analysis methods? Let us explore this idea.

As part of my MA dissertation, I spoke to a number of M&E professionals about their perspectives on how to advance MAE, and I share some insights here.

MAE is the product of a breaking point of dislodging African IKS from western forms of research, where research has been done ‘on’, ‘for’ and ‘about’ Africans but not often ‘with’ or ‘by’ African communities (Goathlobowe et al., 2018). It is about empowering African IKS through knowledge production.

However, what emerged from my research findings was the frustration from respondents not knowing how to implement MAE and IKS components through locally conceptualized tools, methods and approaches. What is challenging is that MAE is still in its conceptualization stage and little has been researched on the actualization of the concept.

As a method of acquisition, African knowledge has a practical, collective, participatory, and social interpersonal characteristics. Characteristics that make them distinctive from Western ways of thinking (Owusu-Anash and Mji’s; 2013). The participatory approach to data collection allows fieldworkers to learn with, by, and from indigenous communities in creating a working relationship which allows people’s priorities and values to be more fully expressed in research.

In participatory methods, the indigenous communities (where the evaluation is conducted) should not be treated as ‘informants’ but rather as ‘significant participants’ and as ‘equals’ in both the process of research and in the decision-making processes about issues that affect their communities (Mpofu, Mushayikwa and Otuluja 2014).

Primary oral data collection is a distinctive, powerful took in African culture. Oral data collection is a primary form of socialisation of hearing and capturing oral primary data that illustrates behavioural trends through storytelling, provers, folktales, poetry, riddles, praises and song.

The ability for the evaluator to immerse themselves in the community’s context makes the data collection less about extracting data but rather finding ways to better reflect, learn and grow as an evaluator (Ibiyemi 2017).

The form of data collection methods can provide inputs to the way the data is analysed.

Evaluation approaches reflects MAE when there is participation, ownership, and community engagement in the evaluation process. As a case study, the research looked at Southern Hemisphere’s take on IKS data analysis techniques, and how their participatory and mainly qualitative approaches are relevant for doing evaluation in Africa.

Southern Hemisphere is an African evaluation consultancy whose evaluation practice has evolved in Africa over the last 22 years. The consultancy uses participatory and reflective tools and approaches for data analyses, informed by techniques such as Outcome Harvesting — a retrospective exercise of collecting evidence in the change of outcomes, to determine results of how the intervention contributed to that change in outcomes (Wilson-Grau 2015).

Another technique is Most-Significant Change — a qualitative exercise of learning from personal accounts collected from participants to table the similarities and differences in what different groups and individuals value (McDonald, et al. 2014). Although these approaches and tools have mostly been developed by Northern Actors based on their experience in development practices, their use is in line with the Made in Africa principles which embrace pluralism. As evaluators we may accept methods developed elsewhere as part of our evaluation toolbox, as long as the evaluation is empowering for Africans. But can we call them Indigenous Knowledge Data collection tools?

While the consultancy does engage stakeholders in key points in the evaluation process, it is seldom that they are able to include community members directly beyond fieldwork — in sense-making, data analysis or validation processes.

It is usually the programme implementers (often a local NGO, or government department) and the donor/grant-makers who are involved in these processes. This is largely because of how the Terms of Reference (ToR) are constructed. ToR’s often have limited budget and tight frames for community level engagement.

Southern Hemisphere engages the local partner NGOs in feedback sessions, often as the proxy for community members. While impressive, the consultancy questions whether this is enough.

There is clearly still a long path to navigate to get MAE into practice. But as African’s we can not do it alone. If the evaluation system that we are contracted into remains focused on accountability to donors, who make decisions about what is good for Africa?

Bibliography

Held, M., 2019. International Institute for Qualitative Methodology 2.767 Impact Factor 5-Year Impact Factor 5.817 Journal Indexing & Metrics » Contents Full Article Content List Abstract Introduction Positionality and Background Philosophical Orientations of Social In. SAGE Journals.

Ibiyemi, A., 2017. A Critical APproach to the study of oral narratives. [Online]
Available at: https://www.academia.edu/34249601/A_Critical_APproach_to_the_study_of_oral_narratives

McDonald, B., Stevens, K., Nabben, T. & Rogers, P., 2014. BetterEvaluation. [Online]
Available at: https://www.betterevaluation.org/en/plan/approach/most_significant_change

Mpofu, V., Mushayikwa, E. & Otuluja, F., 2014. Exploring Methodologies for Researching Indigenous Knowledge of Plant Healing for Integration into Classroom Science: Insights Related to the Data Collection Phase. African Journal of Research in Mathematics, Science and Technology Education, pp. 1–14.

Owusu-Ansah, F. & Mji, G., 2013. African indigenous knowledge and research. National Library of Medicine.

Wilson-Grau, R., 2015. BetterEvaluation. [Online]
Available at: https://www.betterevaluation.org/en/plan/approach/outcome_harvesting

About the Author

Matshediso (Tshidi) Moilwa was an intern at Southern Hemisphere in 2022 for 6 months as part of our partnership with the SAMEA/UNICEF internship programme for emerging evaluators.

Her MA dissertation (2022), is titled “Opportunities and challenges for Made in Africa Evaluation Capacity Development: South African experiences”, completed with the Wits School of Governance, Public and Development Management, as part of her Master’s Degree. She collected data from a series of semi-structured interviews with 15 M&E consultants, academics and commissioners of evaluation, working across universities, NGOs, CSOs, government, consultancies and VOPEs working in South Africa.