An introduction to participatory monitoring and evaluation: the missing link between inquiry and impact
In a previous article, I talked about why it is important to consider inclusion when designing and implementing an M&E system and highlighted some of the key characteristics of inclusive M&E. I made the link between inclusion and participation, both from an instrumental point of view that focuses on the effectiveness of our interventions and from a point of view based on human rights principles.
Just as ‘inclusion’ has become a buzzword in humanitarian and development circles, so too has ‘participation’. Donors, governments, international organisations, and NGOs alike champion participation, promoting it as a guiding principle and good practice. However, it is not always clear what it means when an organisation says that they ‘value participation’ or take a ‘participatory approach’ to their work. At the same time, there are many misconceptions about what participation means and looks like in the context of M&E in particular.
Today, we will look at participatory M&E more closely. We will see how it challenges us to think differently about power, knowledge, and the role of practitioners and affected populations in M&E. We will also consider when we might use participatory approaches in the M&E cycle. I will make the case for participatory M&E in terms of its benefits and its relationship to human rights, and look at some of the challenges we might face in using participatory approaches. In the next article, we will look at what kinds of methods and tools we have at our disposal to conduct participatory M&E, including in virtual settings.
View the French version of this article
View the Spanish version of this article.
If you like this article don't forget to register to the ActivityInfo newsletter to receive new guides, articles and webinars on various M&E topics!
First thing’s first: we need to talk about power
At the heart of all participatory approaches to humanitarian and development programming is the recognition that power relations tend to be unequal, and the related goal of changing the balance of power in favour of those who don’t normally have it. This imbalance is most evident between those funding, planning, implementing, monitoring or evaluating a project (‘us’, the professionals, specialists or practitioners) and those who are said to benefit from it (‘them’, the affected people or communities who are often referred to as the beneficiaries, recipients, targeted population or end-users).
At its most extreme, this power imbalance translates into a top-down approach based on a line of thinking such as this: ‘we’ have the necessary skills, knowledge and worldview to judge what is best. ‘They’ are vulnerable and lack the resources, skills or knowledge to help themselves, and are therefore in need of aid or development support as defined and delivered by ‘us’. The kind of thinking used to justify such a top-down approach is often not quite so obvious and it can influence our actions without us even realising it.
M&E is also subject to unequal power relations, but they can be even more difficult to spot. This is because we tend to think of research, monitoring and evaluation—which are all forms of inquiry, which is the process of asking questions and seeking answers—as completely objective activities. We think of knowledge, data, and evidence as being neutral and unaffected by social relations. Traditional approaches to social science research, whose methods we rely on for M&E, usually involve an outside expert studying people according to a pre-defined research agenda with pre-set methods and tools. The researcher—without the involvement of the people who are being researched—formulates research questions, designs studies, extracts data from research subjects and selects information for analysis and further use.
What is participatory M&E?
To understand what ‘participation’ might mean and look like in the context of M&E, we first need to understand how participatory approaches to social science research are different from traditional approaches. In short, participatory approaches challenge conventional beliefs about knowledge, capacity and the purpose of inquiry. They seek to disrupt traditional power relations between researchers and research subjects. The process of inquiry and the production of knowledge become collaborative activities, instead of being technocratic, one-sided and extractive.
A participatory approach to research is not a specific methodology or a set of particular methods, but a set of principles and practices for undertaking research. The term can describe a variety of frameworks and approaches used in many fields, from health, education and anthropology to humanitarian assistance and development. These approaches share some basic assumptions:
- Everyone has biases, whether conscious or unconscious, and professional researchers and practitioners are no exception.
- You don’t need to be professionally or academically trained to participate in research or to make a valuable contribution to knowledge. The process of inquiry is therefore an opportunity for mutual learning for both practitioners and other stakeholders.
- People who are closest to an issue or intervention tend to know the most about it and how it affects them. They can therefore help generate more relevant and correct insights than an outsider could on their own.
All participatory approaches engage stakeholders who are not usually involved in the process of inquiry and knowledge production. In our work as M&E practitioners, the most excluded stakeholders are usually individuals and communities who are the intended beneficiaries of an intervention. It is the participation of these stakeholders that I will focus on in this article (using ‘communities’ as shorthand).
The extent to which communities participate in a given approach will vary according to the ultimate aims of our M&E process. These can range from being functional to being transformative. We can think of M&E processes as functional when they seek to produce new knowledge in a particular field or about the performance of a particular intervention according to questions and goals set by practitioners. We can think of them being transformative when we use M&E and the knowledge we produce for the purpose of action or change as defined and driven by the community. Accordingly, we can think of levels of participation as being on a continuum, becoming more or less effective and meaningful depending on the aims of the M&E process and the distribution of power within it.
On one end of the scale, the practitioner informs or consults with the community. This is participation in name only. Organisations sometimes use it to legitimise a particular finding or decision, even if there is no co-leadership over the M&E process or joint ownership over results. On the other end of the scale, we see a meaningful collaboration between practitioner and community that includes shared decision-making and leadership, or the empowerment of communities to lead the M&E process themselves and act on its results. The role of the practitioner here is to facilitate rather than dictate.
Examples of more transformative participatory approaches and frameworks, which are used in development contexts especially, include:
- Participatory action research (PAR)
- Community-based participatory research (CBPR)
- Participatory learning and action (PLA)
- Participatory rural appraisals (PRA)
- Transformative participatory evaluation
When could we use participatory approaches in our M&E processes?
For participation to be effective and meaningful, it should be integrated into every phase of our M&E cycle, when we:
- Plan and design the M&E system or activity
- Collect and analyse data
- Act on M&E findings
- Reflect on the effectiveness of the M&E system in order to improve the next cycle
The community should participate in every phase, and we should systematically act on the outcomes of participation during each phase in subsequent phases of the cycle. We should also focus on building and maintaining culturally responsive relationships between practitioners and participants throughout the M&E cycle. This fosters trust and confidence, improves the quality of participation and therefore generates more representative and useful findings for greater impact.
The benefits of participatory M&E
There are both instrumental reasons and human rights considerations for using participatory M&E in humanitarian and development interventions.
Participation improves the quality of M&E and our interventions
There are benefits to having community participation during all stages of the M&E cycle. Combining the theoretical and methodological expertise of practitioners with the real-world knowledge and experiences of participating communities can improve our processes, tools, data and findings.
| Phase | Benefits | Notes & examples |
|---|---|---|
| Plan | - More relevant and meaningful indicators and monitoring and evaluation questions, since they relate to the lives of participants - More community motivation to participate in M&E processes - More realistic expectations about the outcomes of the M&E process and the intervention |
- Indicators for abstract concepts such as ‘improved wellbeing’, ‘decreased distress’, or ‘reintegration of former child soldiers’ - More realistic expectations can in turn lead to less frustration and greater community satisfaction |
| Design | - More useful and culturally relevant tools - Better handling of sensitive or taboo topics in our data collection methods, processes and tools, since the community knows better how to address these topics in a culturally acceptable way - Ability to generate both qualitative and quantitative insights, since participatory approaches are compatible with both types of research - More inclusive data collection and analysis processes, since the community can better identify barriers to participation |
- Using local terminology helps us formulate better survey and interview questions because they are better understood by respondents - Participatory methods for producing numerical data may be preferable to conventional surveys in terms of cost-effectiveness and the ability to put numerical values on complex qualitative categories (such as wealth or well beingwellbeing ranking) - Barriers to participation can include unsuitable time or location for data collection, or feeling inhibited in certain settings because of community power structures |
| Collect | - Greater trust in the data collection process because it is designed and/or led by the community, leading to higher rates of participation and therefore more representative data - Enhanced ability to capture unexpected yet meaningful responses and insights because of an open-ended and flexible process - Enhanced ability to capture data on sensitive topics, since participants may be more open to sharing views and beliefs using different modes of expression and through group activities - Highly accurate data through group data collection |
- Different modes of expression include visual (e.g. body mapping), written (e.g. journaling), oral (e.g. storytelling) and tangible (e.g. counting using small objects) modes - Group data collection is especially accurate if it involves public knowledge, so that participants can correct or add information before reaching a consensus (applications include community and census mapping) |
| Analyse | - Enhanced ability to deal with complexity in analysis by leveraging the community’s understanding of the cultural, social and political context - Enhanced ability to cross-check data and validate findings and conclusions - Enhanced ability to generate insights from data that are locally relevant - Mutual learning and benefit for practitioners and communities |
- In the context of an evaluation, causality is more easily investigated by exploring respondents’ understanding of key factors that contributed to an identified impact (e.g. using participatory flow diagrams) - Findings and conclusions become more valid through community debate and discussion, which allows for differences to be reconciled or for people to ‘agree to disagree’ - Communities can benefit from participation as an empowering process that helps identify and strengthen local capacities, while practitioners and organisations fulfil more functional requirements (e.g. donor reporting) and learn for future interventions |
| Act | - More effective and culturally competent communication about findings - Enhanced ability to translate findings into meaningful action - More opportunity for communities to think about and develop solutions for themselves, and for interventions to integrate local knowledge, strategies and capacities - Potentially positive impact on social cohesion in the community because of collaboration, and shifting power dynamics in favour of marginalised groups and individuals |
- Meaningful action can include making a change within the project based on monitoring data, or the community using M&E findings to gain or improve a service or to assert their rights |
| Reflect | - Better understanding of the effectiveness and quality of our M&E system - Deeper and more useful learning for future improvement |
- Understanding the utility of our data collection tools and methods, and the quality of our M&E processes (in terms of participation and inclusion especially) |
| Adapted from “A Short Guide to Community Based Participatory Action Research: A Community Research Lab Guide” by J.C. Burns, D.Y. Cooke, and D.Y. Schweidler, C., 2011, Advancement Project-Healthy City Community Research Lab; “Participatory Approaches” by Gujit, I., 2014, Methodological Briefs: Impact Evaluation 5, UNICEF Office of Research, Florence; “Who Counts? The Quiet Revolution of Participation and Numbers” by R. Chambers, 2007, Institute of Development Studies; and “Who counts? The power of participatory statistics” by J. Holland, 2013, Practical Action Publishing. |
||
The benefits of participation for our M&E processes have positive knock-on effects for our interventions as well. They become more responsive to actual (rather than assumed) needs and priorities, more effective, and more able to mitigate potential negative consequences. Participation also leads to greater accountability and transparency and can make results more sustainable because of stronger community ownership and empowerment.
Participation is a right
Besides being good practice, participation is a human right. Participation in this sense refers to the effective and meaningful involvement of people in the decision-making processes that affect them. Several international treaties outline aspects of this right, including the International Covenant on Economic, Social and Cultural Rights and the International Covenant on Civil and Political Rights. Other international conventions and declarations have strengthened the right to participate for particular groups, including women, children, persons with disabilities, Indigenous peoples and persons belonging to national or ethnic, religious and linguistic minorities. The Declaration on the Right to Development also refers to “free and meaningful participation in development” (art. 2), recognising that participation is “an important factor in development and in the full realization of all human rights” (art. 8.2). It establishes that everyone is “entitled to participate in, contribute to, and enjoy economic, social, cultural and political development” (art. 1.1).
The challenges of participatory M&E
Although there are many benefits to participatory M&E, it is not without its challenges. Here are a few things to keep in mind if we want to do it well:
Being inclusive and representative
We need to think critically about power throughout the M&E cycle, taking into account not only the power relations between the practitioner and the community but also those that exist within the community itself. Our role as practitioners is to work towards greater equity in these relations through facilitation and to prevent inequalities from being reproduced in M&E processes. Our methods, tools and processes should be chosen and designed to include marginalised groups and those at risk of discrimination. Otherwise, they may be dominated by powerful voices in the community and our findings will not be representative, and therefore less valid. For example, men may dominate women and adults may dominate young people in group discussions. Political, cultural and religious leaders might dominate decision-making and prioritisation in the community.
Do no harm: considering ethics and safety
Doing participatory M&E in a way that is ethical and safe requires careful planning and communication. This is because it often involves long-term and intense social interaction among practitioners and other participants. We need to consider how best to get free, prior and informed consent from the community; establish the norms and ground rules that should be followed during all social interactions; and decide how we collect, store, analyse, interpret and disseminate data so that it does not infringe on participants’ privacy or harm them in any way.
We need to weigh the benefits of participatory methods with the risks. It is important to be mindful that these methods make demands on people’s time and energy, can create expectations that could turn into a source of frustration, and could even increase tensions between individuals and groups of people. We also need to take special care when working with vulnerable populations, such as children. For example, facilitators would need to be carefully vetted and trained on child rights and safeguarding, psychosocial care should be offered to participants, and referral mechanisms should be in place, among others. There are standards, norms and guidelines for participation that practitioners can refer to (please refer to the further reading list below for some examples).
Resources, speed and scale
Skilled facilitators are critical to ensuring the reliability of data gathered through participatory methods. Good facilitation depends on special skills, attitudes and behaviours, and requires training and experience. Community members might also need some training to learn how to effectively use the methods and tools proposed. These requirements can result in potentially higher up-front costs and slower implementation, even if the eventual results are still very cost-effective (compared with alternatives like conventional large-scale surveys, for example). Building and managing good working relationships and trust can also take more time, as does the discussion and negotiation that is required to make shared decisions in each phase of the process.
It may therefore become more difficult to scale up participatory M&E while still maintaining an adequate level of representativeness. It also means that not all participatory approaches will be acceptable in all circumstances: for example, if the immediate need to save lives in a humanitarian crisis outweighs the benefits of participation. Even in challenging circumstances such as these, however, it is still possible and desirable to encourage participation to the extent that it can be safely and ethically done. We can use strategies on the lower end of the participation continuum (which inform, consult or involve) and ensure that all groups are at least represented even if they cannot participate directly.
Scepticism and non-acceptance among decision-makers
Participatory M&E is methodologically flexible and diverse. Research questions, methods and tools are collaboratively designed and adapted to the context. This is in contrast with conventional approaches to social science research, and to quantitative research especially, which tend to have more rigid ways of doing things and are often still seen as producing more valid and reliable findings. Participatory methods are subject to many of the same methodological and epistemological critiques as qualitative methods. Their application to M&E in humanitarian and development contexts is also still relatively limited compared to other fields. If we want to use participatory M&E this often means we have to persuade others of its utility and legitimacy. It is especially important to secure buy-in and support for this kind of participation from decision-makers who have power over the project, such as donors, and those who have the power to effect change, such as policymakers. We cannot ask communities to spend their time and energy participating in M&E if we do not believe that decision-makers will take our findings seriously.
Participation, power and action: some final thoughts
Participatory approaches to humanitarian assistance and development are not new. Participation in M&E processes, however, is not as common as participation in project planning or implementation. This may be because traditional beliefs about inquiry and knowledge production are still more common, and a functional view of the purpose of M&E predominates. It is also, perhaps, because participatory approaches require us to look critically at power and ultimately to redistribute it. Participatory M&E is therefore not just about ‘them’, but also very much about ‘us’. It values local knowledge, perspectives and capacities and seeks to empower communities through M&E. It also urges us to be more open to learn from others and to reflect on our own role in the power dynamic, both as M&E practitioners and as humanitarian and development professionals more generally.
This kind of self-reflection can be uncomfortable and confrontational. The issue of systemic sexism, racism, and ableism in the humanitarian aid and development sectors deserves more words and careful attention than I can dedicate to the subject here. The complex power relations between practitioners and affected communities are grounded in the history of the sectors themselves, which evolved within long-standing colonial and patriarchal structures and discourses. While these issues have recently been talked about more openly in the context of #MeToo, #AidToo, and the Black Lives Matter movement, the need for reflection and conversation is far from over.
Participatory approaches to M&E invite us to think about which belief systems are privileged in our work, whose knowledge and priorities matter most, and why we do the work we do in the first place. If our ultimate aim is to support real-world impact and transformative change through M&E, then the participation of affected communities is a precondition. You cannot have empowerment without participation: empowerment is not something we ‘do’ to other people, but is itself a participatory process that engages people in reflection and inquiry to understand the power they have, and to take action for change as they define it. As technical experts and custodians of data, evidence and knowledge, M&E specialists have the power to influence how M&E is done and whose voice counts. We can make the case authoritatively for participation in M&E both as means to an end, and as an end in itself.
The team of ActivityInfo would like to warmly thank Ms Naomi Falkenburg for this insightful and detailed article on participatory M&E. Ms Falkenburg is a certified ActivityInfo Partner since July 2020.
Naomi Falkenburg is an independent consultant who works with humanitarian and development actors to design, manage and learn from their interventions and to conduct insightful research and analyses. Naomi believes the most impactful interventions are evidence-driven and inclusive, and she is passionate about empowering others to use M&E to this end. Prior to becoming a consultant, Naomi worked at several UN agencies and international NGOs in West Africa, Europe, and East Asia, on themes such as gender equality, disability inclusion, migration and forced displacement, decent work and skills development for youth, and digital inclusion. She is certified in project management for development (Project DPro) and member of the International Association for Professionals in Humanitarian Assistance and Protection (PHAP). Naomi holds an MSc from the University of Oxford’s Department of International Development and a BA in Politics and International Studies from the University of Warwick.
Sources and further reading
Alexander, J., & Bonino, F. (2014). Ensuring quality of evidence generated through participatory evaluation in humanitarian contexts. ALNAP. https://www.alnap.org/help-library/ensuring-quality-of-evidence-generated-through-participatory-evaluation-in-humanitarian
Bergold, J., & Thomas, S. (2012). Participatory research methods: A methodological approach in motion. Forum: Qualitative Social Research, 13(1). https://doi.org/10.17169/fqs-13.1.1801
Burns, J. C., Cooke, D. Y., & Schweidler, C. (2011). A short guide to community based participatory action research (T. Bonilla & T. Farris, Eds.). Advancement Project – Healthy City. https://ktpathways.ca/system/files/resources/2019-02/cbpar.pdf
Catley, A., Burns, J., Abebe, D., & Suji, O. (2014). Participatory impact assessment: A design guide. The Feinstein International Center. https://fic.tufts.edu/wp-content/uploads/PIA-guide_revised-2014-3.pdf
Chambers, R. (2007). Who counts? The quiet revolution of participation and numbers (Working Paper 296). Institute of Development Studies. https://opendocs.ids.ac.uk/opendocs/bitstream/handle/20.500.12413/398/Wp296%20web.pdf?sequence=1&isAllowed=y
Chicago Beyond. (2018). Why am I always being researched? https://chicagobeyond.org/researchequity/
Groupe URD. (2009). Participation handbook for humanitarian field workers. https://www.alnap.org/help-library/participation-handbook-for-humanitarian-field-workers
Gujit, I. (2014). Participatory Approaches. Methodological Briefs: Impact Evaluation 5, UNICEF Office of Research, Florence. https://www.betterevaluation.org/sites/default/files/Participatory_Approaches_ENG.pdf
Holland, J. (Ed.). (2013). Who Counts?: The Power of Participatory Statistics. Practical Action Publishing.
Macaulay, A. C., Jagosh, J., Pluye, P., Bush, P. L., & Salsberg, J. (2013). Quantitative methods in participatory research. Nouvelles Pratiques Sociales, 25(2), 159–172. https://doi.org/10.7202/1020827ar
The Global Initiative for Economic, Social and Cultural Rights (GI-ESCR). (2014). Practitioner’s guide: A rights-based approach to participation. http://globalinitiative-escr.org/wp-content/uploads/2014/05/GI-ESCR-Practitioners-Guilde-on-Right-to-Participation.pdf
UN Office of the High Commissioner for Human Rights. (2016). A human rights-based approach to data: Leaving no one behind in the 2030 development agenda. United Nations. https://www.ohchr.org/Documents/Issues/HRIndicators/GuidanceNoteonApproachtoData.pdf
Vaughn, L. M., & Jacquez, F. (n.d.). Participatory research methods – choice points in the research process. Journal of Participatory Research Methods. https://doi.org/10.35844/001c.13244
Willetts, J., & Crawford, P. (2007). The Most Significant Lessons about the Most Significant Change Technique. Development in Practice, 17(3), 367-379. http://www.jstor.org/stable/25548222
Examples of guidelines for participatory research and M&E
Assembly of First Nations (2009). Ethics in First Nations Research. https://www.afn.ca/uploads/files/rp-research_ethics_final.pdf
Faulkner, A. (2004). The ethics of survivor research. Joseph Rowntree Foundation. https://www.jrf.org.uk/report/ethics-survivor-research-guidelines-ethical-conduct-research-carried-out-mental-health
FAO. (2016). Free, prior and informed consent (FPIC) manual. http://www.fao.org/3/a-i6190e.pdf
Graham, A., Powell, M., Taylor, N., Anderson, D., & Fitzgerald, R. (2013). Ethical research involving children. UNICEF Office of Research - Innocenti. https://www.unicef-irc.org/publications/pdf/eric-compendium-approved-digital-web.pdf
Lansdown, G., & O’Kane, C. (2014). A toolkit for monitoring and evaluating children’s participation. Save the Children. https://resourcecentre.savethechildren.net/node/8102/pdf/me_toolkit_booklet_1.pdf
Rambaldi, G., Chambers, R., McCall, M., & Fox, J. (2006). Practical ethics for PGIS practitioners, facilitators, technology intermediaries and researchers. Integrated Approaches to Participatory Development (IAPAD), 54, 106–113. https://pubs.iied.org/sites/default/files/pdfs/migrate/G02957.pdf
Protocols & principles for conducting research in an Indigenous context. (2003). University of Victoria. https://www.uvic.ca/hsd/research/igovprotocol.pdf