Choosing methods and tools for participatory M&E - Reflections and resources

Last time, I talked about the benefits and challenges of participatory M&E and how it encourages us to think critically about power, inquiry and knowledge. Today, we will look at a few examples of participatory methods and tools for M&E and what we should keep in mind when choosing them. We will also consider how participatory M&E can be adapted to virtual settings. At the end of the article, there are more resources and toolkits.

If you like this article don't forget to register to the ActivityInfo newsletter to receive new guides, articles and webinars on various M&E topics!

Examples of participatory methods and tools for M&E

There are many methods and tools that we can use to make M&E more participatory. Even conventional quantitative or qualitative methods, such as surveys or focus group discussions, can be more participatory if they have been designed and are used in a participatory way.

Today, however, we will look at a few methods and tools that have been designed specifically for participation. This is a short list, since there are already many useful and freely available resources out there on participatory methods and tools for M&E (links are in the table below and at the end of the article).

Method or tool Description
Calendars, charts and timelines Calendars, charts and timelines are visual tools for exploring sequences of events and how they are perceived. Examples of calendars include seasonal calendars, which can be used to gather data on and analyse change over a period of time (like agricultural production patterns and yield). Examples of charts include daily activity charts, which can be used to compare the daily activities of different people (like gendered differences in time spent on employment and household chores). Timelines can be collaboratively drawn to represent the history of a community or past events. They can also explore hopes or expectations for the future. Calendars, charts and timelines can be used for monitoring, evaluation and planning purposes.
Learn more:
Salm, M. (2014, January 13). Seasonal calendars. BetterEvaluation. https://www.betterevaluation.org/evaluation-options/seasonalcalendars
Sequencing, calendars and charts. (2007). Ethnographic Action Research Training Handbook. https://ear.findingavoice.org/toolbox/4-3.html
Most significant change (MSC) A method by which you collect individual stories of change from the people involved in an initiative. Together, participants discuss and analyse each story before selecting the most significant ones. It is particularly useful for initiatives that do not have predefined outcomes and indicators, where outcomes are very different for the people involved, and where outcomes are complex. MSC can be used both for monitoring and evaluation purposes, as well as to support accountability.
Learn more:
Davies, R., & Dart, J. (2005). The ‘Most Significant Change’ (MSC) technique: A guide to its use. https://mande.co.uk/wp-content/uploads/2018/01/MSCGuide.pdf
INTRAC. (2017). Most significant change. https://www.intrac.org/wpcms/wp-content/uploads/2017/01/Most-significant-change.pdf
Outcome harvesting A method to collect (harvest) evidence about changes—whether intended or unintended, negative or positive, and direct or indirect—and determine whether and how an intervention contributed to these changes. Outcome harvesting does not measure progress towards predetermined objectives or outcomes and is therefore particularly suited to complex situations where it is difficult to establish clear cause and effect relationships or where objectives had to be adjusted during the intervention. Although it is more well-known as a tool for evaluation, outcome harvesting can be used for both monitoring and evaluation purposes.
Learn more:
Wilson-Grau, R., & Britt, H. (2012). Outcome harvesting. Ford Foundation. https://www.outcomemapping.ca/download/wilsongrau_en_Outome%20Harvesting%20Brief_revised%20Nov%202013.pdf
Participatory diagramming Diagrammes are visual tools that can be used to represent, analyse and explain social phenomena. They can also help participants to express their ideas. Examples include tree analyses, which can be used to explore root causes and consequences of a problem or conflict; Venn diagrams, which represent the relationships between different things or people; and spider web diagrams, which represent multiple topics that are scored using a common scale and can be very useful to show strengths and weaknesses.
Participatory mapping and modelling Communities create their own maps or models, visually representing their understanding of the community’s geography and the significant features within it. Participatory maps can be used for many purposes, ranging from spatial and resource management planning to safety audits and advocating for resource rights. Advances in technology mean people can make use of Geographical Information Systems (GIS), Global Positioning Systems (GPS) and spatial data and imagery, which lend even greater spatial accuracy and legitimacy to maps.
Learn more:
Macbeth, S. (n.d.). Participatory Geographical Information Systems (PGIS). Participatory Methods. https://www.participatorymethods.org/method/participatory-geographical-information-systems-pgis
Rambaldi, G. (n.d.). Open forum on participatory geographic information systems and technologies. PPGIS.Net. http://www.ppgis.net
Participatory video A group method through which a community learns basic filming skills and works together to produce a film about a subject. Through filmmaking, participants jointly explore an issue or concern and reflect on their experiences together, enabling them to take action based on what they learn and to communicate their story, needs or ideas to others to advocate for change.
Learn more:
Participatory video (PV). (n.d.). Managing for Sustainable Development Impact. http://www.managingforimpact.org/tool/participatory-video-pv
Resources archive. (n.d.). InsightShare. https://insightshare.org/resources/
Photovoice A group activity in which participants capture their experiences related to an issue or concern in the community through photographs, to which they add a caption. The resulting images are usually discussed through group dialogue and can also be shared with decision-makers for advocacy purposes to bring about change. In the context of an evaluation, participants can capture visual responses to a particular question. Their photos can give insights into previously invisible attitudes, practices or beliefs.
Learn more:
Rabinowitz, P. (n.d.). Implementing Photovoice in your community. Community Toolbox. https://ctb.ku.edu/en/table-of-contents/assessment/assessing-community-needs-and-resources/photovoice/main
Rubrics A tool jointly developed by stakeholders that sets out detailed criteria and standards for performance and describes what it would look like at each level. The different levels are assigned scores or labels accordingly (for example, 1 to 5 or ‘very poor’ to ‘excellent’). If developed during the planning phase based on indicators that have been chosen, rubrics can be effective tools for both monitoring and evaluation.
Learn more:
Rogers, P., & Kaplan, J. (2020, March 25). Rubrics. BetterEvaluation. https://www.betterevaluation.org/en/evaluation-options/rubrics
Stone-Jovicich, S. (2016). To rubrics or not to rubrics? An experience using rubrics for monitoring, evaluating and learning in a complex project. Managing For Impact, Practice notes series. https://www.researchgate.net/publication/292501331_To_rubrics_or_not_to_rubrics_An_experience_using_rubrics_for_monitoring_evaluating_and_learning_in_a_complex_project
Sorting and ranking Sorting and ranking are techniques that give insight into how people perceive different phenomena by categorising and ranking them. Cards are often used in sorting and ranking activities. A group of related topics or statements are written out on the cards, which are then sorted by participants according to their own values, priorities and experiences. These exercises can be a useful point of departure for more in-depth discussions about a topic and can be used for both monitoring and evaluation purposes.
Learn more:
Davies, R., & Kaplan, J. (2016, September 22). Hierarchical card sorting. BetterEvaluation. https://www.betterevaluation.org/evaluation-options/hierarchical_card_sorting
Grouping, ranking and comparing. (2007). Ethnographic Action Research Training Handbook. https://ear.findingavoice.org/toolbox/4-2.html

Participatory M&E in virtual settings

M&E practitioners have been exploring the use of ICTs for monitoring and evaluation since well before the onset of the Covid-19 pandemic. Some of the potential benefits of using such tools are to enable real-time data collection and decision-making, to improve the quality control of data, to lower costs and save time - and to facilitate the participation in M&E of those who are not usually involved.

Many of the participatory methods described above can be adapted to virtual environments by using general-purpose digital tools for video conferencing (like Zoom, Webex, Google meet or Whereby) and collaboration (such as Stormboard, Mural, Miro or Google Jamboard). There are also examples of more specialist digital tools that are being used to increase participation in M&E, such as software to develop theories of change or use crowdsourcing and micro-narratives for data collection. ActivityInfo, as a web-based information management system, is a useful tool for collaboration in data collection and reporting.

However, using digital tools comes with its own set of challenges that may create barriers to participation. People may not own Internet-enabled devices, have Internet access, be able to afford mobile data, or have the skills to use digital tools. They may not even necessarily trust these tools or feel comfortable using them, especially if privacy and security are a concern. It is important to consider whether doing M&E in a virtual setting could lead to exclusion.

Choosing participatory methods and tools for M&E

There is no one ‘right’ way to do participatory M&E and no particular set of methods and tools that is right for every situation. Often, you might need to combine several methods and tools over the course of a project. It is helpful to consider the following questions when choosing which are most appropriate:

Question Notes
What are the ultimate aims of the M&E I am engaged in? The aims of M&E processes can range from being functional to transformative; read my previous article on the difference here. The more transformative your aims are, the higher the level of participation will be, and you would therefore select a method or tool that transfers power to participants.
What is the purpose of the M&E activity I am undertaking? Some methods and tools might be more useful than others depending on what phase of the M&E cycle you are in: are you designing your methodology, collecting data, or analysing data?
Who are the participants and what are their needs and preferences? Learning how people think and communicate information will give you important clues about which method or tool is most appropriate. For example, if you are conducting an evaluation that engages children, you would take into account their cognitive development and might therefore choose a method or tool that is more visual, creative and playful. If you are working with a community that has a strong oral tradition, it might be more effective to use a group narrative method in which people tell and collectively interpret stories. Ideally, methods and tools are chosen together with participants. It is always a good idea to first test a method or tool with a small group to see whether it works well.
How much time do I have?
What financial and human resources and support materials are available?		 You need to be sure that it is feasible to conduct your participatory M&E activity well with the resources you have available. Is there enough time and sufficient budget to organise it and ensure the appropriate follow-up? Are there skilled facilitators available? If you are working with specific tools, like digital tools or recording equipment, are all participants able to access and use these?
What is the potential of a particular method or tool to do harm? As with all research, you must carefully assess the ethical implications and potential risks of using participatory methods and weigh the benefits against these risks.

Ultimately, participatory methods and tools are only as useful as the skills and attitude of the facilitator. The ones I have described above do not by themselves guarantee effective and meaningful participation. Indeed, even methods and tools that have been designed for participation can be used for extractive purposes if our aim is not to empower participants. Similarly, good facilitation is not only skilled, but it also draws on a relationship of mutual trust, respect and understanding between practitioners and participants. Our toolbox for participatory M&E is important, but our mindset even more so.

The team of ActivityInfo would like to warmly thank Ms Naomi Falkenburg for this insightful and detailed article on participatory M&E. Ms Falkenburg is a certified ActivityInfo Partner since July 2020.

Naomi Falkenburg is an independent consultant who works with humanitarian and development actors to design, manage and learn from their interventions and to conduct insightful research and analyses. Naomi believes the most impactful interventions are evidence-driven and inclusive, and she is passionate about empowering others to use M&E to this end. Prior to becoming a consultant, Naomi worked at several UN agencies and international NGOs in West Africa, Europe, and East Asia, on themes such as gender equality, disability inclusion, migration and forced displacement, decent work and skills development for youth, and digital inclusion. She is certified in project management for development (Project DPro) and member of the International Association for Professionals in Humanitarian Assistance and Protection (PHAP). Naomi holds an MSc from the University of Oxford’s Department of International Development and a BA in Politics and International Studies from the University of Warwick.

Sources and further reading

Websites

  • betterevaluation.org: information on many approaches, methods and processes for evaluation, including participatory ones.

  • participatorymethods.org: website of the Participation, Inclusion and Social Change Cluster at the Institute of Development Studies.

  • reflectionaction.org: website with a range of participatory tools and techniques for practitioners and activists supporting people to analyse their situation, identify rights violations, confront power and bring about change.

Toolkits and examples

Resources on facilitation

Participatory M&E in virtual settings