7 steps for setting-up a Monitoring & Evaluation system

Designing a Monitoring and Evaluation system, or M&E system, is a complex task that usually involves staff from different units. This article describes the development of such a system in 7 steps (1). Each step is linked with key questions, which are intended to stimulate a discussion of the current state of the M&E system in a project or in an organization. Therefore, the 7 steps do not represent a strict chronological sequence for the development of an M&E system. All steps should be considered from the beginning:

• Step 1: Define the purpose and scope of the M&E system
• Step 2: Agree on outcomes and objectives - Theory of change (including indicators)
• Step 3: Plan data collection and analysis (including development of tools)
• Step 4: Plan the organization of the data
• Step 5: Plan the information flow and reporting requirements (how and for whom?)
• Step 6: Plan reflection processes and events
• Step 7: Plan the necessary resources and skills

Download the tool “M&E gap analysis” to work on these 7 steps.

Do you wish to get more information on designing a project level database for your M&E activities? Then, take a look at "The complete guide for a project level M&E database" too.

Take a look at how a database for development assistance for projects would look like via this database template.

Explore a database for indicators tracking for global M&E in this database template.

If you like this article don't forget to register to the ActivityInfo newsletter to receive new guides, articles and webinars on various M&E topics!

This guide is also available in French and Spanish

Step 1: Define the purpose and scope of the M&E system

It is crucial to define the scope of the M&E system at the very beginning. A question that will likely need to be answered is whether the system should be impact-oriented and whether we even want to monitor higher-level impacts, or whether the project team is satisfied with simply recording the proper implementation of activities and their results. Both can make sense and be correct, depending on the circumstances and what is needed to be able to improve the project in the best possible way. Of course, high-level results-oriented monitoring is usually preferable, but it could also fail due to the available capabilities. Moreover, it should be clear from the beginning who will continue to work with the M&E findings later on.

A challenge in this first step is engaging staff and convincing them that the additional time and effort to set-up an M&E system is worthwhile in order to improve project steering and thus the quality of project or program results. There are many and varied activities that can be carried out to this end. For some project/program teams, a workshop or a presentation may be helpful to convince of the usefulness of monitoring; in other cases, various face-to-face discussions may be more appropriate. The approach must ultimately be decided by the person responsible for M&E and depends on the resources available and the key people involved. Prior consultation on these can be useful.

Step 2: Agree on outcomes and objectives - Theory of change (including indicators)

A Theory of Change (ToC) is a description of how and why activities are expected to lead to short, medium, and long term outcomes over a period of time. This is more than just identifying outcomes and objectives. In fact, a ToC is a set of impact assumptions or hypotheses that can be described as a visual diagram, a narrative, or both. Once there is a ToC or something similar, staff will know better which M&E data to collect.

ToCs are not necessarily complex but they do provide a way to summarize the complexity of a situation and bring clarity to it. At best, this allows a wide range of stakeholders to come to a shared understanding of why and how activities will lead to desired results. Further information and examples of ToC are provided by Culligan & Sherriff in "A Guide to the MEAL DPro".

It is helpful to involve a variety of stakeholders when developing the ToC – this could include staff, beneficiaries, partners, funders and even other experts who are familiar with the technical theme. The development process, and the thinking involved, are often as important as the diagram or narrative produced. However, if this seems too time-consuming, a common, good practice is to produce a first draft, which can be then discussed with other key stakeholders. The result of this work should be a complete but not over-complicated description of the activities and its results, with prioritized outcomes for measurement and SMART indicators (2) to collect data against them.

Of course, ToCs need regular review because as context and needs change so do they. But high level outcomes and impacts are usually valid for some years.

It’s worth noting that an important critique of ToC is that it neglects social realities and possible negative project effects and that it might narrow the view on planned project/program goals.

Step 3: Plan data collection and analysis (including development of tools)

For this step, it is recommended to create an M&E work plan, an M&E matrix or a combination of both. Depending on the needs of a program or project, the design of such a document may vary greatly. However, to ensure that the M&E activities are implemented, it is advisable to determine in such a matrix clear responsibilities with timelines or frequencies of data collection. Evaluation Toolbox provides a template that can be customized according to these needs

The methods to collect the data depend on the information needed. For example, if quantitative information on jobs created is needed, then a survey with a standardized questionnaire may be useful whereas if information on the reasons for behavioral changes of supported groups is required, qualitative interviews or a combination of interviews and a standardized survey may be more useful. Data collection tools (e.g. interview guidelines and questionnaires) should be pre-tested before they are actually used. Important guidance on how to develop such tools can also be found in the aforementioned Evaluation Toolbox and at the websites of INTRAC. Some of the staff involved probably need to have analytical skills (e.g. statistical skills in the case of analysis of questionnaires). If such skills are missing or there is no time to develop the required tools or to analyze the collected data, hiring an external M&E expert could be considered.

For some programs or projects, there might be an M&E officer who coordinates such M&E activities. If this is not the case, then it will be necessary that the staff coordinate the activities among themselves. It will then be even more important to assign clear responsibilities within the team.

Are you trying to decide whether to build an internal tool, work with external contractors or use an off-the-shelf tool such as ActivityInfo? Then, take a look at our White Paper "Off the shelf, external contractors or building your own application: When off-the-shelf is your best option for your information management needs"

Step 4: Plan the organization of the data

To use the collected data, the information needs to be stored and shared with the people involved, regardless of their location. One can store data physically or digitally using an information system. This means that M&E systems and data management go together. The data management system should be designed according to the needs, size and complexity of the project or program. Staff engaged in M&E activities, may need to liaise with the IT support of their organization. In any case, it is important to label and organize items in storage clearly (chronological, by location, by content or any other category considered useful).

Good data management includes storing data securely to avoid unauthorized access, theft, or unintentional destruction of data and to comply with any legal requirements, such as data protection legislation. This often involves IT protection methods, such as passwords, firewalls and virus checks. But it might also simply mean having a lock on a filing cabinet. The global collaboration organization BetterEvaluation synthesizing advice from the UK Data Archive (3) recommends: not to store digital data on externally networked servers or computers; the installation of firewalls and security systems to protect against malware and viruses; the existence of password protected computer systems; the encryption of sensitive materials (even when transferring data by email); the signing of non-disclosure agreements. If it is important for a survey to include personal data, such as address or name, it is essential to obtain the permission of the respondent beforehand.

When using external software for data management needs, the terms of use, data protection and confidentiality, and the servers location should be checked.

Data management is linked with data quality assurance too: It is important to avoid gathering data of low quality and to ensure that data is "cleaned" of any errors. The collected data may be the basis for further decisions. Data quality methods may include the use of multiple data sources, such as triangulation of data and interviewer training and supervision. It should be clarified among the staff who is responsible for data quality assurance and how.

Step 5: Plan the information flow and reporting requirements (how and for whom?)

To be useful, information gained through M&E needs to be communicated to different stakeholders. Most likely, there are certain reporting requirements set by donors. However, it is good practice to disseminate and to discuss findings among other stakeholder groups so that learning from M&E has a wider reach. At the very least, the M&E results should also be discussed with the supported communities and groups.

There are many ways to communicate M&E information with stakeholder groups. The best communication method will depend on the audience and how the information will be used. For example, project managers may require much more detailed information on the progress made; program directors may require regular, short summary reports across different projects and programs, with aggregated tables and statistics; policy-makers might benefit from a short brief summarizing the main issues, and making recommendations for change; a member of the public that supports an organization through donations might prefer to see a story of change, a photograph or a short video that enables to connect with beneficiaries on an emotional level.

Sometimes, especially when communicating information to partners or supported groups, it is useful to discuss communication methods with the audience beforehand. This is a core element of participatory M&E. Consideration should also be given to how information can be communicated to people with audio or visual disabilities, or whether stakeholders are able to access the venues for meetings. When communicating information to illiterate or semi-literate people, presenting information in written form is of little use.

It is also important to know when information needs to be communicated. For example, if decision-making meetings occur on a quarterly basis then it is important to communicate M&E findings before those meetings are held. Similarly, when seeking to influence a government policy, it is important to supply information at the right time so that it has the maximum chance of achieving its purpose.

The use of communication strategies or dissemination plans will facilitate the organization of the information flow. The key point is to be very clear about who needs what M&E information, when and where. Narratives (formal reports, case studies, newsletters, press releases, policy briefs) are the most common way of communicating M&E findings. Other means of communication are through photographs, videos, pictures and cartoons. The big advantage of the latter mentioned communication channels is that they can communicate information from supported communities and groups directly to different audiences, without being filtered through a report. In addition, M&E findings can be communicated verbally in meetings and workshops, through feedback sessions and even through informal conversations. Speaking directly to a target audience allows messages to be tailored to the individual or group, and allows for some discussion of findings as well.

Also, more artistic and traditional methods of communication such as poems, drama, mime and song can be used to share M&E information with others. Using such activities can help prevent M&E becoming a sterile exercise, and can foster a broader understanding and discussions about change.

Recent technological advances offer another way of communicating M&E information. Websites and social media sites, podcasting, and webinars have made it much easier to present and communicate information in new and innovative ways. Communication via mobile phones and tablets offer further opportunities in the communication of M&E information, although to date, this has mainly been used for data collection (for example surveying through text messages) rather than for communication of M&E findings (4).

The dissemination plan below, taken from the website of the Technical Centre for Agricultural and Rural Cooperation provides a good example of how such a plan could look like. Other examples are offered by the websites of BetterEvaluation and MEAL DPro initiative.

Source: Website of the CTA. Technical Centre for Agricultural and Rural Cooperation, licensed under a Creative Commons Attribution-Share Alike 3.0 Unported License

In some organizations, there is a communications officer responsible for external communications and organizing the flow of information; in others, especially smaller organizations, this task is the responsibility of an M&E officer or the project team itself.

Step 6: Plan reflection processes and events

This step goes a bit further and wants to not only communicate results but discuss M&E findings with stakeholders so that everyone learns from each other. Again, the discussion and learning formats can vary widely. These could include, workshops, exchange visits, seminar, conferences, After-Action Reviews (AAR) (5) to name a few. However, learning does not happen in one sitting. It is important that moments of reflection take place regularly throughout the life of the project or program. The incorporation of learning events in the project/program cycle is key. In this regard, annual or bi-annual reviews are critical learning opportunities to reach conclusions about achievement and failures. The optimal sequence of learning events follows reporting lines of decision making. It should be ensured that the right people are involved in such reviews. Therefore, it may sometimes be important to include decision makers in the reviews (so that they learn at the same time as their staff and therefore make appropriate decisions), but it can be a challenge to ensure that this does not affect the openness of the conversations. It could be very helpful if staff are trained in facilitating intentional group learning processes.

Regular team meetings are another important opportunity for reflection. Team members may include project staff, implementing partners, and primary stakeholder representatives – this depends on how the project is structured. Weekly meetings are common but if other stakeholders are involved this may be needed less frequently. In each project context, there are usually forums where implementing partners interact with each other. These events offer another chance for reflection.

It is recommended to assign roles and responsibilities for leading the learning events. In addition, the learning and resulting conclusions for further actions should be documented well, with a focus on documenting “action needed”, “person responsible for implementation”, “deadline”, and “persons responsible for follow-up”. Such documentation could be tabular (see the example below), but any other form is fine as long as it records the most important items.

Source: own composition

Lastly, it is worth mentioning here that learning in the context of M&E is about having a culture that encourages intentional reflection and processes that support this culture. All teams learn as they implement project activities. But to take advantage of this learning and consistently translate it into improved practice, learning must be planned and managed.

Step 7: Plan the necessary resources and skills

It is good to start planning the M&E budget already in the project/program design phase so that adequate funds are allocated and later available for M&E activities. There is no standard formula to determine the budget for a project/program’s M&E system. An industry standard is that between 3 and 10 percent of a project/program’s budget should be allocated to M&E (6). A planning table for key M&E activities can be useful in this regard. It is particularly important to budget for any expensive items, such as baseline surveys and evaluations.

Moreover, an effective M&E system requires capable people. Therefore, when defining roles and responsibilities for M&E, specific consideration should be given to the M&E qualifications and expectations, including the approximate proportion of time for each person to support the system. A first step in planning for M&E human resources is to determine the available M&E experience within the project/program team, partner organizations, target communities and any other potential participants in the M&E system. This will inform the need for capacity building or outside expertise. For long-term and larger projects/programs, it may be useful to create an M&E training schedule. Ideally, data collection, analysis and M&E training involves the people to whom these processes and decisions most relate (7).

One key planning consideration is who will have the overall responsibility for the M&E system. It is important to clearly identify who will coordinate all these M&E activities and to whom others will turn to for M&E guidance. The responsible person or team should supervise the M&E functions, and have an overview of any problems that might arise.

This article was written with the intention to support especially smaller organizations in their M&E activities. Hopefully the information has been helpful and of practical use in setting up M&E systems. The author welcomes suggestions, additions and comments.

Download the tool “M&E gap analysis” to work on these 7 steps.

The team of ActivityInfo would like to warmly thank Ms Susanne Neymeyer for this insightful and detailed guide on setting up a Monitoring and Evaluation system. Ms Neymeyer has been an ActivityInfo Education Partner since July 2020.

Susanne Neymeyer is an independent M&E consultant with more than 15 years experience in the field of development cooperation and humanitarian aid. Her academic background is in social work, adult education and evaluation. Susanne started working as an independent M&E consultant in 2009. Since then, she has evaluated and supported a wide range of development and humanitarian projects and programs all over the world. Before she became an international consultant, she worked for various international development and humanitarian organizations as project coordinator and manager.

Footnotes, references and further reading

• (1): In other guidance notes the development of an M&E-System is described in 6 or 10 steps. I have a preference for 7 steps, but a representation of the process with fewer or more steps is of course just as good.

• (2): SMART indicators are Specific, Measurable, Achievable, Relevant and Time-Bound. More information about SMART indicators and how to design them is provided by the INGO People in Need (PIN)

• (3): Van den Eynden, V., Corti, L., Woolard, M., Bishop, L. and Horton, L. (2011). Managing and sharing data: Best practice for researchers. UK Data Archive, University of Essex: Essex.

• (4): More information on innovative tools can be found at Glenn O’Neil. (2017) A Guide: Integrating Communication in Evaluation

• (5): An After-Action Review (AAR) is a simple process used by a team to capture the lessons learned from past successes and failures with the goal of improving future performance. More information on AAR and other knowledge sharing methods provides the Knowledge Sharing Toolkit developed among others by the Food and Agriculture Organization of the United Nations (FAO) and the United Nations Children's Fund (UNICEF).

• (6): IFRC. (2011). Project/programme monitoring and evaluation (M&E) guide

• (7): Ultimately, the degree of participation of supported communities will vary according to the project/program and context. Some examples of M&E participation include among other activities: vulnerability capacity assessments; involvement of local representatives in the project/program design and the identification of indicators; participatory monitoring where elected community representatives reporting on key monitoring indicators; sharing monitoring and evaluation findings with community members for participatory analysis and identification of recommendations.