The impact of digital technologies on Monitoring and Evaluation; cases from the field, country offices and HQs

Traditionally, M&E activities have primarily revolved around accountability focusing on reporting to donors and what has been achieved with the funds provided. However, thanks to digital technologies, more sophisticated data management capabilities are available and this is enabling a shift. More organizations are now empowered to move beyond collecting data for donor-centric reporting and into more advanced data analysis and that allows them to unlock the insights that inform important decisions about how programs should be run.

We had the opportunity to explore this shift with key stakeholders coming from a variety of humanitarian and development fields in a panel discussion during the gLOCAL Evaluation Week 2022. Megan Passey, Head of Knowledge and Learning at International Cocoa Initiative, Klaus Minihuber, Head of Effectiveness and Innovation at Light for the World, Shodmon Hojibekov, Chief Executive Officer at Aga Khan Agency for Habitat Afghanistan and John Paul Nyeko, Senior Monitoring and Evaluation Advisor at AVSI Uganda shared their insights on challenges and opportunities they came across during the digital transformation journey in their organization and in which cases ActivityInfo supported this transition. You can watch the recording here.

The discussion revealed some interesting insights such as:

Moving away from M&E paper-based systems:

  • improves data utilization, decision making and the quality of programme delivery
  • removes data custodians and reduces risks related to data availability due to staff turnover
  • saves a lot of time and is more cost effective in the long-term
  • is more environmentally friendly
  • significantly improves data protection

Replacing M&E spreadsheet systems with web-based technologies:

  • removes collaboration obstacles such as parallel systems and outdated files
  • reduces data cleaning and harmonizations needs
  • enables information flow to and from all directions
  • reinforces team participation on data related activities
  • saves time and resources leaving time for learning from and acting upon collected data
  • supports understanding better performance in relation to strategy

Moving away from spreadsheets increases opportunities for learning from program data both at HQ and field office level

Using spreadsheets for reporting can become cumbersome and impractical for the global team of an organization when working with large, multilingual teams submitting data from various locations around the world. Implementing partners might start adjusting the reporting spreadsheet according to their needs, by-passing protected sheets and creating parallel versions of the same system. As a result, the M&E officers in the global office would need to spend time sorting out files, looking for the most updated version, cleaning data and integrating various files into one instead of investing their time in learning from and acting upon the collected data.

Mr. Minihuber, working with the global team in Light for the World, walked us through the shift from spreadsheets to ActivityInfo and the way technology is assisting the updated MERLA strategic process which is currently being rolled out in the organization. Feeling insecure about the completeness, validity and recency of the data submitted by the local implementing partners via spreadsheets, the global team needed a new way to summarize core indicators across different programs and countries at the global level. A year ago the organization adopted ActivityInfo to collect simple output indicators such as beneficiaries reached by different activities in each country and soon expanded the use to include the systems change outcome monitoring framework too. Using more digital tools such as Power BI and the ActivityInfo API, the global team became more creative with the data collected. Then, by allowing the data to flow back to the implementing partners they supported their learning too. Everyone in the organization is now able to view trends and how the programs evolve in a multi-annual view thanks to these integrations. The global team produces reports tailored to specific audiences and provides easy access to all local partners. Thanks to digital technologies, Light for the World managed to move a step further from reporting for accountability and today they have sessions on country and global program level so as to analyze the progress of their programs, act upon the data and achieve more qualitative monitoring of progress towards systems change.

Moving away from paper-based M&E systems is cost effective and a catalyst for decision making

The digitilization of the monitoring and evaluation system in an organization can significantly improve data utilization, decision making and the quality of program delivery. Instead of moving around paper forms and spending valuable time in cleaning data and catching up with deadlines, organizations shifting to digital tools can invest that time to improve their efficiency.

Mr. Nyeko and the AVSI Uganda team identified the opportunity and decided to rethink their M&E approach and switch from a paper-based M&E system to a digital one so as to learn faster from the data collected and adapt their actions accordingly. Combining digital tools such KOBO, ODK, visualization tools and ActivityInfo as well as smartphones the AVSI Uganda field teams were able to collect data and receive feedback faster. At the same time, all stakeholders were able to access data and analysis relevant to them. In the field, critical service delivery became faster and more cost-effective thanks to maps which indicated the most efficient ways to combine service delivery points with beneficiaries as well as field staff with participant households. Digital technologies also played an important role during the COVID19 pandemic. Thanks to digital tools, services were delivered remotely and the web-based ActivityInfo platform allowed the team to continue monitoring emerging issues such as GBV and nutrition issues and manage referrals despite the limitations on group gatherings. You can learn more about how AVSI in Uganda elevates evidence-based M&E with a centralized MIS platform in our Case Study.

In Afghanistan, the significantly wide geographic coverage and reach of the programs created similar challenges for the Aga Khan Agency for Habitat Afghanistan according to Mr. Hojibekov. Apart from the unavailability and low quality of the data gathered, the paper-based system resulted in significant expenses too. Following a strategic moment of reflection, the agency decided to adopt a systematic approach to address information gaps, data custodians and data quality. By making ActivityInfo the backbone of the M&E activities, the agency improved data utilization and accessibility, and reduced expenses related to stationary, paper supplies and long-distance travel by 50% while supporting the commitment to achieving net-zero carbon operations. The M&E team achieved more impactful programming as it was possible to see clearly how various variables affect relationships and which areas need more investment. At the same time, they improved data security and data acceptance among reporting teams and this improved the quality of the reporting. You can learn more about Disaster Risk Management and the COVID-19 Response in Afghanistan by the Aga Khan Agency for Habitat Afghanistan in our Case Study.

As far as data security is concerned, Mrs. Passey from the International Cocoa Initiative underlined the importance of data confidentiality and how all partners need to get official assurances such as a data sharing agreement about the data that might be shared to the public following reporting. This is important when a system collects sensitive data but at the same time there is the need to showcase the progress of the organization's work by sharing for example aggregate figures.

Adopting digital technologies leads to more efficiency thanks to the decentralization of processes

Mr. Minihuber, Light for the World, explained that adopting a digital tool such as ActivityInfo allowed the organization to increase their efficiency because they no longer needed a centralized team to take care of the reporting activities. Instead, thanks to the platform, the entire organization, global team and local partners included, has a consolidated centralized view of every program and processes are decentralized. All stakeholders can be aware of how efficiently they move in relation to their strategy on a thematic, country and global level.

In the Aga Khan Agency for Habitat, Afghanistan, thanks to ActivityInfo, M&E people can access different types of data based on their needs and permissions and they can use it for various purposes. The challenge of working in widespread geographies and under challenging conditions is addressed with a centralized system which allows offline reporting and paper-based reports are replaced with links to reports that share real-time data. Then, the whole process of cleaning, processing, analyzing, visualizing and disseminating data becomes more automated and faster. Moreover, in instances of big staff turnover, the data remains available and functional on the platform and this supports business continuity within the organization.

Time to go digital

Compared to traditional paper-based methods, the digitalization of the monitoring and evaluation system allows for data to be updated in real-time, be accessible by multiple parties, and eliminates spreadsheets chaos. As such, deploying digital systems for monitoring and evaluation is a step into the future as it improves the quality of the data collected and reduces time and resources spent on data cleaning, which can instead be invested in decision making and in improving the quality of program delivery.

The ActivityInfo team would like to warmly thank all the panelists for participating in this discussion and providing valuable insights for the creation of this article. We hope that, by sharing these knowledge, other organizations will be inspired to adopt digital approaches within their M&E system to unlock greater potential for learning and impact.