Beyond data collection - Light for the World goes a step further with an information system that supports learning and adapting and monitors systems change at the global level
The ActivityInfo team would like to thank Mr. Klaus Minihuber, Head of MERLA and Knowledge Management and Mr. Nadir Abu Samra Spencer, Data Analyst for Monitoring, Evaluation and Research at Light for the World for their valuable insights in this Case Study. You can also watch the panel discussion “From Reporting To Decision-making: How Digital Technologies Are Evolving Data Utilization To Enable Better Decisions” that took place during the gLOCAL Evaluation Week 2022 for more information on Light for the World and their use of digital technologies.
Light for the World is a global disability and development NGO, supporting eye health services, education and work opportunities for people with disabilities. With offices in Vienna and other European countries, the organization focuses its work on countries in sub-Saharan Africa, including Burkina Faso, Ethiopia, Kenya, Mozambique, South Sudan and Uganda.
The organization collaborates with a wide range of stakeholders including local governments, UN agencies, companies, CSOs and grassroots organizations. The Light for the World teams work and advocate at the international, country and local level.
Strategic Monitoring & Evaluation, providing evidence on the effectiveness of their interventions, and maintaining an overview of the progress towards long-term goals and objectives are fundamental aspects for the organization. To achieve this, the MERLA team at the global office needs to be able to manage, interpret and assess the increasing amount of data collected as efficiently as possible.
For that, a data collection app or a reporting spreadsheet is not enough as the MERLA team needs a comprehensive system that goes beyond data collection.
A global MERLA strategy
In 2021, Light for the World developed their MERLA (Monitoring, Evaluation, Research, Learning and Adapting) strategy at the international level. The main focus of the strategy is on systems change, research, learning and adapting and the use of digital technologies to achieve these goals.
In November 2021, the organization decided to use ActivityInfo as the team’s MERLA platform. There was the need for an integrated digital system that would support the vision of the overall MERLA strategy. Since then, the ActivityInfo platform has been supporting this strategy by ensuring that data comes together in an accessible manner and can be integrated and analyzed easily. Despite the initial intention and following the initial deployment of the platform for the international MERLA team, country offices also started using the platform to manage their own data.
“In the beginning, ActivityInfo was meant to be used as an international database just for the international data but it quickly expanded and we are now actually making the next step of expanding its use in our country offices”, notes Mr. Minihuber.
Systems change: quantifying qualitative data to measure progress
The international MERLA strategy has set the ambition to monitor and evaluate the organization’s progress and contribution towards systems change in key areas such as inclusive education, economic empowerment and eye health.
In systems change, it is essential to cooperate with a variety of stakeholders at different levels to achieve change. To this end, the MERLA team is following the outcome mapping approach and has built a Global Results Framework with established Global Systems Change Outcomes and progress markers for these outcomes. In many cases, the Global Outcomes were built from the bottom up by the country teams and were then aligned and clustered together in Global Outcomes by the MERLA team.
The monitoring tool for this framework has been built in ActivityInfo and it already hosts hundreds of data sets. Using this system, the MERLA team can sit together with the country teams on a regular basis, go over the progress markers and measure change and contribution.
To make progress-towards-change measurable the team created a scoring table to assign values to the progress markers based on the change achieved. This way every progress marker can be scored. As a result, there are measures for every degree of change and this allows the MERLA team to do further analysis and measure the overall contribution of the organization. In addition, the MERLA team captures qualitative evidence on the scoring and information related to each specific case.
Having this tool in place, allows the MERLA team to uncover discrepancies in how contribution and progress are perceived among all stakeholders and then dive into the details of such cases and detect misunderstandings or actions that need to be taken.
Making key information for Research, Learning and Adapting available at all levels
The MERLA team also focuses on learning, research and adapting. The team aims to develop thematic learning and research agendas based on the data they already have at hand.
“The overall purpose of everything we do in MERLA is learning. And ActivityInfo is instrumental for that”, explains Mr. Minihuber.
A more specific example of how ActivityInfo supports learning and adapting is the following. As mentioned, the MERLA team is using the platform to support their Systems Change Outcome monitoring and mapping system. Having completed a mid-term evaluation of the country's strategies after two years, the team used ActivityInfo to collect qualitative data related to the evaluation internally.
This allowed the MERLA team to get information from major stakeholders which they had been trying to influence via systems change. All monitoring data were in one, common place and they could tap into the respective parts to inform teams at all levels, from international management to program management and from country to thematic team level.
“For qualitative data management, you just need to build in some predefined measures that are also quantitative. Then you can derive very interesting analysis or summarize work that is really complex and very diverse in itself into a graph. It reduces complexity a lot, but it also allows you and especially the more distant observer to really get the bigger picture and trends. Still, you can always dig deeper and compare the data sets and the qualitative information”, explains Mr. Minihuber.
Another example is the partner Output Monitoring system Light for the World has put in place to track reach figures across all countries they operate, at the global portfolio level. ActivityInfo is the core infrastructure for this M&E system, supporting its operationalization. This is important not only for monitoring and better program design and follow up but also for external communication and public fundraising.
It makes it possible for the whole organization to be on the same page as to where they stand and how things are moving forward as a whole but also for specific thematic areas such as eye health, inclusive education, economic empowerment and for specific aspects such as humanitarian action or gender.
“ActivityInfo was instrumental in providing a platform that would allow us to come together as an organization with diverse needs and work approaches; we share multiple languages, work ethics and structures. Some country offices are much larger than others and in some instances we focus our work on implementing partners. In other cases, we are the implementers. So there are really diverse needs and ActivityInfo was flexible enough to provide a core infrastructure that can be shared in a modulable manner with everyone. This allows us to speak the same language when we come together to discuss. And indeed, it also provides easier learning because we can also then analyze the data together, and also analyze the tools that we are using”, adds Mr. Abu Samra Spencer.
Having a common platform as a single source of truth allows the organization to look in detail into their processes. They are able to assess whether the data collection process is working, detect bottlenecks and factors affecting data quality as well as improve interaction with partners and data visualization.
“This goes far beyond the reach of ActivityInfo itself, but the fact that we have a solid infrastructure, that is simple and flexible in terms of the data you can put in and the data you can get out really facilitates all of that. In terms of the nitty-gritty work of a monitoring and evaluation group of people, it is interesting to see how once we have a tool that can be shared, ideas come together. By having ActivityInfo in a central role at the level of the international team, we have a practical focus for our discussions and that facilitates us coming together as a group of people with different experiences and skills and discussing how we can improve our M&E systems”, Mr. Abu Samra Spencer concludes.
Building a monitoring system for a global MERLA framework
Using digital tools to achieve the MERLA strategy objectives was part of the strategy itself for Light for the World who wished to move away from Excel spreadsheets exchanges and a less flexible custom built statistics tool. After researching possible solutions internally and externally, the MERLA team decided to try ActivityInfo for their international data monitoring because it was easy to adjust to their needs, there were no barriers to start experimenting with it and the customer support was engaged from the very beginning.
“I really liked the fact that it's a stand-alone system designed for Monitoring and Evaluation. I could shape it according to our needs and it wasn’t costly to get started. Also, the excellent customer support made us stay with it. It's the solution for getting rid of our Excel based data exchange which was not feasible nor sustainable anymore. And then it was part of our MERLA strategy to look for a solution. So it was convenient because it wasn’t a sophisticated system that covers other areas of project cycle management, something we didn’t need at that time. I really just needed a database to manage M&E data. And that's exactly what we got”, explains Mr. Minihuber.
An important factor that made working with multiple, varying data sets easy was the relational database model. Data security was also crucial as the organization did not wish to host data on their own servers. In addition, the available API allowed the MERLA team to explore new directions in data analysis and visualizations and the collection links made data collection much simpler for stakeholders who weren’t ActivityInfo users.
“The relational database is the core aspect of ActivityInfo along with the security associated with it. We are not hosting all the data on our servers. That makes everything a lot easier in terms of the underlying infrastructure. The API makes it versatile and allows us to explore new usage. All of the database visualization that we do rests on the API functionality. The API allows us to explore new directions and make ActivityInfo more central in our work. The collection link was also really useful for us because you send it out and the data comes in. It is also obviously easier because you do not have to manage all the users, you do not have to train them as much”, adds Mr. Abu Samra Spencer.
The platform accommodates the two core monitoring tools the MERLA team is using, SCOM (collecting and quantifying qualitative data to measure progress towards system change objectives) and APOM (output monitoring). The relational database simplified the system as it made it possible to add modularity to the data collected and link different datasets to each other based on relevance.
“I think this is the success factor; instead of working with a huge Excel spreadsheet and then having to sort of mentally keep track of how things link to each other, you could instead have part of the core information in one form and then the relational database would allow you to link it to another form based on whether a form is relevant to an answer. This reduced the information load on partners and colleagues”, explains Mr. Abu Samra Spencer.
The fact that data is immediately available allowing live tracking of reporting has significantly reduced the time otherwise spent in exchanging spreadsheets and chasing the latest updates.
“A very important aspect is that you have the data available immediately. I have live tracking of data reporting. Now we see the data flowing in when it comes in and all levels can look at it at the same time and then make changes. I think that from the collection and analysis point of view, that's a massive advantage. We still need to capitalize on that a bit more but it's definitely already visible”, adds Mr. Minihuber.
As for the initial setup process, the MERLA team started with a monthly subscription to experiment with the options and features and the ActivityInfo team supported the exploration phase whenever that was needed.
“For the support we received, we had a couple of calls to set things up and it was a really amazing experience given that we were just a very small client but still getting lots of guidance and support. That was really something that convinced me. Not treating a single user as less relevant to someone coming-in with a big demand. A good experience”, concludes Mr. Minihuber.
Country level M&E and data collection in a decentralized organization
At the country level, each country has its own MERLA team and can decide on what tools to use for their primary data collection system as the organization is quite decentralized. However, ActivityInfo is increasingly gaining traction at project level too and the international MERLA team is currently in the process of aligning the needs of the different country offices who wish to use ActivityInfo.
“We are a very decentralized organization. This is our strength and in some aspects also our weakness but in terms of ActivityInfo, it's mostly a strength because our colleagues in the country offices are autonomous and work with their national databases individually. So we do not have to keep oversight of every single little change that occurs and that is really useful. On the other hand, it requires an effort for alignment and for information flowing in collective learning”, explains Mr. Abu Samra Spencer.
“We did not enforce the platform to be used by country offices. Initially, we needed it as an internal tool for our international MERLA team to manage our data but now we see that it's increasingly gaining traction on the country level. So now we look into coding standards, user management and all these things because the demand is just increasing rapidly on the project and program level as well”, adds Mr. Minihuber.
The needs of the country level teams are different from those of the international MERLA team. They are more focused on the primary data collection aspect and less in the database management aspect. To this end, the MERLA team shares information and adapts the tool to meet these needs.
“So there is this adaptation phase that we are currently engaging with. We get requests by colleagues to show them what the possibilities with ActivityInfo are and how it could be used for specific tasks. It's quite interesting to see how this single tool started from the top and now is diffusing through the MERLA system. And we can later on come together and have this sort of collective learning and improvement of our practices, identify limitations and strategies to tackle these limitations, see which tools are complementary under which circumstances and so on. It's quite fascinating, the life, the shape it is taking”, notes Mr. Abu Samra Spencer.
The MERLA team is currently crafting a strategy on how the platform should be used by interested country offices. They look into ways to keep everyone aligned despite the wide variety of different activities and to ensure that the usage of ActivityInfo follows best practices. For example, finding the most effective form design for every case, connecting the platform to other analysis systems such as Power BI or R, detecting things that make this type of integration easier or harder, building a user experience for all users of the platform.
“Basically we have in mind to offer curated tools or strategies, documentation, training material, and an overall infrastructure, that makes it easier for people to then take up a different tool. So, we are at this moment where you can see that people are really interested in the tool but cannot yet, fully picture how they will integrate it in their work. Some have started, some not yet but broadly speaking there is demand. We are still working out how to provide the best possible user experience in the end. It is also a user experience from our end so that people can focus on content and carry out their work as easily as possible and do not have to reinvent the wheel. You can imagine how it would be, reinventing the wheel each time for data collection tools in 13 countries, so we are in this transition phase but it is very interesting to see the change actually happening”, explains Mr. Abu Samra Spencer.
At the same time the MERLA team leaves a lot of freedom for country teams to explore the capabilities of the platform.
“We try to map things out instead of providing rigid directives on how things should operate. In some cases, people take up new technologies, new approaches very quickly, in other cases it’s really not like that. Also it really depends on how good our communication can be with an implementing partner. When an implementing partner is a government, obviously it is not the same type of communication you can have and establish and maintain as when it is some like-minded civil society organization. So there is a lot of diversity out there. There is a lot of heterogeneity in terms of who the person entering the data will be, the capacity they have and the willingness they have for adopting a new system”, adds Mr. Abu Samra Spencer.
Conclusion
All in all, the MERLA team in Light for the World placed learning and adapting in the center of their MERLA strategy focus and leveraged the capabilities of an integrated information system to derive value for all stakeholders.
Moving beyond standard reporting and data collection for accountability, the organization uses key features in ActivityInfo such as relational databases, collection links, different types of user roles and access, and the API to combine and analyze large data sets so as to monitor their contribution towards systems change, assess the overall reach of their work, learn and adapt.
At the same time, viewing the increased interest for the platform from country offices, the MERLA team develops an organized, integrated approach to provide support and align the needs across country teams.
The ActivityInfo team would like to thank Mr. Klaus Minihuber, Head of MERLA and Knowledge Management and Mr. Nadir Abu Samra Spencer, Data Analyst for Monitoring, Evaluation and Research at Light for the World for their valuable insights in this Case Study. You can also watch the panel discussion “From Reporting To Decision-making: How Digital Technologies Are Evolving Data Utilization To Enable Better Decisions” that took place during the gLOCAL Evaluation Week 2022 for more information on Light for the World and their use of digital technologies.