From global indicators monitoring to strategic evaluation and learning - how a common MIS platform is enabling IREX’s global MEL strategy

Download printable version

The ActivityInfo team would like to thank Mr. Charles Guedenet, Senior Technical Advisor for MEL and Ms. Sharon Lang’at, Data Management and Learning Specialist, at IREX for their valuable insights in this Case Study.

You can also watch a presentation by IREX where representatives of the organization discuss how they approach Monitoring and Evaluation and Information Management at global level and share insights, lessons learned, and best practices as well as their experience in introducing ActivityInfo into their processes.

Information management and MEL needs in IREX

IREX is a global development and education organization that builds more just, prosperous and inclusive societies around the world. With an annual budget of more than a hundred million dollars and approximately 700 staff, it works with partner organizations in over 100 countries in every continent. IREX works in four areas essential to progress:

Cultivating leaders at all levels of society
Strengthening educational, civic, and governing institutions
Engaging and empowering youth
Extending access to quality education and information

As a mission and strategy-driven organization, IREX engages in evaluation and learning related to its organizational priorities and at the same time supports programs to engage in evaluation and learning related to their specific programmatic needs. IREX’s diverse portfolio of programs and the diversity of contexts and countries in which it works requires an adaptable Monitoring Evaluation and Learning (MEL) and information management strategy. Country contexts and program teams have different challenges and opportunities and a range of data and learning needs. To meet these diverse needs within and across programs, and at the organizational level, a strategic MEL system that is adaptable while maintaining standards around data quality and security is needed. In the words of Mr. Charles Guedenet, Senior Technical Advisor for MEL at IREX “a one-size-fits-all data management system does not exist”.

Following the launch of IREX’s new 2025 strategy and, in parallel, a new approach to strategic evaluation and learning, IREX’s Center for Applied Learning and Impact (CALI) decided to pilot ActivityInfo in late 2020. ActivityInfo is now the standard database system used by all IREX programs and centrally for global and strategic evaluation and learning.

A bottom-up approach: moving away from standard indicators for all programs to program-specific indicator data for cross-program learning

IREX’s current system for cross-program and institutional level evaluation is the result of many years of experimentation with different approaches and data management platforms. IREX Global Indicators were originally conceived in 2016 with the goal of tracking global reach and key outcome markers of progress relative to the organization’s mission. Objectives also included stimulating learning and high-level reflection, supporting strategic communications, and strengthening institutional capacity to monitor and evaluate impact. A Salesforce-based database was developed with support from consultants and programs were required to report on the Global Indicators quarterly. However, using Global Indicators to evaluate reach and outcomes across programs was limiting in many respects –a lack of granular data on program-level impact and the additional reporting burden on programs, for example. The Salesforce database was also perceived as inflexible and, because it was developed using code, IREX felt overly reliant on Salesforce experts to maintain and update it.

Following an internal review of the organization’s rollout and use of Global Indicators and interviews with peer organizations in 2018 and 2019, IREX decided to change its approach. Global Indicators and the Salesforce database were not sufficiently meeting the diverse data management and learning needs of IREX programs and the organization’s leadership.

In 2020, following the launch of the organization’s 2025 strategy, IREX launched a new strategy for evaluating the IREX 2025 Strategic Plan (SP). To address shortcomings of its former database system identified in its internal review, IREX also piloted ActivityInfo at this time, eventually adopting it in 2021 as its centralized database system.

A timeline for moving from standard global indicators to strategic evaluation and learning

Putting a Strategic Evaluation and Learning (SEL) system in place to meet global MEL needs

IREX now follows a multi-level strategic approach to global MEL that starts with strengthening program-level MEL to improve outcomes and to enable more robust cross-program and institutional-level evaluation. A centralized data management system like ActivityInfo has been crucial to carrying out this vision of connecting program-level data to the global strategy and to building a culture of learning and data informed decision making.

According to Mr. Guedenet:

“Previously, we were using Salesforce just at the institutional level. Now we are able to collect program-specific indicator data at the program level and connect that to the global strategy. This data is also accessible to more people so that they can query it for cross-program learning. Right now we have about 60 active programs using ActivityInfo and there are up to one or two new programs in startup each month. The ActivityInfo database system has helped us support programs with their data management from the very start, setting them up for more robust data analysis, adaptation and reporting throughout the program lifecycle. This is helping raise the quality of MEL across all programs. And lastly, having a centralized data management system with more accessible, timely data that can be used to tackle some of those cross-program learning questions supports our organizational goal of strengthening our culture of learning and data informed decision making.”

IREX’s new SEL approach moved away from collecting data across programs on a set of standard Global Indicators to 1) collecting program-specific indicators and 2) conducting cross-program evaluation studies. Program indicator data is collected quarterly, then coded to outcomes in our strategic plan and analyzed to evaluate progress toward our strategic plan. IREX also conducts cross-program evaluation studies, recognizing that some strategic outcomes cannot be evaluated with indicator data alone.

A central component of its SEL approach has been requiring that all programs enter and manage their indicator data in the ActivityInfo database. Programs are additionally encouraged to and supported with creating their own databases to manage and analyze other program MEL data more effectively. Having all program data available in one centralized platform allows different stakeholders at IREX –from programs to Practice Area Directors and the Strategic Communications Unit– to query the database for relevant data that will meet their specific learning needs.

IREX’s vision to create a system that eliminated duplicative efforts, streamlined MEL processes, and met diverse stakeholder needs has required a whole system approach. This approach includes incorporating IREX MEL standards and institutional learning priorities in proposals and active programs, mobilizing MEL experts and the appropriate technology at program startup, and capturing data and learning at closeout. All parts of the program life cycle are impacted. To make this vision a reality, multiple experts and teams at IREX are also engaged, including the Information Technology (IT) team, Global Programs leadership, program teams, especially MEL staff, and the proposal development team.

Creating a global MEL system - IREX’s SEL System

Piloting ActivityInfo as a global MEL system

IREX started piloting the ActivityInfo platform in late 2020 with 10 users who tested the data entry forms. Program indicator data was also imported into ActivityInfo from over forty programs and coded to strategy outcomes. Based on learnings from the pilot, the IREX team revised data entry forms, created easy-to-use import templates in Excel, and drafted instructions for entering or importing data. Today, over 65 programs are entering indicator data directly into the ActivityInfo platform that has 115 active users.

In addition to using ActivityInfo for indicator data, a number of early-adopter programs started using the platform in 2021 to manage, analyze, and report on other MEL data such as training participation, survey results, and assessment data. Today, all active programs are encouraged to migrate their data to the platform and new programs are setting up their databases in ActivityInfo right from the start. In 2023, IREX moved from piloting and scaling up ActivityInfo across all programs to ‘institutionalizing’ it by taking steps to strengthen the system across multiple areas.

From a pilot exercise to an institutionalized solution

To support the broader integration of the ActivityInfo platform, IREX is targeting 6 key areas: program adoption of the platform for all MEL data, data consolidation, data access and use, expert technical support, data quality, and data-informed insights and learning.

Institutionalizing a MEL IM system by addressing multiple key areas

Program adoption

During the pilot and the first year of scaling-up the platform, the use of ActivityInfo beyond indicator data management was limited to a small group of early-adopters. As IREX began to institutionalize, it invested more resources (e.g., tutorials, help guides, database templates) to support the development of program-specific databases for all new programs during startup. Additionally, support with migrating data is being provided to programs that are in full implementation and have already been collecting data. Program adoption is being accelerated now that ActivityInfo has become the default go-to data management system for the organization.

Data consolidation

The organization is also exploring ways to consolidate data in ActivityInfo with other sources across the organization to facilitate access to data for learning and decision-making. Examples include exploring ways to integrate internal program reporting within ActivityInfo and using tools like Power BI to join data from ActivityInfo with data in other platforms such as Microsoft Sharepoint and online survey tools. Data consolidation is breaking down data silos between units or teams that use different platforms, reducing duplicative data entry, and facilitating information-sharing across the organization.

Technical support

IREX uses a network approach to support ActivityInfo users, drawing on a mix of MEL and technology experts to meet different needs. Program MEL teams vary in size and not all staff are familiar with database systems. Ms. Sharon Lang’at, a Data Management and Learning Specialist, was recently hired to provide custom technical and MEL support to programs interested in developing their own databases. For support needs related to quarterly indicator data reporting, programs can also reach out to designated MEL experts in their Practice Areas i.e., program groups that share common approaches or goals. At a global level, Mr. Guedenet is responsible for developing the overall data management strategy, testing new features or resources, gathering and learning from feedback, and tracking progress towards short and long-term implementation goals. Finally, Mr. Guedenet and Ms. Lang’at work closely with an implementation specialist at ActivityInfo who is available to address technical questions and requests and to develop IREX’s strategy and implementation plan.

“One of the reasons we selected ActivityInfo is that there is very little to no coding needed. It’s fairly intuitive. However, if you want to go beyond just managing indicator data, and certainly if you plan to migrate all your MEL data, it is good to have some technical support from someone with expertise and experience using the platform. We are building out our in-house expertise to better support programs and also have a customer success package with ActivityInfo that gives us direct access to an ActivityInfo implementation specialist as well as support with an implementation strategy. We can talk to the ActivityInfo team for help and if there is a feature we would like to see, we can bring it to them.” Mr. Charles Guedenet, Senior Technical Advisor for MEL at IREX.

Data quality

The utility of the database system for learning and decision-making is only as good as the data that is entered into it. For that reason, ensuring the quality and completeness of MEL data in ActivityInfo has been a priority. IREX has approached this need by promulgating MEL and data standards, building-in validation and other controls in the database to reduce errors, and creating dashboards in Power BI to monitor and flag missing data in near real-time.

Learning from data

IREX has been engaging in a series of activities and experiments to make the data in ActivityInfo easily accessible and useful within and across programs for learning and decision-making. An important first step was integrating ActivityInfo with Microsoft’s Power BI tool to create dashboards that refresh daily with new data. These interactive dashboard reports make it possible for anyone in the organization to use filters, word searches, and tags to easily search for program data salient to different learning needs. For example, an initiative to advance Gender Equality and Social Inclusion (GESI) in IREX programs uses indicator disaggregated data and GESI-tagged indicators to understand what data programs are collecting and to advance conversations around how, when, and why programs should collect this data. Other examples include evaluation studies for IREX Strategic Plan outcomes where indicator data is being used to identify programs to include in studies, and for proposal development where program data can be used to describe IREX’s reach and outcomes within and across similar programs.

“Some of our data users have identified specific areas that they are interested in. In the area of Gender Equality Social Inclusion (GESI), for example, we started tagging indicators that are measuring a GESI outcome so that we can more easily filter data and see just those indicators. We have a database of 1000+ program indicators so we need ways to quickly find the data we are looking for. We do that by adding these tags.” Mr. Charles Guedenet, Senior Technical Advisor for MEL at IREX.

Finally, data collected and managed in the ActivityInfo database system is used to analyze progress towards IREX’s 2025 Strategic Plan outcome targets. IREX analyzes the data, drafts an evaluation report, and shares results via an interactive dashboard. Using ActivityInfo’s API to work within the R programming language has been particularly useful for quickly analyzing and summarizing indicator data across the organization.

“ActivityInfo makes it possible to not just push data to Power BI for dashboard reports but also to work with data in R programming language environments. This is a big pro for IREX as it opens up a lot of possibilities for automating processes around data analysis, visualization, reporting, and consolidation with other data sources.” Charles Guedenet, Senior Technical Advisor for MEL at IREX.

Rolling out ActivityInfo to programs to meet their data and learning needs

By the end of 2022, IREX had started rolling out ActivityInfo at the program level. By the first quarter of 2023, 8 programs had begun developing their own database to manage, analyze, and report on their MEL data. Due to the diversity of IREX’s program portfolio, both the technical support and the data needs vary greatly. Therefore, while all programs are required to manage their indicator data in the system, their integration of other MEL data in ActivityInfo is customized to their specific needs. Here are a few examples of how programs are using ActivityInfo at IREX:

Pre and post test analysis: Leveraging ActivityInfo’s survey and data analysis tools, one program used the system to conduct pre- and post-test surveys and analyze the results.
Tracking and analyzing training participants data: Programs that run trainings for partner organizations or individual leaders in conjunction with other activities use ActivityInfo to track unique participants, avoiding a common challenge of double counting and making use of ActivityInfo’s functionality to collect signatures, images, and other documentation.
Collecting and managing partner or grantee data and reports: ActivityInfo can be used to collect data directly from partner organizations and grantees. However, in other cases, programs have preferred to collect partner reports in excel and word, conducting a review before uploading final results into ActivityInfo.
MEL indicator data analytics and visualization: Programs are also making use of built-in tools in the platform such as Notebooks and Dashboards to visualize and present data; others use R and Power BI integrations to query data outside the platform.

Onboarding process for programs

IREX has been developing its process for onboarding new programs that express an interest in using ActivityInfo to monitor their program MEL activities. The process includes an orientation call to introduce programs to ActivityInfo and discuss specific program needs followed by the design and testing of a dummy database before building out the actual database design and providing training. The development process is iterative with multiple cycles of design, testing, learning, and improving. Program teams that adopt ActivityInfo have had an MEL officer responsible for working closely with Ms. Lang’at to develop their program database. Ms Lang’at describes the onboarding and development process:

“We start with an orientation call. Once a program’s MEL plan has been approved, they reach out to us and we do an orientation of the platform.This session familiarizes them with the platform and its functionalities.

Following the orientation, we create and send a dummy database to the program. The purpose of this dummy database is to allow them to interact with it and explore its features before proceeding to the next step, which involves designing the actual database. We believe it is important for them to be actively involved in the database creation process.

During the database design phase, we guide them through the creation of the entire database schema. This includes creating forms, tables, and establishing database relationships. If they intend to utilize the database for data collection and visualization purposes, we also provide training on these aspects. Throughout this collaborative process, our goal is to empower them and ensure they will be capable of independently managing the database with their MEL team.

Subsequently, we conduct a pilot of the tool before commencing data collection. This step ensures that the database is functioning correctly and that all integrity features and validations have been appropriately configured. The nature of the pilot depends on whether the program will collect the data themselves or if their partners will directly input the data into the database.

Once the pilot phase is completed, we offer training on database management practices. This training is designed to equip them with the necessary skills to access the tool, understand data security within the platform, and address any potential challenges that may arise.

Finally, we provide ongoing support to the program's team. If they encounter any issues or require assistance, they can reach out to us for guidance and resolution. Our aim is to ensure a smooth and successful implementation of the database and support their MEL efforts effectively.” Ms Sharon Lang’at, Data Management and Learning Specialist, at IREX.

Database development process for programs in startup and implementation

The onboarding steps are similar for programs in active implementation. The main difference is that a data migration plan is crafted with the program team for moving the data from previous systems, such as Excel or Salesforce, into their ActivityInfo data. At this stage, the organization remains flexible as to the kind of data the programs would like to add to the database.

“The migration plan serves as a systematic reference for the program's MEL teams, outlining the steps involved in data migration. For new programs, we guide the MEL team through database design, migrate the data, conduct validation, and provide training on database management practices. Once completed, we hand over the migration plan and continue supporting the team as needed. This includes troubleshooting, addressing queries, and offering guidance on data management best practices, ensuring a smooth transition and ongoing support for the MEL team.” Ms Sharon Lang’at, Data Management and Learning Specialist, at IREX.

Conclusion

With the rollout of its 2025 Strategic Plan in 2020, IREX doubled down on its commitment to being a learning organization, introducing a new strategic evaluation and learning approach that met both program and organizational-level strategic needs and investing in a centralized database system that is increasing data access and use.

"We settled on ActivityInfo for our information management system after researching several options. Two important criteria we were using to assess our options included ease of use –we didn’t want it to require coding experience– and wide applicability for different users and use cases. What has been most exciting about the adoption of ActivityInfo is how programs are learning on their own how to adapt it for their needs or to integrate it with other tools and then teaching us something new!" Mr. Charles Guedenet, Senior Technical Advisor for MEL at IREX.

The ActivityInfo team would like to thank Mr. Charles Guedenet, Senior Technical Advisor for MEL and Ms Sharon Lang’at, Data Management and Learning Specialist, for their valuable insights in this Case Study.