MANAGING PERFORMANCE DATA
USAID’s renewed focus on rigorous M&E means that a significant amount of data is collected throughout the R/CDCS, project, and activity life cycles. The amount of data being collected raises important questions about how this data will be effectively managed and used for reporting, analysis, and learning purposes.
The Mission’s performance monitoring information system is a data system that serves as a repository for all performance indicators (at the strategy, project, and applicable activity/implementing mechanisms (IM) levels), including baseline values and timeframes, targets and rationales for targets, and actual values. The indicator data stored in the performance monitoring information system is an essential component of the PMP. Performance data is dynamic and will be updated as baselines are measured, actuals are collected, and
performance indicators are added, dropped or revised. At a minimum, performance data should be stored in Excel. Ideally, the Mission will have and use a performance monitoring information system (e.g., AIDtracker, FACTS Info, or another Mission system) that offers more functionality in analyzing the Mission’s performance data.
Some performance monitoring data must be reported to Washington under the Government Performance and Results Modernization Act (GPRMA), largely via the Operating Unit’s annual Performance Plan and Report (PPR). Other performance data is captured and reported via other processes, depending on legislative
requirements, presidential initiative requirements (e.g, Feed the Future), hard and soft earmarks, and to satisfy other technical, policy, and stakeholder needs.
Table 11 provides a summary of some of the different performance monitoring information systems currently being used by the Agency. Note that this is just a snapshot of different performance monitoring information systems available to Missions and does not include the many Mission-specific performance monitoring information systems currently being used.
OVERVIEW
Given the volume of data involved in the monitoring and evaluation of a Mission’s programs, it is imperative that these data be effectively stored and managed. This module provides an overview on managing
performance data, including available performance data systems and considerations for making sure that data is managed effectively.
TOOLS
Part 2 Module 10: The Mission’s Performance Data Management System
69
Interim Guidance on Mission-Level Performance Monitoring
Information Systems
Per USAID Standardization Project Notices, AIDtracker is suggested as an interim solution for Missions until a permanent Agency system is developed. It will include geo-mapping functionality as well as a field-tested Indicator Wizard. See ProgramNet for more information and FAQs on AIDtracker. Also, per Executive Notices, spending money to develop a new system to manage data is prohibited. Missions may continue to use an existing system, adopt an existing system, or adopt AIDtracker. Table 11: Example of USAID Performance Monitoring Information Systems
Information that a performance monitoring information system can help the Mission manage includes:
Data Quality Assessments (required for all indicators reported to Washington)
Indicator data inclusive of targets and actuals (including disaggregates)
Site visit reports
Implementing partner performance reports (monthly, quarterly, semiannual, annual)
Performance Indicator Reference Sheets (PIRS) Evaluations
Note that per the Mission Order on Performance Monitoring, the Mission must track the geographic location of each activity/IM at
the administrative level (state/province/department). Missions can require further detail or greater specificity in geographic location, if desired. USAID’s GeoCenter is available to provide expert consultations and capacity support for analysis of geographic information that aids in strategic decision-making.
Name of Report/System Description
AIDtracker
AIDtracker is a system managed by M/CIO that enables Mission‐level project and activity management and monitoring to include real-time status of project indicators, beneficiaries, and other frameworks. Mapping of project and activity data to user‐specific locations is enabled via a geographic “point and click” interface. See Annex 20 for a screenshot of AIDtracker.
Foreign Assistance Coordination and Tracking System (FACTS) Info
FACTS Info is a central USG data system that combines into one central repository all planning and tracking of foreign assistance funds over which the Director of U.S. Foreign Assistance (F) has authority. The system includes information for each country or headquarters office that manages foreign assistance funding. FACTS INFO allows both State and USAID to get data for all F managed foreign assistance funding in various ways in order to make decisions, prepare required OMB and Congressional reports, and respond to information requests.
Feed the Future Monitoring System (FTFMS)
FTFMS allows the tracking of FTF programming and M&E data, which
encompasses internal and mission Bureau for Food Security (BFS) programs, Global Health nutrition programming, and BFS funded development
programming for FFP. It also provides tracking for the 57 indicators that BFS uses to monitor their program's performance. This data is used to create a variety of reporting tools that BFS uses to measure progress and tell the FTF story as a bureau, agency, and a cross-Agency government initiative.
Helpful Hint
Per the Mission Order on Performance Monitoring, to ensure visibility and transparency the PMPOC will establish a common electronic location and naming conventions for all Project M&E Plans and Activity/IM M&E Plans. The PMPOC also will ensure that the latest Mission-wide PMP is stored in a common location. Similarly Project Managers and COR/AOR will also ensure that the latest Project and Activity/IM M&E Plans, respectively, are stored in the common location. UPDATING THE PERFORMANCE MONITORING INFORMATION SYSTEM
The COR/AOR/AM/G2G is responsible for ensuring timely data collection of performance data along the schedule outlined in the award agreement and Activity/IM’s M&E plan (quarterly, semi-annual or annual). After the COR/AOR/AM/G2G, in consultation with the Project Manager, reviews the data in implementing partner reports and verifies that the data is of acceptable quality and accurately reflects actual achievements, the performance indicator actual values need to be updated in the performance monitoring information system. Since indicator data is collected not just at the activity level but also at the strategy and project levels, a number of individuals across the Mission in addition to COR/AOR/AM/G2Gs, including the PMPOC, Project Managers, and DO team leads, may be responsible for ensuring that data for a particular indicator is collected and updated in the performance monitoring information system. The Performance Indicator Reference Sheet should clearly document who in the Mission is responsible for a given indicator.
Depending on the type of system the Mission uses to manage its performance data, in some cases the system will have a web-based “partner portal” which implementing partners can use to report data directly into the system. In other cases, data from partner reports will need to be manually entered into the Mission’s performance monitoring information system.
VALIDATING PERFORMANCE DATA
Even if the Mission uses a web-based platform to collect data from implementing partners, the
COR/AOR/AM/G2G and Project Manager should still regularly review and assess the “face validity” of the reported data or, in other words, that the data looks “right” and makes sense. If, for example, partner data suggests that crop yields increased during a period in which there was flooding, the COR/AOR/AM/G2G may want to contact the IP to confirm that the data reported is indeed accurate. For Missions that do not have a web portal for partners to enter data, it is recommended that the Mission use a standardized template to collect data from implementing partners in order to facilitate data review and aggregation across activities. DO Team Leaders/Project Managers should periodically review project performance data, checking for consistency and quality. The Program Office is responsible for ensuring that DO/Project Managers and COR/AOR/AM/G2G collect and review indicator data consistently, and that these data are entered in the performance monitoring information system on a timely basis. (See Module 2.7 for a more detailed discussion of data quality assurance procedures). Table 12 summarizes illustrative key Mission roles and responsibilities for performance monitoring data management. Mission staff should also refer to their Mission’s Mission Order on Performance Monitoring.
DATA MANAGEMENT BEST PRACTICES
Linking data collection with data analysis, reporting, and utilization is all part of the performance monitoring information system, and is therefore a critical part of the M&E process. The way that data is stored, managed, and accessed is a significant determinant of its utility. Consider the following to ensure that data remains secure, practical, and user-friendly.
1. Data format. Seek to record, store, and report data in standardized formats across Mission programs.
2. Logical organization. Organize data to facilitate easy analysis and reporting. For example, the Mission will likely want to be able to sort data by DO and IR, as well as by project and activity.
Part 2 Module 10: The Mission’s Performance Data Management System
71 collected data and ways to facilitate access and sharing across the Mission.
4. Data quality. The Mission Order on Performance Monitoring details the regular review and verification that the Mission should undertake of its performance data. In addition, the Mission should assess opportunities to foster a culture of high quality data collection.
5. Data accessibility. Where possible, utilize technology to organize and manage data. A common data system can help staff organize and analyze data, as well as highlight trends and identify emerging patterns.
Table 12: Key Roles and Responsibilities for Capturing Performance Data
Level Performance Monitoring Responsibility Who
Activity/IM
Collect performance data.
COR/AOR/G2G and Activity Managers Verify implementing partner performance reports.
Review data in IP reports or third-party sources.
Enter performance indicator actual values into the performance monitoring information system.
DO/Project Periodically review performance data, checking for consistency and quality across activities. DO Team Leaders/Project Managers DO/Project/Activity
/IM
Ensure that DO/Project Managers and COR/AOR/G2G collect and review indicator data consistently, and ensure that the data is entered into the system on a timely basis.
Program Office Review COR/AOR/G2G and DO/Project Manager indicator data for
quality and consistency. Non-project (e.g.,
Goal, context) Collect data as needed for non-project performance indicators (e.g., Goal level indicators and context indicators) or third-party data. Mission-wide Review data from third-party sources (e.g., M&E support mechanisms) SUMMARY
By now you should have an understanding of:
How and why we document performance data
The different platforms available to document and manage data for analysis and utilization Considerations for effectively managing performance data across the Mission
REFERENCES
ADS 203
Good Practice
While learning efforts often focus most directly on improving
development interventions, effective management and support operations are critical to the success of those very development interventions. Missions are encouraged to plan for and foster wide-scale systematic learning that also supports learning within the Mission’s support offices including the OAA, EXO, OFM, RLA and HR.