Performance Measurement as a Critical Leverage Point for Improving International Development Impact

Individuals and organizations need good performance feedback in order to be effective and meet their objectives. In the private sector there is the luxury of sales, profit and other clear measures that indicate output and outcome performance. These measures are used to identify and reinforce successful business patterns, such as effective strategies or high performing individuals, and to disrupt and correct unsuccessful patterns. They are critical to learning, development and long-term success. To survive, private sector organizations have to meet the performance expectations for these measures that are set by their funders. This mechanism helps to ensure that high-performing organizations and individuals are able to succeed and expand their impact while low performing organizations and individuals are not. 

Unfortunately, measuring output, and particularly outcome performance is very difficult and generally poorly done in the international development sector. Outputs are often complex and highly variable, there are rarely any clear measures of outcome performance and outcomes are usually affected by a vast array of factors outside the control of any single organization or even the international development sector as a whole. In the absence of high quality output and outcome feedback information, people naturally focus on things that are easy to measure, such as activity and costs. 

Consider this example. You have 100 euro to donate to charity. Care says that 95% of your money will go directly to projects for people in need. Save the children says that 50% of your money go directly to projects for people in need. Given this choice, most people would immediately say "that's a no-brainer, I'll give my money to Care!". However, without knowing the track record of the two organizations for meeting outcome objectives, how do we know that a 5% investment in organizational capability will benefit those in need more than a 50% investment? How much should be invested in understanding the dynamics of the problems affecting the people in need? How much should be invested in training staff, in supporting technology, in measuring performance, etc? 

Without reliable performance feedback, no one knows. So, the overriding assumption that drives behaviour in the system is that more projects is good, overhead is bad. The consequence of this dynamic is that international development organizations almost invariably over extend themselves across too many projects and massively under invest in long-term strategic capabilities. It is for this reason that I have been trying to help international organziations, and the system as a whole, improve the way that outputs and outcomes are measured, and how this information drives accountability, learning, improvement and long-term impact. I believe that this is a critical leverage point for achieving rapid and dramatic improvements in the performance of the international development system as a whole. 

In a future blog I'd like to discuss some of the areas of investment I think are required, but a few of the key ones are robust measurement methodologies; a conceptual framework for continuously improving the collective understanding of the systemic dynamics affecting development outcomes; widely adopted data standards; a culture and practice of data sharing within and across organizations; tools to facilitate data and knowledge generation and sharing; and a culture of accountability. 

With regard to the tools, I have been working on some ideas for improving the efficiency and effectiveness of collecting the data required for output and outcome performance feedback. I have been working with a technology called xforms, which enables the creation of very user-friendly electronic forms that can be accessed through a web browser and used off-line. Using electronic forms makes data collection easier and faster, improves the quality of data and enables broader data sharing. 

A number of experiments using electronic forms are being conducted but the forms are typically on a small hand-held device and the small screen is a major constraint. I have been thinking that the ideal interface for the forms would be an A4 or Letter sized touch screen tablet computer. It would only need enough power to run a cut-down version of Linux and a Firefox browser. It could have a relatively small flash memory card for storage and wouldn't need most of the bells and whistles that make computers complicated and expensive today. 

One option I've been thinking about is the XOXO pc from the One Laptop Per Child project. It's their second generation kid-focused laptop that is under development. It has two touch screens that fold together. One of the screens acts as a keyboard when necessary. The attraction of this device is that it is cheap, has the specs required and enough screen real estate to make form completion fairly easy. It also is a device that hopefully will be very common and familiar in the settings where the data collection would take place. 

Another option is an initiative launched by Michael Arrington of Tech Crunch to produce a "dead simple web tablet for $200". This is only at the dream stage but would be a fantastic device for field data collection. Check out the initiative here. Search for "Fraser" to see my comments on this blog where I describe how the device could be used by UN agencies. After a wise crack that the device could be developed with UN slush funds I had to come to the defence of the UN because there is such low awareness of what the UN does and the positive impact that it has.