The Intel

Collecting evidence of performance is crucial for understanding the impact and effectiveness of development initiatives. However, traditional monitoring, evaluation and learning (MEL) systems often rely heavily on quantitative data and bureaucratic processes. These processes can be time-consuming, complex, and fail to capture the full picture of what is happening on the ground. Whatsmore, these MEL systems are generally entrenched in power structures that can limit the inclusion of diverse voices and perspectives, and risk being driven by compliance rather than a genuine desire for program improvement.

Better alternatives to the MEL-industrial complex should focus on shifting power dynamics, building trust and decolonising practices, enabling more voices to contribute to the evaluation process. Making MEL more accessible is key, and video can play a huge role in this. By listening to what people want to tell us, we can design MEL systems that are more inclusive and reflective of the true impact and outcomes of development initiatives. This requires an exercise in deep listening and recognition that traditional MEL approaches may not tell the full story.

To truly understand impact (beyond MEL frameworks), we need to focus on the authentic, lived experiences of people. This involves facilitating platforms for them to share the information that matters most to them. By incorporating community voices into evaluation processes, we can drive better outcomes and make more informed decisions. Shifting power ensures that evaluation processes are not only more inclusive, but also more effective in capturing the true impact of development initiatives.

Weakness in monitoring, evaluation and learning (MEL) is regularly cited as a key reason for the Department of Foreign Affairs and Trade (DFAT) identifying development investments as unsatisfactory. The MEL-industrial complex is one contributor to this, but performance evidence alone cannot produce more equitable or effective programs. Better practice is contingent on how, when, where and by whom MEL is commissioned. The key question is not ‘is MEL done well’ but ‘how is it used’ by managers. 

The MEL-industrial complex tends to silo performance evidence in an ivory tower. Issues with commercial in confidence reports and inaccessible DFAT information make richer discourse between actors in the development community difficult. At the policy level for example, there is a lack of dialogue between programs, stakeholders and DFAT to generate evidence on the strategic and foreign policy outcomes latent in the aid program. Much is discussed implicitly. Whilst there is no one way to get meaningful and accurate evidence in front of a decision maker, a more open and organised approach to information management would be a good start. 

That said, what this all boils down to, is that good MEL is good management. Without integration into ongoing program management, outsourcing performance evidence collection can be both performative and costly. Some types of MEL require niche skills; these may lie in both external consultants and communities. However what we really need is decision makers of all life experiences with the social and analytical smarts to question how a program is tracking against its objectives, and who turn to evidence to make their decisions. Development needs critical friends, but performance evidence is everyone’s business.

There is deafening dissonance between the aid sector’s proclamation that performance evidence is vital, and its limited acknowledgement of why it is often absent. This is reflected in the thin enamel of sophisticated-sounding monitoring, evaluation and learning (MEL) tools invariably proposed in program design, only to be chipped away by uncomfortable operational truths come time for the mid-term review.  

Rectifying this is less about more MEL or better MEL. Instead, it means breathing life into the widespread recognition that effectiveness of aid depends on effectiveness of the organisations delivering it.

First, Government needs to better understand the operational implications of key practices and systems through which aid is delivered, and which are more amenable to redress – procurement regulations, for example. This tends to be impeded by the bureaucrat’s inclination to prioritise the sanitised, high-level arena of policy-making and strategising over the messy details of implementation.

Second, MEL experts and organisations need to calibrate the depth and breadth of our analysis to the issues at hand (puzzles or mysteries?). This will help us to determine our tool of analysis and how they can be used for good judgment and decision-making. 

Third, development organisation leaders must dive deeper into the black box of organisational culture that penalises humility, stifles durable contestation, and provides fertile ground for baseless blagging. It is encouraging to see organisational culture become more prominent in discussions on aid effectiveness in recent years. But we must work to place this concept more firmly in the operational hands of our leaders – at all levels – as a vehicle for shaping organisational functionality on a daily basis.