Evaluating OCP’s work to turn natural resource revenues into effective public infrastructure and services

For the past five years, OCP has been working in resource-rich countries around the world to help governments spend their money more effectively. As the project funded by the BHP Foundation’s Global Signature Program on Natural Resource Governance comes to a close, Oxford Insights has independently evaluated successes, gaps and lessons learned across the program drawing on interviews with OCP partners from government, civil society, academia and journalism across program countries. As part of this final evaluation, we also looked into the potential to use open contracting to promote environmental sustainability and a just energy transition – a particularly crucial concern for resource-rich countries.

Here are our main findings and recommendations.

Overarching successes: achieving impact, increasing subnational engagement, and a stronger focus on green and equitable procurement.

The evaluation confirmed that OCP has met, and will perhaps even succeed, its overall target for quantitatively demonstrated impact as part of the grant agreement. OCP has documented seven instances of impacts in Colombia (2); Chile; Buenos Aires, Argentina; Indonesia; Ecuador; and Nuevo Leon, Mexico.

OCP also successfully responded to recommendations made in the midterm evaluation just over two years ago, including increasing engagement with partners at the subnational level, in particular in Nigeria, Colombia and the US; and supporting partners to make more immediate use of open contracting data to meet their present needs. 

Over the past two years, OCP has successfully also expanded its focus to include green (or environmentally sustainable) procurement practices and social and economic inclusion, with partners exploring initiatives in both areas.

Finally, OCP has made progress scaling up, as the evaluation found evidence of OCP projects being independently replicated elsewhere. 

Overarching challenges: What’s holding back faster progress

Partners shared what is holding back their work towards progress and impact. These barriers include: 

• government officials lack time and resources to devote to open contracting; 
• poor quality of published open data; 
• low levels of data literacy in government and civil society; and
• changes of political administration, along with regular turnover in government roles. 

Overarching recommendations

Based on the insights from interviews with partners and desk research, Oxford Insights has given OCP several recommendations that could help address these barriers:

1. More showcase projects on climate and inclusion: OCP should use the Lift impact accelerator program to further build an evidence base around green procurement and open contracting for social inclusion, by working with governments and civil society organizations to compile success stories and lessons learned. 

We found that OCP’s Lift impact accelerator has already been particularly successful in helping partners to explore more targeted objectives, such as anticorruption, green procurement, and social inclusion initiatives. Lift can provide the framework, resources, and project management support to allow partners to experiment with new applications of open contracting principles.

2. Refine impact definitions and timelines: Acknowledging its high bar for impact, OCP should consider how it can ensure that it doesn’t risk deprioritizing work with partners who could reach impact, but across a longer timeframe. 

The evaluation found that OCP’s strict definition of impact didn’t resonate with all partners, particularly those working in contexts where open contracting is less mature. While we recognize the need for OCP’s strategic emphasis on measuring impact quantitatively, OCP should still offer limited support to lower-capacity partners who find themselves earlier on the path toward impact. 

OCP should also make a renewed effort to foster and celebrate the progress made by other partners reaching intermediate milestones on the path towards impact – it has begun to do this since the 2021 Strategy Refresh, especially through data use and progress stories documented on the website, but more can be done to celebrate these milestones along the way to impact. 

3. Facilitate CSO connections: OCP should work to reach more civil society organizations outside of capital cities in a resource-efficient way. 

We heard from multiple partners that their national open contracting ecosystems would benefit from further support for civil society organizations and journalists outside of capital cities and that OCP could build greater connections in each country and help expand this network to include other organizations interested in working on related issues. 

4. Foster replication: OCP should work closely with partners to help them understand how to apply lessons from other projects to their own work, so progress can be replicated across different countries and contexts. 

In a number of interviews, partners highlighted that they would benefit from more help identifying case studies that apply to their work. This need is two-fold: some partners were asking for case studies which more closely resemble the legislative and political contexts in which they operate; other partners said that they would appreciate more OCP case studies which more clearly communicate certain use cases of open contracting, such as driving internal efficiencies in government.

The evaluation recommended that OCP should work closely with partners to apply lessons from other case studies to their own work and replicate approaches to progress, perhaps with clearer guidance on how to use existing case studies or better organization and coding of case studies on the website.  

5. Support partners with resource mobilization: OCP should rethink the way in which it helps partners plan for financial longevity. 

The evaluation found that OCP should do more to formalize their existing efforts to connect partners with other organizations who might support their work in the future. This could involve developing resources for partners which outline potential sources of future funding for their work and the requirements to apply. 

6. Continued support: Finally, OCP should continue to support partners, or empower others to do so, beyond the end of the next strategy cycle.

Across the board, we heard how OCP is filling an important gap. Partners highlighted how valuable OCP has been to them, in terms of technical help, project management support, and financial backing. It was clear from our interviews and the results of OCP’s own partner surveys that most partners feel they would benefit from continued OCP support beyond the next planned strategy cycle.

Therefore, the evaluation recommends that OCP should consider how best it can continue to support partners – or empower others to support partners – beyond the end of the upcoming strategy cycle. Continuing to consult with partners as these discussions develop will be crucial to making sure that the open contracting community continues to thrive.

Overarching reflections

At Oxford Insights, we’re delighted to hear that OCP will be feeding our recommendations into its next strategic cycle. You can read the summary report of Oxford Insights’ final evaluation of Open Contracting Partnership’s work supported by the BHP Foundation here (also available in Spanish). 

And stay tuned as OCP works through its strategy process over the coming months to better address some of these challenges and recommendations!