Learning Insights: Open contracting – a helping hand in Nepal
Photo: One government agency in Nepal shares their citizens charter for the procurement process on their wall.
How will open contracting help you? A deceptively simple question perhaps, but it’s vital. Identifying and understanding the needs of our partners is crucial to making sure that open contracting data gets used and genuinely adds value to their work. It’s the first step in our 7-step process – design the key goals. We’ve learned from experience that if it gets jumped, we can end up with transparency just for its own sake and with siloed information.
With that question, we kicked off our work in Nepal with partners from government and civil society after the devastating earthquake as part of our Showcase and Learning project. With so much aid and public funds being spent on reconstruction, a transparent, accountable and efficient procurement process will help deliver much-needed infrastructure, goods and services more effectively.
In this Learning Insights blog, we want to provide a look into the process of how we typically develop our monitoring, learning and evaluation plans. We see four phases: research, design, implementation and iteration. However, we had to adapt this plan in Nepal as usable data was lacking to measure baselines and determine impact.
First up, we worked with both the public procurement agency and Young Innovations – a super capable social enterprise – in 2016 to nail down how open contracting could help. This began our research phase for the project. We brought government, civil society and business together to collaboratively explore and research the use cases, which means understanding which specific problems they want to fix through open contracting. Overall in Nepal, local actors have three aspirations: to improve efficiency, competition and integrity. No surprise given the reconstruction effort. The underlying theory of change that we all agreed is that increased disclosure of useful procurement data in useable formats combined with strong stakeholder collaboration and capacity building will encourage the use of the data, and ultimately foster trust between the government and the public.
This research led us to our second phase, co-designing our MEL plan for the project. As we discussed in our last Learning Insights blog we try to center our MEL plans around disclosure, use and impact of data. However, data availability was much worse in Nepal than it was in Ukraine. In order to determine our indicators and get credible baselines, we had to implement a different, more qualitative, design approach for Nepal. So we spent a fortnight with Young Innovations conducting a series of semi-structured interviews with seven governmental agencies to find out more about their contracting processes and challenges.
These interviews helped us paint a rich picture of common challenges that our partners are aiming to fix through open contracting and rolling out the new procurement system. The top challenges we identified are:
- Inefficiency of the contracting process: Six agencies noted that the inefficiency of the contracting process is a serious challenge. For example, it is not uncommon for a contractor to submit a request for an extension of contract in the final days of the contract period, thwarting agencies’ plans to proceed with other related projects. Agencies are lacking tools and resources to track the contracting process effectively and to be able to react in a timely manner.
- Lack of clarity about potential/current suppliers: Four agencies mentioned a lack of information about potential suppliers. They reported they were not able to judge the capacity of potential suppliers because they are missing information on previous performance. Additionally, agencies lack the tools and information to decide whether a particular vendor is right for the job – there is no blacklisting or another process that would allow discrediting bidders based on a poor previous performance.
- Lack of competition: Half of the agencies we interviewed said that they suffer from a lack of sufficient competition, and there is anecdotal evidence that suppliers aren’t diverse enough. While the lack of quantitative data prevents us from taking a hard look at the numbers that would indicate this (number of bidders, average suppliers, etc.), the fact that this was a recurring theme in our interviews indicates that it is a significant problem.
- Lack of communication & collaboration among stakeholders: All agencies mentioned that communication between them, suppliers, bidders and the general public is still mainly paper-based. Gathering feedback and addressing questions and complaints is often time-consuming and burdensome.
We then co-created an MEL plan that translates these qualitative statements into quantifiable indicators. In line with our (relentless?) focus on user needs, the indicators center heavily on efficiency, competition and integrity.
Some indicators that we have our eyes on are:
- Efficiency of procedures (conversion rates) and share of contracts completed on time – These two indicators are particularly important when it comes to knowing whether or not efficiency within the procurement process is improving, which was a key goal for nearly every agency with spoke with.
- Number of companies registered, Average number of bidders per tender and Average number of unique suppliers per procuring entity: These three indicators work together to give us a good picture of whether or not competition is increasing in Nepal.
- Number of complaints per tender and Number of portal users reporting that procedures and expectations in the system are clear: Both of these indicators speak to an increase in integrity and trust in the system.
In addition to performance, we are also eager to measure the use of data and tools. To have the amount of data needed to determine how we are progressing towards the above indicators, people must actively use, and engage with, the system. Additionally, consistent engagement with the system is necessary for long term open contracting success. Regarding data use and engagement, some of the indicators we plan to track are the number of visits to the monitoring portal, number of online tools created using contracting data and number of public sector agencies using newly created tools and information for monitoring.
Right now we are in the third and fourth phases: implementation and iteration of our shared MEL plan for open contracting in Nepal. The first baseline data are coming in, and we realize that the data quality is still challenging. First of all, almost all agencies used their procurement platforms and will continue doing so until July 2017. Many institutions have been conducting paper-based procurement with e-procurement systems used voluntarily and sporadically. So for now many of the baselines will show as zero as we will only be able to calculate these indicators in August after the new system has become mandatory. In the meantime, we will continue our interviews with the seven agencies and track progress and challenges through their feedback.
We are also planning project activities that will help address data availability and quality issues. For example, we are working with Young Innovations to help the procurement agency launch a new open contracting portal that will provide government, business, and civil society with user-friendly access to all of the available public procurement data. In addition to making this data available for the first time, it will give insight to metrics and trends such as top procuring entities and top suppliers across agencies.
We were in Nepal last week to co-design a pilot project with community monitors to better understand how open contracting data monitoring will work in Nepal and to further underpin our shared MEL plans. We’ll tell you more about this project in a few days in another blog, and we’ll keep you updated on the progress made throughout the project. Additionally, we continued to iterate on our MEL plans, so we also collected more stories from some of the agencies we’ve been working with, tweaked a few indicators, and met with an evaluation expert from Kathmandu University. We’re working hard to compile a complete picture of how the government in Nepal currently uses procurement data in their daily work so that we can help ensure that this new system improves their contracting processes.
As we continue our journey with our Nepali partners, we will continue to iterate and adapt this approach. We will do so with our use cases as a guide, navigating to our best attempt to answer How is open contracting helping?
