Learning insights: The latest impacts emerging from Ukraine’s Prozorro reforms

Three years into Ukraine’s post-Maidan reforms, Prozorro has become a trademark of transparency in the country (although wider political challenges are still significant). An open contracting approach has been integrated across public procurement and has a healthy ecosystem of tools have been built around it,  including a citizen monitoring platform in www.DoZorro.org, a powerful business intelligence tool in http://bi.prozorro.org and a tool for identifying corruption risks http://risk.dozorro.org/. Together, these platforms attracted around 220,000 users in 2017. And each month, around 200,000 Ukrainians search for Prozorro or related keywords on Google.

We know that we have already shared a lot about Ukraine’s reforms in the past but, as one of the most advanced and complete series of open contracting reforms in the world, it provides a valuable opportunity to learn about the sustainability and institutionalization of open contracting and offers lessons to emerging reforms in Latin America, Asia, Africa and the OECD.

Our recently published, second Monitoring, Evaluation & Learning report on Prozorro shows some interesting new impacts in terms of competition, value for money, integrity, engagement, and other indicators. Here’s the how the hard numbers have changed from July 2016 to July 2017 (one year on from the baseline):

• public interest in procurement as measured by Google queries grew by 90%,
• the number of private e-procurement platforms (or “marketplaces”) more than doubled (from eight to 20),
• at least seven government institutions changed their monitoring policies,
• The average procuring entity grew its supplier base by 20%,
• there is evidence that more competition leads to greater savings (and for tenders with more than five bidders, savings grew up to 30%),
• and general perceptions about Prozorro remained very positive (80% of the business community believed procurement has improved with Prozorro).

For more detail on these and other impacts, check out the full MEL report.

Compiling that report involved several conversations about what has contributed to Prozorro’s success, including the complementary role that each stakeholder group played to support the politics and policies behind Prozorro. Here’s the latest on how government, business and civil society are engaging as of the end of 2017.

Government

Some 21 local administrations and 61 supervisory institutions created accounts at bipro.prozorro.org, a business intelligence tool for procurement professionals. To our knowledge, seven institutions drastically changed their monitoring practices, including hiring new staff and using the www.dozorro.org civic platform for monitoring feedback on their procurement. Four of them are large state-owned enterprises, such as Ukrainian Post.

Our latest research also reinforces our findings from July 2016 on responsiveness to public feedback. When an institution responds to queries, their procurement is 5% more likely to be successful. Considering the significant volume of overall procurement (over a million procurement procedures annually, worth around EUR 26 billion), this figure can have a meaningful impact on procurement quality.

So, in short, dozens of public sector entities are trying to use better procurement analysis, data and business intelligence tools and a handful of them already changing their monitoring policies. Acting on their own initiative, a handful of state companies piloted the OCDS data-based monitoring early in the process. This monitoring approach has caught on and subsequently become mandatory under a new monitoring law (discussed below). More targeted interviews should help uncover why, but we have a feeling that it has taken a shift in mindset spurred on by positive evidence from early adopters, and sustained cajoling with civil society actors, such as Transparency International Ukraine.

Civil society

The growth in traffic to www.dozorro.com (more than 133,000 visitors and 14,000 feedback submissions recorded since November 2016) is impressive, but what really matters is the fix rate – the number of problems that get resolved. Dozorro currently unites 22 CSOs that are actively monitoring procurement and have used to platform to find over 5,000 cases of suspicious activity in the last six months.

Around half of these cases have been resolved so far, including over 1,200 cases where tenders were changed as a result of the feedback. So the fix rate is approximately 48% of resolved cases or 25% of all cases. In addition, 22 criminal charges and 79 sanctions have also been issued.

Following recommendations by Transparency International Ukraine, the parliament recently adopted the first-of-its-kind procurement monitoring law (4738-d), which establishes data-based monitoring procedures and mandates the use of digital tools, including corruption risk indicators.

These developments can be looked at from many different angles. What we know for sure is that in Ukraine procurement monitoring is based on good data, has scaleable tools such as www.dozorro.org, is coordinated by the excellent leadership of Transparency International Ukraine and is having a measurable impact on both solving problems and on public accountability. The fix rate is low but there are plans to increase it further: Transparency International Ukraine and others want to focus on increasing monitoring and feedback quality in 2018.

The Dozorro concept is currently being tested in other countries, including Moldova, and the code and approach are open for anyone wanting to test it out in their country. It will be interesting to see how it travels. It will also be really interesting to see how the new law accelerates the existing impact from citizen monitoring.  

Business

Since July 2016, 12 new private marketplaces were created under Prozorro (from eight to 20 in total as of July 2017). Many of these not only facilitate participation in procurement, but also sell market research and analysis services based on OCDS data.

Most companies use the marketplaces more than once. Around 60% of them bid at least one other time and many companies (around 10%) over the last year have participated in more than five tenders.  Businesses are also getting better at picking their opportunities: on average, bidders won 42.85% of tenders in which they participated (Aug 2016 – Jul 2017), up from 36.47% (Feb 2015 – Jul 2016).

Around 80% of participants are SMEs, 75% of which won at least one public tender in 2017. That is a remarkable figure – many European or North American governments would dream of having such a high rate of SME participation (or indeed, anything over 25-30%).  

We surveyed around 200 companies that use Prozorro about impacts on their business. 80% said that the new system positively impacts their business by, for example, lowering administrative costs (6%), saving time to participate in procurement (18%), increasing quality of business decisions (19%), or making new connections (43%).

Final thoughts

We should remember the sheer scale of government procurement in a relatively centralized, post Soviet economy like Ukraine: this is a market of around 26 billion EUR with over 27,000 procuring entities, around 127,000 bidders and over a million procedures annually. Prozorro seems to have nailed the incentive mechanisms that work for each of the three main stakeholders: it helps government get deals done efficiently and effectively, it helps companies win business, and it allows CSOs to monitor government dealmaking in an easy way.

None of this can happen without collaboration between key groups; for example, the government not only allows for an enabling monitoring environment, but also actively encourages CSOs, by mandating institutions to use CSO-run tools, such as www.dozorro.org. This should help reinforce the long-term collaboration and sustainability underpinning progress so far.

Finally, here is the list of more notable indicators we observed. We will report back in 2018 too. We hope this blog helps inspire others as we explore what it takes for long-term sustainable open contracting reforms.

Photos: All pictures taken at DoZorro Fest, the first event of Kiev’s procurement monitoring community.