Government procurement is opening up, but progress is slow

2015 was the first time Open Knowledge’s global open data review included procurement. It is one of government’s most valuable datasets – every third dollar that your government spends is on public contracts – so it was great to see it being included. Since 2015, we’ve observed lots of commitments to publishing open contracting data and some progress in implementation. But are the actual data user-friendly, machine-readable and open?

A first look at the 2016 Global Open Data Index and its review of the state of open procurement data launched this week is encouraging: In 2016, average openness stands at 51% across all countries surveyed. This makes procurement the third most open dataset in the ranking.

Some 27 countries – about one in three – scored 70% or higher. At first glance, this is not a bad result. Clearly, governments think publishing this information as open data is important. New champions have emerged and increased their score notably by making their data fully open: Hong Kong, Singapore and Slovakia.

The index only scores the availability of a limited selection of data points but, even then, fewer countries are publishing fully open data. Only eight countries received a perfect score of 100%, compared to 11 the previous year, which raises the bigger question of how countries are putting out data. To use data effectively to analyze, track and monitor procurement information, it needs to be timely, machine-readable and accessible via bulk download or an API. And of course, it needs an open license.

The quality of the procurement data is also a big concern (much of what we said about this in 2015 still applies). Here’s what stands out for us in this year’s assessment:

• Timeliness

Timeliness of data seems to be a much bigger issue than a year ago. Even countries that tick the box when it comes to what is published lag in terms of when the information is shared. One in six datasets, or 18%, of public procurement information reviewed is old or updated irregularly. To track public deals, it matters when you can access data on tenders and awards of contracts, and if patterns of fraud and collusion can be flagged early enough to change course. Countries such as the UK or Slovakia give daily releases. But still, seven of the top 30 countries fail to provide at least regular monthly updates.

• Embedding open data in agency processes

The Philippines, which dropped out of the top scorers, is an example of how open data initiatives alone are not enough. The country used to provide an up-to-date dataset of machine-readable and downloadable information, but now, timeliness is an issue. Open data needs to be fully embedded within government agencies and their systems for managing data to ensure it is updated and timely, for example through automatic releases. When leadership changes, open data initiatives tend to suffer.

• Members of the European Union

The European Union is a special case for procurement information, as tender and award data is available through its central register, Tenders Electronic Daily. However, countries you would expect to be leading in the Open Data Index, such as Denmark, Germany, Netherlands, Sweden or Switzerland, publish little, if any, coherent procurement information nationally beyond what is registered above EU thresholds. This is surprising, as countries like Denmark or Sweden are perceived as highly transparent and less corrupt.

So why would they not publish this information? Is transparency necessary for ensuring integrity in public procurement or are many of the countries perceived to be of high integrity actually much less clean than we believe? Mihály Fazekas, who leads the Digiwhist project, has been studying this conundrum for a while. Single bids on public contracts are one of the key indicators known to increase the risk of corruption – and easily trackable if open data is available. He shows that lack of competition is a growing phenomenon in Europe. Digiwhist analysis in England, for example, shows how a lack of scrutiny leads single party councils to uniformly spend more money on basic contracts that multiparty councils. Mihaly finds similar patterns in Sweden and Romania. The complacency of governments who fail to publish open data in public procurement may be misplaced.

• Linking contracts to aid

We also noted that many aid recipient countries are clustered at the bottom of the table. Donors should be more proactive in demanding that tender and award information be public and linked to aid flows. With increased need for private investment, especially around huge infrastructure projects, transparency is essential. We’ve looked into how to encourage this by connecting aid money to public contracts using the Aid Transparency Standard and the Open Contracting Data Standard. At the same time, donor countries like Germany, Denmark, Sweden, and Switzerland should be leading by example.

Our recommendations:

1. Data on public contracts should be open by default. It is a key dataset that adds value to all stakeholders involved.
2. Focus on feedback so the data is put into action. Public procurement is a process that starts at the planning stage and ends only with the final goods or services are delivered. In Ukraine, the Dozorro.org portal complements the government e-procurement portal, Prozorro, to provide an open feedback channel to get problems fixed.
3. Use an open standard, such as the Open Contracting Data Standard, to ensure data gets used effectively. This not only increases data quality but also allows for connecting to other key datasets such as budget and spending to complete the accountability cycle.

Do you live in a country where there is at least some data to work with? We want your best idea on how to use this data for more transparent and effective public procurement. Join our global Open Contracting Innovation Challenge now.