Government contracts: Still a long way from open

For the first time ever, the Global Open Data Index is assessing open data on tenders and awards in this year’s index. This is crucial information. Government deals with companies amount to $9.5 trillion globally. That’s about 15% of global GDP. Schools, hospitals, roads, street lights, paper clips: all of these are managed through public procurement. 

This is why we were delighted that the Open Contracting Partnership was asked to evaluate the submissions from the more than 120 countries participating in this year’s index. Here are some of the highlights and aggregated findings of what we learned.

Public procurement is the number one risk of corruption and fraud in government. Too often, when government and business meet, public interest is not the highest priority. Scandals from failed contracting processes abound: ‘tofu’ schools, constructed to substandard specifications in an earthquake zone, that fell down on their students; provision of fake medicine and medical equipment that kills patients; phantom contracts siphoning off billions of dollars dedicated to national security; or kick-back schemes in contracts that steal direly needed monies from school children.

To make sure this money is spent fairly and honestly, it is essential that data is disclosed on how much, when and with whom governments spend money on.

This is why we were delighted that the Open Contracting Partnership was asked to evaluate the submissions from the more than 120 countries participating in this year’s index. The Index assesses whether monthly information on both tenders and awards is available and basic information disclosed. This should include name, description and status of a tender, as well as the name, description, value and supplier information of an award or a contract. Ideally, this information should not only be public, but also machine-readable and downloadable as a complete dataset.

Here are some of the highlights and aggregated findings of what we learned.

The bad news first: Government contracts are a long way from being open. Less than 10% of the countries surveyed provide timely, machine-readable and openly licensed data on tenders and awards of government contracts. Without this information available to the public, there is little opportunity to scrutinise and monitor public spending.

Now the good news: We can see a trend towards publishing more data and doing so in an open format. 11 countries is definitely progress and it is clear that some global champions are beginning to emerge. Some of the credit, we hope, could go to the Open Contracting Data Standard, that is being implemented in countries such as Canada, Paraguay, and the ProZorro procurement portal being currently developed in Ukraine. Other countries are implementing open data on contracting after committing to them in their National Action Plans under the Open Government Partnership, such as Colombia, Mexico and Romania.

The Index doesn’t score quality, so this is important to keep this in mind when looking at the data. At the Open Contracting Partnership, we are aware that the quality of disclosed data on tenders and contracts varies strongly. The better the quality of contract information and the more recent is, the more valuable it is for evaluation, analysis, and monitoring. While timeliness of the data in general was relatively good, the quality of the open data is still problematic.

Digiwhist, a research initiative, has looked at the amount of information actually available in tenders and contracts in the EU and found common id’s are rarely available to match tender and award data, government agencies and business. a key quality issue of this data.

Other findings of the Open Data Index include:

• More countries publish tenders rather than awards data: Focussing on publishing tender data makes business sense. More competition drives savings. In Slovakia, opening up the procurement process has led to doubling the number of bidders and reduced costs. But to be able to track public spending, matching awards to tenders is crucial. Over 100 countries and places now publish both tender and award data.
• Information can be public, but is not necessarily open. The majority of countries have well developed e-procurement platforms through which they publish information. In the majority, these systems are closed and so is the data. You could have a lot more public openness from a relatively small investment in publishing the data in reuseable, accessible formats. Contract data needs to be aggregated to ensure all information is available in one place. In many countries only some sectors of government publish tenders and awards data.
• Countries shouldn’t hide behind official thresholds of beyond which amount contracts should be publish. All contracts should be made available publicly. However, thresholds for publication of tenders and awards varies strongly by country. EU countries, for examples, have to publish tenders above specific amounts via the electronic tender database TED. Systems for publishing nationally or locally are much less standardised. This allows for all EU submissions to qualify as publishing open procurement data even though some countries, such as Germany, do not publish award value for contracts below those thresholds, and others have closed systems to access specific information on contracts awarded.
• Timeliness and completeness of downloadable data on awarded contracts, is an issue. Formats for downloads are diverse as well, going anywhere from a PDF download to a full set of XML or JSON data. We have given brownie points to 18 countries that publish some bulk downloads, but this criteria definitely needs tightening. To qualify, the datasets needed to be updated more frequently than just yearly to ensure data is relevant to analyze recent contracts.

Lots of work needed, as these results of the index shows. Looking forward, at the Open Contracting Partnership we believe that next year’s results can be only improved by implementing the following policies.

1. Open data on contracts is a key dataset that adds value to all stakeholders involved. This is why contracts should not be legal and enforceable until published. There are too few countries who have made publishing contract data open by default. Credit to those who have such as Colombia, Georgia, Slovakia and, hopefully, the Czech Republic.
2. Public procurement is a process that starts at the planning stage and ends only with the final service or construction being delivered. To harness the most of open data, engagement and feedback channels need to be available to get problems fixed. Data on complaints and contract adjustments needs to be made public as well.
3. By publishing more data on government contracts, citizens will be able to follow the money from planning to implementation of policies, from the budget to the actual contracts, to showing how taxes are being spent, and how this benefits citizens.

While this year’s Open Data Index first assessment of open data of awards and tenders highlights many of the deficiencies, some positive trends can be detected where information has been made public. Everyone has to gain from open and better quality data on government contracts. Government receive more value for money in their investments. Business, especially small and medium enterprises and minority-owned enterprises, gain access to a fair marketplace of contracting opportunities with the government. Citizen can evaluate how money is spent and managed. Going the extra mile to make government deals with companies truly open might be just the way to show the real impact of open data.

This post has been cross-posted from the Open Knowledge blog. Lots of great insights into the state of open data globally at the full Global Open Data Index here.