A new registry for open contracting data
Sourcing open data can be hard. Sourcing standardized, quality open data is harder. Public procurement data can be particularly difficult, as it is often published by different government administrations, each having their own process and platform. You may need scraping tools, when the information is published on a webpage to access the whole dataset. Data published in different languages makes this even more challenging.
Our new Open Contracting Data Registry makes accessing re-usable public procurement data easier. We hope that this service will help you better analyze and monitor public procurement in your country and across countries, whether it’s red flags for corruption or competition and inclusion indicators and whether you prefer working with JSON files or CSV files.
Why a Data Registry?
As of February 2023, there are 59 Open Contracting Data Standard publishers globally including national and subnational governments. While this is good news for standardized open data, in our work with partners around the world, we see that users sometimes find it hard to access OCDS data, since each of the publishers discloses their data in different portals, or with different access methods such APIs or bulk downloads.
We have developed tools for accessing and storing OCDS data from the different publishers. But after talking to many of you, we decided that a simple search facility would help users to easily identify and access available datasets, in line with our work to make data more accessible, usable and – as a result – more open. That’s the data registry.
Finding and understanding the data
Public procurement data can be tricky to work with. A contract’s start date might be expressed differently in the Czech Republic compared to Mexico. In the UK the Foreign, Commonwealth & Development Office was recently the Foreign & Commonwealth Office. Before that it was the Foreign Office. In the UK datasets, all of these names are used, as are the acronyms FDCO, FCO, FO – and then the data might also contain mis-spellings. The result has been to keep the search functionality and filtering simple in the data registry. The focus has been to help users to see if data for a certain country is available – rather than, for example, to enable users to filter contracts from a specific buyer.
A key issue that we tried to address was that users want to have an idea of what data is contained in what can be very large files before they proceed to downloading it. For instance, some users are particularly interested in the tender information, but it could be that only information about the contract has been published. Other users might only be interested in procurement data because they want to investigate the supplying companies or because they are searching whether specific documents to check that planning for sustainability took place.
Finally, to help improve the user experience of working with OCDS data, we wanted to share our knowledge about the data quality issues and quirks of the different datasets, that we know of from using the data and advising partners on how to improve its quality. As part of ‘on-boarding’ any dataset into the Data Registry, information about the dataset includes challenges with the data that we have identified.
We’ve launched the Data Registry with the hope that it is just the start for a new wave of using procurement data. We will keep adding new datasets and features and are keen to hear thoughts and feedback. How are you using the data? Do you have any suggestions for what we should add next? Let us know at jmckinney@open-contracting.org.
