Standards, eServices, and open contracting in the EU

Open Contracting principles and data practices could be an attractive solution to support monitoring and analysis of how public money is spent across Europe. Because an open contracting approach facilitates comparability and interoperability across borders, this use case becomes even stronger in an economic union such as the EU, or when dealing with trade agreements. This week we had the opportunity to see how the Open Contracting Data Standard (OCDS) could complement a growing suite of interoperability standards for the whole contracting chain in the EU.

CEN BII, a programme that looks at harmonizing electronic procurement in Europe, has developed a suite of profiles describing detailed business requirements and technical specifications for contracting information exchange. It covers the information that businesses, governments and others want to be able to communicate electronically, from tender notices, to orders, catalogue entries, and invoices. Where the OCDS describes processes that may have taken place both electronically or on paper, the CEN BII suite is designed to capture 100% electronic processes. To put the specifications into practice, CEN BII has developed bindings to express key profiles in UBL (Universal Business Language) or CEFACT XML structures.

Fully adopted, the components of CEN BII could cut out all of the paper-based processes of invoicing, signing contracts or ordering from a catalogue, whilst maintaining security and reliability. Although full adoption of all the components of the CEN BII stack is fairly limited right now, and some of the existing implementations such as TED (see below) use earlier schema and standards, as each component of a fully electronic procurement process is implemented it will offer new sources of structured data. This data will cover many of the different sections and building blocks that are important for procurement transparency and open contracting.

For example:

• Tenders Electronic Daily (TED) provides tender and award notices for all government procurement processes in the EU above the EU threshold. The majority are submitted directly via an eSenders framework of machine-to-machine communication, with the rest input via an online notice form.
• The ePrior project has been developing modules for submission of tender responses online, piloted with European Commission procurement processes.
• The PEPPOL project has been developing a secure and open infrastructure for exchange of eInvoices, leading to the direct exchange of request for payment between systems and cutting out manual payment processes.

We’ve already been exploring mappings between the TED XML format (used to published current and archived EU procurement notices) and OCDS, and we’re in the process of working up a number of extensions to capture information of particular relevance to the EU within OCDS (such as lots). This should support a common approach to present TED XML notices as OCDS data. You can join the discussion on this process via the OCDS issue tracker.

However, TED notices will only provide some of the stages of a contracting process. The CEN BII suite covers structured data all the way through to invoicing and spending transactions.

The CEN BII process has also specifically considered the issue of how the data generated by digital exchange of information might be published, producing report CWA 1234 part 106, Open Procurement Data report. The report notes that right now «Open data publication [in the EU] does not generally include Procurement data with the exception of TED (Tenders European Daily)» and argues that «the highly structured CEN BII transactions could be the best source of good data on public procurement». However, it also raises some concerns that not all the information contained in the transactional CEN BII notices will be suitable for open publication, and points to ways to handle this. For example, in Norway, bidders can submit original and redacted documents, and the procuring authority can then judge whether the redaction is justified in choosing which version to publish. A similar approach to confidentiality in public procurement documents is in place in the UK. The report focuses primarily on publication as Linked Open Data, although our experience in developing the OCDS has been that providing data in a range of formats is vital to ensure maximum accessibility and use, and starting from JSON and CSV may be more practical.

We believe that the OCDS has an important complementary role to play in opening up EU procurement information. The OCDS is designed for open data disclosure, around the priority data that users need, and around the technical requirements of business, citizen and government data users. TED, CEN BII and related tools are either designed around the legal forms prescribed in EU law, or based on machine-to-machine transactional processes. Mapping data to the OCDS for publication can help identify what should be published, and can provide it in a more accessible form for those using the data. The feedback we’re receiving from OCDS users also underscores why it is so important to have clear and accessible online documentation. This enables a wide range of those who actually use contract data, from individuals, to SMEs and large system integrators, to generate and work with open data effectively.

With EU Procurement data available in OCDS, we can unlock opportunities to monitor data quality, both by the European Commission internally, and by external users of the data. Through a combination of top-down and bottom-up engagement with contracting data we can have a  greater chance of overcoming key data quality challenges, such as inconsistent linking between tender and award notices, or lack of common identifiers for legal entities.

As the work of CEN BII moves into a new phase, within a new Technical Committee, and the European Publications Office work on continued development of the eSender and eNotice forms for data capture, we look forward to see how the OCDS can support better use of contracting data across the EU.