How to make sure open contracting data gets used: A guide to defining the use case

The success of implementing the Open Contracting Data Standard should not be measured by compliance but by how much data is being used. The data standard was designed to help people make sure contracting information achieves:

• Better value for money;
• Fairness of accessing the market of public tenders;
• Detection of corruption; and
• Monitoring on effectiveness of delivery.

In our documentation and guidance we note repeatedly how important it is to define use cases for local implementation of OCDS. The real results of open contracting come from the engagement of diverse government actors, citizens, businesses, technologists, journalists and academia. With their participation we can achieve better contracting processes and outcomes that solve problems of government agencies, companies and citizens. But it’s often not so straightforward to see how this can be done.

Over the course of the last few months, through our support to implementers around the globe, we have learned a few lessons that might be helpful and we’d like to share.

I’ve done my best to break it down into some specific steps, but not necessarily easy steps: Identifying stakeholder groups, figuring out what they want, mapping supply of data with demand, and then documenting use and impact.

Before looking at the details, let me first share an example from Ukraine. One of the key objectives of the multistakeholder ProZorro coalition was to increase the fairness and integrity of the public procurement system. The problem articulated was that not enough businesses were participating in the public procurement system because the system had developed a reputation for being opaque, difficult to enter, and corrupt. They agreed on key indicators to measure improvement, including:

• Average number of bidders per opportunity
• Number of new bidders bidding
• Average number of suppliers per procuring entity

The project stakeholders went on the road to meet company representatives and understand their concerns and reservations. Why weren’t the companies bidding? How could trust be rebuilt? The team improved the procurement process, introduced open source electronic auctions, and developed a business intelligence tool to facilitate the use of procurement data. They piloted new approaches, raised awareness of the new system and its unprecedented transparency and trained potential participants. A new law was developed, introduced and implemented. The data they used to measure showed the substantial improvements and helped gain more support for their reform effort.

Averages for completed tenders, number of tenderers and bids per tender are going up!

Identify your stakeholders

Successful reforms depend on actively and appropriately engaging stakeholders throughout the entire process. To make the OCDS as impactful as possible, it needs to display information and analysis that government agencies, businesses or civil society want to see and are asking for. You want to avoid developing a tool that is not useful (and is not used). The best way to do so is to involve key stakeholders from the very beginning in creating the user cases.

Who do you think is likely to care about public contracting information? Are they already using it? Should they be?

Some of the folks that tend to be potential users of the information include:

In some countries, donors are also a user of public procurement information.

Figure out what information they want and why

The next step is to distill their information needs and prioritize them as part of your OCDS implementation. There are a couple of ways on how to do this.

First, you can ask them. When we developed the OCDS, we developed a short questionnaire and shared it with representatives of all of the above groups. We also conducted workshops at several events around the world. We highly recommend questionnaires, interviews and workshops.

In a workshop setting, you can gather a subset of stakeholders and use your imagination to develop use cases through a ‘personas’ exercise. The first stage of this process involves brainstorming the long list of stakeholders that could be interested in a specific implementation.

User brainstorm

For example, here is a picture of a recent brainstorm completed with the Red Compartida PPP team in Mexico. They were thinking of all the stakeholders who could possibly be interested in the largest telecom PPP in Mexico’s history.

User persona exercise

Then, you prioritize a subset of those stakeholders and try to identify in some depth a specific imagined person. Who are they? Why do they care? How do they usually interact with this type of information? What they would want to know, why they would want to know it, when they would need the information, and how they would appreciate receiving the information. One example of a resulting use case from the Red Compartida exercise can be seen here. You can notice a lot of detail, from the specific requirements for a monitoring dashboards to the illustration of imaginary 43 year old user Juan Camarera! More resources on personas exercises can be found here and here.

Once you know what your stakeholders are looking for, you can take additional inspiration and guidance at examples of uses of data that have been developed by others.

An example of a private sector use case is explained in the above example from Ukraine. With regard to anticorruption use cases, you could check out the detailed methodologies for detecting integrity risks in public procurement developed by experts at the Corruption Research Centre Budapest and Cambridge University that are being put to work in the Digiwhist project. In Indonesia, Indonesia Corruption Watch, through an MoU with the procurement authority has similarly developed a monitoring methodology and tool using e-procurement data to detect ‘red flags’ in procurement.

A government that would like to evaluate value for money could look at the smart visualizations by the Paraguayan procurement authority that analyse spending patterns and progress of contracts along the contracting cycle. The visualizations of spending by contracting authorities helps compare the volume among each public agency, timing of contracts, and highest value contracts. It also shows spending by category providing the government with a much better overview of where and especially when contracts are being awarded. Detailed view of each contract shows the current progress in each contracting process. A calendar highlights when tenders and offers are made.

Methodology for Procuring Entity Benchmarking from Zindex.cz in Czech Republic

In Czech Republic, zIndex.cz is a civil society-developed public procurement benchmarking tool. It’s methodology uses real data to measure each contracting authority’s rate of transparency and efficiency in public procurement. zIndex’s ambition is to publish evaluations of different contracting authorities (municipalities, government departments, state-owned enterprises, etc.) on a regular basis, and in doing so, inspire them to improve their public procurement practice.

All of these mentioned consultations, research processes, and brainstorms, can help you to identify what are the problems (or opportunities) facing your public contracting system.

Document your user requirements

Once you have your use cases written up (and here is an example from the OCDS development process), you can distill them into a list of requirements (here is an example of the requirements matrix we developed for the OCDS).

In a more simple example, let’s say your stakeholders are interested to implement the methodology of Misi Fazekas and Istvan Toth to detect corruption risks. One of the requirements written as a ‘use case’ might look like this:

Once you have the requirements documented, it’s easier to see where you have some patterns and overlap in terms of the data needed to support your use cases. You can also share them back with your community of users for validation.

One of our and our partners’ goals over the next year or so will be to map and provide more guidance on existing methodologies for data use to facilitate this process.

Map demand to supply and make a plan

As part of your OCDS implementation, it is important to map 3 things:

• The data and documents you have in your systems
• The information you are required (or prohibited) to publish under your legal framework
• Your user requirements

We are currently undertaking this same mapping to publish our own contracting data and documents using the OCDS Mapping template and color coding the fields as Blue (what we have), Yellow (what our open contracting policy requires) and Pink (our user requirements).

It may be that some of the data your users require is not currently contained in the OCDS. The mapping template indicates a space for extra information. If you feel that this information is important enough that it should be part, you can tell us through our community issue list. It may be that multiple partners are defining the same demands and we can learn from each other.

Once you can clearly see what you have and what you want, you can make a plan to determine what you will actually publish. This plan will involve a consideration of feasibility and prioritization. For example, will you begin collecting new data to meet a user need? What is the cost involved? What are the trade offs?

It’s a good idea to share your publication plan back with your community of users. This can help manage expectations but also facilitate the development of tools and approaches for data use once publication starts.

Document use and impact for adaptive learning

Your use cases should have helped you to identify some issues related to your public contracting. You can take it a step further by articulating problem (or objective) statements and define some targets for improvement.

You can also take a baseline of your key indicators. Then, stakeholders can engage in change management and monitoring processes to use the data to make improvements (like it was done in Ukraine). This data can serve to track improvements and share learning.

Whatever approach you take to your OCDS implementation, we highly recommend engaging your stakeholders in this planning and prioritization process. By being clear about what will be published as part of your OCDS implementations, everyone will know what to expect and can start to think about how they will use the data for their purpose and needs.

Open contracting workshop in Mongolia. Great things happen when we work together

We are of course happy to answer any questions and share your progress with the community. Get in touch with us at data@open-contracting.org