Cardinal, an open-source library to calculate public procurement red flags

Our new solution, Cardinal, makes it easier to calculate common risk indicators of corruption and collusion in procurement. We’re already working with partners to use it to examine contracting procedures in Ecuador and the Dominican Republic. But the solution is ready to be applied to help detect red flags in the procurement of more than 50 governments whose contracting data is published in the Open Contracting Data Standard (OCDS) format.

Cardinal builds on our previous work to help our partners publish and use open contracting data to achieve greater public integrity. In 2016 we launched our Red Flags for Integrity Guide, a methodology to calculate a list of risk indicators that identify corruption risks throughout the procurement process from planning to implementation, mapped to the OCDS. 

Our partners have used this guidance to build red flags analytical tools and dashboards and implement civic monitoring initiatives. However, each project took a different approach and used different tools to access and pre-process the data, calculate the red flags, and share the results. 

With more than 50 publishers of OCDS data and with a growing community of data users, we realized there was a need for a solution that could help OCDS users perform data cleaning tasks more easily and automate the calculation of red flags. This information could then be used in business intelligence tools to enhance procurement monitoring. 

The solution: With support from the Microsoft ACTS program, we built Cardinal, a free open-source “public procurement red flags library.” 

Our goal was for Cardinal to make it easier for developers to build their own red flags tools by using standardized procurement data in OCDS, to promote effective data use related to procurement integrity, and to contribute to further data improvements. 

So, what is Cardinal? 

Cardinal is an open-source library and command-line tool that uses OCDS data to:

1. Calculate the coverage of an OCDS dataset, listing the fields and the number of non-empty values it has. This helps you identify quickly if the dataset you are planning to use to calculate the indicators has the needed fields. For instance, to calculate collusion risk indicators you will need individual bid values.

Example of a sample of data fields and their coverage, using Cardinal’s coverage command.

2. Correct data quality issues in your dataset. The prepare command in Cardinal, is designed to warn about quality issues (1) that it can fix and (2) that interfere with the calculation of indicators. For instance:

• Correct structural errors
• Normalize ID fields
• Fill in missing values, for example if the currency or the item classification scheme is not set. 
• Redact incorrect values, including monetary amounts and organizational IDs.
• Re-map invalid codes, for example if the bid status is set incorrectly:
  
  [codelists.bid_status]
  Qualified = valid
  Disqualified = disqualified
  InTreatment = pending

• Standardize unconstrained values
• Replace incorrect award statuses

This has been very helpful to easily detect and correct data quality issues in the datasets we used to test Cardinal. In the Dominican Republic, using the prepare command we were able to quickly make multiple fixes: fill in default values for currency (DOP), item classification scheme (UNSPSC) and bid status (valid) over one million times; redact placeholder values, like zero amounts and dummy identifiers; fill in the missing award status according to the provided contract status; and map non-standard codes to standard codes, like ‘Calificada’ to ‘valid’. 

3. Automatically calculate procurement indicators and red flags. Currently Cardinal supports 10 indicators. We are in the process of incorporating eight more in the short term. 

Depending on the indicator, the flag is raised for the contracting process, the buyer or procuring entity, or the tenderer. 

Below, you can see an example of the output. In this case, the contracting procedure has the flag R036 (lowest bid was disqualified), and the buyer was flagged for having a high rate of bid disqualifications (R038). 

​​{
 "OCID": {
 "ocds-6550wx-JRFPFA-DAF-CM-2021-0012": {
  "R036": 1.0
 }
 },
 "Buyer": {
 "DO-RPE-55216": {
  "R038": 0.8
 }
 }
}
 

You can then use these results to analyze patterns and develop tools. In the chart below, with sample data from the Dominican Republic, we found 292 bidders had a red flag indicating that all the bids they submitted were disqualified.

How have we used it? 

We have already used Cardinal with OCDS data from the Dominican Republic and Ecuador.  

We worked with our partners in those countries to understand what were the most common public procurement risks in their contexts that could be detected using red flags indicators and how using data could help them improve their monitoring practices.  

We then developed those indicators in Cardinal, and with the results built a Power BI dashboard to visualize the risks.  We already launched the business intelligence tool internally in the Dominican Republic, and it is currently being used by the public procurement agency Dirección General Contrataciones Públicas (DGCP), as part of their set of tools to monitor risks using data. 

We are finalizing the BI tool for Ecuador, which will include new red flags relevant to its specific context.  

What have we learned? 

1. Standardized data is very helpful. Thanks to OCDS, developing a red flags library that can be easily tested and used with procurement data from different countries was possible. Standardized data eased the process of identifying the feasibility of implementing red flags across countries, prioritizing indicators, thinking about edge cases, identifying common data quality issues, and making a library that can work with different datasets. 

2. Cardinal can adapt to different contexts. Implementing useful red flags in practice requires a good understanding of the context. To develop Cardinal we worked closely with partners to understand their procurement processes that inform what red flags made sense and explore potential edge cases and configurations of the indicators, e.g. which procurement methods to exclude. For instance, Ecuador has reverse auctions, while in the Dominican Republic bids are present for other open procedures. We made Cardinal work for both cases, and we want to expand to more. 

3. Don’t forget about visualization tools. In the Dominican Republic and Ecuador we used the results from Cardinal to build business intelligence solutions that could help procurement authorities explore the red flag results and help them in their monitoring process. 

Are you interested in using Cardinal with your OCDS data? Visit our Data Registry. 

Do you have some feedback or new indicators you want to include? Do you want to contribute to the library? Reach out to our Head of Technology! And in a future blog post, we will describe all the technical details of how we developed Cardinal!