How to make better public procurement decisions with business intelligence: Insights from Ukraine

Almost a decade after its inception, Ukraine’s Prozorro system remains at the forefront of procurement transformation and innovation. Open contracting data and principles were embedded in the project from the start, enabling the development of powerful analytical tools and the growth of an ecosystem of procurement stakeholders who rely on these solutions for a range of different purposes, such as improving integrity, generating savings, and promoting better outcomes in a high risk, opaque sector.

A new case study we developed together with Transparency International Ukraine (TI Ukraine) examines how Prozorro’s data analytics tools have become an integral part of the work of more than 30,000 people across government, monitoring institutions, the private sector and civil society, even as the country grapples with Russia’s full-scale invasion.

Explore more in the full study

Data on public procurement in Ukraine can be analyzed through two business analytics modules: 

• The public BI Prozorro module, a free, publicly accessible analytics module, currently includes 49 dashboards, each designed to address specific use cases and issues within the procurement process, and covering all stages of the procurement cycle. There are specific dashboards that analyze prices on the central e-catalog, medical procurement, buyer evaluations, Energy Services Company (ESCO) procurement, or internal auditing practices. 
• The professional ProBI module is a report-building tool that allows advanced Qlik users to create and save their own selections and customize the appearance of reports based on all available parameters in the Prozorro procurement system. 

Both modules allow for comprehensive, in-depth analysis of public procurement using an exhaustive range of parameters (such as goods and their classifications, tenders, lots, and complaints; for any selected period; and any combination of selection criteria). 

The platforms are administered through a unique partnership between the government and civil society, run by DOZORRO, a project of TI Ukraine that promotes fair play in procurement. They interact with the central database of Prozorro, a transactional e-procurement system.

Built on Qlik Sense technology, the public module is accessible for free, without registration, to all users. Data can be exported in Excel format from custom selections for further exploration in any system. TI Ukraine, with the support of Qlik, has provided 300 licenses for access to ProBI to Ukrainian policymakers, regulatory bodies, the Prozorro state-owned enterprise, the Antimonopoly Committee of Ukraine, centralized procurement organizations, researchers, NGOs, and large buyers to enhance their efficiency.

The new case study explores some of the measurable achievements of the Prozorro business intelligence platforms, insights from different user groups, and reflections from those involved in creating the system.

These are some of the top tips for implementing procurement business intelligence solutions from the team at TI Ukraine:

1. Solve problems of specific users

The public analytics module was designed to address the needs of all stakeholders simultaneously – policy makers, procurers, businesses, and public monitoring. While not the most effective approach initially, it provided valuable insights into the procurement system’s operation and stakeholders’ needs. If we were to create BI Prozorro today, we would start by developing a tool tailored to address specific problems of one stakeholder group, which would likely have a much stronger impact.

2. Prioritize your data

It is crucial not to attempt to use all the data from the procurement system, but rather to focus on high-quality data. For instance, we discovered that planning stage data does not work in Ukraine as we anticipated (because procurement plans are published almost immediately before the announcement of lots and contain significantly less data than the lots announced on the same day from that amended plan), and thus, the analysis of this data is not useful. 

Similarly, at the contract stage, the procurement system generates inaccurate and incomplete information, despite the time and resources invested in developing contract and planning analytics applications. However, these applications helped highlight these issues to stakeholders and inform their opinions about the effectiveness of these stages. For example, we researched and found that overlaps in procurement-related reporting might be very costly – the annual budget losses are estimated at around UAH 30 million, (~US$820,000), providing clear and compelling arguments for why the planning stage of public procurement in the Prozorro system needs significant reform.

3. Understand your target audience’s needs 

We had high hopes for using the report constructor (ProBI) by large procurement entities and regulatory bodies, where they could configure the sophisticated set of indicators and report layouts to fit their organization’s needs. However, this hypothesis largely did not hold. A more detailed analysis of the processes revealed that for effective use of ProBI, users need a dedicated data analyst who continuously works with the tool and improves their expertise. Only a few organizations can afford this, typically those with diverse tasks that allow managers to continually enhance their data analysis skills and the market salary to retain skilled specialists. Nevertheless, organizations with the necessary resources have created several good solutions in ProBI, which we have incorporated into the public analytics module (e.g., “Events in Lots”). Recently, TI Ukraine found an interim solution that combines the flexibility of ProBI with a low «entry threshold.» We added «Constructor» and «Custom Comparison» features to the public analytics module, allowing even users who do not fully understand the structure of procurement data to create their indicators and reports.

4. Study the capabilities of the target audience

To launch a successful analytics module, it is necessary to regularly communicate with the target audience, understand their needs and problems, and how their processes are structured. Such knowledge allows for the optimal use of limited resources on what is truly needed by the target groups and in a convenient format for use and learning.

5. Understand how processes work not only according to the law but also in reality 

For example, knowing that the State Audit Office focuses on large sums during procurement audits helped us correctly develop an indicator for assessing violations. Similarly, the knowledge that there are companies specializing in challenging tender documentation (but not participating in tenders) allowed us to create an indicator that identifies such cases.

6. Maintain and manage relationships with system developers 

Regardless of how good the e-procurement portal or data documentation is, there will always be moments when you do not understand where to obtain the necessary data or how to interpret it correctly. Thanks to good relationships with Prozorro e-procurement system developers, we can always consult with them to create accurate indicators that account for all nuances and are reliable.

Furthermore, the Prozorro system is dynamically evolving, with new indicators emerging and changes in the logic of existing ones. Thanks to advance notifications from the SoE Prozorro about all changes, we can keep the module up to date. Without such collaboration, we would learn about changes only after the fact, when incorrect analytics would appear in the module or errors would start occurring from data loading algorithms (which would significantly impact our reliability).

7. Align methodological approaches and terminology

Government institutions in Ukraine are skeptical of data provided by non-governmental entities, especially in sensitive areas like procurement. Therefore, to expand the use of BI Prozorro, it is necessary to improve coordination on methodological issues between the procurement methodology unit at the Ministry of Economy, the data publisher (SoE Prozorro), the main audit institution (the Accounting Chamber), and the BI Prozorro team. Finding a common vision for solving methodological problems, procurement terminology, approaches to calculating procurement indicators, data completeness, and quality in the procurement system can help resolve these issues and expand the use of analytics among government bodies.

Equally important is communicating in a language that all users understand. We have encountered numerous cases where users misinterpreted data and made incorrect conclusions due to poorly chosen indicator names. This undermined trust in the tool as users began to distrust their own analyses done using the analytics module.

8. Have a capable team and sufficient funding

At TI Ukraine, the team supporting the analytics module consists of three people and external developers: a Project Manager—who coordinates development; a Business Analyst—who gathers and formalizes requests, researches data, and translates them into technical specifications; and an Analytics Trainer—who prepares training materials on new functionalities, cases, consults users, and conducts training for various target audiences.

Finding funding is always challenging, as maintaining the team, servers, and supporting and developing the module requires significant financial resources (>US$90,000 per year). However, real examples of how the analytics module helps various stakeholders become better and more effective, make well-informed decisions more quickly, and attract funding for maintaining these capabilities make the effort worthwhile.

“Cooperation of the government, civil society, and socially responsible businesses can bring a positive economic effect worth hundreds of millions of dollars,” says Viktor Nestulia, ex-coordinator of the Prozorro project at TI Ukraine and currently Head of Ukraine Support at OCP.

A decade of smart, open, transformational procurement in Ukraine

• Policymakers: The Ministry of Economy of Ukraine and the SoE Prozorro use BI Prozorro and ProBI modules for policy development, analysis, reporting, and informing stakeholders about procurement reforms.
• Buyers: Use BI Prozorro to analyze the market, prepare for procurements, develop procurement Key Performance Indicators (KPIs), and assess their own performance.
• Internal Oversight Bodies and Centralized Procurement Organizations: Use BI Prozorro for real-time monitoring of potential deviations in the procurement processes, risk identification, and internal control over their network of subsidiary organizations.
• Suppliers: Use BI Prozorro to find new business opportunities, analyze buyers, monitor competitors, and increase business profitability.
• Think tanks and NGOs: Use BI Prozorro to identify issues in the procurement sector, conduct civic monitoring, and prepare research papers.