Collaborating toward open contracting

By Josh Powell, Director of Innovation at Development Gateway.

Last week, Tom Steinberg caused a buzz in the open data community with his strong commentary on the flaw of political correctness in the government transparency space, positing:

“Perhaps most importantly, and worryingly, I’ve not seen any meaningful attempt to systematically overhaul procurement rules to ensure that new government computer systems produce decent open data by default. This omission is especially important since building open data feeds into new government computer systems remains the only way that most government data will ever get released cheaply, quickly, and in appropriate formats.”

In response, John Wonderlich’s counterpoint response on the Sunlight Foundation blog argues (emphasis mine):

“In the face of slowly winning a cultural fight, and as ‘contracts should require the creation of open data’ is broadly adopted as common sense, the bigger barrier will likely be technical. This is part of why we’re so glad people like Waldo Jaquith, founder of U.S. Open Data and adviser to Sunlight, are chipping away at this very problem on a software level. They’re anticipating public officials’ and data managers’ needs and designing systems that bake in openness from the beginning.”

In my experience – largely in the context of collaborating with developing country governments – these statements are both partially correct, and partially problematic. Taking them in the order presented:

New computer systems, with openness by default, are the only way for most governments. (Steinberg paraphrase)

This statement runs into two problems:

1. Often, no central “system” is used to collect or aggregate procurement data. Instead, governments typically have many systems, fragmented across ministries and subnational administrations, which use everything from paper filing to Excel sheets, from databases to proper eProcurement systems. Furthermore, many “e-procurement systems” are more post-hoc reporting tools, rather than transactional systems that capture live data. (This has implications for data quality through lazy/sloppy reporting and insufficient budget for validation – see below.)
2. In many cases, the investment in the “system” has already taken place. Many countries have established eProcurement systems over the past 5 years, only to find them quickly out-of-date and unable to adapt to new norms of openness and data standards (like the Open Contracting Data Standard). In these cases, the best solution often involves creating intermediary platforms that can pull data from the eProcurement system and republish as open data. This process is far from straightforward, and likely requires collaboration across vendors.

“The bigger barrier will be likely technical.” (Wonderlich)

Whenever someone suggests that technology is the biggest hurdle, I am immediately uneasy. While wonderful that openness by default is a growing norm, governments can only create useful open data if they have useful data to begin with. In many countries, governments themselves do not have centralized, reliable, timely data on procurement activities.

Three political economy challenges which must be solved before broaching the technology/software barrier (from my perspective) include:

1. Typically, weak legal mandates exist for data collection in central procurement agencies. Even weaker are budgets for collecting, validating, and producing data.
2. In the absence of centralized reporting mandates, substantial sunk costs have typically been invested in dozens of government procurement systems within a country – systems which were never designed to talk to each other. So while true that many governments are now purchasing e-procurement systems (raising the real opportunity that Wonderlich predicts for simple features to create openness by default), dozens more are “stuck” with systems that are not new enough to be useful, but not old enough to justify scrapping.
3. Standards and regulations for data quality, content, and schema for interoperability and disclosure are not defined – and when they are, often ignored.

Procurement reforms are currently present in 47 OGP National Action Plans – this strongly supports Wonderlich’s statement that this ideal of open contracting is the new norm, rather than something that must be “beaten out of governments.” However, the standards, processes, and legislation to make this a reality have a long way to go. Those focused on procurement transparency must collaborate with governments to: understand the practical hurdles to open contracting, advocate and advise on good practices for streamlining lines of reporting and standardizing data schemas, and (yes) help introduce useful technology as the “last mile” tool for converting these changes into sustainable and useful open data.

At Development Gateway, we – like most – are early in our programming on procurement transparency, and are trying to apply lessons learned from a decade of helping governments collect and open aid information. Through ongoing support to the government of Vietnam and studies of the governments of Nepal and the Philippines, we are actively learning the challenges governments face when operationalizing their decision to open. Over the next few weeks, we will have a series of posts on our experience in Vietnam, discussing processes, incentives, technical tools, and implementation of the OCDS – we look forward to sharing and learning together.

The blog post was previously published at Development Gateway’s blog.