An “ODD” chance to bring open contracting to life: open contracting front and center at Open Data Day 2018

Authors: Katherine Wikrent, Nicolas Penagos, Carey Kluttz, Jorge Florez (Global Integrity), Chelsey Lepage (Reboot) with contributions from Aura Martinez. Cover image: Kate Paez during #RallyColombia

One day, 406 events, and thousands of brilliant minds working on tackling social issues with open data: the latest installment of an annual celebration of open data brought the magic to all corners of the world.

We jetted across the Americas to support three stellar events co-led with our in-country partners in Mexico, Colombia and New York City (you can read up on each event in detail below after our overall top takeaways).

Open contracting is not monolithic. Different laws, the extent of open contracting commitments, internal capacity, and available resources mean open contracting takes on a different face in each of these three countries. What was powerful about these three events (an Open Data Rally and Data Expedition in Mexico, an Open Data Hackathon in Colombia, and a Treasure Hunt in New York City) was collaborating directly with publishers and local users to deep dive into some of the most pressing challenges in each context and harnessing the unique strengths of each country’s open contracting efforts and data availability.

Data use events provide insights into how citizens interact with publishers to appropriate open contracting data for their unique use cases. We are keen to see what will come out of the projects and conversations started on this Open Data Day, which was registered by Open Knowledge International and supported by our friends at Hivos, SPARC, Mapbox and the FCO.

What we learned

Mexico: The two Mexican events, apart from being truly fun to participate in, were also a good exchange space for cross-agency and cross-sectoral learning. The live demonstrations of portals, data extraction techniques such as API calls, and visualization platforms by government and CSOs aimed at learners from all levels and provided the opportunity for deep engagement between data publisher and data user in a safe setting. Not only did the users benefit from a detailed explanation of how to use the data and tools already available, but the publishers also received feedback on the types of information and data available and the formats in which they are released.

These opportunities for honest and productive cross-sectoral exchange are rare but incredibly valuable to make open contracting really thrive. We hope this event is just one of many spaces for these kinds of exchanges in Mexico this year.

Colombia: We found that potential solutions to problems in public policy are sometimes overlooked by the government and sometimes can be simpler than what they seem. Using open data to show which food items should be delivered to Colombian schools, and allowing parents to complain directly if the quality or quantity is subpar, could greatly improve the school meals program in many regions.

Even among university students, training on how to use data, build attractive visualizations, and work with large datasets is a must. In a world where the buzz about new methods and technologies like blockchain and artificial intelligence dominate the conversation, for open data practitioners to unlock the real possibilities of these tools, they must start with the basics of data science. Though these technologies might become game-changers for certain problems, they may not apply or be useful for every problem we face, and can even alienate certain users due to their complexity.

Finally, several teams identified duplicated data. They also realized that they could not easily track spending between the contract database of the public procurement agency, Colombia Compra Eficiente, and the budget and payment database of the Ministry of Finance. There is ample work to be done to streamline these two datasets, which would give citizens a better understanding of how their tax pesos are being used for procuring public goods and services.

New York City: We were able to identify concrete data challenges and gaps, such as lack of unique identifiers, problems linking up multiple data sources, ambiguities in field names or categories related to demographic information. We revealed opportunities to enable more effective access to and use of open data to address problems in the city.

For example, we developed insights that could inform the city’s work engaging Minority and Women-Owned Businesses and help those businesses increase their participation in government procurement.

Mexico has been pioneering open contracting – from global leadership in groups like the Contracting 5 (C5) and numerous open contracting efforts at the federal and subnational levels in Jalisco and Mexico City, to linking up contracting and budgeting data through use of the Open Contracting Data Standard and Open Fiscal Data Package.

But just opening data isn’t enough: real open contracting comes from government working together with civil society, journalists, academia, and others to make sure that the data published is used for monitoring government spending and finding ways to improve the quality of goods and services.

Leading up to the Open Data Day, the Transparencia Presupuestaria team of the Ministry of Finance launched an open call for public monitoring of spending on government works, which culminated in the Data Rally #DatosEnLaCalle. Rally participants searched the Transparencia Presupuestaria portal for current contracts underway across Mexico and spent two weeks cross-referencing information and visiting the project sites in person to report on progress. Teams shared this information on social media seeking to spark public interest in public contracting, including among non-technical users. And spark, they did! By the end of two weeks, the numbers spoke for themselves:

Registered participants: 487
Finalists: 37
Participating Mexican states: 17

Projects visited: 214
Publications: 843
Videos: 111
Infographics: 68
Photographs: 775

Total interactions on Twitter and Facebook: 8202

On 3 March, a panel of judges from Coordinación de Estrategia Digital Nacional, Global Initiative for Fiscal Transparency (GIFT), Red Viral, Secretaría de la Función Pública, SocialTic, and the OCP rated the teams on a variety of factors such as the number of submissions and creative sharing of information. They awarded three prizes to teams who went above and beyond:

• 1st place: CIEP Mx (Centro de Investigación Económica y Presupuestaria). This Mexico City team plunged deep into multiple portals and linked up data across sources.
• 2nd place: Jorge Alfredo Domínguez Pedraza, who single-handedly monitored key works in the Benito Juárez area of Mexico City and conducted captivating interviews with citizens whose lives have been touched by public contracting projects.
• 3rd place: Triángulo de Acción. This team from Mérida explored projects in their city and distributed informational packets to invite their fellow citizens to learn more about public works.
• Special mention: Las Visionarias. This team tirelessly tracked projects and displayed the captivating results in their hometown of Torreón, Coahuila.

The OCP recognized the hard work of each of the winning teams with gift cards, so team members could buy technological equipment like cameras to continue documenting public works in their communities.

Open contracting in action: A sample post by special mention winner Las Visionarias showing progress made on the construction of the Metrobus Station at Laguna Estación Nazas, which began in 2016. Some 30% of the funds have been spent for the project, which is 75% complete.

To complement the Data Rally, the OCP also supported Transparencia Presupuestaria’s Data Expedition, which invited users of all technical levels to plunge into Mexico’s plethora of open contracting data and create customized visualizations about the results of their analyses. During the workshops that led up to the Expedition, we, along with our government partners as well as organizations such as PODER and Google Developer, gave demonstrations of how to work with Mexico’s open contracting data, from making API calls to analyzing massive quantities of information. (As a side note, the Expedition sounded so interesting that I (Katherine) even tried to sneak into a group to create a project. I was quickly found out by the organizers and asked to please not be such a nerd. I reluctantly conceded.)

The Expedition was just as successful as the Rally. Users from all levels extracted real data from the Mexican portals and created captivating graphs, infographics, statistical analyses and regressions, and narratives. The winners of this challenge (Santiago 3rd place, Los Regios 2nd place, Luis Javier Ortega 1st place) can be seen below, grinning over their prizes.

Following Mexico’s example, an infrastructure data rally #RallyColombia was also set up in Colombia. Teams from around the country searched for public works contracts and checked on progress in the construction of roads, sports venues, and parks, among other infrastructure projects. The competition was fierce to the end, with several teams dashing to sites up until the last minute, documenting the stories with videos and photos and sharing the data they could fetch from the Portal de Transparencia Económica and SECOP. Visiting 26 sites and documenting his findings in clever reporter-style videos, Juan Fernando Libreros Lozano emerged victorious.

The race to the finish was hardly the end, as the real treat came with the 24-hour hackathon #AlimenData, hosted at the Ministry of Finance. Public interest in school meals programs has spiked after the Secretariat of Education in Bogota successfully overhauled the contracting scheme for its public school meals program with the help of open and transparent data, and because of shocking irregularities found in food contracts for schools in several regions. It is now clear to many that solutions to the school meals problem must come from a joint team of civil society and government, and this is what the #AlimenData Hackathon was all about.

Ten teams of finalists from across the country were selected to begin their deep dive into the budget and contracts databases at 6 pm on Friday, 2 March. The dedicated teams stayed there, working hard and getting a few hours of sleep by taking turns on the building’s couches. Mentors were available to provide support with everything from analyzing data, helping eliminate stage fright, and making preparations for delivering their final pitch at 5 pm on Saturday. Tired but enthusiastic young participants washed their faces and straightened their clothes as they got ready to present their findings and solutions in front of a jury made up by the organizers and representatives from the Ministry of Education and the National Planning Department. The three-minute time limit proved challenging for most of the groups, and a tough set of questions followed each of the presentations.

Exhausted finalists soldier on as the hour of the final pitch approaches

A team from Medellin present their solution

At 7 pm the jury made a decision, after having evaluated aspects including presentation, impact, and use of suggested databases such as the OCDS datasets published by Colombia Compra Eficiente. The winners were two students from Bogota’s Universidad Distrital, who presented a solution to show contract data and the corresponding food items that were being bought for children, along with a module for parents to complain if these items were not delivered. They even proposed to keep every piece of information secure using blockchain. This work will be reviewed by the Ministry of Education to see how some of these ideas can be incorporated into a new scheme for contracting purchases in school meals programs across the country.

In recognition of their effort, the OCP awarded the team a trip to attend the International Open Data Conference (Abrelatam) in September in Buenos Aires. Open Knowledge Colombia made good use of the mini-grant they received to support this event, and invited the thirsty and tired finalists to some well-deserved refreshments at a bar in the north of the city, where teams could unwind and reflect on their experiences.

The Open Data Day in Colombia was organized by a joint team of the Ministry of Finance, the Ministry of Information Technology, the Transparency Secretariat, GIFT, the Open Knowledge Foundation Colombia and the Open Contracting Partnership.

New York City is already a leader on open data at the city level in the US – evidenced by their Open Data Week (not just a day!) as well as an open data law, open data portal, and an open data team housed in the Mayor’s Office of Data Analytics. Given that more than $14 billion dollars per year is spent through contracts in the city – nearly 20% of annual city spending – we are convinced that extending this open data leadership to cover open contracting will be a quick win for NYC.

On March 9, we paired up with the brilliant brains at Global Integrity and Reboot to launch a Treasure Hunt, a deep dive into the City’s contracting data to find hidden nuggets of data gold. The central theme of the Treasure Hunt was using open data to promote greater economic inclusion of Minority and Women-Owned Businesses (M/WBE) – a key focus of city agencies against ambitious targets announced by the Mayor’s office of awarding 30 percent of the value of City contracts to M/WBEs by 2021, and $16 billion to M/WBEs by 2025). The city has put in place important actions such as: publishing procurement plans targeting M/WBE contracting opportunities; setting goals for M/WBE participation in procurement by agency and overseeing its implementation; publicly tracking agency spending on M/WBE contracts through platforms such as Checkbook NYC; and conducting capacity building and matchmaking activities.     

During the hunt, users from all technical levels and familiarity with city procurement policies formed teams to plunge into NYC’s open data and develop practical ideas on how to enable M/WBEs to get a bigger share of government spending. After demoing some of the available open contracting data on procurement opportunities, we guided participants –a range of data scientists, government representatives, students and civil society actors and advocates–  through a data challenge in which they used their sleuthing skills to either help M/WBEs identify procurement opportunities within the public and private sectors or to inform M/WBE efforts in preparing bids and finding the most promising avenues and strategies to grow their businesses.

Our fearless leaders directing the data expedition on providing guidance to M/WBE about preparing procurement bids

The groups concluded the hunt by presenting their findings and ideas to address some of the gaps they observed while working with the data and transform existing datasets and sources into more actionable resources for minority- and women-vendors. Spanning tips for relationship-building with prime vendors, to introducing common identifiers and filters to better enable data joining across the procurement cycle, to the development of new data products to allow more strategic analysis of successful vendor profiles and strategies, teams were eager to share stories of how their observations of common trends and challenges might be used to bridge between the City’s extensive public contracting data sources.   

Highlighting opportunities to link New York City’s different data sets in ways that can help expand opportunities for M/WBEs, this event sparked new discussions and opportunities for collaboration between the City’s public and private sector actors to make procurement more efficient and transparent extending far beyond Open Data week.

Until next year!