FUNES, the corruption-detecting algorithm that ‘never forgets’

A website, a search engine and more than 245,000 public contracts. By simply entering the name of a company or entity, in the blink of an eye an algorithm checks hundreds of documents and calculates the corruption risk as a percentage. This is how FUNES works, the tool that the Peruvian news outlet Ojo Público created to investigate government contracts. 

FUNES has already played a key part in many of Ojo Público’s investigations. In 2019, for example, they revealed that the main milk supplier in Peru was the sole bidder for 90% of the contracts it obtained and the government paid him more than US$70 million.

The platform is the result of more than 10 people’s work over one year. With the support of a grant from the ALTEC initiative, journalists, programmers and specialists in statistics and law formed a multidisciplinary team with a clear objective: to investigate the public procurement system in Peru in a more systematic way. In 2020, the tool won first place in the “Innovation” category of the Sigma Awards, the most prestigious prize in data journalism. 

Why the name? “Funes the Memorious” is the protagonist of a short story by Jorge Luis Borges who had the so-called sage syndrome: the ability to remember everything down to the smallest detail. This is the spirit of the algorithm that Ojo Público’s newsroom created. An instrument to read thousands of documents and detect signs of “collusive” corruption: instances in which a public official is linked to a company and uses his power and influence to win a public tender.

The FUNES algorithm is unique. It is based on a methodology, developed by Mihaly Fazekas, which proposes a definition of corruption according to certain indicators. More than 20 parameters, each carrying a certain weight, are used to calculate the corruption risk of the company in a few seconds. The tool draws on four databases with more than 245,000 documents: it analyzes records of public contracts, electoral campaign contributions, lists of suppliers and even the Official Gazette of the country. When the calculation is complete, the alerts or “red flags” allow Ojo Público to tell stories that expose cases of corruption to promote changes and improvements. Ernesto Cabral, one of the journalists in charge of the project, says that FUNES not only helps to shorten the time it takes to conduct an investigation, but it also gives a better overview of the contracting system by combining different databases. Nelly Luna, co-founder and editor-in-chief of Ojo Público, says that FUNES is much more than a tool for journalists. FUNES allows to “exercise greater citizen control.”

The birth of Ojo Público’s FUNES: behind the scenes of the project

But the project did not happen overnight. The information used by the tool is the result of years of searching for data, cleaning and assembling public databases that were not readily available. “When we asked the departments for access to their information, they told us that it was on the website, but they only had general search engines and the databases we were looking for weren’t there,” says Cabral. This led the team to create scripts to download the data that the government sites did not show and that was critical to the newsroom’s investigations. However, the process for obtaining information on public contracts was even more complicated: “With the body that supervises government contracting, we had more difficulties because when they detected our IP, they blocked it. That was the most difficult,” Cabral says. 

Organizing and cleaning the data was another challenge that Ojo Público faced. Although the team had worked on stories with data, they had never encountered such a large volume of information that combined several data sources at the same time. The FUNES algorithm also included testing processes that were carried out simultaneously against the data that Ojo Público structured and organized to upload to the platform. 

Validating the results produced by the tool was a critical part of developing the algorithm. At this point, the weighting given to the various indicators was adjusted to reflect the Latin American context more accurately. For example, Cabral says that the weight of the “campaign contributions” variable was increased. They also had to make adjustments to how the algorithm weighted regional contracts: “FUNES originally gave greater emphasis to local governments due to the number of contracts, but later we saw that a ministry had three contracts that, combined, represented much more money than the contracts in those places.” 

Since the Open Contracting Partnership also developed a red flag system to detect risks and inefficiencies in contracting, Ojo Público used their model to re-validate the algorithm analysis. “We always ran the same data against the OCP model; if we got the same results, it was a sign that we were on the right track,” Cabral says. The Open Contracting Partnership alert scheme evaluates, among other aspects:

• Short bidding periods
• Low number of bidders
• Low percentage of contracts awarded competitively
• High percentage of contracts with amendments
• Large differences between the value of the award and the final amount of the contract

Confronting pre-existing beliefs

FUNES also had a big impact on the journalists who worked on its development: “The results that the algorithm produced confronted you with your pre-existing assumptions… we were faced with our own prejudices. You might have though the biggest risk of corruption occurred in a certain entity, but the analysis threw something else out. At that time [when developing the tool], we had to verify whether it was our prejudice or indeed if we should adjust the weighting of the indicators, ” says Cabral. The Ojo Público team recalls that the duration of the project was also a challenge: “It was a full-time job for a year and we had to keep up the pace, drive and motivation of the team throughout.”

Automation and new stories

“There are still many stories to tell,” says Cabral about the use of FUNES and adds: “Our plan is also to automate the process for loading the data onto the platform”. The COVID-19 crisis affected the way newsrooms work all around the world, but it also showed the importance of journalistic investigations into public contracts. For Cabral, the current crisis is a turning point: “With the pandemic it became clearer that procurement is not something abstract, it affects people. For example: if there is corruption in a contract to purchase masks, within days many people can die. The pandemic highlighted the direct relationship between contracting and violations of human rights, especially those of the most vulnerable.” 

FUNES is now an essential part of the investigations carried out by Ojo Público within the framework of the RED PALTA network, an alliance of Latin American media outlets scrutinizing public procurement. 

What was the main lesson from FUNES? Cabral thinks for a few seconds before answering: “We journalists are used to working on specific contracts. When we do, the only thing that happens is that they remove the corrupt official and thus, the corrupt system continues to operate. So we need to say: let’s not look at one case, let’s not look at ten, let’s put the focus on 200,000 and find the common pattern in all of them. That’s what FUNES does. It allows us to have a holistic view of the system, to fight corruption in public procurement more effectively.”