Rallying the power of our communities: Open contracting’s field grew 9 times in 2017

2017 was a big year for open contracting. We brought over 200 people from our community from across 37 countries together at the first OCGlobal for a movement-wide ‘check-in’, expanded our team, added more programmes and took a deeper look at how we organise ourselves for more impact. This allowed us to celebrate progress so far but also regroup with our wider community on our next step forward.

The themes that came about during conversations we had with our partners throughout those big changes got us thinking about who is in our community, and how we can better support them. As an organization committed to learning, we conducted our annual network analysis. We want to put ourselves out of business so we need to foster deep connections between different groups within our community. So are we growing our existing networks growing, strengthening and are we helping bring new groups into the fold?

We measured the following indicators:

• network growth: measured as the number of new users with at least two links to existing actors, compared with the network at a previous time
• network strength: measured as average network connections between members, over time this will tell us if individuals in our network are becoming more connected.
• new communities: measured as ‘as a cluster of at least 25 people who share a common interest’. These were identified by using our core email network and analysing their conversations and networks on twitter (as a proxy for wider discourse).

Here’s what we found:

• The number of actors in our top 10 communities has grown by almost 9 times! We went from 771 actors in 2016 to a whopping 7,160 in 2017. We’ve characterized the main communities as open data, open government, anti-corruption, media & journalists, other transparency actors, technology solutions & vendors, resource governance, funders, business and learning. On a stricter comparison, email network alone grew by 310% (from a list of 1723 to a list of 5336 in 2017). 65% of this email network was also present in 2016 which means we’re retaining our core communities.
• Our network is highly interconnected, with an average of 10.7 connections per actor. This was 9.7 in 2016. This is a good sign, suggesting as new members join, they are also plugging into a tighter more connected network.
• We added 13 new communities. This includes engaging media and journalists, tech & vendors organizations, the private sector, open data and civic tech, anti-corruption organizations, civil society groups and impact and learning organizations in a more meaningful way. We set ourselves a relatively arbitrary target of engaging with 5 new communities to force us to bring new actors into the fold. We have exceeded that.

This is great news all around. To shift power from closed to open, opaque to transparent and to embed our work in long-term strategic and political change – we have successfully connected with more people in diverse communities in more meaningful ways. These results also surface a challenge we are eager to face: how to maintain our mission of building the field, not being the field, with so many players in the game, many of whom are new for us.

If you are deputy mayor of Nepal, a procurement agency in Paraguay or a journalist in the UK, you will need support in different ways. This analysis is just the first step in learning how we can best support these actors, not the last.

By coupling this with our global field survey, and event feedback – we’ve been able to tailor our field-building efforts (see results here or summarized below) last year to gave our communities a big boost. The global field survey revealed that we needed to play a more active role in building connections between groups and convening the field more often. As a result, we leveraged #OCGlobal17 and emphasized on involving more actors (and coalitions) in our country work too. That increase in engagement is now evident.

The analysis also shed some light on challenges for our team and the whole field.

Engaging Communities

Our field survey and #OCGlobal17 told us that we need to do more to engage businesses and journalists more for example.

On business, we’ve only seen a small uptick – our community rose from 50 to only 67, so we have started incorporating more business engagement into all our country work, especially in Nigeria, Colombia and UK. Though we continue working with fantastic partners such as The B Team and IACCM to engage with global businesses, we’re investing in a new role that ensures this approach is reflected in all our programmatic strands.

When we started measuring the growth of our field, the media and journalist community we engaged with was so small that it wasn’t even a category we tracked. It was clear we needed to do a lot more in 2017 and so we did. As you can see below, now almost 12% (1,666 people) of our field is filled with media organizations and journalists who care about the opening of contract information. In 2017, more than 250 stories about open contracting were published in 26 countries and international outlets and a number of journalists joined us at #OCGlobal17. And this year, we’ve trained journalists at the BBC in UK and others in Nigeria, hosted a data dive in the UK, data investigation in Colombia. There’s a lot more to come!

Following the many commitments of 2016, we had a big task ahead of us. Following up with 40 governments, and turning those words into projects and implementation became our mission in 2017. You can see this clearly in the numbers. Following the many commitments of 2016, we had a big task ahead of us. Following up with 40 governments, and turning those words into projects and implementation became our mission in 2017. You can see this clearly in the numbers.

In one year, we grew our community in open data and open government by 27 times (3054%), and anti-corruption and resource governance by 6.2 times (520%).

Mapping also shows that our advocacy with the US government and partners has decreased. This is a direct result of a shift in focus or political landscape there. That said, we’ve just recently launched a program on encouraging US cities to adopt open contracting principles and data standard with the Sunlight Foundation.

Keeping an eye on twitter

Twitter activity inevitably comes in bursts peaking at large events and announcements. We noticed a drop in our influence from 2016 to 2017, which is a shame but perhaps understandable when you think we have the major events of the Anti-Corruption Summit and the OGP Summit in Paris. Our Twitter community is still growing or our own activity is up but over the last year, we’ve had less major Summit announcements etc (and, to be fair, there is a wall of bad news out there that may also be distracting our key allies).

That said, we are after shifting a global norm in policy and practice so we want the idea of open contracting to be omnipresent so we dug more deeply into it to tell us what’s happening and we should do about it. One of the insights is that we have a series of super-connected influencers on twitter who we need to reach out to and engage more. They are the iris of Sauron’s Eye in the diagram below. When we look at the most influential advocates on Twitter, there are many names that would not be regarded as such in traditional terms. Among our top 10 influencers, there were no celebrities or politicians.

The Kevin Bacon Method: 6 degrees of separation

Eyes of Sauron: Core 300 twitter members of Open Contracting’s community surrounded by the entire network

We pulled a complete history of our email records and the last 6 months of our Twitter content and all of the people who @mentioned opencontracting on Twitter, or who we @mentioned. Though “following” is a common metric, we chose “@mentioning” as it is a far better measure of direct influence. This yielded a total of 23,277 names (see the ‘Eye of Sauron’ chart below). The core 300+ followers can be seen at the center of the Eye. However, to make sense of the active core, we only analyzed people in the network who had @mentioned at least two other people in the network, just a few thousand names. To see how this analysis was done, you can read Marc’s blog here.

This is the third year we have commissioned a network analysis. We shifted gears to focus on identifying the most important metrics that are both reliable and useful for changing our strategy, and how the measurement of those can be operationalized. We’ve written code scripts and built web apps to allow systematic measurement, reducing the time required to run this analysis again. So expect more regular updates from now on. A couple of quick notes for the network nerds out there.

1. Defining communities is still an art and not a science. It is still relatively arbitrary. We used a community of about 25 people with tighter connections between them and us than to other parts of the network. In 2016, we were defining them by events (IODC, OGP) but have altered this for 2017 as it is was not useful.

2. Core email growth. Creating granular assessment for this indicator such as the number of staff, programmatic objectives and quality of interactions would provide the context needed to make this metric more robust. We sent 170,380 emails sent in 2017, 139% more than in 2016 and the number of people OCP staff emailed grew by 310%. These are striking numbers but context is key. The total number of emails sent is the easiest to compute, but could be seen as a vanity metric.

3. Twitter has a short memory span. If you’re undertaking Twitter analysis, the timing and frequency of data collection can completely change the analysis. Twitter has a rather short memory span, so most users’ histories can only be retrieved for about the last 3,000 tweets. This makes year-over-year comparisons difficult and impractical in practice. We found 6-month intervals to be more feasible and our analysis represents this. Some users will not be benchmarkable over time (looking backwards), either because they have no history, or they tweet too often and their history doesn’t go back far enough.

4. Don’t forget the context. It may seem puzzling that Open Contracting’s influence decreased on twitter even though the community had grown. This makes complete sense though when you consider that in 2016 our advocacy efforts were much bigger due to summits. In future, we will look at #opencontracting and see if the conversations on there are different to those that involve @opencontracting mentions.

5. Humans are irreplaceable (sorry AI!). Although we have been able to operationalize email and Twitter engagement analysis through building web applications, a human eye is needed to create an in-depth insightful report. Only OCP staff were able to categorize relationships (especially when a person belongs in two groups) into the community groups that matter to OCP’s theory of change (i.e. open government, open data, anti-corruption, etc). A context-based analysis was key to this study and that cannot be stressed enough.

Network analysis is a new and developing field. Better theories of measuring impact, engagement and the interaction between networks are emerging. Though there are a few tools out to do this analysis, most are expensive and few are open-source. This is why we’ve invested in creating tools that will power such work quicker, and more efficiently for us. From redefining how we categorize communities to detangling threads of connection between groups, we’ve come a long way in our understanding of both the importance and limitation of such analysis. We hope that this long read provides some food for thought for others in our movement or wanting to analyze their networks too. You should take pride in the fact that over the past year, you helped the open contracting community grow by 9 times. We shot above our targets and with a richer understanding of our network and tactics to build the field, we’ll do our best to do the same next year.