Five things we heard from you & what we are going to do about it

If you’ve read our blogs, strategies or other communications before, you will probably know that we plan to put ourselves out of business by building a global field of policy and practice on open contracting. Public contracting involves trillions of dollars of public money, and the only way that we are ever going to scale enough to have an impact on that is by building a field of other folks who can help catalyze open contracting around the world.

To understand better how you think we are doing in achieving this goal and more importantly how we can help our partners maximize their impact, we’ve launched our first ever global partner survey to see how our partners feel we are doing in that regard. We’ve partnered with Keystone Accountability to help us carry it out. You can read their full, unredacted report here.

But in this blog, we would like to highlight what we heard from you and what we will do in response. We take feedback seriously, collecting and responding to it at all our events and designing our entire forthcoming global conference around it.

And this is not just a puff piece: keep reading as there are a few points where we get a bit of kicking!

1) We asked the “ultimate” partner question: How likely would you be to recommend the OCP to a friend or colleague working at a similar organization to yours?

This is the classic, ‘killer’ question, looking at whether partners are deriving benefit from working with us.

You said: Very likely. 64% of respondents were highly likely to recommend the OCP to a friend or colleague; another 22% were likely to do so.

Keystone commented: “This is a clear indication that the OCP is valued and that allies find tools and other products useful in furthering their own work.” Responders also felt that we are having a substantial influence on our area of work.

So far, so good.

2) We asked: Are our arguments credible?

You said: Yes.

We got strong positive feedback again. When we asked what particularly resonates there was a pretty strong response that arguments around efficiency and freeing up time and resources for government resonate strongly. As one person put it: “It is listened to quicker and easier than ethical ones, or ones about the quality of actual services delivered, sadly.”

There was also a good push to encourage us to join the dots more between advocacy and implementation. “Some middle steps between advocacy/commitments and implementation are not clearly addressed, which makes it hard to work with middle managers and implementers.”

What we will do: We hear you and are retooling a lot of our guidance to fill that “middleware gap.”

3) We asked: To what extent has your interaction with the OCP helped you to connect with other organizations who support your work?

You said: Hmm. There was a genuine split in the answers here. Interestingly, both positive and negative scores were quite high, with 62% saying that we have been helpful or very helpful, and 39% saying that we have not been helpful at all.

The open responses also show a mix of views, with some people establishing positive relationships outside the OCP, and others less so.

For every positive – such as “OCP has done a great job organizing or co-organizing side events and sessions at key gatherings, such as the EITI summit, OGP events, and the IODC. We have met and/or solidified relationships with organizations through these events’’ – there were more guarded responses – “OCP could stand to convene the community more often and more accessibly.” A couple were directly negative or skeptical: “We create our own groups.”

What we will do about it: It’s clear that we need to do much more work here. Hopefully, the global event on open contracting that we are organizing with Hivos, Article 19, CoST and the B Team will be a great time to consolidate that sense of community and networking. It was also one of the key requests that we found in the pre-event survey. We’ll continue to take advantage of global and regional events (watch out for the Open Knowledge Festival, OGP Global Summit, Abrelatam/Condatos and IODC next year) to bring together existing and new implementers and partners to work together in advancing open contracting.

The key emphasis for us in the question is: who then supports your work? We’re making facilitating financial support to our partners an important part of our strategy. As of September, we track that there have been ten separate organizations funding open contracting projects, and will continue to work on bringing more potential funders into the open contracting community.  

We are also going to begin a new strand of work to focus on building our community connections better. We have put an awesome new team member, Hera Hussain, who joins us from OpenCorporates, to work on it. We carry on measuring regularly measuring our field and how it is growing too (next update on that will be January).

In addition to our OCDS mailing list or the Github, these community connections include some more of the basics, for example, as suggested by one respondent, providing better fora to pose questions and discuss and build a supportive community of users that can work without the OCP. Through a Spanish-speaking helpdesk managed by a great team at the Latin American Open Data Initiative (ILDA), we can do this in multiple languages too. We can also commit to surveying everyone in six months to see if you numbers here poll better and will report back.

4. We asked: Is OCP empowering you to do the things you want with less outside help?

You said: No, not yet.

Those who are convinced that we help are outnumbered by those who are not yet sure. We got a similar response about the level of impact on people’s work too.

On empowerment, we scored above our peer group in similar surveys, and we came in a bit below them on impact, but we think the absolute level is probably more important. This is a miss for us.

What we will do about it: So the second big lesson for us from the survey is that we are going to have to try harder to support our partners to deliver open contracting. Our helpdesk and other services polled very highly in the survey both for its responsiveness and the quality of its support but clearly, we need to do more on that next step of helping partners move beyond using helpdesk to then help others elsewhere.

You’ve told us that you need help to better shape your projects and measure the outcomes better. So for one, we are redesigning our website to bring all the guidance and tools together in one space and make it accessible and actionable. One of our priorities is better guidance on use cases, and we should have something to share with you all in time for November’s #OCGlobal17 gathering.

It is also partly connected to feedback we get on the OCDS. It’s a powerful tool, but we need to balance the tech with policy reform, which can be tough, especially in politically charged climates. The flexibility of the standard means that it can be challenging to know where to start and ensure it is sustainable. On a good day, the OCDS is a trojan horse for much wider systemic reforms. On a bad day, it can be taken as a short-lived technofix. We want it to be as painless as possible to get the best bang for buck, but we also want it to work, which means it needs to produce effective, granular information so implementation demands time, care and user-centered programming.

5. We asked. To what extent is the diversity of those involved in open contracting discussions adequate for achieving the strategies and meeting purposes of the OCP?

You said: Not enough.

We agree too. The survey comments agree with our sense that while we are doing well with bringing technical actors to the table, we need to involve (a lot) more businesses and demand-side players such as CSOs and journalists. They tend to be country specific so need extra effort to engage. Part of the reason that working on public contracting is so powerful is that it touches everything and everyone. Still, we chose it because it was hard, not because it was easy.

One of the comments: “Definitely a need for more private sector engagement and support. A challenge in this regard is the potential perception that what open contracting is, is set and only one thing – and that it is a technical fix and not something to be experimented around or shaped.”

What we will do about it: We need to put (a lot) more emphasis on folding these actors into our catalytic engagement work in countries. We are especially pleased that we have some new funding coming in to enable us to do this better with partners in Latin America and elsewhere. This also very much chimes with our ambition to make sure that open contracting is not just about the data but focuses on long-term strategic change. We will be changing our templates for programming to specifically reflect this, to better weave these actors into coalitions at the local level.

At the international level, we heard that businesses feel that the conversation has already partly developed on open contracting without them, so they struggle to engage outside general cheerleading for the principle of fair play. We want to ponder this one, especially with allies like the B Team. We’d love your ideas and will go back to explore how business can play a stronger role too.

What next?

The conversation doesn’t stop here. Hopefully, we’ve clearly laid out what we have learned and what we are going to do about it.

Fessing up, we will go for shorter and sweeter surveys in the future. We asked people a lot of hard questions that made them think quite hard about exactly how OCP has helped them. This can trigger cognitive fluency bias which isn’t always great. We’ll explore the tools we can leverage to make the survey process less onerous next time.

Over the coming year, we will deploy smaller surveys to check things are going in the right direction. We will also directly integrate this feedback into our upcoming global event, our learning program and our next strategy cycle.  

So, lots of good lessons to make us stronger, smarter and better and at least three good ideas that we can immediately put into practice. Great feedback folks, keep it coming!

Your feedback in one quick table

This table summarizes your feedback based on the methodology by Keystone Accountability. It separates the respondents into Promoters, Passives and Detractors based on the values of the scale (Response of Promoters: 9-10, Passives: 7-8, Detractors: 0-6). This is then calculated into an average Net Promoter Score that can be compared to results of similar surveys run by Keystone Accountability.

We like the Net Promoter concept as it is sharper and clearer than just an average scoring system, providing a single score for the interpretation of data. It’s tough but fair. We want people to be stoked about working with us, not that we are just another in a long list of ‘me too’ collaborations.  

This benchmark includes organizations such as Transparency International, Practitioner Hub, Action for Community Development and Gender at Work.

Photo: Marco Verch. Big headphones, close up (CC BY 2.0)