2020 vision: your feedback has set a clear plan for the year ahead
Happy New Year! At the end of last year, we conducted our annual partner survey and received an incredible 185 responses.
Firstly, a bit about the diverse group of partners who answered the survey:
- We had more than double the feedback from our last survey (185 respondents, up from 85)
- The most respondents come from Africa (24%) followed by Europe (23%) and Latin America (19%)
- Respondents work in a variety of areas: public policy (26%), data use (24%), advocacy (19%), and data publication (13%)
- They are relatively familiar with open contracting. Most (about 60%) said they have been working on open contracting for three or more years which gives them a lot of experience to share.
- We also did focused follow-up interviews with 20 individuals.
We’re very grateful to everyone who took the time to participate. You gave us some great feedback that will shape a clear plan for 2020. Here’s where we’ll be making our big investments this year based on your responses.
1. We asked: How are we doing? You said: Do more to empower us in our work
In our new strategy and beyond, we’ve invested heavily in co-design, getting feedback and making quick course corrections when we don’t get things right. We’re thrilled that this seems to be paying off. When we asked you the ultimate partners question, “would you recommend working with OCP to others working in the field of open contracting?”, you said yes and gave us a strong score of 8.6 out of 10 in total.
Our net promoter score – calculated by taking all the positive responses of those who are enthusiastic (8 or more out of 10) and taking away all those who aren’t excited (who scored us a from 0-5) – is 76%, a big jump from 50% last time.
But, of course, pride comes before a fall. We also asked another vital partner question: is our support enabling you to do more of what you want with less future support from us?
You only gave us an average score here: 5.2 out of 10. We’ve made progress from last time on our score but we want to do much better. If we plan to put ourselves out of business, we need a consistent positive score on this (indeed, our net promoter score is not yet positive, enthusiasts don’t outnumber those less convinced). When we dug into the numbers, we found that we definitely scored lower in Asia and Africa, but that we score much higher when we do specific targeted training.
What we’ll do: It’s clear we need to work harder to empower you to work independently of us. This is going to form the core of our program of activities for the year. We’ve begun to correct course in our new strategy and put this score as one of our key performance indicators. We explore how we can better support you under point 2, and we are investing in new programs like Lift that provide comprehensive, sustained support and peer engagement.
2. We asked: What can we do better to support you? You said: Provide more support with advocacy, data use and connections
You wanted us to help you make the case for change, connect you to other community members (especially funders) and support you with data use. These core support needs chime very well with the changes that we made to our work in our new strategy on building political will, focusing on problems, not proposing OCDS as a cure-all solution, and using data.
While you would think it’s the advocates who are looking to us to make a better case, they gave us among the highest scores when asked ‘would you recommend’ and ‘are we helping’.
Where the need was more evident was among data users (and in fact our net promoter score was negative), so it is clear that we have to offer more basic resources and support services.
What we’ll do: Our first response (anticipated by our strategy) has been to invest in two new hires at the end of last year – the fabulous Camila Salazar and Andiegong Okon as our lead data analyst and community and capacity building manager respectively – directly to meet those needs.
Some partners also told us in the comments that our support can get too technical too fast and can focus too much on the Open Contracting Data Standard, which isn’t helpful if you are only starting to make the case for more openness, in a low tech environment, or are a civil society activist who is pushing for change from outside the government. We can do more to help ‘countervailing power’ as our friends at the International Budget Partnership eloquently put it.
We are testing a new approach under our Lift program where we are doing a lot more theory of change support and analysis with our partners. That has been really positively received so we will scale this up to reach more of you. We also think that a new approach of providing really granular diagnostics of what a local procurement market actually looks like (i.e. how much sole sourcing, how many single bid tenders, how concentrated a government’s key suppliers are) will help to make a more powerful case for change. But there is no substitute for sustaining support in the trenches and great political campaigning so we will work more with local partners on this. A great example of an approach that works – just out this week – comes from our friend and ally at the European Bank for Reconstruction and Development Eliza Niewiadomska who explains how she seizes on a good crisis to embed systemic reforms. We shared another great example last year of how the Kyrgyz journalist collective Kloop uses procurement data to hold the government to account.
We will think hard about what else we can do to facilitate more peer connections and more links to funders. We have launched learning circles that we will continue in 2020 and gather feedback on if they are helpful. We are also going to focus more on regional peer gatherings: we have several of these planned in 2020 in Korea, Amsterdam, Cali, Kenya and elsewhere. We also care about connecting you to funders. We track this in our strategy but clearly not everyone is feeling the love yet.
3. We asked: How useful are our key resources? You scored them high overall.
All of our resources got positive feedback from the community, with the most frequent response to each resource being either 4 or 5 out of 5. You found our website and the OCDS documentation most helpful with a high net promoter score. Among the 13 resources surveyed, Flatten Tool was the least used resource.
We received a few comments expressing surprise at the range of resources on offer. Clearly we can do more to make these accessible and visible.
What we’ll do: Over the winter holidays, we have been working on a complete revamp of our website to really put these front and centre. It should go live at the end of January but until then you can search all our resources in this library.
We are going to be improving the implementation guidance section of the OCDS documentation to better highlight available guides and tools. We will also add a new section to the OCP website to guide OCDS data users with information about tools, methodologies, and guidance materials designed to support data use. In addition, we’ll invest in further improvements to the functionalities of key tools (including the Data Review Tool and Flatten Tool) and develop an OCDS data registry. We have already announced improvements to OCDS Kit, which is our suite of command-line tools for working with OCDS data, for example to create compiled and versioned releases and record packages, upgrade from an old version of OCDS to a new version, detect the format of an OCDS file, or load packages into a database.
Finally, we noticed that people were asking us for more policy guidance. To address this, we plan to soon launch the Open Contracting Playbook: A Reformers Guide, and are embarking on a major project with Trustlaw to develop open contracting legislative guidance based on a review of legislation from around the world. You can read a draft of the Open Contracting Playbook.
4. What we also learned: Provide more multilingual and gender-sensitive engagement
English responses to the survey were more than 80% and we only received a few responses from Asian partners although it is a growing area for our work, especially on infrastructure. We will put much more significant effort into multilingual engagement and resources, starting by translating this blog into English, Spanish and Russian.
Only 39% of responses were from women so clearly we need to do more to get our responses at least gender balanced especially as so many women work in procurement, open government and open data. We have a new research program looking at gender in procurement led by one of our open heroines Hera Hussain that will generate insights and more engagement with our work from a gender perspective.
5. Finally you said: Be careful, be inclusive and don’t become the field!
A few of you shared frankly with us that, despite our best intentions, we are risking to become the field. As we’re building up new programs and resources on infrastructure, health or advocacy, you rightly see a risk of OCP taking on more topic areas, staff and funding. We hear you. It’s a tricky balance that we are trying to carefully navigate. We will keep investing in co-designing with you, featuring your stories, approaches and tools and linking you to funders.
Our investment in storytelling and featuring local partners is appreciated and you told us that you want more of this.
Our work to support advocacy and data use more broadly will hopefully help shift the perspective that we are still often seen as mainly working with government actors. You also said that we should try to engage more with local partners instead of with consultants from the North.
You offered some good suggestions on how we can develop meaningful partnerships and deeper capacities, for example by supporting local advocacy campaigns, data workshops or research projects.
In closing, it’s game on for 2020. You’ve set a clear direction for us. We will partner with frontline advocates, and with them, develop more and better guidance on how to make the case for open contracting and build political support. Building on learning from Lift so far, we will try to help you navigate reform management challenges and share our insights with the community.
We will keep gathering tools and approaches for high quality data publication and analytics and bring them to you. We will help you tell your stories, document your progress and share them widely. Whenever we can, we will connect you to each other and partner with you in developing new resources. Most importantly, we will keep asking you how we are doing and what we can do better. We promise, we will listen and act on your feedback again and again. Together we can go further and faster in the year ahead!
