• 15 July 2011
• in Conferences, Ideas
• 6 min read

“Everything I need to know about open data, I learned from open source”

BoF “Open data in development” at OKCon (via Tobias Eigen)

But what did we learn from open source? Two days of Open Knowledge Conference gave lots of food for thought. And lots of inspiration as well: plenty of projects doing interesting work, and experiences to share. And to add a cherry to the cake, we had a great “open lunch for development” with several people active in development aid. My (delayed) take-aways for Open for Change.

From data.gov.* to data.your.org

Nigel Shadbolt and Andrew Stott shared their lessons from setting up data.gov.uk, and Tom Lee talked about data.gov and the recent threats of its budget cuts.

It’s crucial to have top-down support, bottom-up activists, and middle-tier connectors, to bring everything together.

• We need to continue nurturing a network of people active in open data for development, to make sure we have the tools, ideas, reports, cases, and standards we need, and to support the early adopters within organisations.
• We have done some work on an “open data briefing”, based on the OKFN open data manual, and we need to continue work on that: in four pages, explain the why, how, what and who of opening up your organisation.
• It’s important to understand the decision-making and budgeting processes to make the case at the right time and the right place. We could review the best material on “how to convince your boss to use open source” as a starting point.

There are many reasons to embrace open data, don’t rely on a single one to make your case: tranparency and accountability; economic value, growth and innovation; efficiency and cost-effectiveness; improving (public) services; (public) engagement; and civil society and social capital.

• We should have good stories illustrating each of these reasons, and not rely on just one. Aid transparency is hot right now, but many actors want to go open for one or more of the other reasons (and may even be scared off by too much transparency). The paradigm shift is what matters to us.
• Andrew had a great slide with excuses to justify “data hugging disorder”. We could turn it into a data hugging bingo game for workshops.
• People tell stories about people. Numbers can be good (in the US case, showing that the IT dashboard project resulted in $3 billion savings on IT spending makes a strong case).
• Some stories may be too scary for some organisations. The UK, for instance, now provides the organograms of all departments, including for instance salaray ranges at each level, and creating a web address for each job post to include it in the web of linked data.

Close the feedback loop: the “build it and they will come” approach won’t work here either. Try to publish data that matters to people, but also consider that “data probably has a long tail”.

• Releasing data early and incrementally creates a steady news flow, and also enables you to work with feedback and to champion people to create peer pressure on the “refusniks”.
• Friedrich Lichtenberg tries to turn the lack of opportunity to debate what you see in “ Where does my money go ” into an opportunity for “ Open Spending ”. In essence: how do we provide support to create a data cycle instead of a data pipeline.
• There is a challenge to create or appoint authoritative sources and URIs. In fact, this was a key topic in the online workshop organised by David Pidsley during the Open Data for Development Camp.
• In a broader sense, the emerging standards and work done on open data in development cooperation needs to settle down in one or more ontologies. This will also pave the way to build more focused data enrichment services that can tap into the existing wealth of documents we have, to help professionals navigate those.

Statutory requirements matter.For governments, this mainly has to do with legal frameworks and obligations. But every organisation could (and should) enshrine crucial elements of open data in their policies: how to ensure “open” stays open, and how to prioritise.

• The UK government re-used lots of existing policies, such as the National Statistician’s guidelines about reasonable measures to prevent identification of individuals. The open data manual could be a good portal to such policies.
• Principles and policies should not be set in stone (at least: not early on), to prevent weasel words like “to the extent feasible” creep in. The essential lesson: in the first phase, compromise on the data that will be published, but not on the license applied to it. (Again: the paradigm shift in thinking and acting is what matters first.)

Why we want to be open: the fallacy of community collaboration

In open data, a lot can be learned from “open source”. In terms of tools and practices, I think we are, but in terms of the stories we tell ourselves and others about why we do it, Benjamin Mako Hill gave some interesting insights on the promise of open source to create better software because more people will be able to see, comment on, and improve the code.

In reality, this hardly happens. He showed graphs of projects on Sourceforge, the first major hub of open source software. The median number of developers working on a project is one. If you only look at “mature projects” (multiple releases, longer history), the median is still one. If you look at the most popular projects (10% of most downloaded), the median is two.

In other words: there are very, very few projects where mass collaboration did happen.

And we actually don’t really understand why some of them succeed.

It doesn’t mean we should not do open source, but we should not promote it with the story that it leads to peer review and better code. There are plenty of other reasons, though, and we should make sure we capture those in our Open for Change Manifesto as well:

• It gives the users freedom: autonomy, control and empowerment. The technology constrains how and what we can communicate as people. Openness allows you to remove barriers.
• It is resistant to “anti-features” (limitations built in to charge for removal). With an open license, anyone can process a data set to make it more useful for themselves or others.
• It makes failure cheap: since the investment to be open is low, there already is reward in just making your own solution available.
• It also makes success cheap: some products failed to have a big enough market to sustain a company producing it, but a community of users can produce and maintain it.
• It is not dependent on persons or organisations: even if the original producer(s) stop working on it, others can continue and keep it available.
• It sometimes does lead to mass collaboration. And it then can produce something that would be impossible to organise through traditional means.

Why we want to be open: a stronger vision

In our beta Manifesto, we tried to capture the essence of why we want to be open, and OKCon was a chance to reflect on it.

I liked a definition given by Jose Alonso of the Web Foundation: the web is humanity connected through technology. And as Brewster Kahle of archive.org said: the last generation put a man on the moon. Pretty cool, but our generation can make all knowledge available to all people on earth, for always and for free. That’s a powerful ambition too.

It is crucial to also translate the promise of open data, open access and open knowledge to “effective use”: how do we make sure we create autonomy, control and empowerment, but more even so: security for the ones who want to realise their “ right to access”? “Open” is part of a struggle for human rights.

Hopefully, a joint “Slash Open” campaign can unite the efforts of many organisations working for humanity in shaping the technology we need and put it to effective use.