Skip to content

Updates

It’s the linking, stupid!

The current discussion around open data often boils down to releasing data sets, and seeing nice visualisations and apps. But lets not forget that the full phrase is linked open data. The real power comes from linking the data. This week, the Open Government Data Camp in Warsaw lets us explore this more.

Just as web pages today link to other pages for further information, the data sets of tomorrow will link to other data sets, for more data. Your browser will help you navigate the data space.

The BBC is ahead in this game, and working on a “Digitial Public Space” project, linking together many sources of cultural data. Jake Berger writes on the BBC blog:

Early versions of this data model indicate that – as hoped – there will be many, varied and often unexpected journeys that can be made through these catalogues and the material they describe. For example, a user starting out by watching a film of a production of Macbeth from the Royal Opera House might then look at a scan of a rare musical manuscript from The National Archives, then browse similar manuscript scans held at the British Library, watch a clip from a BBC documentary about how paper was produced in Shakespeare’s era, before ending up learning about the plants used to make the paper using information from The Royal Botanic Gardens At Kew. In a [Digital Public Space], all of this could happen in the same online space.

That may still sound a bit like the current web of pages. Except: the publishers only provide standardised links for “Shakespeare”, “paper”, etcetera, and your browser makes the connections to offer you ways to move forward:

Mo blogged about the development of a web browser-based user interface, which navigates through these catalogues using the concepts of “people”, “places”, “events”, “things” and “collections”.

In international development aid, the IATI standard is an effort to work towards a similar “digital public space” in which you can navigate through “organisations”, “activities” and “results”.

An important part of establishing that space is to work towards joint standardised identifiers. At our ODDC conference in May, David Pidsley’s Virtual Workshop on Linking Development Data was focused on that, and next weekend, Tim Davies is organising an Organisation Identifiers Workshop as a fringe event for the Open Government Data Camp, in Warsaw, to continue working on this. And we’ll have more general Open Data for Development: Open Space session on Saturday morning.

Peter Eigen: Overcoming the fear of transparency

We've participated in two interesting events in the last two weeks: the Open Aid Data Conference in Berlin, and the IATI in the Visigrad countries conference in Prague. A proper post is still due, but here's already a video of the closing talk from Transparency International's founder Peter Eigen, "Overcoming the fear of transparency":

[openaiddata.de] Peter Eigen – Overcoming the fear of transparency from Open Knowledge Foundation on Vimeo.

  • in News
  • 2 min read

Dutch development data and results online

../../../assets/posts/bb21848612edc9fd9e03b903a0047562_MD5.jpgToday, Ben Knapen, the Dutch State Secretary for development cooperation, presented the “Resultatenrapportage”, the “reporting of results” on Dutch efforts in development aid in the period 2009-2010. He used the occassion to also present the first release of Dutch government data in the IATI standard format, making The Netherlands the fifth signatory to deliver on its commitment for phase 1 of the IATI agreement.

The “Resultatenrapportage” report itself is the result of the collaboration of some two hundred writers from government and NGOs, in a process led by the Ministry of Foreign Affairs and Partos, the Dutch platform of organisations in development cooperation.

To make the report a stepping stone towards sharing more data online, it is no longer being printed, but instead made available as PDF document, with links to offer easy access to background documents and data.

At the same time, the State Secretary announced the opening of http://www.minbuza.nl/opendata with the first set of spending data released, up to date to the end of the second quarter of 2011.

Behind the scenes, the last bits are put in place to automatically register the data at the IATI registry. The data will be udpated quarterly, so the next release is expected to already happen in early October.

It will be interesting to see how the regular publication of the data leads to continuous improvement of the data: less jargon, less abbreviations, clearer descriptions.

In addition to the IATI data, there also is a set of documents of embessies, reporting on the progress of particular projects in countries around the world. An open challenge is to now find ways to link the information in these documents to the data available in IATI.

Knapen expects civil society to also publish their data in IATI, so that it becomes easier to look directly into the operations and processes happening at the core of international development cooperation.

The Guardian has made a website to slice and dice the data of the World Bank, DFID, etc. In collaboration with Akvo, the Ministry is doing a pilot project to show their water project portfolio, but Knapen hopes that parties in The Netherlands will also explore the expanding universe of data to answer their own questions and develop their own perspective on what is happening.

After offering the report on a (symbolic) USB stick to Nebahat Albayrak of the Parliamentary committee for foreign affairs, she said she hopes the open data will make the political debates more informed and perhaps even a bit more objective, but also stressed the importance of reflecting on actual impacts as done in the “Resultatenrapportage”.

Albayrak also hopes to also evaluate the “resultatenrapportage” and the “open data” initiatives and standards from the perspective of effectiveness for their work in the “kamercommissie”.

What are the effects of open development?

At “Open Data for Development Camp” in Amsterdam, Marijn Rijken of the Dutch research institute TNO presented on “open data opportunities in development”. Together, we’re now drafting a research proposal to gather answers on pertinent questions around open data in development: “What are the social, organizational, technological, financial and legal effects of open development?”. It’s part of our efforts to build network as the basis for a Dutch knowledge platform.

../../../assets/posts/ecc185b1d82babb476abb813afdd9442_MD5.png

TNO has done similar sector-wide research around open data in other sectors, and would like to take the existing research in this area a step forward. AidInfo published a cost-benefit analysis on open data, and a framework for this. The Transparency and Accountability Initiative recently published a report on introducing open data in middle income and developing countries. And we also like to include “effective use” and impacts on e.g. privacy and security, and a possible “data divide”.

We plan to look at existing literature and research, sketch a vision on what “open development in 2020” could look like, and provide a framework for a social cost-benefit analysis, and the ground work for a road map to help organisations embrace open data and “do it right” (e.g. critical success factors, activating crucial stakeholders and infomediaries).

Of course we like to hear of other research projects with recent publications or currently underway.

Aid Transparency Barcamp Nepal on August 4th

Aid Transparency Barcamp Nepal, run jointly by YoungInnovationPvt. Ltd and aidinfo, is a conference to raise the awareness of the foreign aid scenario in Nepal. It intends to create a platform to initiate conversations and connections on the effective use of ICTs to support aid transparency and effectiveness. There will be a chance for organisations and individuals to showcase innovative ideas and tools that promote effective accessibility and visualization of aid data. Further, it is hoped that it will create a platform where the best technology products around aid data can be collaborate, supported, sponsored and promoted. It is also an opportunity to raise awareness of the International Aid Transparency Initiative’s (IATI) standard for publishing aid data.

Featured speakers will include Bibek Raj Kandel, Simon Parrish, Anjesh Tuladhar, Aman Shakya, Bibhusan Bista, Hemanta Sapkota, and Prabhas Pokharel. Sessions will include:

  • Linked data and semantic web technologies for aid transparency
  • Civic engagement: creating a feedback loop on aid effectiveness
  • Community led development projects: Information processes and pitfalls
  • What is IATI format and how it enhances aid effectiveness?
  • Social media for aid awareness
  • Taking aid transparency local: radio and SMS-based transparency opportunities
  • Crowd-sourcing for better data: geo-coding and traceability

This event is targeted at the tech community (programmers, app developers, FOSS enthusiasts, mobile developers), INGOs, aid donor communities, government officials, the media and aid transparency professionals and practitioners.

For more information on this event, please visit the website: http://nepalaid.yipl.com.np/

“Everything I need to know about open data, I learned from open source”

../../../assets/posts/de22a71eac0dc493960f5cd090f09975_MD5.jpg

BoF “Open data in development” at OKCon (via Tobias Eigen)

But what did we learn from open source? Two days of Open Knowledge Conference gave lots of food for thought. And lots of inspiration as well: plenty of projects doing interesting work, and experiences to share. And to add a cherry to the cake, we had a great “open lunch for development” with several people active in development aid. My (delayed) take-aways for Open for Change.

From data.gov.* to data.your.org

Nigel Shadbolt and Andrew Stott shared their lessons from setting up data.gov.uk, and Tom Lee talked about data.gov and the recent threats of its budget cuts.

It’s crucial to have top-down support, bottom-up activists, and middle-tier connectors, to bring everything together.

  • We need to continue nurturing a network of people active in open data for development, to make sure we have the tools, ideas, reports, cases, and standards we need, and to support the early adopters within organisations.
  • We have done some work on an “open data briefing”, based on the OKFN open data manual, and we need to continue work on that: in four pages, explain the why, how, what and who of opening up your organisation.
  • It’s important to understand the decision-making and budgeting processes to make the case at the right time and the right place. We could review the best material on “how to convince your boss to use open source” as a starting point.

There are many reasons to embrace open data, don’t rely on a single one to make your case: tranparency and accountability; economic value, growth and innovation; efficiency and cost-effectiveness; improving (public) services; (public) engagement; and civil society and social capital.

Close the feedback loop: the “build it and they will come” approach won’t work here either. Try to publish data that matters to people, but also consider that “data probably has a long tail”.

  • Releasing data early and incrementally creates a steady news flow, and also enables you to work with feedback and to champion people to create peer pressure on the “refusniks”.
  • Friedrich Lichtenberg tries to turn the lack of opportunity to debate what you see in “ Where does my money go ” into an opportunity for “ Open Spending ”. In essence: how do we provide support to create a data cycle instead of a data pipeline.
  • There is a challenge to create or appoint authoritative sources and URIs. In fact, this was a key topic in the online workshop organised by David Pidsley during the Open Data for Development Camp.
  • In a broader sense, the emerging standards and work done on open data in development cooperation needs to settle down in one or more ontologies. This will also pave the way to build more focused data enrichment services that can tap into the existing wealth of documents we have, to help professionals navigate those.

Statutory requirements matter.For governments, this mainly has to do with legal frameworks and obligations. But every organisation could (and should) enshrine crucial elements of open data in their policies: how to ensure “open” stays open, and how to prioritise.

  • The UK government re-used lots of existing policies, such as the National Statistician’s guidelines about reasonable measures to prevent identification of individuals. The open data manual could be a good portal to such policies.
  • Principles and policies should not be set in stone (at least: not early on), to prevent weasel words like “to the extent feasible” creep in. The essential lesson: in the first phase, compromise on the data that will be published, but not on the license applied to it. (Again: the paradigm shift in thinking and acting is what matters first.)

Why we want to be open: the fallacy of community collaboration

In open data, a lot can be learned from “open source”. In terms of tools and practices, I think we are, but in terms of the stories we tell ourselves and others about why we do it, Benjamin Mako Hill gave some interesting insights on the promise of open source to create better software because more people will be able to see, comment on, and improve the code.

In reality, this hardly happens. He showed graphs of projects on Sourceforge, the first major hub of open source software. The median number of developers working on a project is one. If you only look at “mature projects” (multiple releases, longer history), the median is still one. If you look at the most popular projects (10% of most downloaded), the median is two.

In other words: there are very, very few projects where mass collaboration did happen.

And we actually don’t really understand why some of them succeed.

It doesn’t mean we should not do open source, but we should not promote it with the story that it leads to peer review and better code. There are plenty of other reasons, though, and we should make sure we capture those in our Open for Change Manifesto as well:

  • It gives the users freedom: autonomy, control and empowerment. The technology constrains how and what we can communicate as people. Openness allows you to remove barriers.
  • It is resistant to “anti-features” (limitations built in to charge for removal). With an open license, anyone can process a data set to make it more useful for themselves or others.
  • It makes failure cheap: since the investment to be open is low, there already is reward in just making your own solution available.
  • It also makes success cheap: some products failed to have a big enough market to sustain a company producing it, but a community of users can produce and maintain it.
  • It is not dependent on persons or organisations: even if the original producer(s) stop working on it, others can continue and keep it available.
  • It sometimes does lead to mass collaboration. And it then can produce something that would be impossible to organise through traditional means.

Why we want to be open: a stronger vision

In our beta Manifesto, we tried to capture the essence of why we want to be open, and OKCon was a chance to reflect on it.

I liked a definition given by Jose Alonso of the Web Foundation: the web is humanity connected through technology. And as Brewster Kahle of archive.org said: the last generation put a man on the moon. Pretty cool, but our generation can make all knowledge available to all people on earth, for always and for free. That’s a powerful ambition too.

It is crucial to also translate the promise of open data, open access and open knowledge to “effective use”: how do we make sure we create autonomy, control and empowerment, but more even so: security for the ones who want to realise their “ right to access”? “Open” is part of a struggle for human rights.

Hopefully, a joint “Slash Open” campaign can unite the efforts of many organisations working for humanity in shaping the technology we need and put it to effective use.

"World Bank Institute: We’re also the data bank"

At the Activate Conference, Aleem Walji of the World Bank Institute gave a brief overview about their first experiences with open data (their data catalogue website gets more visitors than their home page now, and Google translated the top indicators they saw people were searching for into 39 languages), and how they hope to connect their aggregate data with the detailed service delivery we can now collect and make available, to build a “Yelp for Development”.

  • in Ideas
  • 2 min read

Open for discussion: the "Open for Change" Manifesto

Ever since we started networking as “development 2.0 pioneers”, we wanted to express our core values, so we can grow our network into a movement.

In January we adopted the name “Open for Change”, and worked to organise the world’s first “Open Data for Development Camp” (ODDC) in May, to bring together the people in that movement.

Based on the conversations we had with many of you (before, during and after ODDC), and inspired by other manifestos, we now have created a beta version of the Manifesto for Open for Change, and love to hear your feedback.

The Manifesto is the foundation for more plans and activities to grow our global movement. Schematically, it could look like this:

../../../assets/posts/ecc185b1d82babb476abb813afdd9442_MD5.png

We see “Open for Change” as an open source brand and as an international ecosystem, based on the Manifesto and with a light-weight organisational and technical structure to connect various projects and activities:

Most of these are in their conception phase, some are already starting to take shape with people working on it.

Our focus now is to get feedback on the Manifesto and to hear your thoughts on the “open source brand” and the organisational form it should take.

Click here to go to the Manifesto and give your feedback! Leave a comment, and forward this message to people you think should be involved! Thanks!

Getting my GSM modem working under Ubuntu

../../../assets/posts/ef4b092c09313cac69cca10fc5ce9eae_MD5.pngAnother “hack post”, to capture how I got mobile broadband working on my Sony laptop. Sony makes laptops with cutting-edge features (small, solid-state disk, full HD screen) and a stylish look, but doesn’t like to help you take full advantage of it unless you’re on Windows. Undocumented tweaks to the hardware, hard-to-find technical information, and so on.

I bought a Sony VPCZ1 (to be more precise: VPCZ13C5E) with a WWAN module installed, with the idea that I could be online anywhere, without any dongles sticking out, or having to connect by tethering it to my phone over Bluetooth or USB. I’ll pay the extra fee… provided it works.

I have installed Ubuntu 10.10 (which was an effort in itself), and went on an excursion to get mobile broadband running.

First, let’s find out what the hardware is: it’s not listed on Sony’s specs page, but according to the specs of a similar model, it’s a Qualcomm Gobi 2000 (PDF link). It apparently needs to be loaded with a firmware before it will operate. I had to install the Ubuntu package gobi-loader as a starting point.

apt-get install gobi-loader

However, that package does not provide the actual firmware. It expects the firmware in /lib/firmware/gobi, but that directory doesn’t even exist.

The discussion about a Qualcomm problem on Launchpad made me look at a way to get the firmware from the Microsoft-based Qualcomm Gobi2000 (WWAN) Driver 1.1.80 that Sony provides. After unzipping that file, we have a directory with a file GobiInstaller.msi… the firmware is somewhere in there.

After installing either p7zip-full or cabextract, we can extract the contents of the.msi file, to end up with a long list of crypticly named files.

Thanks to a post by Madox, and the discussion after it, I saw what to look for:

/lib/firmware/gobi$ ls -l
total 13888
-rwxr-xr-x 1 root root 11096116 2009-12-11 21:10 amss.mbn
-rwxr-xr-x 1 root root 3104812 2009-12-11 21:10 apps.mbn
-rwxr-xr-x 1 root root 9284 2009-12-11 21:10 UQCN.mbn

As it happens, there is just one file with length 11096116, just one with length 3104812, and 18 with length 9284 bytes. The first two are easy, the UQCN.mbn file contains the specific setup for a region or provider.

When using a hex editor (like hte in the package ht) to inspect the various variants of 9284 byte files, there is a string at the end revealing a bit what it is intended for: umts_gen, umts_orange_nogps, umts_tmo_noxtra and so on. See below for a table of strings, file names, and md5 checksums in my setup.

../../../assets/posts/7fa7d93755494aad8e6c412ba8926dde_MD5.pngAfter copying the appropriate three files with the right name into /lib/firmware/gobi, I flipped the “wireless” switch off, waited some 10 seconds, and switched on again. And was greeted with a pop-up to enter the PIN code for my SIM card: indicating that the modem had been detected, the firmware had been loaded, and my SIM card was working.

Under the Networking menu, I could add a new mobile broadband network, and connect after a few simple steps of selecting my country and provider.

Notes: various posts for earlier versions of Ubuntu mention hacking in qcserial, or compiling your own kernel modules. I didn’t have to do any of that.

string file name in.msi md5sum
umts_gen _61F1C9E9670341009A49DFBF7ED9308B e601a7bf3c55104badcdf21bcbb0bfa9
umts_gen_nogps _82F1F7B633254DD8943C3C66695180D4 633bed88c29244683635c261849d0e88
umts_gen_noxtra _B1755EF712704F6EA05AD29399FDAAD2 f1911bcefc4bd5bf8d8fd401082c1a5a
umts_orange _D086B5600A824F88A7B5B4DA9AEC7393 0044ef086b828c30689b899a3570dd56
umts_orange_nogps _EAA5B766B8FC42258CF0903EB29B3866 1dbb1ce26cb59f9d7b551e54c9f71c80
umts_orange_noxtra _97CDF82428924F739C17CEC7DB7642C5 668d1e8903f362b4fc5ec66145ab9b36
umts_telital _FF884A864DC04D3B9BE3B80CA0D6365D 6f575f681ffad81bf3159c7b2d7122a9
umts_telital_nogps _F5A8481D6D0141DDB5AC04E02C5D6B77 bf6b02a2e4ac42c40b028519ed5db487
umts_telital_noxtra _CE104EC699FA4012AF9BF053838EFEEA 6a1b2b342a9e3548dc02f882f156ec21
umts_tellfon _F85609A0A9B64C399F670AE6D78A9EDD b0edb9f5ee92204f9d0e455ff860ca84
umts_tellfon_nogps _AB685C9BC9BF406691ED3CC70C0EA2F8 345c4671242f94d31e3161ead89227db
umts_tellfon_noxtra _D9B76BE055B04137AB32632049501DF1 f064a0c0c7806d30dacd33b4672661cc
umts_tmo _FC981235AEDB429BA1F601941B97E11C 1061d15ca89d0d8f66919c99cb67cc45
umts_tmo_nogps _A6C51028090341929FAC167B2938F19C 6d7b94fed93f47ceafc9ba0c7889fc1f
umts_tmo_noxtra _9358B6316845471886EF8B592B400046 4132ebbea25e4014043d902d7e272f71
umts_vod _A51C11D307D344229DD775AD527BA6DA 4d1b58cb79817dbe111194dfc286e57a
umts_vod_nogps _890ED25310C543B483CA0E67C40B9C54 d06886a62c5c42e2076e0d2a055d1675
umts_vod_noxtra _A66130F57F1E4EFCAA571D5BFBF84CE4 39f0b2663f682b5c9d97cdaddaa72813