Skip to content

Updates

Deploying a website with lftp

a3e419a79945282c3a90deb03218793f_MD5.png There still are web hosting providers offering only ftp access to your website files. No fun if you’re used to version control systems and shell access.

I had to deal with that situation, and used Linux’s strength: combining several small tools.

I have a laptop running Ubuntu, a website in WordPress, git for version control, and use Eclipse as my development environment. I first looked at Aptana and other options for Eclipse, but I wanted a more light-weight solution that I could also use outside Eclipse.

lftp

Ubuntu comes with lftp, an ftp client that can be scripted and has a “mirror” command to basically get a target location synchronised to a source.

(It actually can do a lot more, and work over http or bittorrent too, but that’s outside the scope of this post.)

.gitignore

Not all files need to be uploaded. Typically, the.gitignore file already has a list of files and directories that are not under version control and wouldn’t go live when using git to update a server.

The lftp mirror command lets you exclude files and directories too, but curiously has no option to read a list of exclusions from a file. Martin Boze wrote how he fixed that, by using sed and tr.

scripting

I didn’t want to write a series of lftp commands, but instead would prefer to connect once, then run a series of transfers, inside a single script.

Specifically for WordPress, I also like to have a local mirror of images and documents uploaded on the live site.

It is possible use lftp as the shell to run a script, but unfortunately, it’s not possible to use environment variables or Martin’s “sed” trick in such scripts.

But it’s not to hard to do using the Heredoc syntax.

git branches

I use branches in git to separate my development version from a preview and a live version. By adapting the upload script in each branch, I can simply call “deploy” to upload the files to the right place.

#!/bin/bash
lftp <<EOF
user ftp-username ftp-password
open ftp.provider.com

# "mirror" from local copy to server, use .gitignore to excude files (sed, tr), delete remote files if needed
mirror -R -e -v -x \.git.+ -x scripts \`sed 's/^/-X /' .gitignore | tr '\n' ' '\` /var/www/dev_sites/website.org /www

# wordpress-specific
# "mirror" uploaded images on live back to local, don't delete local files if not on remote
mirror -v /www/wp-content/uploads/ /var/www/dev_sites/website.org/wp-content/uploads/
EOF

The only thing left to desire is a way to speed up ftp deployment…

Captchas and crowdsourcing

What if… you can help 100 million people to learn a new language, and let them help translate the web in return?

TEDxCMU — Luis von Ahn — Duolingo: The Next Chapter in Human Computation (by TEDxTalks)

Newsletter February 2012

In this newsletter:

  • a wrap-up of the past Open Tea and announcement of the next one.
  • an update on international aid data networking
  • current developments on the IATI NGO working group

Open Tea

On December the 8th the Open for Change network had its first Open Tea at the Amlab in Amterdam, to look back at the past year and discuss where we will be heading in 2012.

Mark Tiele Westra from Akvo presented openaid.nl, an initiative that makes open data on Dutch development aid visualized and searchable.
Marijn Rijken from TNO informed  and invited us to participate on a research project on the effects of open data for the development sector.

We discussed the organization of the Open for Change network. Evident in the discussion was the important role the Open for Change network holds in connecting, exchanging and supporting open data initiatives and knowledge in the development sector. How this role should be filled in is something we are working on in 2012.

We want to thank the 1%CLUB, Akvo and TextToChange for hosting the open tea in the gorgeous Amlab, and hope to see you there again at the next Open Tea: March 8th, 15:00- 17:00 and after that Open drinks!

International networking

At various meet-ups at conferences in 2011, we discussed ways to strengthen the international network of open aid data activists.

In November, we submitted a proposal for a European Aid Data Network to the EuropeAid budget line of the European Commission, led by AidInfo in the UK, with Partos (NL), FORS (CZ), ACEP (PT), IGO (PL) and the Open Knowledge Foundation (UK). We hope to hear by early March whether we are invited to submit a full proposal.

In the meantime, AidInfo has asked Claudia Schwegmann of OpenAid.de to continue building out this emerging European network. We had a first conference call last Tuesday, and plan to have the next one on March 2nd.

To create joint channels of communication, we invite you all to:

  1. Join the open-development mailing list to discuss international open aid data: http://lists.okfn.org/mailman/listinfo/open-development
  2. Track our guide of open aid data-related events, and submit yours: http://lanyrd.com/guides/open-aid-data/

IATI and NGOs

The IATI NGO Working Group is a CSO-led forum that was created with the approval of the IATI Steering Committee to discuss the application of the IATI Standards to the work of CSOs and to present practical proposals on CSO-specific approaches to publication of IATI compatible data.

The CSO Working Group is co-chaired by Beris Gwynne, representing the International NGO Charter of Accountability Company, and Brian Tomlinson, representing the CSO Open Forum.

Both Partos and Open for Change are represented in the group, and we’re aiming to organise an “intervision meeting” for Dutch NGOs in March.

The next peer reference meeting is planned for beginning of March. The first general consultation will hopefully take place in April 2012. Read more on our blog!

Got news?

If you want to bring in subjects or interesting news for next newsletter, you are more than welcome: send your contributions to info@openforchange.info

Stable communities and the Suck Principle

../../../assets/posts/2b6ffe4744471450f8a98e8e4bf692ab_MD5.jpgsecretlonden123")

“For anyplace to stay cool it has to suck.” Dmytri Kleiner wrote up some interesting thoughts on making communities more continuous and stable. When a place becomes too cool, it becomes too crowded, and the regulars are replaced by a more transient crowd that has no deep bonds to the community of regulars. Referring to a quote by Yogi Berra, “ nobody goes there anymore, it’s too crowded “.Dmytri draws on his experience with 28C3, the latest Chaos Computer Congress, held yearly at the end of December, in Berlin, and proposes The Suck Principle:

Only places that suck can really have a continuous community, because if nothing about the place sucks, it will attract more and more people until it sucks because of crowding. So if you want a continuous, closely knit community, something about the venue or event must suck, your only choice is what should suck or how it should suck.

I think it’s a variation on what I regularly tell organisations who want to build an online platform around their mission: community is about exclusion, not about inclusion. It’s about constructing boundaries, whether enforced or self-selective.

Offline, physical location or dimensions are usually one of the barriers. Dmytri dismisses the option of relocation CCC, but my experience with for instance Web of Change at Hollyhock, on Cortes Island. off the West coast of British Columbia, is that “traversing the barrier to entry together” is a bonding experience in itself, creating community bonds that extend well beyond the few days that participants spend together.

Online, there usually is plenty of stuff that sucks. But little of it forms barriers to entry that double as “social objects”, around which participants can come together and bond, both with each other as well as with the social norms of the community you then enter.

Wordpress for Presentations (part 1)

For a while now, I’ve been wanting to get rid of using presentation software (like Microsoft Powerpoint and LibreOffice Impress). Since I’m mainly presenting stuff on the web, about the web, I want to use a web-based tool. Like my blog. And now I can! Here’s part one of the journey.

Doing presentations with WordPress, part 1

  • Why?
  • How?

Why a web-based presentation tool?

Current presentation software sucks…

  • Poor conversion of formats
  • Hard to publish on your own site
  • Need to switch to a browser to demo anything

Obviously, I’m not only one with this itch. I’ve explored a couple of options.

I’ve also tried a few plugins for WordPress. The one I liked the most was (a)SlideShow.

Cool effects and all, but hard to have a post live as a blog post (with texts between the slides), and being able to switch it to presentation mode.

The web standard for style sheets (CSS) provides a mode for presentations: @media:projection. It should be possible to switch to a “projection style”, and use CSS to eliminate all the text in between the slides, provide basic pagination, and so on. But browsers still don’t really support this:

  • Opera uses the “projection” style (if available) when going into full-screen mode.
  • Firefox has a FullerScreen add-on that allows similar functionality, but needs to be installed separately.

With the Firefox add-on, it wasn’t hard to start working on a style sheet for presentations that works on my own laptop.

But… I’d like to be able to present from another machine as well, and to enable web visitors to my blog to also see how the slides look.

So the search continued for a simple library that would do the trick.

In the end, I mostly liked Zack Grossbart’s simple and clean version in the browser. And it should work well with the style sheet already prepared.

I’ve spent a bit of time integrating it into my web theme, and to strip Zack’s version further: no extra interface, just the same as the F11 key in my own “Firefox+FullerScreen” set-up.
It took a bit of hacking to put it into WordPress.

Doing presentations with WordPress, part 1

What the code does now?

  • Are we presenting a single post?
  • And is it in the category “Presentations”?
    • Is there a URL parameter “projection=1”?
      • Load the “projection” style sheet for all modes.
      • Load javascript to enable keys for “forward, back”.
    • Else
      • Load the “projection” style sheet for “projection” only.
      • Load another style sheet to present the slides in “screen” mode.

It can do pretty cool stuff already. Use standard embed codes (but you’ll have to add the div code with class="slide" yourself).

http://www.youtube.com/watch?v=PIa0n8rbUFQ

This is part one. There still is work to be done 🙂

Who is implementing the aid transparency agreement?

Owen Barder published an overview by the International Aid Transparency Initiative (IATI) on how far countries and donors are on their road map to publish their aid spending data before the High Level Meeting in Busan, end of this month.

If you’re curious about how the International Aid Transparency Initiative (IATI) can help aid effectiveness, have a look at their video with some stakeholders:

The International Aid Transparency Initiative (IATI) from Development Initiatives on Vimeo.

Describing organisational relations

One of the side-events of the Open Government Data Camp, last week, was an Organisational Identifiers Workshop put together by Tim Davies and Chris Taggart. The meeting discussed the various challenges in linking information about organisations held in separate data sets. Although participants were careful to avoid the word “ ontology “, one of the break-out groups did look at describing relations between organisations. Since I graduated on research into “part-of” relations in an ontology, and what you can infer from them, I joined that discussion. Here’s what we came up with.

../../../assets/posts/64e5bb7c94bcbdffd0206ff2b6c693cc_MD5.png

Outset

The workshop was a good chance to catch up with where things are right now, with several organisations at the table and participating online that have to deal with information about organisations:

  • The IATI standard needs organisational identifiers to refer to individual donors and recipients of grant money and payments. IATI does not want to provide this standard, but rely on an external one. They will need some way to represent up to the level of government departments as part of an upcoming pilot project, to capture intended donor flows in a meaningful way.
  • The Open Corporates website, and its companion the Open Charities website, capture information about organisations, but also lack a common identifier scheme, as well as ways to describe relations between organisational entities (especially the complicated relations between companies).
  • Within the open government data movement, and the Open Knowledge Foundation, there is a need to represent organisational units such as departments, and be able to deal with renaming and reorganisation of such units over time.
  • The Sunlight Foundation is dealing with for instance DUNS numbers, which often are too detailed for the purpose of identifying a larger organisation (every outlet of a supermarket chain will have its own number).
  • GlobalGiving, OpenSpending and IATI are looking into decentralised registrars, but each registar basically expresses a different type of relation between a legal or organisational entity and a purpose, such as tax registration or legal entity.
  • Everyone faced a difficulty of dealing with entities which cannot register as such (e.g. informal associations), and so are not in any registrar’s database.
  • To end this list, many people will talk about a known brand as if it is a company, and would expect to access information that way, but even these have no single register.

How to create identifiers for organisations across the world, which might not be registered anywhere, and which relate to each other and to more generic concepts, in such a way that we can capture all the meaningful relations and data we want to capture?

How to make sure it works with the schemas already in use in big organisations? And that it works with data stores that are not open? Without introducing another naming authority?

  • You should be able to determine an ID without requesting it from anyone.
  • You should be able to resolve it to commonly known registrars.
  • You should know where to find the list of those registrars.
  • You should be able to represent the granularity (aggregating detailed levels of information, allowing for splitting up individual entities into smaller ones)
  • Who decides what is a good registrar?

We split up in a couple of groups, one looking at identifying public bodies, another at the technical architecture that might be needed, and a third at common terms to describe relations between organisational entities. I joined that third group.

Inputs

We spent some time discussing various types of relations, and I also looked around to find possible candidate schemes, but without much luck. I couldn’t find an obvious example, like the FOAF standard for personal relations. A few standards, like OrgPedia, or the Organizational Ontology, seem likely candidates, but don’t cover this area (yet?).

We looked at some use cases:

  • A company wants to show their supply chain, to demonstrate that their suppliers are ok, or perhaps to “crowd-source” the question whether they are: “these are our suppliers, if you think they’re not ok, let us know”.
  • A campaigning organisation wants to express what they know about organisational ties, to support their arguments on why the ties should be broken.
  • A reporting entity wants to express their donation relations, for instance to a government department, and be able to deal with changes due to reorganisation.
  • A watch-dog organisation wants to express that a certain company has changed names or merged or split operations, but still remains to pursue the same activities.
  • A consumer wants to find out what a certain company has done, but basically only knows that company through a name or brand, without knowing the exact structure behind it.

We acknowledged additional cases, like finding influential relations between corporate or organisational entities based on board membership or roles of individuals, but decided not to take that on in this discussion.

Output

We came up with a first-version typology of relations. The naming and exact semantics will need further review.

Persistent relations

These are relations between entities that have a “permanent” and “structural” character. Of course, all these relations are bound in time, but the beginning and end points may not be known.

We distinguished two sub categories.

  • Organisational relations express membership, ownership, or hierarchy.
    • “is member of” (an association, group, cabal); “is affiliated to”; “is organisational unit of” (department, location); “is shareholder of”; “is owner of”
  • Contractual relations express transactions between entities. For instance, a relation “donates to” would express a sizable or structural donation from one entity to another. In the IATI standard, this would mean there should be (one, but probably more) “activities” records or “transactions” records.
    • “has contract with” with eg. subcategories “owes money to” (long-term debts, mortgages), “is supplier to”, and “licenses to”; “in legal conflict with”; “donates to”

(This typology still fails to capture something like a brand as abstract entity.)

Temporal relations

These are relations that express a change in the structure or responsibilities of some entities, often the beginning or the end of particular entities. We identified four basic types:

  • Split into: A splits into B, C, … A ceases to exist, B, C, … come into existence.
  • Spin-off off: A creates B as a separate entity
  • Merger: A, B, … merge into C. A, B, … cease to exist, C comes into existence.
  • Acquisition: A acquires B and moves B’s assets into A. B ceases to exist.

Further work

More work is needed to mold this into a useful standard (relations are currently described from the perspective of one end, there is still plenty of room for interpretation, things have not been tested on real-life examples described as use cases, and so on).

And, of course, we’d need those organisational identifiers to refer to other entities, and find ways to delegate resolving identifiers to services that can provide additional information on those identities. See the whole report of the workshop on the OGDCamp wiki for the results in the other discussions as well.

But thinking about and discussing relations between those entities brought back memories of all the fun in making machines infer and report unknown relations 🙂