Open Data: So Much Potential, So Little Progress

While most organizations are still enamored with Big Data, there’s another major data disruption budding: Open Data.

Saturday was International Open Data Day, with cities around the world hosting hackathons to develop applications with open datasets. At this point, that basically means government data and Twitter feeds.

It seems to have garnered little coverage, however. I did find an article on the Oakland event, which drew a mere 80 developers, activists and government employees. Here are a few of the ideas generated at Oakland and other locations:

• A map showing where owners of blighted properties live
• A list of locations offering free computers and Internet services
• A map using green and red circles to mark locations where Twitter users recently made positive and negative Tweets
• An app that tracks street trams in real time
• An app allowing users to check data about school sanitation and immunizations

Most of the datasets targeted for Open Data fall under the public domain, but the idea is to make it more accessible and easy for developers to use. Governments are starting to roll out data through open portals, although so far the results have been less than stellar and very uneven.

In the UK, critics have charged that the data has serious data quality issues and other problems that make it difficult to use. Germany’s Open Data portal came online recently, but proved very disappointing to Open Data activists, because much of the data was either not freely available — meaning you had to request it — or the government outright charged for it.

Australia’s effort was recently described as “patchy and transitional” by its own Information Commissioner. India’s effort is underwhelming to DNA India commentator Shyamanuja Das, who noted its site “has just about 115 datasets; what’s worse, all those datasets are from only 11 government department/agencies.”

The U.S. government is one of the forerunners in the Open Data movement, but no one seems to know or care: The government recently released a new redesigned prototype after learning no one really knew about the Data.gov, the U.S. portal.

Some global agencies also offer open datasets from studies, including the World Bank World Bank, IMF, and UNESCO, reports DNA India.

It’s not hard to see how this data could be put to use, either in apps or as an addition to business intelligence efforts. So why is Open Data taking so long to take off?

Das points to two major gaps:

1.      The government agencies — or at least the employees in charge of these efforts — do not understand end users, or at least did not develop these datasets with them in mind. That’s led to major gaps between what they’ve developed and what the market needs.

2.      These initiatives aren’t generating enough money or benefit to move the effort forward.

Maybe something great and disruptive will emerge out of Saturday’s event. Until then, experts seem to think Open Data will remain a movement with great potential, but little realization of it.