Twitter API Version 1.1 Switchover

In late May/early June Twitter finally took some long standing API components out of service as they switched to Version 1.1. This shift had been advertised for some time – and most developers should have already have made the appropriate changes – but the new API represents a couple of important changes so I thought it might be useful to review these.

The most significant change for many will be the final withdrawal of the last Twitter RSS feeds. The visibility of RSS was scaled back several API releases back (there used to be a link on each user’s page) but they remained available – and actually easier to call on – for  those looking for them. They have now been entirely switched off with Twitter redirecting interest directly to the (less easy to use/play with) API or the (much more tightly controlled) Twitter widgets and tools.

The withdrawal of all RSS feeds is understandable in business terms – Twitter will now be able to monitor activity around a tweet much more easily and to, perhaps, push promoted tweets more directly, but it is a frustrating move in terms of the openness and re-usability of Twitter data. The recent introduction of a much expanded suite of Twitter widgets plugs many of the gaps left by the RSS withdrawal however the switch off proper will be surfacing other custom scripts, embeds, etc. that still need updating or replacement.

Here at EDINA we needed to make some updates for the RSS switch-off. We display the most recent blog post (still brought in via RSS from our aggregated blogs) and tweet on the EDINA homepage as this gives visitors a sense of the most recent news and updates. For the Twitter update we have switched from a custom RSS feed from Yahoo! Pipes which aggregated and filtered content from the feeds for each of our 18 organisational accounts, to a widget highlighting the latest tweet from a Twitter List of our accounts. This new solution works reasonably well after some quick but crucial customisations to ensure a good fit with out homepage (whilst still obeying the Twitter display guidance), but the new widget, whilst convenient and Twitter-approved, does restrict our ability to filter out specific noise (retweets from other accounts, which may look more out of place on our homepage, in particular).

The EDINA Twitter list stream

The EDINA Twitter list stream

The other change to accessibility and flexibility of data is less obvious but also frustrating for those who like to do a quick and low tech mash up or prototype. Whilst Twitter have been using OAuth for some time this update ensures that effectively no Twitter tool, app or widget can use anything other than OAuth – rendering the RESTful API a rather different beast to use. With that last change in mind users of Martin Hawksey’s excellent Twitter Archiving Google Spreadsheet may want to make sure they have upgraded to Version 5.

Useful Links

DeliciousShare/Bookmark

Geospatial API’s and other tools for developers

Since the release of Open data back in 2010, and through the rise of mobiles and the App, the use of spatial data by developers has increased significantly.  Spatial data is no longer the preserve of the geospatial scientists, it is used by mainstream developers who see the value of location. I this post we will look at some of the “free” data and tools that you can use.

Data

OpenstreamAPI:

The Digimap OpenStream service provides access to a Web Map Service (WMS) offering Ordnance Survey OpenData products, including GB Overview, Miniscale, 1:250000 Colour Raster, VectorMap District Raster and OS Streetview.

EDINA aims to provide the latest version of OS OpenData via the OpenStream service. Once registered, you can use the Digimap OpenStream API to do things like:

  • Mashups, combining OS Opendata with maps and data from other sources.
  • adding OS Opendata to Google Earth.
  • Embed maps in your website.
  • provide OS mapping in your own applications.
  • provide OS mapping in a Desktop GIS project.

Openstream viewed in Google Earth

If you want to find out how to use Openstream please check out the folowing useful links:

Advantages – Free. Easy to use and provides regularly updated OS mapping. Available in OSGB and WGS1984 projections
Disadvantages – Not as easy to use in WGS84 as google maps, no satellite imagery available through API.

Google

Most people will be familiar with Google maps, specially if they have a smart phone. What can you say. Global cover (almost) and improving all the time. Packed with detail in urban areas. I suppose the weakness is in rural areas where detail may be sparse. Google maps is enough for many applications but if you may not be as accurate as the data from a national mapping agency, such as the OS.
Google maps is generally available for free in most cases. You have to pay if you use lots of data or if you want the data behind a login.

OpenStreetMap

OpenStreetMap is global community mapping project that wants to make a freely available map of the World. 10 years ago this would have been a crazy idea but today it is almost a reality. It suffers in the same way as Google maps in that it is great in urban areas and less good in rural areas. But, if the detail is not there, just sign up to the cause and add the detail yourself. Your edits will then be available for your, and everyone else’s, map.

Bing

Bing have a mapping stack that you can access through an API in a similar way to Google. Bing was quick to offer a bird’s eye view option for its satellite data and also has OS 1:50 000 mapping available as well as offering detailed street mapping for London. It is worth looking at the data available through the Bing Maps API as it is different to that offered by Google and may better suit your needs.

British Geological Survey

The BGS offer a number of Web Mapping Services (WMS) for developers and GUS users. These include 1:50 000 scale mapping, 1:625 000 mapping (on and offshore), 1:625 000 Hydrogeology, Soil data and contaminated land data. Full information on the use of these datasets and the GetCapabilities statements needed to access them is provided on the BGS website.

API Services

An Application Programming Interface (API) is a protocol intended to be used as an interface by software components to communicate with each other. An API is a library that may include specifications for routines, data structures, object classes, and variables. Thee are a number of API’s that give developers to spatial data and tools to create interactive web maps.

Unlock Text

Unlock Text is a powerful geoparser that can search text hosted on the web in .txt or .html format for references to locations. These locations are then returned ready for use in your results page, web map or any other application.

Unlock

Unlock Text is a RESTful API, you can use it by first setting up a username and password using the POST operation. You can use the GET operation to find out information about this account such as the number of Text documents that have been parsed. POST and GET operations can be used to set up new documents to be parsed and to retrieve the urls of results pages created by the parser.

You will need to use cURL commands; or a REST Console or Client which are available as add-ons for Chrome and Firefox web browsers, to use Unlock Text.

Advantages: Free! There is not really anything else out there to compete with it, unless you pay!

Limitations: Not always a perfect set of results, but then no one else does it perfectly either.  Wikipedia currently blocks it!

Unlock Places

Unlock Places is a place search web service, its API helps developers to find locations for place names. You can also use it to convert postcodes into coordinates, look up electoral boundaries, and to find shapes that overlap or intersect one another.

Unlock Places can provide geographic information describing places – or other features – as points (latitude-longitude coordinates) or larger bounding boxes and more detailed shapes (where possible).

Unlock Places can be used in your web project, app or mobile information service to add a geospatial location on to users’ data.

Advantages: Free

Limitations: Some searches can be a bit slow

National Library of Scotland

NLS

The NLS has a fantastic api which offers access to Historic Maps of the UK. Thus service is great and the mapping is based on out-of-copyright Ordnance Survey maps, dating from the 1920s to the 1940s. Full details of how to implement the map on a webpage or on a mobile app are available on the NLS website.

Google

Google has a host of API’s that allow developers to tap into tools and content delivered by Google. On the maps side of things, there is:

  • Google Maps JavaScripts API v3 – It provides you with the framework to add a number of unique features and content to a Google Maps interface and embed Google Maps in your own web pages. Version 3 of this API is especially designed to be faster and more applicable to mobile devices, as well as traditional desktop browser applications. The API provides a number of utilities for manipulating maps (just like on the http://maps.google.com web page) and adding content to the map through a variety of services. The JavaScript Maps API V3 is a free service, available for any web site that is free to consumers.
  • Mobile API and SDK – Google offers an Google Maps for Android API and a Native iOS SDK for Apple.  Both will allow developers access to google map content and functions such as titl, 3D buildings and Streetview.
The Google maps api has a free use cap of 750k maps a month.  If you think you will exceed this, then you can buy capacity.

Advantages: Google is by far the most popular API for mapping. The public are generally aware of Google Maps and are familiar with both the controls and the content. Mobile Friendly.
Disadvantages: Just because it is the most popular doesn’t mean that it will suit your needs. In particular, the level of details is patchy and may not provide your users with the richness that they need.  For example, you wouldn’t head off into the hills with Google Maps as your main navigational aid. Not entirely free.

Bing Maps API

Bing Maps API offers developers access most of the mapping and satellite images that are available through their main map site.

Bing Maps API as it is different to that offered by Google and may better suit your needs. Did i mention that Bing also does directions? Well it does for car, walking and public transport. The Bing api is has a free option but this has a usage cap and remove the birds-eye view function.  Education/not for profit accounts are available and commercial organisations can sign up to a range of service packages.

Advantages: Alternative mapping stack to Google API and similar user experience. Imagery pretty new.
Disadvantages: User community not as large as Google’s, not free.

Here API

Not heard of Here Maps API? Well Nokia is behind the Here API and that is a good start. Long before Google got serious about delivering spatial data to mobile phones, Nokia were supplying their handsets with mapping pre-loaded. OviMaps, as they used to be known, were pretty good and Nokia have developed them into the Here API service. The mapping is subtly different to Google and uses a pastel pallet that is easy on the eye. In addition to the cartography, there is also relief shaded Digital Terrain Models (DTM) and Maps 3D which is a bit like Google Earth but not quite as easy to use. However, there is a neat 3D option if you have a suitable screen and glasses. Free accounts are capped at 1 million map views a month, slightly higher than the google limit.
Mobile friendly version is available
Advantages: Clean cartography. Looks different to the usual maps. Mobile friendly api
Disadvantages: User community even smaller than Bing maps. Maps 3D not as intuitive as it could be. DTM has artifacts in it that may be annoying. Aerial Imagery not as up-to-date as Bing/Google. Not entirely free.

Openlayers

OpenLayers is an open source JavaScript library for displaying map data in web browsers. It provides an API for building rich web-based geographic applications similar to Google Maps and Bing Maps. OpenLayers is used by the OpenStreetMap project for its slippy map interface which is very similar to Google maps in terms of usability. OpenLayers is used in many web map interfaces including EDINA’s Digimap services.

OpenLayers

The nice thing about OpenLayers is that the user community is both large and very active. This means that you can usually find the answer to the questions that you have when developing an interface. In addition, there are plenty of example maps which allow you to view the code and therefore learn how to implement solutions quite simply.
From Version 2.10, OpenLayers supported built in Navigation and TouchNavigation. Other features such as Map Dragging and Pinch Zoom are also supported. provider the mobile browser support touch events.

Leaflet 

Leaflet

Leaflet is a modern open-source JavaScript library for mobile-friendly interactive maps. It is very similar in style and function to the Openlayers API. It has slightly less features than the Opnelayers API, however this has the advantage of possibly making the API more straightforward. The other biggest, or smallest, advantage that Leaflet has is it’s size. The library is a svelte 64K (Openlayers is over 700k).

One of the trickier aspects of Leaflet is using your own local projection, such as OSGB, rather than the standard Web Mercator.  If you want to do this, then i would suggest reading this blog post which documents the process.

Modest Map

To be added

Map Quest

To be added

Tagger – Final Blog Post

As part of the UK vision for supporting open discovery principles in relation to education materials, JISC has sponsored a number of projects to assist in discovering and enriching existing resources.Tagger (variously referred to previously as GTR or geotagger), was one strand of the JISC funded umbrella DiscoverEDINA project. For the two other strands to this work see here.

The primary purpose of Tagger is to assist in enriching and exposing ‘hidden’ metadata within resources – primarily images and multimedia files. Images for example embed a lot of descriptive and technical metadata within the file itself and very often it is not obvious that the main focus of interest – the image, is carrying a ‘secret’ payload of information some of which may be potentially compromising. For example, the recent embarrassment suffered by Dell after a family member uploaded images to social media sites with embedded location information, thus frustrating the efforts of a multi-million pound security operation. Or take the case of the US military when an innocently uploaded photograph of a new assignment of Apache helicopters led to their destruction when insurgents used the location information embedded in the image to precisely locate and destroy the helicopters.

There are many other instances of people being innocently or naively caught out by these ‘hidden’ signposts in resources that they distribute or curate. Tagger helps by providing tools to expose those hidden features and makes it easy to review, edit and manage the intrinsic metadata routinely bundled in resources. It has concentrated on, but not been limited to, geotags.

Tagger has delivered three main things:

  • A basic web service API based around ExifTool, suitable for 3rd party use to geo-tag/geo-code image, audio, and video metadata.
  • A demo web site enabling user upload, metadata parsing (from resource) and metadata enrichment (map based geo-tagging/geo-coding);
  • An Open Metadata corpus of geo-tagged/geo-coded enriched records with a REST based query interface. Currently, this corpus consists of approximately a quarter of a million creative commons licensed geotagged images mainly bootstrapped from Geograph.

Tagger supports the open discovery metadata principles and has made extensive use of open licensing models.

Along the way we started thinking about specific use cases. The ‘anonymise my location’ seemed an obvious case and Tagger’s API and website reflect that thinking. Additionally, in talking to colleagues involved in field trips it was clear that there was potential in providing integrated tooling and we experimented with Dropbox integration.

Taking this further and building on EDINA’s more general mobile development work, we then started to think about how Tagger could be used to assist and enrich in-field data capture use cases and post-trip reflective learning. We continue to explore this beyond the project funding as the enrichment facilities Tagger provides allows for flexible integration into 3rd party services and projects.

Of course, Tagger will never be a complete panacea for all the ills of metadata nor should it aim to be. However by building on best-of-breed opensource tools (Exiftool) Tagger, or more accurately the Tagger API, provides a facility for other service providers and projects to make use of to enable better manipulation and management of those ‘hidden’ metadata.

Therein lies the rub – the perennial  question of embedding and take up.

That’s are next challenge.

OpenStream WMS service for ac.uk

A new web mapping service is now available that makes the Ordnance Survery Open datasets available through WMS. The service offered by EDINA is initially available to anyone with an “ac.uk” email and requires a simple email activated registration. Other than that, there are no restrictions on use. The website includes an example mapping app using OpenLayers and isntructions on how to use the service with Google Earth. We also created a mobile app using this datasource, as described in a previous post on the TouchMapLite interface.

the data products available are:

  • GB Overview
  • Miniscale
  • 1:250000 Colour Raster
  • Vector Map District Raster
  • OS Streetview

you can register for the service (initially ac.uk email addresses only) here.

If you would like to find out how you can use Digimap OpenStream with desktop GIS you should read the Go-Geo! Blog.