Clipper @ National Library of Scotland

Today I am at the National Library of Scotland for a Clipper project workshop (info here). Clipper is a project to create a content creation tool for multimedia, with funding from Jisc.

After an introduction from Gill Hamilton Intro it’s over to John Casey who will be leading this through the day…

Introduction – John Casey

The tagline for the project is basically Clipper 1. 2. 3: Clip, Organise, Share.

We want your input early on in the process here but that means we will be trying out a prototype with you – so there will be bugs and issues but we are looking for your comments and feedback etc. The first outing of Clipper was from 2009, as a rapid development project which used Flash and Flex. Then it went to sleep for a while. Then we started working on it again when looking at Open Education in London

Trevor: I’m Trevor Collins – research fellow at the Open University. My background is very technical – computer engineering, HCI but all my research work is around the context of learning and teaching. And we have a common interest in HTML5 video. And my interest is working, with you, to ensure this will be helpful and useful.

Will: My name is Will and my background is engineering. Originally I worked with John on this project in Flash etc. but that’s really died out and, in the meantime HTML has really moved on a long way and with video in HTML5 we can just use the browser as the foundation, potentially, for some really interesting application. For me my interest today is in the usability of the interface.

With that we have had some introductions… It is a really interesting group of multimedia interested folk.

John Casey again:

This project is funded by Jisc as part of the Research Data Spring Initiative, and that is about technical tools, software and service solutions to support the researchers workflow, the use and mangement of their data. Now it’s interesting that this room is particuarly interested in teaching and learning, we are funded for researcher use but of course that does not proclude teaching and learning use.

The project partners here are City of Glasgow College as lead, The Open University and ?

So, what is Clipper? One of the challenges is explaining what this project is… And what it is not. So we are punting it as a research tool for digital research with online media / time-based media (ie audio/video data). The aim is to create a software toolkit (FOSS) deployed in an institution or operated as a n national service. We are about community engagement and collavorative design delivering a responsive design. And that’s why we are here.

So, why do this? Well time-based media is a large and “lumpy” data format, hard to analyse and even harder to share your analysis. There are barriers to effective (re)use of audio and video data including closed collections (IPR) and proprietary tools and formats. So we want to be able to create a “virtual clip” – and that means not copying any data, just metadata. So start and stop points on reference URI. And then also being able to organise that clip, to annotate it, and group into cliplists. So playlists of clips of excerpts etc. And then we can share using cool URIs for those clips and playlists.

This means bringing audio and video data to live, enabling analysis without breaking copyright or altering the soure data. We think it had streamlined workflows and facilitate collaboration. And we think it will lead to new things. It is secure and safe – respecting existing access permissions to data and does not alter or duplicate the original files. And it creates opportunities for citizen science/citizen research; user generated content – e.g. crowd sourcing etdata and user analytics. Colleagues in Manchester, for instance, have a group of bus enthusiasts who may be up for annotating old bus footage. The people who use your archives or data can generate analytics or para data and use of that can be useful and interesting as well.

So Clipped is… An online media analysis and collaboration tool for digital researchers (ie it supports human-based qualitative analysis, collavoboration and sharing. It is not an online audio/video editing tool. It is not a data repository. It is not using machine analysis of time based media. 


John: The best way to understand this stuff is to demonstrate and test this stuff out. We are going to take you through three workflows – these are just examples: (1) One source file, many clips, (2) Many source files, many clips, (3) Many source files, many clips, and annotations.

Over to Trevor and Will for examples.

Trevor: Hopefully as we work through these examples we should get more questions etc. and as we look through these examples.

Do bear in mind that what we will show you today is not a finished product, it’s a prototype. We want you to tell us what is good, what needs changing… You are the first of our three workshops so you get first say on the design! We want clear ideas on what will be useful… We hope it is fairly straightforward and fairly clear. If it isn’t, just tell us.

So, example (1): Analysing a source file – the idea is an app developer (researcher) interviewing a user when testing an app. So the flow is:

  • Create and open a new project
  • Add the source file to the project
  • Preview the file – to find emerging themes etc.
  • Create clips – around those themes.
  • Add clips to cliplist



Integrating Openlayers and HTML5 Canvas (Revisited)

The WordPress stats tell me there is still a lot of interest in our previous post on integrating OpenLayers and HTML5 Canvas from way back in 2010.
Time has passed, technology has moved on and I’ve started buying shoes in bulk like Mr Magorium. So below, I provide an update on how I integrate OL and HTML5 Canvas 3 years on.

Previously my approach was to replace each individual tile image with a corresponding canvas element, the same size as the tile (typically 256*256). Also we used JQuery to capture tile rendering events. The updated approach is to capture tile images as they are rendered by OpenLayers using OL built in event listeners and then draw these onto a single HTML5 canvas element.
Apart from being more efficient and producing cleaner, more robust code, this approach has the advantage that you can use HTML5 to draw shapes, lines and manipulate pixels on a single canvas tile, crossing tile boundaries. This is particularly useful for drawing lines and shapes using paths (e.g. lineTo() , moveTo() functions).

To demonstrate this I’ve set up a simple demo that shows the HTML5 Canvas adjacent to a simple OpenLayers map, where the canvas version (on the right hand side) is manipulated to show a grayscale and inverted version of the original map image (grayscale is triggered by loadend and the invert function by moveend) The source code is available on EDINAs gitHub page. ( and on JS Fiddle page.
The solution hinges on using the OpenLayers.Layer loadend event to capture the tiles when OpenLayers has finished loading all the tiles for a layer, and also the OpenLayers.Map moveend event, which OpenLayers triggers when it has dealt with the user panning the map. The former is shown in the code snippet below:
// register loadend event for the layer so that once OL has loaded all tiles we can redraw them on the canvas. Triggered by zooming and page refresh."loadend", layer, function()

// create a canvas if not already created


var mapCanvas = document.getElementById("mapcvs" ) ; // get the canvas element
var mapContainer = document.getElementById("OpenLayers.Map_2_OpenLayers_Container") ; // WARNING: Brittle to changes in OL

if(mapCanvas !== null)
var ctx = mapCanvas.getContext("2d") ;
var layers = document.getElementsByClassName("olLayerDiv") ; // WARNING: Brittle to changes in OL

// loop through layers starting with base layer
for(var i = 0 ; i < layers.length ; i++)

var layertiles = layers[i].getElementsByClassName("olTileImage") ; // WARNING: Brittle to changes on OL

// loop through the tiles loaded for this layer
for(var j = 0 ; j < layertiles.length ; j++ )

var tileImg = layertiles[j] ;
// get position of tile relative to map container
var offsetLeft = tileImg.offsetLeft;
var offsetTop = tileImg.offsetTop ;
// get postion of map container
var left = Number(,"p"))) ; // extract value from style e.g. left: 30px
var top = Number(,"p"))) ;
// draw the tile on the canvas in same relative postion it appears in OL map
ctx.drawImage(tileImg, offsetLeft + left, offsetTop + top) ;


greyscale(mapCanvas, 0, 0, mapCanvas.width, mapCanvas.height) ;
// uncomment below to toggle OL map on /off can only be done after layer has loaded
// = "none" ;


Note that some of the code here comes with a health warning. The DOM functions used to navigate the OpenLayers hierarchy is susceptible to changes in the Open Layers API so you need to use a local copy of OpenLayers (as is case in GitHub sample) rather than point to the OpenLayers URL (as is case in the JS Fiddle version).  Also note that all layers are drawn to the Canvas, not just the one that Open Layers triggered the loadend event for. This is necessary to ensure that the order of layers is maintained. Another issue to be aware of when using Canvas drawing methods on maps  is the likelihood of a CrossOrigin tainting error. This is due to map images being loaded from a different domain to that of the HTML5 code. The error will not get triggered simply by drawing the tiles to canvas using the drawImage() function, but does fail when you attempt pixel manipulation using functions such as putImageData() . OpenLayers handles this using the Cross-Origin-Resource-Sharing protocol which by default is set to ‘anonymous’ as below. So long as the map server you are pointing to is configured to handle CORS requests from anonymous sources you will be fine.

layer.tileOptions = {crossOriginKeyword: ‘anonymous’} ;

Would be interested to hear if others are doing similar or have other solutions to doing Canvasy things with OpenLayers.

OpenLayers Canvas Capture

Mbtiles and Openlayers

Mbtiles and Openlayers

I was testing the feasibility of adding an overlay to openlayers map that is displayed on a mobile/tablet device .

The overlay is going to be in mbtiles format the made popular by MapBox.

The mbtiles db will be accessed locally on the device this useful when bandwidth is poor or non 3g tablets .

The mbtiles format is described here.

Its is basically a sqlite database that holds a collection of  x,y,z indexed tiles.

Webkit based browsers including mobile versions support this although its not actually part of the Html5 spec.

The main issue of using mbtiles locally is actually getting the database into the right location.

Another is the speed at which the device can render the images. The overhead in extracting blob images to the resulting  base64 encoded images.

There are a couple of ways this can be done however.

Getting Mbtiles on Device/Browser

With Phonegap

You can use the  FileTransfer object in phonegap to copy the database locally from a server. It will be downloaded to the Documents folder on the iphone by default.

example code to download an mbtiles db.

var fail = function (error) {

var doOnce = window.localStorage.getItem("doOnce");

   window.requestFileSystem(LocalFileSystem.PERSISTENT, 0, function(fileSystem) {
       fileSystem.root.getFile('testDB2.db', {create: true, exclusive: false}, function(fileEntry) {
           var localPath = fileEntry.fullPath;
           if (device.platform === "Android" && localPath.indexOf("file://") === 0) {
               localPath = localPath.substring(7);
           console.log("LOCAL PATH  "+ localPath);
           var ft = new FileTransfer();
           localPath, function(entry) {
               console.log("successful download");
           }, fail);
       }, fail);
     }, fail);

Use the phonegap web sql plugin  and open the database like.


The benefit of using a phonegap sqllite plugin – allows flexibility where you download the mbtile db to and removes the device dependant limits on database size.

Also if a browser drops native web sql support then it doesn’t matter.


Rather than download a remote database you could copy over a local database at startup.

The simple way to add a prepopulated SQLite DB in PhoneGap from this blog

If you want to keep it an entirely non-native web app based solution or desktop browser (webkit based – Chrome Safari you might be able to use a tool like.

There are more suggestion on stackoverflow here but I not tried them.

By using the syncing by creating an empty local mbtiles database and then populating it by inserts via data from the server is going to adversely affect performance. I have not tried this so I dont know how well it would work.

OpenLayers integration

First thing is to subclass an Openlayers TMS class.

* Map with local storage caching.
* @params options:
*     serviceVersion - TMS service version
*     layerName      - TMS layer name
*     type           - layer type
*     isBaseLayer    - is this the base layer?
*     name         - map name
*     url            - TMS URL
*     opacity        - overlay transparency
var MapWithLocalStorage = OpenLayers.Class(OpenLayers.Layer.TMS, {
   initialize: function(options) {

       this.serviceVersion = options.serviceVersion;
       this.layername = options.layerName;
       this.type = options.type;

       this.async = true;

       this.isBaseLayer = options.isBaseLayer;

           this.opacity = options.opacity;

       OpenLayers.Layer.TMS.prototype.initialize.apply(this, [,
   getURLasync: function(bounds, callback, scope) {
       var urlData = this.getUrlWithXYZ(bounds);
       webdb.getCachedTilePath( callback, scope, urlData.x, urlData.y , urlData.z, urlData.url);
   getUrlWithXYZ: function(bounds){
          bounds = this.adjustBounds(bounds);
       var res =;
       var x = Math.round((bounds.left - this.tileOrigin.lon) / (res * this.tileSize.w));
       var y = Math.round((bounds.bottom - / (res * this.tileSize.h));
       var z = this.serverResolutions != null ?
           OpenLayers.Util.indexOf(this.serverResolutions, res) :
  + this.zoomOffset;

       //inverty for openstreetmap rather than google style TMS
       var ymax = 1 << z;
       var y = ymax - y -1;
       var path = this.serviceVersion + "/" + this.layername + "/" + z + "/" + x + "/" + y + "." + this.type;

       var url = this.url;
       if (OpenLayers.Util.isArray(url)) {
           url = this.selectUrl(path, url);
       return { url: url + path, x:x, y:y, z:z};

   getURL: function(bounds) {
       return OpenLayers.Layer.XYZ.prototype.getURL.apply(this, [bounds]);


this.async = true;

as it will have to receive images from the local sqlite database asynchronously  as  web sql has an asynchronous callback style API.

       var ymax = 1 << z;

       var y = ymax – y -1;

All this does is invert the y axis tile to handle openstreetmap not required for google style TMS.

The is a good site that describes the various types of TMS around.

The Database Setup

"use strict";
var webdb = {};

function getWebDatabase(){
   if(typeof(openDatabase) !== 'undefined'){
       webdb = undefined;
   return webdb;
} = function() {
 var dbSize = 50 * 1024 * 1024; // 50MB
 webdb.db = openDatabase("'testDB2", "1.0", "Cached Tiles", dbSize);

webdb.onError = function(tx, e) {
 console.warn("There has been an error: " + e.message);

webdb.onSuccess = function(tx, r) {
 console.log("Successful Database tx " );

webdb.createTablesIfRequired = function() {
   console.log("Creating DataBase Tables");
 var db = webdb.db;
 db.transaction(function(tx) {
   tx.executeSql("CREATE TABLE IF NOT EXISTS " +
                 "tiles(zoom_level INTEGER, tile_column INTEGER, tile_row INTEGER, tile_data TEXT, mapName TEXT)", [], webdb.onSuccess,

                 " tile_index on tiles(zoom_level, tile_column, tile_row, mapName)", [], webdb.onSuccess,

function hexToBase64(str) {
   var hexString = str.replace(/([\da-fA-F]{2}) ?/g, "0x$1 ");
   var hexArray = hexString.split(" ");
   var len = hexArray.length;
   var binary ='';
   for (var i = 0; i < len; i++) {
       binary += String.fromCharCode( hexArray[ i ] )
   //getting a stack error on large images
   //var binary = String.fromCharCode.apply(null, hexArray);
   return window.btoa(binary);

webdb.getCachedTilePath = function(callback, scope, x, y, z, url ){
   var db = webdb.db;
   var resultsCallback = function(tx, rs) {
       console.log('resultsCallback *********************' );
       console.log('rs.rows.length ' + rs.rows.length);

       if(callback) {
           if( rs.rows.length > 0 ) {
               var rowOutput  = rs.rows.item(0);
               var tile_data = rowOutput['tile_data'];
               //strip off the hex prefix
               tile_data = tile_data.substring(2);

           } else {
     , url);
   db.transaction(function(tx) {
       tx.executeSql("SELECT quote(tile_data) as tile_data FROM tiles where zoom_level=? AND tile_column=? AND tile_row=?", [z,x,y], resultsCallback,


When you have larger blobs in the database you can’t use the overloaded array version of String.fromCharCode as I was getting stack memory issue on the device. (iphone).

So you have to loop through and build it manually.

You have to use the quote function on the tile_data blob to turn it into a hex  string.

“SELECT quote(tile_data) as tile_data

Then trim the hex prefix X’ of the hex string before base64ing.

Testing if you just want to test the javascript /html5 with mbtiles you can copy your mbtiles database to the correct folder .

/Users/murrayking/Library/Application Support/iPhone Simulator/6.1/Applications/667F70EF-D002-425D-86C9-5027C965C518/Library/WebKit/LocalStorage/file__0/0000000000000001.db on a mac

or Chrome on mac as well.

Users/murrayking/Library/Application Support/Google/Chrome/Default/databases/http_localhost_8080/13


This approach is a bit convoluted.

Esp  the conversion of the blob to base64 and performance is a bit poor on older devices. But on newer devices its acceptable.  And as devices become more powerful it will become less issue as with all html5 javascript type things.

Not tried it yet on Android but should work. Worked in the Chrome browser on the linux box.

It does allow you to use rich openlayers framework cross platform without having to invest in native versions.

Also you can debug and test using a desktop browser which is fast before doing proper testing on the actual device.

Example Screenshot working on iphone3g using Phonegap and Mbtiles.

Development version based on our Fieldtrip GB app available on android and iphone.

Overlay is historic map in mbtiles format from the National Library of Scotland.


Debugging on Chrome non-native

working on chrome

Fieldtrip GB App

First of all – apologies for this blog going quiet for so long. Due to resource issues its been hard to keep up with documenting our activities. All the same we have been quietly busy continuing work on geo mobile activity and I’m please to announce that we have now releases our Fieldtrip GB app in the Google Play Store  


We expect the iOS version to go through the Apple App Store  in a few weeks.

Over the next few weeks I’ll be posting to blog with details of how we implemented this app and why we choose certain technologies and solutions.

Hopefully this will prove a useful resource to the community out there trying to do similar things.

A brief summary. The app uses PhoneGap and OpenLayers so is largely using HTML5 web technologies but wrapped up in a native framework. The unique mapping uses OS Open data including Strategi , Vector Map District  and Land-Form PANORAMA mashed together with path and cycleway data from OpenStreetMap and Natural England.


Fourth International Augmented Reality Standards Meeting

I’m just back from the Fourth International AR Standards Meeting that took place in Basel, Switzerland and trying hard to collect my thoughts after two days of intense and stimulating discussion. Apart from anything else, it was a great opportunity to finally meet some people I’ve known from email and discussion boards  on “the left hand side of the reality-virtuality continuum“.

Christine  Perry, the driving spirit, inspiration and editor at large of  AR Standards Group has done a fantastic job bringing so many stakeholders together representing Standards Organisations such as the OGC, Khronos, Web3d Consortium, W3C, OMA and WHATWG  Browser and SDK vendors such as Wikitude, Layar, Opera, ARGON and Qualcomm AR and hardware manufacturers ( Canon, SonyEricsson, NVIDIA) as well as several solution providers such as MOB Labs and mCrumbs – oh and a light sprinkling of academics ( Georgia Tech, Fraunhofer iDG ).

I knew I’d be impressed and slightly awe struck by these highly accomplished people, but what did  surprise me was the lack of  any serious turf fighting. Instead, there was a real sense of pioneering spirit in the room.  Of course everyone had their own story to tell (which just happened to be a story that fitted nicely into their organizational interests), but it really was more about people trying to make some sense of a confusing landscape of technologies and thinking in good faith about what we can do to make it easier.  In particular, it seemed clear that the Standards Organizations felt they could separate the problem space fairly cleanly between their specialist area of interest (geospatial, 3d, hardware/firmware, AR content, web etc). The only area where these groups had significant overlap was on sensor APIs, and some actions were taken to link in with the various Working Groups working on sensors to reduce redundancies.

In seemed to me that there was some agreement about how things will look for AR Content Providers and developers (eventually). Most people appeared to favour the idea of  declarative content mark-up language working in combination with a  scripting language (Javascript) similar to the geolocation API model. Some were keen on the idea of this all being embedded into a standard web browsers Document Object Model. Indeed, Rob Manson, from MobLabs has already achieved a prototype AR experience using various existing (pseduo) standards for web sensor and processing APIs. The two existing markup content proposals ARML and KARML are both based on the OGC’s KML, but even here the idea would be to eventually integrate a KML content and styling model into a generic html model, perhaps following the html/css paradigm.

This shared ambition to  converge AR standards with generic web browser standards is  a recognition that the convergence of hardware, sensors, 3d, computer vision and geo location is a bigger phenomenon than AR browsers or augmented reality. AR is just the first manifestation of this convergence and “anywhere, anytime” access to the virtual world as discussed by Rob Manson on his blog.

To a certain extent, the work we have been discussing here on geo mobile blog, using HTML5 to create web based mapping applications, is a precursor to a much broader sensor enabled web that uses devices such as camera, GPS, compass etc. not just to enable 2d mapping content but all kinds of application that can exploit the sudden happen-chance of  millions of people carrying around dozens of sensors, cameras and powerful compute/graphic processors in their pockets.

Coming back from this meeting, I’m feeling pretty upbeat about the prospects for AR and emerging sensor augmented web. Let’s hope we are able to keep the momentum going for the next meeting in Austin.

OpenLayers Mobile Code Sprint

Last week EDINA had the opportunity to take part in the OpenLayers Mobile code sprint in Lausanne. A group of developers from across the world gathered to add mobile support to the popular Javascript framework.

After a week of intensive development we have been able to add a number of new features allowing OpenLayers to function on a wide range of devices, not only taking advantage of the touch events available on iPhone and some Android mobiles to allow touch navigation, but also enabling the OpenLayers map to be responsive and useful on other platforms, or even unexpected devices!

Jorge Gustavo Rocha and myself worked on adding support for HTML offline storage. Covering storing maps and feature data on the users local browser using the Web Storage and Web SQL standards. Here is the example sandbox which allows the user to store map tiles for the area they are viewing, which are automatically used instead of downloading the online image when possible.  More details on this and other features added can be found on the OpenLayers blog.

I have to say I wasn’t sure what to expect, and I have certainly found it rewarding contributing to OpenLayers and working with such a dedicated and talented team of developers. Far more was achieved than I would have thought possible in such a short space of time. Very inspiring stuff!

Caching and OpenLayers Mobile Code Sprint

We think that a key feature for a mobile mapping application is the ability to view maps offline. One of our key use cases, a educational field trip exercise, requires access to maps in locations where network connectivity might be poor or non existant. Having access offline also widens the kinds of device we can use to include WiFi only devices such as the iPod Touch and no SIM iPad tablet.  Threfore we’ve been working hard over the last couple of months on working out how we can cache maps.  We’ve made alot of progress on this, developing the caching ability in the TouchMapLite library to be more robust and work with our WMS mapping server.

This is possible in a web app environment thanks to 3 seperate  HTML5 initiatives that assist web developers in caching data listed below.

It’s quite easy to get these three flavours of caching mixed up (I’ve done so myself), so one of our engineers has blogged on the subject, explaining what each one does and offering some very helpful hints and gotchas. The first post focuses on the Application Cache, with similar posts on SQL database and Web Storage to follow shortly [update 22 feb 2011: Our engineer has now blogged part 2 ( web SQL database) and part 3 (Web Storage) – some very useful comparisons.

We are going to share the techniques we’ve developed for caching maps during the upcoming OpenLayers Code Sprint in Lausanne later this month. It’s really great to see the OpenLayers development community tackling mobile in earnest this year. Respect to CampToCamp and other sponsers for organizing and sponsoring the event. Geomobile is looking forward to sharing progress with its readers!