Making Digimap data easier to use: OS VectorMap Local

As part of ongoing efforts to improve the usability of data delivered by Digimap, we have made some significant enhancements to one of the Ordnance Survey licensed datasets.

OS VectorMap® Local is a relatively new product from Ordnance Survey providing slightly less detail than OS MasterMap but still very useful for detailed backdrop mapping. Individual buildings are shown as well as roads, rivers, landcover and contours. The vector version of the data is provided by Ordnance Survey in GML 2.1.2. This is a standard geospatial data format developed by the Open Geospatial Consortium; however it generally needs to be converted into a proprietary format before being used in GIS and CAD software.

To make using this data easier we are now providing OS VectorMap Local in both Shapefile and DWG formats. Shapefiles can be easily opened in ArcGIS and most other GIS programs including OpenSource GIS software such as QGIS and gvSIG. DWG is the native format for AutoCAD and can be used in many other CAD systems as well.

This data is now available for download from the Data Download application in the Digimap Ordnance Survey Collection. VectorMap Local is in the Vector Data section; to change the format of the data you are downloading click on the Change link under Options once you have added it to the basket.

Digimap Data Download Basket

Click image to view full size.

Creating the data:

TO achieve the best results when recreating the data in different formats we had to use different software products for different formats. To convert the data to Shapefile we used the OpenSource Software GDAL program called “ogr2ogr” and for the DWG conversion we used FME from Safe Software. The image below shows part of the FME Workbench project used to convert the data to DWG.

Image showing FME Workbench converting GML to DWG

Click image to view full size.

1: Shapefiles

To convert the GML to Shapefile we created a lookup table between the Feature Codes in the GML data to Shapefile Layers.  As Ordnance Survey do not provide any guidance in their User Guide as to how to group features in VML , we analysed the data and experimented with various groupings, so that in the end we split the data into a possible 22 separate Layers. These layers may contain multiple feature classes but as they all have the original feature codes on them as attributes as well as all the other original attribution, further distinction between features can be achieved for representation and querying.

Example of the Shapefile data in ArcMap:

Image of VML in ArcMap

Click image to view full size.

A simple set of Layer Files has been created for use in ArcMap and can be downloaded from the VML Help page.

2. DWG

To convert the data to DWG we had to go through a similar process (as well as our data team becoming much more familiar with AutoCAD). We originally intended to produce DXF format data but this turned out to be a problem. We had to make some decisions about how to store attribution in the output data and one of the main considerations was the size of the resultant files. To begin with when we converted a tile of VML that was 60Mb of GML, it expanded in size to 240Mb when converted to DXF containing attributes as Inserts. This was unusable in systems we tried so we had to work on ways of making the files smaller. The first thing we did was change the format from DXF to DWG. As both formats are commonly read by various CAD systems this seems a reasonable thing to do although it may make the data slightly less interoperable. This changed the file size to 44Mb for our sample dataset. This was still quite large and some systems were having problems with this, especially when dealing with tiles in dense urban areas. We therefore decided to change the way we were storing attributes in the data. Instead of using Inserts we used Extended Entity Data (or XData) which was one of the options available in FME. This stores a fixed amount of information against an entity in the drawing. We use this to store the attribute information that was in the original GML data, including the original feature code. It’s a bit more difficult to work with this data but it can be viewed in AutoCAD  Map 3D by using the Express Tools –> Tools –> List Object Xdata or typing xdlist in the command prompt.

Using XData brought the size of our sample tile down to 10Mb and was usable in all the systems we tested it with. If you need access to the attributes either as Inserts or as Map 3D Object data please let us know.

As part of the conversion process we defined the representation for features in the DWG file, including point symbols and area fills. Features have also been grouped together into separate Layers as we did with the Shapefiles.

Example of the data in AutoCAD:

Image showing VML in DWG in AutoCAD

Click image to view full size.

Finally, after getting the project set up in FME and a thorough testing of the data, we ran the batch process to convert the data. This took nearly 3 days per format to convert the full load of our national coverage of VML data. This data will be updated on the normal update schedule, as we get updated VML from Ordnance Survey as Change Only Updates so future conversions will be much quicker.

Remember,both these datasets are available through Data Download by selecting VectorMap Local and then changing the format after you have added the data to your basket.

We intend to produce alternative formats for other products, including OS MasterMap, so we would very much like to hear your thoughts on the data that we have produced so far and let us know if this meets your needs.

 

 

 

EmailShare

EDINA Geoforum 2013 Review

Geoforum 2013EDINA’s Geoforum 2013 was a great success with over 50 academic and support staff attending four very well received presentations:

Geoforum aims to bring together staff who support the use of Digimap and other geoservices from subscribing institutions around the country and showcase what’s new and upcoming from EDINA. It also gives people an opportunity to ask the Digimap team questions and to chat with others about how support and promote geoservices.

There was a live blog running throughout the day which is still available for you to read and find out what happened.

Keynote Address

The keynote was delivered by Shelley Mosco of The University of Greenwich and was titled: Digimap Data and a non-traditionalist approach.  Shelley is a member of the The School of Architecture, Design and Construction and  described the ways in which spatial data could be used to inform design. Shelley was keen to stress the importance of spatial data and GIS in the implementation of Building Information Models (BIMs).  BIMs have been used in large engineering projects for some time, however the government is making them mandatory for all publicly funded building projects in England and Wales. This means that commercial organisations will be looking for students to have been trained in the concept of BIMs and the software that drives them.  You can find out more about BIMs through the following links:

Geofurum 2013 keynote

Two of Shelley’s current MSc students also gave brief overviews of their experiences of learning about GIS and using spatial data in their projects.  Both David Parfitt and Robert Park were self-confessed GIS newbies, but they managed to get data from Digimap and use it in their conservation projects. The data allowed them to visualise and analyse the environment and provide evidence to support their proposed designs.  Their demo’s were excellent a they really showed the power of simple GIS analysis.

You can view the slides from this presentation here:

Slideshare Shelley Mosco Presentation at Geoforum 2013Digimap Data and a non-traditionalist approach – Shelley Mosco

Open and “Free” Geo Software & Data

After the Keynote was a presentation that focused on Open or “Free” resources for geospatial teaching and research. The presentation looked at  software, web-mapping and data. The main resources are listed below:

Data

  • OS Open data is available through the Digimap Data Download service.
  • ShareGeo Open is a repository for open geospatial data. It has lots of useful and interesting datasets on a variety of subjects such as wind farms, crime, boundaries and DTMs
  • Data.gov.uk – the UK government’s open data store

Software

  • QGIS – one of the best open source GIS out there. Lots of functionality and plugins that allow you to perform complex spatial analysis. It is also well supported by forums.
  • gvSIG – anther fully functioning GIS.
  • GRASS – a remote sensing package aimed at serious remote sensor’s. If you are a newbie to remote sensing, you can access GRASS tool through the GRASS plugin for QGIS which makes things really simple.

Web-mapping

Digimap is a great web mapping tool, but how can you create your own interactive web map for your website?

  • MapBox – simple intuitive web site that helps users build interactive web maps. Basic functionality is free, more advanced functions are available for a small fee.
  • Leaflet – the engine behind MapBox, it is free but requires user to do a “bit” of programing
  • Openlayers – an alternative to Leaflet which is more flexible. Openlayers powers Digimap. Requires a fair amount of programming knowledge.
  • MapServer – implements Openlayers for enterprise scale operations. MapServer is also used for Digimap services.

You can view the slides from this presentation on slideshare here:

OpenSourceOpen & “Free” Geo Software & Data – Tom Armitage

EDINA Geoservices Review

After lunch the lead of EDINA’s geoservices Support team, Emma Diffley, shared what we have been busy working on.

Interfaces

Digimap is being streamlined, each collection will eventually have just a Roam and a Download. These interfaces will all look and work in a very similar way. The aim is to make the service more consistent and easier to use, but also much easier to maintain.

EDINA will be withdrawing Digimap Carto on 31st July 2013. Carto, launched in 1996, is harder and harder to maintain, so we have taken the best bits and put them into Roam.

On top of the Carto functionality already added to Roam the following improvements are coming soon:

  • an improved way of saving and opening maps and annotations
  • a Basemaps button to change the style or even mapping product you are viewing at each scale
  • PDF format prints in Ancient Roam
  • all Roams having the ability to print up to A0
  • a cleaner interface, consistent across all Roams
  • additional overlay options coming soon (boundaries, contours…)

The Download interfaces will also going through a similar process.

Data

  • A single consistent style for all the vector products through all the scale levels accessible through using the basemaps button in the new Roam.
  • VectorMap Local (VML) in shapefile and DWG formats
  • ArcGIS layer files for symbolising VML shapefiles coming soon
  • OS MasterMap Topo available as DWG is being planned

Support

  • There is a new Resource Centre, which does not require login, with answers to questions, videos, case studies etc.

Beyond Digimap

  • GoGeo has added more resources and now highlights the “Editor’s Picks”
  • ShareGeoOpen now has over 210 resources all of which are open and free to use
  • FieldTrip GB is a mobile app for capturing data, see below
  • GeoTagger – a tool to allow you to edit the metadata for your photos
  • Cartogrammar – upload your data and visualise it in different ways
  • UKBORDERS is now the UK Data Service Census Support Unit

On the horizon

We are still awaiting news on funding, but are lots of things we would like to do. Currently our focus is to “mobilise” more services and we will be continuing to add new support materials once the latest round of Roam and Download changes have been made.

If there are things you would like us to do we really want to hear about them. Recommendations from the user community carry real weight for us, and we are keen to hear ideas on new data or services we should be providing.

You can view the slides from this presentation here:

Slideshare Emma Diffley Presentation at Geoforum

EDINA Geoservices Review – Emma Diffley

Fieldtrip GB – data capture, simplified

The final part of the day was about EDINA’s new mobile app for carrying out field work Fieldtrip GB. It simplifies the process of capturing data in the field against quality cartographic mapping. It is equally at home in urban environments as it is in rural ones. Custom forms allow users to design their own data capture projects and collect exactly what they need for their research.  The session gave a brief overview before running a “live” group data collection exercise.  A custom form was created and deployed to participants mobile phones. They then headed outside and captured data on things like building fabric and design.  After 15 mins everyone reconvened and the collected data was “synced” and exported to Google Earth.

You can view the slides from this presentation on slideshare here:

FtGB Slideshare

FieldtripGB – data capture simplified – Addy Pope

As mentioned earlier, there is a transcript of the whole day in the form of a live blog  so if you didn’t manage to attend and want to find out what happened please have a look.

EmailShare