Replacement of MasterMap Download

Posted on March 26, 2013 by ecls

The functionality and datasets offered by MasterMap Download and Boundary Download are now available through the standard Data Download application. This can be found in the Download OS mapping data section of Digimap’s Ordnance Survey Collection.

This means that both MasterMap Download and Boundary Download will be withdrawn at the end of May 2013.

If you have teaching materials or course notes which are currently based on the old MasterMap Download and Boundary Download applications, please make sure you update them before these facilities are withdrawn.

If you have any concerns about this or any other issue then please contact us:

Email: edina@ed.ac.uk
Tel: 0131 650 3302

Preview the New Look SUNCAT in Early April

Posted on March 25, 2013 by SUNCAT Team

We’ve been blogging about the new look SUNCAT for a few months now, giving you a look at the new homepage, some background on the technical architecture and also some idea of the new features which will be available. However, we now hope to let you have a preview of the new service itself!

The current service will continue as the primary service until further notice, but you can check out the redeveloped service from 8th April when we preview it at the UKSG Conference (Bournemouth 8-10 April). We will let you know the url on the day and from then on we would encourage you to take a look, experiment and let us have your feedback, good and bad! The service is still in development so the more feedback we can gather at this point the better.

If you are attending UKSG, please come along to the EDINA stand 26 and we will be happy to demo the service for you and answer your questions in person. However, we would also encourage you to let us know what you think or if you have any questions or suggestions via the EDINA helpdesk at edina@ed.ac.uk.

SUNCAT Redevelopment: Focus on Limiting your Search by Institution or Location

Posted on March 11, 2013 by SUNCAT Team

This is the first in a series of blog posts which will highlight some of the new or improved features which will be available in the redeveloped SUNCAT. In this post we are going to focus on how the institutional and geographic limits will improve with the new service in comparison to the current service.

Old SUNCAT: You can only select one institution and/or one location per search.

New SUNCAT: Select as many institutions/locations as you like per search. Also, in the future, we hope to allow you to select and save your preferred group of institutions as your default search preference. We would also hope to set up some pre-selected groupings based on institutional consortia or geographic area. For example these might include UK Research Reserve (UKRR), Scottish Higher Education Digital Library (SHEDL) or Wales, North West England etc.

Old SUNCAT: Only one location (normally the main library) is recorded for each institution on the service, so the geographical location limit is, therefore, not always exact.

New SUNCAT: The new service records all of our contributing institutions’ physical libraries so limiting by location is now much more granular and precise.

Old SUNCAT: You can’t select an institution if it’s outside a location limit you have also selected. For example if you choose to limit your search to the location of “Edinburgh” you cannot then, in addition, select and search on an institution outside Edinburgh, e.g. the University of Glasgow

New SUNCAT: You can combine your limits to create very specific searches tailored for your individual requirements. For example, you can create a search limited to the location of “London” but then also include individual institutions outside London, e.g. University of Oxford.

Old SUNCAT: Although a search with limits applied does only return titles held by that institution or within that location, holdings from other institutions/locations outside of the selected limits will also be displayed.

New SUNCAT: Only holdings within the institutions or locations selected will be returned and displayed, creating a clearer more focussed result and record display.

Naturally, the improved recording of institutions’ individual library locations and the improved functionality of the limits are both vital for the new SUNCAT mobile app which will also be released later this year.

If you have any questions or suggestions about how the limits will work on the new SUNCAT or on the development in general, please contact us at edina@ed.ac.uk.

SUNCAT Redevelopment: Technical architecture overview

Posted on February 13, 2013 by SUNCAT Team

Back in a post in November 2012 we announced that EDINA is redeveloping the existing SUNCAT search platform. We also gave you a preview of the new design and now we want to follow these posts up with more technical information about the new architecture.

New Search Platform

The cornerstone to the SUNCAT service is the ability to search for libraries’ MARC records and we wanted this to be as efficient as possible within the new architecture. We have therefore decided to use the open source enterprise search platform ‘Solr’ from the Apache Software foundation given its popularity and feature set.

We have established a workflow, which exports records from Aleph (currently used for loading and de-duplication of libraries’ serial records) and indexes these records within Solr.

The diagram below illustrates the process:

Individual MARC records are exported from Aleph and these are then grouped into MARC collections. A MARC collection consists of records from the libraries holding a particular serial.

These MARC collections are then indexed in Solr using a modified version of the open source solrmarc code.

Currently the Solr index consists of over 5 million individual library MARC records and in terms of storage sizes, this equates to over 30GB on disk.

New Interface

Storing all of the records in Solr however is only part of the new architecture. We are developing a new user interface for the SUNCAT service, taking the opportunity to also incorporate some new features that exploit the power of Solr.

A combination of the Java programming language, Groovy programming language and the Grails framework has been chosen as the software stack as it enables a rapid development process and also leverages the experience of developers within EDINA.

Grails follows the highly popular software design pattern of Model View Controller, which allows separation of concerns and a clean software design.

The following diagram illustrates at a very high level the architecture we are using and how Solr is involved:

We are developing a number of ‘controllers’, which are used to process user requests and issue queries to Solr as and when required. These controllers are designed for specific tasks e.g.

• Handling search requests

• Handling API requests

• Handling requests to view details of a specific MARC record etc.

We also have a number of templates that comprise the ‘views’ of the new system. When a controller has completed the processing of the user request and it is ready to return something to the user (e.g. a web page), it uses the relevant template and injects the correct data e.g. the search results. Using a template approach for all of the views of the system provides a huge amount of flexibility as we have full control of all visual aspects of the service and we can even support different output formats based on the user request e.g. supporting HTML, XML, JSON etc.

Enabling Searching at Library Level

We are also spending time in constructing a database to store institution, library and location data and integrating this within the new user interface in-order to allow users to perform more detailed searches within SUNCAT. For example searches can be performed at the library level whereas previously users could only search at the institutional level. We have also tagged every library with its GPS coordinates, which allows us to show all SUNCAT contributing libraries in a map interface and will allow proximity based searching in the forthcoming SUNCAT mobile application for iOS.

We will be keeping you up to date with more posts, about the redevelopment and the mobile app, to follow over the next few months.

OS MasterMap and Boundaries now in Digimap Data Download

Posted on February 8, 2013 by Tom Armitage

We have now put the datasets that were in MasterMap Download and Boundary Download into the Data Download facility. The datasets added are:

OS MasterMap® Topography Layer
OS MasterMap® ITN (Integrated Transport Network™) Layer
Boundary-Line™
Codepoint®
Codepoint® with Polygons
National Boundaries
National Grids

MasterMap Download and Boundary Download will continue to run until the end of May after which they will be removed from service.

You should find it quicker and easier to get the data you need from Data Download, and we have made it easier for you to come back and get updated versions or different formats too.

This functionality can be found in the My Previous Downloads section where you can also make a Change Only Update (COU) to your OS MasterMap data. See this blog post for more details:

Digimap Data Download: New Datasets

Another big advantage of getting your OS MasterMap data from Data Download is that the interface allows you to select data from previous years, going back to 2009.

You can select an area, add MasterMap data from 2009, then use the Add more data button to include more MasterMap data from different dates. This is a great feature if you want to study how an area has changed over time.

SUNCAT Redevelopment

Posted on February 7, 2013 by SUNCAT Team

We reported in a blog post in November that a new interface for SUNCAT is being developed and since good progress has been made we are keen to share with you more details on what we are doing and perhaps more importantly when we plan to show what has been done. We are working towards having a public preview of the new interface early in April.

Building up to this we will have a blog post next week on the technical architecture which has been adopted. This will be followed by blog posts on the functionality, which will be available initially, and outlining the details for the public preview. We are very keen to find out from our users what they think of the development and accordingly will be asking for feedback after users have had a chance to view the interface.

Future blog posts will include sharing the roadmap of our planned developments until the end of the year and providing information on specific aspects of the technical developments.

It’s clearly a busy time for us but we feel that it is time and effort well spent as we will be in a position to develop and implement requested functionality much more quickly than we have been able to in the past.

Digimap Data Download: New datasets coming in January!

Posted on December 18, 2012 by Tom Armitage

In January 2013 the OS MasterMap^®, Boundary-Line™ and Code-Point^® data products will be available from the Data Download service. We have tried to keep the changes necessary to accommodate these products to a minimum as the current interface has proved so successful (190,000 map tiles and files downloaded Nov 2012).

Here is a list of the changes and additions you will see in the interface; please note that they may still be subject to some change prior to launch.

Both Boundary-Line and MasterMap have layers so we have had to make changes to the basket:

Instead of selecting Version and Format using drop-down menus there is a new pop-up for changing both these and the layers for each of the products in your basket. You can access this pop-up by clicking on the Change link in the Options column:

We have also improved the “My Account” section which is now called “My Previous Downloads”:

Once in the interface you can access any of your past downloads and reopen them at the basket stage:

It is now much easier to update your previous downloads to the latest version, get the same data in a different format or simply reorder data you have previously taken. In addition to these functions we have incorporated the ability to make Change-Only-Updates (COUs) to MasterMap Data:

Both Boundary Download and MasterMap Download will be kept in service for a transition period but as there are so many benefits to using Data Download we hope everyone will make the switch quickly. We expect to have switched off these services by the end of April.

New Design for SUNCAT

Posted on November 28, 2012 by SUNCAT Team

As part of the SUNCAT Redevelopment we reported on last week, we will be introducing a new contemporary design for the SUNCAT web interface.

A number of designs were produced by EDINA’s in-house designer, based on a design brief submitted by the SUNCAT team based on

User feedback we have received over the past few years.
A survey of a number of commercial search engines as well as library and union catalogues in the UK, Europe and beyond. These sites included:

Feedback suggested that the current design was well liked but was starting to look somewhat dated so some key elements of the brief included:

Keeping the SUNCAT logo and colours for continuity
Keeping the search functionality at the heart of the design
Incorporating other elements of the site into the search homepage to reflect a more portal style approach
Trying to keep the design clean and simple.

After the initial set of designs was discussed and adapted by the SUNCAT team, the selection was narrowed to two preferred designs which were then circulated to EDINA colleagues for comment.

We were then keen to consult our SUNCAT contributors to involve them in the final selection and to gather feedback which could be incorporated into the ultimate design. A short online survey was made available to SUNCAT contributing libraries for a week at the beginning of November. Sixty responses were received in total, from at least 26 different libraries.

Seventy-two percent of respondents preferred the design displayed below, one percent liked both designs equally and happily none of the respondents reported that they didn’t like either design. The main reasons for preferring the design below included:

Simpler, cleaner and more user friendly
Preferred colours and images
Layout of search and filters on the screen
Having a map of contributing libraries and a newsfeed easily accessible on the homepage

New Design for SUNCAT Interface

We will now be looking at some of the suggestions on how to improve the design, such as:

Changing the colour of the “Find Now” to better differentiate it from the limit options
Moving the Advanced Search and Browse buttons closer to the main search box
Moving the Reset button to beneath the “Find Now” button.

The new design will form part of the beta release of the new platform in spring 2013, please let us know if you have any comments about SUNCAT’s new look in the meantime.

Redevelopment of SUNCAT Platform

Posted on November 21, 2012 by SUNCAT Team

EDINA has embarked on a programme to redevelop the existing SUNCAT search platform. The impetus for this redevelopment emerged from a long held desire to not only provide enhanced functionality but also to be able to be more responsive to user feedback regarding suggested improvements.

Work commenced on the first phase of this development in spring 2012 as EDINA developers started to design and implement an entirely new bespoke user interface for the SUNCAT service.

In this initial development phase SUNCAT will continue to rely on Ex Libris’ Aleph software to load and de-duplicate contributing libraries’ serials records. The web interface however, will be developed in-house leveraging the open source enterprise search platform, Solr to facilitate highly efficient searching across the millions of SUNCAT records.

The developers considered a number of options to facilitate record searching, but Solr proved to be the best solution for dealing with the complex issues around searching and displaying records grouped into matched sets, a central component of the SUNCAT service. Moving to this open source platform should allow EDINA to have greater control and flexibility over the functionality and presentation of SUNCAT.

One key area of improvement, which will be available from the outset, is the ability to limit search results restricted to holdings from multiple libraries and locations. These limits will include all the individual locations of each of our contributing libraries, rather than just locations at an institutional level as with the current service. Another benefit will be that users will be able to select multiple locations and/or institutions to limit their search by, so giving them great flexibility. The limits will now also ensure that users only see the holdings from locations or institutions they are interested in, as any extraneous holdings will no longer be displayed. These improvements mean that in the future EDINA will be able to provide customised views onto the service, configurable at both the individual user level, and also at a higher geographic, subject specialist or consortial level.

The improvements to the geographic limits are particularly important for the mobile application which is also currently in development. EDINA conducted some early user testing with a small group of volunteers earlier in the year and it is hoped that a beta version will be made more widely available early in 2013.

Other key areas of new functionality will follow throughout the next year. The SUNCAT team have identified a wish list of features based on user feedback and also on a survey of some of the best functionality available in commercial search engines, library and union catalogues in the UK, Europe and beyond.

The feedback and survey also informed the design brief for the redeveloped service. Having considered a number of designs the SUNCAT team have narrowed the selection down to a few favoured options and we are currently consulting with our contributing libraries to decide on the final design.

It is hoped that a beta version of the new platform will be available in spring 2013, when we will be asking our users to provide feedback on progress. We hope that you will approve of the changes to come!

Digimap Carto: We need your help!

Posted on October 12, 2012 by Tom Armitage

Digimap Carto has been making maps for nearly 13 years now, though it last received a major facelift in 2006. Although it is still well used the underlying technology is no longer a viable platform to develop on. It has simply become too difficult to maintain in today’s environment of almost monthly changes to web browsers and the Java Runtime Environment.

Thankfully these advances in browser technology, and our underlying GIS and web mapping software, mean that we can now provide a Carto like service without the Java applet. In 2013 we will be launching a replacement for Carto that will run directly in an up-to-date web browser and still include advanced functionality. Until the new interface is launched we will keep Carto running, please go here if you are having trouble getting it to work: Carto Notes

With work starting on the new advanced mapping application we would really like to know which functions you value most in Carto. We’d also like to hear your ideas for any new features you would like to see available. We can’t promise to include everything, but knowing what you want helps us to make informed decisions.

Please send your thoughts to: edina@ed.ac.uk

EDINA Blogs

A Blogs.edina.ac.uk weblog

Category Archives: interface