SUNCAT Redevelopment: Focus on Limiting your Search by Institution or Location

Posted on March 11, 2013 by SUNCAT Team

This is the first in a series of blog posts which will highlight some of the new or improved features which will be available in the redeveloped SUNCAT. In this post we are going to focus on how the institutional and geographic limits will improve with the new service in comparison to the current service.

Old SUNCAT: You can only select one institution and/or one location per search.

New SUNCAT: Select as many institutions/locations as you like per search. Also, in the future, we hope to allow you to select and save your preferred group of institutions as your default search preference. We would also hope to set up some pre-selected groupings based on institutional consortia or geographic area. For example these might include UK Research Reserve (UKRR), Scottish Higher Education Digital Library (SHEDL) or Wales, North West England etc.

Old SUNCAT: Only one location (normally the main library) is recorded for each institution on the service, so the geographical location limit is, therefore, not always exact.

New SUNCAT: The new service records all of our contributing institutions’ physical libraries so limiting by location is now much more granular and precise.

Old SUNCAT: You can’t select an institution if it’s outside a location limit you have also selected. For example if you choose to limit your search to the location of “Edinburgh” you cannot then, in addition, select and search on an institution outside Edinburgh, e.g. the University of Glasgow

New SUNCAT: You can combine your limits to create very specific searches tailored for your individual requirements. For example, you can create a search limited to the location of “London” but then also include individual institutions outside London, e.g. University of Oxford.

Old SUNCAT: Although a search with limits applied does only return titles held by that institution or within that location, holdings from other institutions/locations outside of the selected limits will also be displayed.

New SUNCAT: Only holdings within the institutions or locations selected will be returned and displayed, creating a clearer more focussed result and record display.

Naturally, the improved recording of institutions’ individual library locations and the improved functionality of the limits are both vital for the new SUNCAT mobile app which will also be released later this year.

If you have any questions or suggestions about how the limits will work on the new SUNCAT or on the development in general, please contact us at edina@ed.ac.uk.

SUNCAT Redevelopment: Technical architecture overview

Posted on February 13, 2013 by SUNCAT Team

Back in a post in November 2012 we announced that EDINA is redeveloping the existing SUNCAT search platform. We also gave you a preview of the new design and now we want to follow these posts up with more technical information about the new architecture.

New Search Platform

The cornerstone to the SUNCAT service is the ability to search for libraries’ MARC records and we wanted this to be as efficient as possible within the new architecture. We have therefore decided to use the open source enterprise search platform ‘Solr’ from the Apache Software foundation given its popularity and feature set.

We have established a workflow, which exports records from Aleph (currently used for loading and de-duplication of libraries’ serial records) and indexes these records within Solr.

The diagram below illustrates the process:

Individual MARC records are exported from Aleph and these are then grouped into MARC collections. A MARC collection consists of records from the libraries holding a particular serial.

These MARC collections are then indexed in Solr using a modified version of the open source solrmarc code.

Currently the Solr index consists of over 5 million individual library MARC records and in terms of storage sizes, this equates to over 30GB on disk.

New Interface

Storing all of the records in Solr however is only part of the new architecture. We are developing a new user interface for the SUNCAT service, taking the opportunity to also incorporate some new features that exploit the power of Solr.

A combination of the Java programming language, Groovy programming language and the Grails framework has been chosen as the software stack as it enables a rapid development process and also leverages the experience of developers within EDINA.

Grails follows the highly popular software design pattern of Model View Controller, which allows separation of concerns and a clean software design.

The following diagram illustrates at a very high level the architecture we are using and how Solr is involved:

We are developing a number of ‘controllers’, which are used to process user requests and issue queries to Solr as and when required. These controllers are designed for specific tasks e.g.

• Handling search requests

• Handling API requests

• Handling requests to view details of a specific MARC record etc.

We also have a number of templates that comprise the ‘views’ of the new system. When a controller has completed the processing of the user request and it is ready to return something to the user (e.g. a web page), it uses the relevant template and injects the correct data e.g. the search results. Using a template approach for all of the views of the system provides a huge amount of flexibility as we have full control of all visual aspects of the service and we can even support different output formats based on the user request e.g. supporting HTML, XML, JSON etc.

Enabling Searching at Library Level

We are also spending time in constructing a database to store institution, library and location data and integrating this within the new user interface in-order to allow users to perform more detailed searches within SUNCAT. For example searches can be performed at the library level whereas previously users could only search at the institutional level. We have also tagged every library with its GPS coordinates, which allows us to show all SUNCAT contributing libraries in a map interface and will allow proximity based searching in the forthcoming SUNCAT mobile application for iOS.

We will be keeping you up to date with more posts, about the redevelopment and the mobile app, to follow over the next few months.

Redevelopment of SUNCAT Platform

Posted on November 21, 2012 by SUNCAT Team

EDINA has embarked on a programme to redevelop the existing SUNCAT search platform. The impetus for this redevelopment emerged from a long held desire to not only provide enhanced functionality but also to be able to be more responsive to user feedback regarding suggested improvements.

Work commenced on the first phase of this development in spring 2012 as EDINA developers started to design and implement an entirely new bespoke user interface for the SUNCAT service.

In this initial development phase SUNCAT will continue to rely on Ex Libris’ Aleph software to load and de-duplicate contributing libraries’ serials records. The web interface however, will be developed in-house leveraging the open source enterprise search platform, Solr to facilitate highly efficient searching across the millions of SUNCAT records.

The developers considered a number of options to facilitate record searching, but Solr proved to be the best solution for dealing with the complex issues around searching and displaying records grouped into matched sets, a central component of the SUNCAT service. Moving to this open source platform should allow EDINA to have greater control and flexibility over the functionality and presentation of SUNCAT.

One key area of improvement, which will be available from the outset, is the ability to limit search results restricted to holdings from multiple libraries and locations. These limits will include all the individual locations of each of our contributing libraries, rather than just locations at an institutional level as with the current service. Another benefit will be that users will be able to select multiple locations and/or institutions to limit their search by, so giving them great flexibility. The limits will now also ensure that users only see the holdings from locations or institutions they are interested in, as any extraneous holdings will no longer be displayed. These improvements mean that in the future EDINA will be able to provide customised views onto the service, configurable at both the individual user level, and also at a higher geographic, subject specialist or consortial level.

The improvements to the geographic limits are particularly important for the mobile application which is also currently in development. EDINA conducted some early user testing with a small group of volunteers earlier in the year and it is hoped that a beta version will be made more widely available early in 2013.

Other key areas of new functionality will follow throughout the next year. The SUNCAT team have identified a wish list of features based on user feedback and also on a survey of some of the best functionality available in commercial search engines, library and union catalogues in the UK, Europe and beyond.

The feedback and survey also informed the design brief for the redeveloped service. Having considered a number of designs the SUNCAT team have narrowed the selection down to a few favoured options and we are currently consulting with our contributing libraries to decide on the final design.

It is hoped that a beta version of the new platform will be available in spring 2013, when we will be asking our users to provide feedback on progress. We hope that you will approve of the changes to come!

EDINA Blogs

A Blogs.edina.ac.uk weblog

Category Archives: search platform