SUNCAT Redevelopment: Focus on Limiting your Search by Institution or Location

Posted on March 11, 2013 by SUNCAT Team

This is the first in a series of blog posts which will highlight some of the new or improved features which will be available in the redeveloped SUNCAT. In this post we are going to focus on how the institutional and geographic limits will improve with the new service in comparison to the current service.

Old SUNCAT: You can only select one institution and/or one location per search.

New SUNCAT: Select as many institutions/locations as you like per search. Also, in the future, we hope to allow you to select and save your preferred group of institutions as your default search preference. We would also hope to set up some pre-selected groupings based on institutional consortia or geographic area. For example these might include UK Research Reserve (UKRR), Scottish Higher Education Digital Library (SHEDL) or Wales, North West England etc.

Old SUNCAT: Only one location (normally the main library) is recorded for each institution on the service, so the geographical location limit is, therefore, not always exact.

New SUNCAT: The new service records all of our contributing institutions’ physical libraries so limiting by location is now much more granular and precise.

Old SUNCAT: You can’t select an institution if it’s outside a location limit you have also selected. For example if you choose to limit your search to the location of “Edinburgh” you cannot then, in addition, select and search on an institution outside Edinburgh, e.g. the University of Glasgow

New SUNCAT: You can combine your limits to create very specific searches tailored for your individual requirements. For example, you can create a search limited to the location of “London” but then also include individual institutions outside London, e.g. University of Oxford.

Old SUNCAT: Although a search with limits applied does only return titles held by that institution or within that location, holdings from other institutions/locations outside of the selected limits will also be displayed.

New SUNCAT: Only holdings within the institutions or locations selected will be returned and displayed, creating a clearer more focussed result and record display.

Naturally, the improved recording of institutions’ individual library locations and the improved functionality of the limits are both vital for the new SUNCAT mobile app which will also be released later this year.

If you have any questions or suggestions about how the limits will work on the new SUNCAT or on the development in general, please contact us at edina@ed.ac.uk.

SUNCAT Survey Report 2013

Posted on February 14, 2013 by SUNCAT Team

We’ve just published the report from our Benefits and Impact Survey for 2013. The survey was launched back in November and closed at the end of last month.

The report confirms SUNCAT’s primary role as a centralised source of serials information and UK holdings, with 75% of responses stating, either locating serials and articles, or checking bibliographic information, as the primary purposes for using the service.

The key features of the service which are most valued by the respondents include:

SUNCAT’s comprehensive coverage
Its aggregation and display of serials information and UK holdings
The accuracy and currency of the data provided
The speed and ease of use of the service

We are very happy to see that the vast majority of respondents find SUNCAT not only easy or very easy to use (86%) but that it also saves them time (89%). Further, 97% indicated they would recommend the service to others.

“It is such a comprehensive reference source it would take me much longer to check information elsewhere.”

“SUNCAT is easy to use and its coverage of UK serial holdings is great.”

“Very trustworthy, fast and comprehensive. Records are to a very high standard.”

“Find Suncat invaluable. If it wasn’t available I would try to source an alternative. Don’t know what though!”

We also used the survey to find out what improvements our users would like to see in SUNCAT so that we can use this information to plan and prioritise our future developments. A number of the suggestions related to the data provided by the SUNCAT Contributing Libraries (CLs), both requesting more detailed holdings information and also providing additional information such as licensing restrictions for electronic serials. While this area is not one EDINA has immediate control over we do take note of these ideas, in the hope that if our CLs start recording and linking such information to their serials records, we would be able to start incorporating this into SUNCAT.

However, with regard to suggested improvements related to the SUNCAT interface we are happy to report that a number of these will be addressed as part of the redevelopment of SUNCAT, announced in November and mentioned in the previous posting. These include:

Moving the search functionality to the homepage of the service to integrate the service and website more closely
Providing a print/electronic serials limit to the search functionality to enable users to pre-filter their search by format
Adding print/electronic icons to the holdings display to enable users to more quickly and easily distinguish between print and electronic holdings

It is also good to know that our users are keen to see the service continue to expand and we will indeed be continuing to add new libraries throughout 2013.

Additional suggestions, described in the report, were also received and we will be investigating the feasibility of these and where possible adding them to a list of requirements for future development work.

SUNCAT Redevelopment: Technical architecture overview

Posted on February 13, 2013 by SUNCAT Team

Back in a post in November 2012 we announced that EDINA is redeveloping the existing SUNCAT search platform. We also gave you a preview of the new design and now we want to follow these posts up with more technical information about the new architecture.

New Search Platform

The cornerstone to the SUNCAT service is the ability to search for libraries’ MARC records and we wanted this to be as efficient as possible within the new architecture. We have therefore decided to use the open source enterprise search platform ‘Solr’ from the Apache Software foundation given its popularity and feature set.

We have established a workflow, which exports records from Aleph (currently used for loading and de-duplication of libraries’ serial records) and indexes these records within Solr.

The diagram below illustrates the process:

Individual MARC records are exported from Aleph and these are then grouped into MARC collections. A MARC collection consists of records from the libraries holding a particular serial.

These MARC collections are then indexed in Solr using a modified version of the open source solrmarc code.

Currently the Solr index consists of over 5 million individual library MARC records and in terms of storage sizes, this equates to over 30GB on disk.

New Interface

Storing all of the records in Solr however is only part of the new architecture. We are developing a new user interface for the SUNCAT service, taking the opportunity to also incorporate some new features that exploit the power of Solr.

A combination of the Java programming language, Groovy programming language and the Grails framework has been chosen as the software stack as it enables a rapid development process and also leverages the experience of developers within EDINA.

Grails follows the highly popular software design pattern of Model View Controller, which allows separation of concerns and a clean software design.

The following diagram illustrates at a very high level the architecture we are using and how Solr is involved:

We are developing a number of ‘controllers’, which are used to process user requests and issue queries to Solr as and when required. These controllers are designed for specific tasks e.g.

• Handling search requests

• Handling API requests

• Handling requests to view details of a specific MARC record etc.

We also have a number of templates that comprise the ‘views’ of the new system. When a controller has completed the processing of the user request and it is ready to return something to the user (e.g. a web page), it uses the relevant template and injects the correct data e.g. the search results. Using a template approach for all of the views of the system provides a huge amount of flexibility as we have full control of all visual aspects of the service and we can even support different output formats based on the user request e.g. supporting HTML, XML, JSON etc.

Enabling Searching at Library Level

We are also spending time in constructing a database to store institution, library and location data and integrating this within the new user interface in-order to allow users to perform more detailed searches within SUNCAT. For example searches can be performed at the library level whereas previously users could only search at the institutional level. We have also tagged every library with its GPS coordinates, which allows us to show all SUNCAT contributing libraries in a map interface and will allow proximity based searching in the forthcoming SUNCAT mobile application for iOS.

We will be keeping you up to date with more posts, about the redevelopment and the mobile app, to follow over the next few months.

SUNCAT Redevelopment

Posted on February 7, 2013 by SUNCAT Team

We reported in a blog post in November that a new interface for SUNCAT is being developed and since good progress has been made we are keen to share with you more details on what we are doing and perhaps more importantly when we plan to show what has been done. We are working towards having a public preview of the new interface early in April.

Building up to this we will have a blog post next week on the technical architecture which has been adopted. This will be followed by blog posts on the functionality, which will be available initially, and outlining the details for the public preview. We are very keen to find out from our users what they think of the development and accordingly will be asking for feedback after users have had a chance to view the interface.

Future blog posts will include sharing the roadmap of our planned developments until the end of the year and providing information on specific aspects of the technical developments.

It’s clearly a busy time for us but we feel that it is time and effort well spent as we will be in a position to develop and implement requested functionality much more quickly than we have been able to in the past.

New Design for SUNCAT

Posted on November 28, 2012 by SUNCAT Team

As part of the SUNCAT Redevelopment we reported on last week, we will be introducing a new contemporary design for the SUNCAT web interface.

A number of designs were produced by EDINA’s in-house designer, based on a design brief submitted by the SUNCAT team based on

User feedback we have received over the past few years.
A survey of a number of commercial search engines as well as library and union catalogues in the UK, Europe and beyond. These sites included:

Feedback suggested that the current design was well liked but was starting to look somewhat dated so some key elements of the brief included:

Keeping the SUNCAT logo and colours for continuity
Keeping the search functionality at the heart of the design
Incorporating other elements of the site into the search homepage to reflect a more portal style approach
Trying to keep the design clean and simple.

After the initial set of designs was discussed and adapted by the SUNCAT team, the selection was narrowed to two preferred designs which were then circulated to EDINA colleagues for comment.

We were then keen to consult our SUNCAT contributors to involve them in the final selection and to gather feedback which could be incorporated into the ultimate design. A short online survey was made available to SUNCAT contributing libraries for a week at the beginning of November. Sixty responses were received in total, from at least 26 different libraries.

Seventy-two percent of respondents preferred the design displayed below, one percent liked both designs equally and happily none of the respondents reported that they didn’t like either design. The main reasons for preferring the design below included:

Simpler, cleaner and more user friendly
Preferred colours and images
Layout of search and filters on the screen
Having a map of contributing libraries and a newsfeed easily accessible on the homepage

New Design for SUNCAT Interface

We will now be looking at some of the suggestions on how to improve the design, such as:

Changing the colour of the “Find Now” to better differentiate it from the limit options
Moving the Advanced Search and Browse buttons closer to the main search box
Moving the Reset button to beneath the “Find Now” button.

The new design will form part of the beta release of the new platform in spring 2013, please let us know if you have any comments about SUNCAT’s new look in the meantime.

Redevelopment of SUNCAT Platform

Posted on November 21, 2012 by SUNCAT Team

EDINA has embarked on a programme to redevelop the existing SUNCAT search platform. The impetus for this redevelopment emerged from a long held desire to not only provide enhanced functionality but also to be able to be more responsive to user feedback regarding suggested improvements.

Work commenced on the first phase of this development in spring 2012 as EDINA developers started to design and implement an entirely new bespoke user interface for the SUNCAT service.

In this initial development phase SUNCAT will continue to rely on Ex Libris’ Aleph software to load and de-duplicate contributing libraries’ serials records. The web interface however, will be developed in-house leveraging the open source enterprise search platform, Solr to facilitate highly efficient searching across the millions of SUNCAT records.

The developers considered a number of options to facilitate record searching, but Solr proved to be the best solution for dealing with the complex issues around searching and displaying records grouped into matched sets, a central component of the SUNCAT service. Moving to this open source platform should allow EDINA to have greater control and flexibility over the functionality and presentation of SUNCAT.

One key area of improvement, which will be available from the outset, is the ability to limit search results restricted to holdings from multiple libraries and locations. These limits will include all the individual locations of each of our contributing libraries, rather than just locations at an institutional level as with the current service. Another benefit will be that users will be able to select multiple locations and/or institutions to limit their search by, so giving them great flexibility. The limits will now also ensure that users only see the holdings from locations or institutions they are interested in, as any extraneous holdings will no longer be displayed. These improvements mean that in the future EDINA will be able to provide customised views onto the service, configurable at both the individual user level, and also at a higher geographic, subject specialist or consortial level.

The improvements to the geographic limits are particularly important for the mobile application which is also currently in development. EDINA conducted some early user testing with a small group of volunteers earlier in the year and it is hoped that a beta version will be made more widely available early in 2013.

Other key areas of new functionality will follow throughout the next year. The SUNCAT team have identified a wish list of features based on user feedback and also on a survey of some of the best functionality available in commercial search engines, library and union catalogues in the UK, Europe and beyond.

The feedback and survey also informed the design brief for the redeveloped service. Having considered a number of designs the SUNCAT team have narrowed the selection down to a few favoured options and we are currently consulting with our contributing libraries to decide on the final design.

It is hoped that a beta version of the new platform will be available in spring 2013, when we will be asking our users to provide feedback on progress. We hope that you will approve of the changes to come!

Article in DPC / DCC What’s New

Posted on August 7, 2012 by Adam Rusbridge

Adam Rusbridge has written a short article on ‘Accessing ceased or cancelled content with LOCKSS‘ in the August 2012 edition of DPC/DCC What’s New. The article provides an update on recent developments around link resolver integration and introduces the benefits that results from supply of access.

New Title List Features in version 1.55

Posted on June 27, 2012 by neil.mayo

The Title List function in your LOCKSS box allows you to report on the holdings of your LOCKSS box or the content available in the LOCKSS network as a whole. The latest release of the LOCKSS software includes some new options, and this post provides a general guide to usage of the Title List feature.

Please see the post View your holdings with LOCKSS titles export for an introduction to the Title List feature.

Title List main screen

Basic Report Options

The main Title List screen presents you with several options for the report, described below. Note that the defaults will produce a standard KBART report, but these options allow you to customise that to better suit your needs.

Scope

This option allows you to specify the scope of the data you want to report on:

Available – all titles available in the LOCKSS network. This is all the content LOCKSS users can select from.
Configured – the titles that you have configured for collection on your institution’s machine using Add Titles.
Collected – the titles that have successfully collected on your machine. A title must be fully collected before it will appear in this list; partial completion is not enough.

Type

This option allows you to filter the titles to include only journals or books if you need to:

Journals – show only journal titles
Books – show only book titles (those that have an ISBN)
All – show all titles

Data Format

This option specifies the format of the data that is output from the reporting tool; the default Title Ranges (KBART) format displays a line for every complete range in a title, while the other options allow you to consolidate these into a single line per title.

The KBART coverage_notes field will contain a description of the complete ranges covered by each row, therefore documenting where the coverage gaps are in the consolidated formats.

Title Ranges – show a row for each unbroken range within a title (this is the default KBART format).
Titles – show a row for each title, consolidating all the individual unbroken ranges into a single range using the outermost start and end volumes. Note that the single range specified in this format may contain coverage gaps.
SFX DataLoader – show a row for each title, with the coverage_notes field listing coverage ranges in the SFX DataLoader format.

Output Format

This option specifies the digital format in which the report will be produced:

TSV – tab-separated format
CSV – comma-separated format
On-screen – the report will be turned into HTML and displayed on-screen

Having selected these options, press the List Titles button to generate the report.

A Title List report showing the customise option

The above shows the top of an HTML report. Notice there is a link to return to the main Title List page, and a button for customising the report you have generated.

Customisation Options

To provide finer-grained control over what gets into your report, there are customisation options allowing you to specify which fields to output and in what order.

Select the Customise Fields button either on the Title List screen or the HTML report screen. You will see these extra options appear at the bottom of the Title List screen:

Title List customisation options

Coverage Range format

The default KBART output reports each unbroken range of a title on a separate line. Sometimes it is more useful to have a single line per title, with start and end points. The coverage_notes field is therefore used to elaborate which specific ranges are fully available within a title listed in a particular row. This field can display data in a number of formats:

Year Ranges – a comma-separated list of ranges, e.g. 1996-2002, 2004-2006, 2009-
Year(Volume) Ranges – a comma-separated list of ranges including volume in parentheses e.g. 1996(1)-2002(7), 2004(9)-2006(11), 2009(14)-
Year Summary – a concise format showing only the start and end of the whole range, e.g. 1996-
Year(Volume) Summary – a concise format showing only the start and end of the whole range, e.g. 1996(1)-
SFX DataLoader – this is the format defined by ExLibris for use with their SFX DataLoader utility, e.g. $obj->parsedDate(“>=”,1996,1,undef) && $obj->parsedDate(“<=”,2002,7,undef)

Note that year ranges can have an empty end point, meaning they extend to the present. The summary formats are for convenience and do not indicate where there are coverage gaps. The SFX format represents the same information as the year(volume) format but in a more verbose format.

Field Ordering

The field ordering box lists all the default KBART fields in their default order. You can edit the contents of this box to change which fields appear in the output. The ordering will also be reflected in the output report. On the right-hand side is a list of all the valid field names – be sure to include at least one of the identifying fields or there will be no way to organise the title records!

Note that the first two fields will be used to sort the resulting list.

Omit empty columns

Some columns have nothing in them due to a lack of appropriate data. Select this option to automatically omit such columns and simplify the report.

Reset/Cancel

If you decide you don’t want to customise the output after all, or have made a mistake, you can select either:

Reset – to reset the field ordering list to its original values
Cancel – to cancel customisation and go straight (back) to the report output

Finally, press List Titles to produce the report, or Hide Customise Fields to skip the customisation and use the three default report configuration options of scope, data format and output format.

Direct URL access to reports

It is possible to access a range of reports directly without manually selecting options on the Title List screen. This is useful for those applications where an automated update of the LOCKSS box’s holdings is required, for example when updating a knowledge base periodically. The default report can be retrieved by requesting the URL

http://lockss.box:8081/Titles?format=tsv

There are five configuration options for direct URL reports, which can be specified as URL parameters. The first four correspond to the basic report options described earlier:

format – output format: tsv, csv, html
scope – data scope: available, configured, collected
type – title type: journals, books, all
report – data format: kbart, titles, sfx
coverageNotesFormat – format for the coverage_notes field: year, year_volume, year_summary, year_volume_summary, sfx

The bare minimum is a format parameter. So for example a TSV report, in SFX report format, on titles configured in your LOCKSS box, can be retrieved from:

http://lockss.box:8081/Titles?format=tsv&report=sfx&scope=configured

Using the report

There are also several common uses for the reports generated by the Title List feature:

Reporting what is available on the LOCKSS network. This can be used to populate a knowledge base in a link resolver such as SFX, or cross-referenced with an institution’s subscriptions to produce a list of what should be added to LOCKSS.
Reporting what titles an institution has configured their LOCKSS box to collect. This can be used in institutional reporting on LOCKSS, or cross-referenced with subscriptions and available titles to find out what is missing from your configuration.
Reporting what has been successfully collected by LOCKSS. This report can indicate where there might be problems, or be used in institutional reporting.

We plan to produce format options for other link resolver products as necessary, though the standard KBART report should be sufficient for many applications. After the current phase of user interface development, the reports should also be useful in configuring your collection list with less manual effort.

If you have any further use cases for these reports do let us know at edina@ed.ac.uk.

Report on Private LOCKSS Networks

Posted on June 25, 2012 by Adam Rusbridge

A report investigating community demand and requirements for setting up a UK Private LOCKSS Network is now available.

The report summarises a survey of members of the UK LOCKSS Alliance carried out by EDINA during October and November 2011. The survey investigated the potential value of Private LOCKSS Networks to UKLA members and focused on assessing the type of content members wish to preserve in a PLN, together with the cost and resource implications.

Agreed at the UKLA Members’ Meeting in May 2012, next steps are to prepare a short proposal identifying possible routes forward.

A Members’ Meeting of the UKLA was held in York on 10 May 2011 where attendees expressed an enthusiasm for further assessment of PLNs. The approach agreed was to conduct a survey of members to assess the level of interest for establishing a PLN, and to gather more detailed information on community requirements: how members could envisage the PLN being structured, the content they proposed using it for, and how they expected to benefit from the PLN. A full report of this event is available on the UKLA website. The PLN survey was carried out during October and November 2011 and focused on identifying content for preservation, costs and resources together with potential infrastructure models. The purpose of this report is to summarise the results of the survey and apply them to the various factors that need consideration when establishing a new PLN.

We would be grateful for UK HE community feedback on this report, so please either submit a comment to this post, or contact EDINA directly at edina@ed.ac.uk.

Introducing the UKLA Personas

Posted on May 31, 2012 by Adam Rusbridge

At last week’s Members’ Meeting we introduced a set of personas to describe the background, aims and roles of staff working with the LOCKSS system. Personas are “descriptions of typical users along with stories about how they would use the product to meet their goals.” The personas we have established will inform our user interface development work. By developing an understanding of staff roles we can make sure the planned developments satisfy those roles.

We’re now making available the set of draft personas for further input and comment.

These personas also have wider application for the UK LOCKSS Alliance community. They will be useful for prospective members planning their participation in the LOCKSS initiative and deciding how to assign LOCKSS-related work, and they will help us ensure we target support documentation at the appropriate level.

We wish to ensure that the characteristics of current institutional deployments of LOCKSS are reflected in the personas. If you feel we are missing tasks and goals that apply at your institution, we would be grateful if you could share these by commenting on this blog post or emailing edina@ed.ac.uk.

EDINA Blogs

A Blogs.edina.ac.uk weblog

Category Archives: development