Post by Richard Wincewicz, Software Engineer for SafeNet at EDINA.

In the previous post we saw an overview of the whole SafeNet project. In this post I wanted to dig a little deeper into the technical side of the project.

Key Components

We are currently developing the SafeNet Service Interface component by extending the LOCKSS software (http://www.lockss.org/), a software platform which allows libraries to store and provide access to locally managed copies of electronic content such as e-journals. The LOCKSS software was originally designed to work from within an institution’s network and provide access only to users that are part of that network. A key component of the SafeNet service is to introduce a centrally-managed Private LOCKSS Network that can be used by UK HE institutions to provide assurances of continuing access to their subscribed content, without having to run a server locally. The SafeNet model will allow institutions to participate in a shared service offering but there are a number of challenges that need to be addressed for this to work at scale.

The first challenge is determining who can have access to what content. In standard LOCKSS deployments, access is restricted by IP ranges (e.g. the university network) and so all users can access the same content. With a centrally-managed service this is no longer the case and we need a mechanism to ensure that a user is entitled to access content that they request.  For this purpose, we are designing and deploying an Entitlement Registry which holds information about the subscriptions that institutions have for specific journals. The Entitlement Registry provides a REST API that allows a user or application to query its database. Some of this data may be made openly available, such as lists of publishers and titles, and some of it will be restricted, such as the journals that an institution is subscribed to. We are extending the LOCKSS software to include a query to the Entitlement Registry whenever a user requests some journal content. As a pre-requisite to this, a user will be required to identify themselves by logging in and providing the LOCKSS software with identifying information about their institution. Using this information, we can then determine whether a user is entitled to access the requested content.

The Entitlement Registry has broader potential value as a reference tool used by both libraries and publishers.  To this end, we are also designing a user interface on top of the Entitlement Registry to allows users to interact with the entitlement data. Users will be able to view general information about titles and publishers as well as entitlement information specific to their institution. In addition to this, we are assessing use cases about external access to the Entitlement Registry API, so that other applications can make use of the data without having to collect and host it themselves.

Deployment Infrastructure

If a service is going to be successful it first needs to be reliable and responsive. These are two aspects of a service but can be solved with similar approaches. Having redundant copies of a service in different locations allows for one site to fail while still allowing users access the service from the second site. This also helps when dealing with heavy traffic because there are now two servers able to handle requests. This approach works well up to a point, but if different parts of a service all start to require large amounts of resource then creating more copies of the service doesn’t help.

At this stage the architecture of the service becomes important. If the service consists of a single application then the only way to deal with increased load is to run the application on a more powerful server. If the service is comprised of many small applications that communicate with each other then copies of these components can be created independently of each other. This leads to a much greater flexibility and allows the service to handle hardware and software failures as well as heavy traffic.

With the different components we wanted to make sure that each could run efficiently, scale well and be updated without disruption to the service. In order to do this we have created each component separately and given them their own environment to run in. Using Docker (https://www.docker.com/) each component runs in a container that isolates it from the rest of the processes that are running on the server. This means that we can have a lot of different components running in the same place without worrying about how they will affect each other.

Use of Docker also gives us a portable object meaning we can create as many identical versions of the component as needed to provide resilience or to deal with load. These portable objects can be started and stopped very quickly allowing us to deal with failed components or manage updates without affecting the running service.

For this model to be successful we had to put some thought into the design of the components so that they work under these conditions. In particular all of the information that the application uses is stored in an external database. In fact, a minimum amount of data is stored with the component to allow for it to be shutdown or restarted without having to worry about what will happen to the data. At EDINA we are lucky enough to have access to two datacentres meaning each of our services is spread across two sites. A load balancer deals with each request and passes it on to an available server in one of the datacentres. If one of the servers is down then all requests are passed to the other available servers ensuring that the service remains accessible.

Now that we have the basic structure of the service set up it is important that we continue to develop the service in a way that maintains the reliability and resilience. Docker makes it easier to rapidly deploy multiple copies of an application in different locations but it brings its own complexities. The goal now is to use Docker to make our lives easier rather than more complicated.