Library Impact Data Project Webinar from Dave Pattern – LiveBlog

I am attending a webinar from Dave Pattern, Library Systems Manager at University of Huddersfield today.  He will be talking about their Library Impact Data Project and I will be taking notes throughout. The usual caveats apply that this is a liveblog and potentially a bit for of typos etc. The session has been organised by Scotland’s Colleges and you can find out more about this webinar and other events on their Webinars page.

The hashtag for this project is #lidp and the whole team is available via Twitter. We have been using the data we collect for a number of years. We are often guilty of making a lot of assumptions about how students use our resources. We have also been trying to measure the impact the library has on student attainment and experience, particularly in the current funding era.

Defining Usage Data – this might be circulation transactions, e-resources usage and also building entry stats indicating use of physical facilities. JISC recently completed an activity data programme and they broke it into user activity data and attention data.

About 6 years ago we started to look at our own library data, with a view to thinking about those students that do not tend to use the library, to look for trends, see what has changed… things like seeing a huge increase in use of e-resources. Not a huge declince in print usage despite this. For the librarians we have been pulling out quite detailed data on usage. We use a system called MetaLib for eresources. In 2005/6 you see very little use of eresources but each year it increases. We also see stats for our Horizon system – our catalogue – and for Sentry, the system for access. It’s useful to look for patterns and trends here.

Screenshot of Dave Pattern's Webinar

One of the other things we wanted to do was identify students who would want library training etc. When we talked to student reps they asked if there was a link between how students do and how much they use the library. When we looked at the stats we didn’t see any significant difference between use of the library and grades, we’d just had a refurb so sort of hoped for a pattern but just reenforced that the library is for all.

BUT when we looked at borrowing levels there were double the number of loans for those who got a first and those who got a third. e-Resources seemed to have a linear relationship. Definitely something there. Looking in more detail we could see that some students who never used the library got a 1st! Many logins to the eresources system was strongly correlated to grades of a first (borrowing more than 180+ books over 3 years). This is interesting stuff…

Complex graph of usage and grade

As we started to show these around we had questions about whether this is just at Huddersfield or the same elsewhere. And we had lots of questions about whether what was happening was statistically significant. So we applied to the JISC Activity Data Programme. Our project aimed to prove there is a statistically significant correlation between library use and attainment amongst other things.When the project kicked off we set up a hypothosis to test – that more book use results in better grades in a statistically significant ways. We had multiple prokect partners who each were asked for appropriate data (most could provide at least two of the three types of data requested).A few challenges here. Concerns on the UK Data Protection Act 1998 (University Student Handbooks), Data anonymisation challenges, data extraction (one security system regarded this is as their own data), finally we did manage to release the data though ??

Broadly speaking it transpired that Huddersfiels’d usage data is broadly representative of UK HE Libraries. But there was an issue here… continuous data made it hard to indicate a true correllation. Some libraries could include renewals as well as original loans, but not all.

So if we look at our own data from Huddersfield… it appears that students who get a first are using an increasing number of books and resources. One of my favourite graphs shows that students who get a first are already doing so in their first year, they seem ahead already. Are those students also used to using public libraries? Do they have better study skills? Really interesting to more in this area.

Slide showing stats on first year usage of resources

Looking at non and low usage of materials… Seems to be a real clustering around non and low usage… and indicates a real need to do more to get students aware of the library.

Non and low use of resources vs grade


Phase 2 of this work is now underway and we are looking at Huddersfield data specifically and using final %age marks and looking in much more depth to try and show significance and causation. We are also looking at UCAS entry points and if improvement between entry points and degree result includes any sorts of patterns. Is non or low usage a warning of drop out or incompletion numbers?

 

Looking at EZProxy stats. Here we seem to have a strong correlation in the average usage and final %age grade. Same for item loans. Looking at the time of day accesses we see most heavy use in the core daytime hours. In the very late night we see those with a third using more eresources than their peers. It’s not a significant effect but notable. And we see early morning as the time when those likely to get firsts are using eresources and physical library resources.

Usage and grade by hour of the day

Looking at nationality we see that the UK born users use books more often whilst EU students seem to have much higher use of resources and really higher use of PDFs and eResources, particularly those from New Europe.

Usage by nationality

Students who drop out appear to be 10 times more likely to not use eresources. This may be an indicator of those that are struggling with courses. Want to look at stats like VLE usage etc. to see what else we can learn here.

It seems that prior library usage before university is pretty important. There is a statistically significant correlation between HE library activity and final degree outcome. And evidance elsewhere seems to confirm this. See ACRL Value of Academic Libraries work for instance – in the US they don’t tend to collect usage data because of issues such as the Patriot Act.

So, what next?

More data to be released. Keep an eye on the blog: http://library.hud.ac.uk/lidp. And keep in touch via Twitter (#lidp).

Q&A

Q1 – Nicola) Is that the stat on use of non e-resources also suggesting something about physical stock and relevance not just usefulness of eresources?

A1) Yes, it probably is… we know students do borrow more books and use good resources… but what value they get out I’m not sure. We need to do focus groups… it may be that if you pick good resources you may borrow fewer books but are more focused on what you need to download etc.

Q2 – Lynn) Dave, assuming all your students joined in 1st year? Any research as to library use/attainment for those joing eg 2nd year via college articulation?

A2) We haven’t considered that but it’s a really good question for us to think about.

Q3) Are many students distance learners at your uni? Does this influence eresource usage?

A3) We do have data at partner colleges and abroad – the eresources data would be really interesting to look at and see if there are big differences.

Going back to that first year data… there is a sense that it’s almost too late to teach students information literacy at that stage, they are already set in their ways and studying practices

Q4 – Nicola) Have you had any feedback from students about this data?

A4) We have made findings available to academics. We have had them trying to scare their students a bit but we have been careful to say that it’s early data though. Will be interesting to

Q5 – Lynn) With library loans, have you looked at number of different titles (ie breadth of usage) as opposed to simply number of loans?

A5) Potentially we could use that data. For a number of years we’ve had recommendations in our catalogue… perhaps we should do a “students who got a first and borrowed this also borrowed that”. And

Q6 – Nicola) Do you have any way to compare the usage in the library with usage of non library stuff e.g. Google Scholar/repositories elsewhere

A6) A lot of the eresources data is via EZProxy. We moved to Summon rather than MetaLib. We push both on-campus and off-campus use of that. We obviously can’t capture use of Google Scholar etc. so I think that will need to be something for focus groups I think. Would be interesting to look at 3rd year students, yet to graduate but likely to get a first, and see what their habits are, how they are using things like Google Scholar etc.

Dave will be at CILIPS Scotland Conference in a few weeks so do say hello if you are too!

 

 

DeliciousShare/Bookmark