Digitisation Workflows and Tools

During the final stages of the County Surveys project, we are shifting our attention to the process of digitising some volumes.  From the outset, a key aim of the project has been to scope the resources required to bring a full set of the county surveys together in a convenient digital format. The creation of the online bibliographic search tool was the first step towards this long term aim, as it allows us to assess the potential of printed books for digitisation and the quality and access conditions of extant digitised copies. A second step was to carry out trial digitisations of a few key surveys. As we have noted elsewhere,  this is a pilot, through which we will explore the potential requirements of a fuller, high quality, full text online collection.

Using the information surfaced through the bibliography, we identified a number of candidate surveys and discovered their locations. We found that one of the rarest volumes was held right here in Edinburgh, in the collection of the Royal Botanic Gardens Edinburgh, who kindly allowed us to work with their copy and use their state of the art digitisation equipment. The work was carried out by Phil Mellor, currently a PhD student at the University of Strathclyde, who did a great job of thoughtfully documenting the requirements and exploring different options for carrying out each step of the process.

The workflow that Phil sketched out  involved four stages: locate, capture, edit, OCR.

The first step was to locate the material: this involves finding an accessible volume (which may be easier said than done in some cases), assessing its binding, checking for folded plates and other potential issues such as uncut pages. We chose two volumes: the General View of the Agriculture of the Shetland Islands (1814) by John Shirreff, which is a revised survey and has 228 pages; and the General View of the Agriculture in the Southern Districts of the County of Perth (1794) by James Robertson, a first series survey with 140 pages, bound with other volumes and featuring the long ‘s’ (historically used where modern English uses a double s).

The capture stage, which we were expecting to be the most time-consuming, turned out to be fairly quick. Using the RBGE equipment, two pages at a time can be photographed and once the initial set-up is done, the cradle and cameras remain in place: Phil was able to capture an entire volume in under an hour.   During the capture stage, we had the choice of creating RAW files or JPEG.  There is an advantage to using RAW, as having a loss-less format from the outset enables editing from a high quality original at any point in the future. However, we also found that the quality of JPEG produced was high enough for good OCR and the JPEG files were easier to edit and quicker to upload and transfer.

The next stage of the workflow was editing. After saving the page images, they were uploaded into the editing software provided with the the capture software and the skew of the pages amended, cropping where necessary. Skewing is an inevitable result of capturing a bound book, in which the pages will always be presented at an angle that increases and decreases depending on the page at which the book is opened. This can be addressed during the capture process by changing the camera angles, or more quickly during the editing phase. Editing can be done on a page-by-page basis or on a chapter-by-chapter basis, and we found the latter to be sufficient for our needs. This stage could be quite time-consuming, depending on the quality of image required and the quality of images captured, but Phil found that editing a whole volume on a chapter by chapter basis took around two hours.

The final stage of the process was OCR-ing the images to produce text files. We tried a couple of different software packages, including the open-source programme Tesseract and a proprietary software called ABBYY Fine reader. Tesseract performed well and we were able to produce searchable pdfs and text files quickly and easily. However, it struggled with the historical print and the many unique Scottish names found in the surveys. ABBYY handled these comparatively well, and also automatically formatted the text to mimic the page image which saved a considerable amount of editing work. Overall, then, the digitisation process was quicker and smoother than we had envisaged, which bodes well for future projects to complete the collection.

The above gave us a workflow which we would use as a template for any future digitisation. However, for comparison’s sake we digitised two further volumes with equipment kindly made available to us by Edinburgh University Library Centre for Research Collections. Using this, we digitised two volumes: a different copy of the 1814 John Shirreff survey of The Shetland Islands and the survey of Ayrshire from 1811 by William Aiton. It might, at first seem strange to do the Shetland survey again but this allows us to do a direct comparison both in term of process and quality of result. The conclusion, in terms of the process was that the equipment at RBGE made the process considerably quicker and easier. Specifically, the most important characteristics of the scanner were:

  • The ability to capture two pages at once reduces the time considerably.
  • A cradle which does not require the book to be flat is both quicker and probably safer in terms of handling fragile and older books.
  • An integrated editing software which allows for corrections of cropping and skewness to be applied at the same time makes the workflow smoother.

The comparison not only gave us some more material which we can make available online but also confirmed that the process we outline above is a good way to proceed.

A final note on the Ayrshire survey: when this book was retrieved from the archives, the pages were still uncut. The library staff were able to assist in opening the book up but there is something which gives pause for thought: this was the first time this book had been read, and it was being read by an Optical Scanner for online processing, a ‘reader’ unimaginable to those people whose effort went into producing it.

The digitised volumes can be found within our service and below:

General View of the Agriculture of the County of Ayr, by William Aiton , 1811

General View of the Agriculture  in the Southern Districts of the Country of Perth, by James Robertson, 1794

General View of the Agriculture of the Shetland Islands, by John Shirreff, 1814

 

Posted in Uncategorized

County Surveys Project in the News

We’re delighted to report that the County Surveys of Great Britain project has been attracting some media attention over the last few months.

magazineIn May, professional family historian and author Chris Paton wrote about the project in a post on the British Genes blog, a wonderful and widely-read resource for those interested in genealogy and local history.

In August, there was a brief write up in the genealogy magazine, Your Family Tree, who picked up on the fact that the surveys could be very valuable in providing context to those researching their ancestors.

summer 14 botanics cover list sizeThe project will also feature in the autumn edition of Botanics, the magazine produced by the Royal Botanical Gardens Edinburgh, which will soon be available for download from their website.

Posted in Uncategorized

Statistical Accounts at the Fringe: Back to the Statistical Future

As always, August in Edinburgh is abuzz with lots of exciting theatre and shows. We were delighted to have the opportunity to present our own show again this year, once more as part of the Beltane network’s Cabaret of Dangerous Ideas. Written and presented by EDINA’s Nicola Osborne and Helen Aiton, a member of the Statistical Accounts editorial board, ‘Back to the Statistical Future’ explored parallels between the ‘New’ Statistical Accounts of Scotland (1834-1845) and our contemporary cultural and political contexts. The wonderful comedian Susan Morrison was master of ceremonies and ‘minister of the parish’ as the discussion ranging over topics such as education, social deprivation and welfare and libraries.

2015-08-26 15.40.21

Helen Aiton, Nicola Osborne and Susan Morrison on stage.

With the aid of a time machine and the fantastically-imagined ‘hover-board of social policy’ (a reference for the film buffs!) we posed the question of how different Scotland in 2015 is to Scotland in 1835. Might we be returning to a time, we asked, when libraries are only sustained by subscriptions? Is it possible that, as some of our ancestors believed, the poor are being ‘corrupted, by being taught to read and write’? As good education becomes increasingly costly and inaccessible, are our modern ‘lords and gentlemen’ motivated once more by the belief that the masses would be ‘more obedient and dutiful, were [we] more ignorant, and had no education’?

Pondering such subversive suggestions, the audience came up with some rather brilliant proposals including introducing dancing sessions to libraries, building more sustainable energy-driven social housing, allowing ordinary people to sit in parliament, and even taxing celebrities based on the column inches they generate.

Many thanks to all who made it possible, and to those who came and contributed their own dangerous ideas!

Share

Final Programme Now Available for Taking the Long View: International Perspectives on E-Journal Archiving

We’re delighted to announce that the final programme is now available for

Taking the Long View: International Perspectives on E-Journal Archiving

A One Day International Conference at the University of Edinburgh, hosted by EDINA and the ISSN IC

7th September 2015

9:00 – 17:00

Speakers Include:

  • Clifford Lynch (CNI)
  • John MacColl (RLUK)
  • Victoria Reich (The LOCKSS Program)
  • Kate Wittenberg (Portico)
  • Randy Kiefer (CLOCKSS Archive)
  • Andrew MacEwan (British Library)
  • Mike Furlough (HathiTrust)
  • Steve Marks (University of Toronto/Scholars Portal)
  • Barbara Sierman (KB, Netherlands)
  • Ted Westervelt (Library of Congress)
  • Zhenxin Wu (National Science Library of China)
  • Vincent Wintermans (UNESCO)

A roundtable discussion ‘Looking Forward to Looking Back: New Horizons for the Scholarly Record’ will be chaired by William Kilbride (Digital Preservation Coalition).

More details on the programme and how to register are available on our website. 

Share

Posted in Uncategorized

More of Sinclair’s Questions Discovered

The first blog about the Statistical Accounts of Scotland service described how in May 1790, Sir John Sinclair wrote to every Church of Scotland Minister in each of the 938 parishes in Scotland with a list of 160 questions plus an addendum of 6 further questions. Sir John intended to use the responses to his very thorough range of questions to elucidate the Natural History and Political State of Scotland or “the quantum of happiness� of its people.

Whilst researching the origins of the first Statistical Accounts for a performance at the Cabaret of Dangerous Ideas on the Edinburgh Fringe, myself and Nicola Osborne from EDINA discovered an additional five questions in the next very persuasive circular letter from Sir John Sinclair, which the Ministers received in January 1791, 8 months after the initial request.

SIR,

   IT is with infinite pleasure I have the honour of acquainting you, that by the zeal and patriotism of the clergy of Scotland, I have already in my possession materials for drawing up a Statistical Account of a considerable part of the whole kingdom…

..But I am anxious that the Clergy of Scotland should not only do it well, but quickly; so that the state of the whole country should be known, if possible, at nearly the same period of time.  I therefore hope, Sir, that, for the honour of our national church, you will make every exertion in your power to send me as full, and as accurate an account, as possible of your  parish…

…In the queries formerly sent, some particulars were omitted, of which I should be glad to be informed, even from those gentlemen who have already favoured me with their answers: as,   

1. What is the state of the schools in the parish; the salary and perquisites of the schoolmaster; and the number of his scholars?  

2. What is the number of alehouses, inns, &c.; and what effect have they on the morals of the people?  

3. What is the number of new houses or cottages which have been built within those ten years past; and how many old ones have been pulled down, or have become uninhabitable?  

4. What has been the effect of employing cottagers in agriculture, or of working by hired servants in their stead? and,  

5. What has been the number of prisoners in any jail in the district, in the course of the year 1790; and for what causes were they imprisoned?   

Tables of births, marriages, and deaths, kept in any particular parish would be very desirable.  Nor can the information respecting all points connected with the population of the country, be too accurate and minute.

 

We are not aware of what the “patriotic and zealous� Parish ministers thought when they received this request for yet more information, especially those who had already responded very promptly! It would be a further eight years before all of the twenty-one Volumes of the First Statistical Accounts of Scotland were published.

These questions, along with images and transcripts of the first 166 questions can be found within the Related Resources Section of the Statistical Account Service.

– Helen Aiton June 2015

Helen Aiton and Nicola Osborne present ‘Back to the Statistical Future‘ as part of  The Cabaret of Dangerous Ideas at the Stand Comedy, Edinburgh. 

How different is Scotland in 2015, to Scotland in 1835?

As good education is increasingly costly and inaccessible to the poor, are we seeing our modern ‘lords and gentlemen’ believing we will be ‘more obedient and dutiful, were [we] more ignorant, and had no education’?

Might our poor potentially be ‘corrupted, by being taught to read and write’? Might we be returning to a time when libraries are only sustained by subscriptions?

Join us for a whistle stop hover-board ride through the bizarre parallels between modern Scotland and the ‘New’ Statistical Accounts of Scotland (1834-1845).

@CODIfringe

 

We hope you have enjoyed this post: it is characteristic of the rich historical material available within the ‘Related Resources’ section of the Statistical Accounts of Scotland service. Featuring essays, maps, illustrations, correspondence, biographies of compliers, and information about Sir John Sinclair’s other works, the service provides extensive historical and bibliographical detail to supplement our full-text searchable collection of the ‘Old’ and ‘New’ Statistical Accounts.

 

 

Share

Keepers Registry featured on The Signal

We’re delighted to be featured on The Signal, the digital preservation blog published by the Library of Congress, this month.

Mapping the Digital Galaxy: The Keepers Registry Expands its Tool Kit‘ is an interview with the team conducted by Ted Westervelt, manager of the eDeposit Program for Library Services at the Library of Congress, who will be speaking at Taking the Long View: International Perspectives on E-Journal Archiving.

The resulting conversation ranged from the history of the Keepers Registry to our new release and current plans for engaging more archiving agencies and libraries: below we repost a few extracts.

 

Ted: Most of the participating agencies are Western European or North American, which makes sense given the origins of the Keepers Registry. How actively are you looking at adding members from other parts of the world?

Keepers team: We have become well-traveled in our quest! The initial focus was on the UK, Europe and the USA as it was much easier for us to encourage participation from agencies in Europe and the US as we had existing contact and relationships with many of the original agencies. However, we are very conscious of need for more international participation as mentioned earlier. There is now engagement with China and Canada and with active outreach to India and Brazil, as well as more countries across Europe.

 

Ted: You are just now releasing a new version of the Keepers Registry, with some interesting new functionalities. One of these is the Title List Comparison, which you mentioned above. Who do you hope will use this and what do you hope this will do for them and for the mission of the Keepers Registry in general?

Keepers team: We anticipate that the Title List Comparison facility will prove very popular. […It] should allow libraries to have insight into the archival status of collections in order to assist informed decision making about subscriptions, cancellations and print rationalization. We hope that the tool will also improve communication between the library community and the Keeper organizations themselves, as libraries make known their priorities for the serial titles that they discover are not being kept safe. The Title List Comparison service is part of our Members Services; access to these requires membership, which is free of charge.

 

Ted: Another major functionality in the new release is the Machine to Machine Interfaces. Who do you expect will use this? What outcome would you like to see from launching this?

Keepers team: Librarians interact on a daily basis with a wide range of services and tools for serials. We want the information on archiving that we bring together in the Keepers Registry to be available and useful at the point of need – when there is need for a quick reference to make a measured decision. Those machine-to-machine interfaces are there to support linking tools from those other services, such as union catalogs, and even OPACs, as well as vendor platforms. In general, those ‘APIs’ are there so that others can do unimaginable things with our data – so please get in touch!

Read the original post.

Share

Posted in Uncategorized

Who Wrote the County Surveys?

Some of the significance and much of the character of Sir John Sinclair’s ‘great pyramid’ comes from the many authors involved in reporting and writing up the surveys.  In the case of the  Statistical Accounts of Scotland, Sinclair drafted in local ministers to describe their parishes. Knowing their parishioners intimately, these men of the cloth were able to answer detailed questions about the place and the people, and frequently gave their individual opinions and perspectives on local tales, customs and morals.

The authors of the County Surveys, in contrast, were not of one profession or social position. The surveys were commissioned  from a wide range of  ‘intelligent gentlemen’, including university professors, farmers, landowners, clerics, professional writers, and political activists. Moreover,  it was planned that “every farmer and gentleman in the district” would have the opportunity to read and remark on the first series, which would be revised to incorporate all their insights before final publications in the second series. It was, in other words, to be a collective undertaking by many hands, designed to provide the board with “a greater variety of information and a greater mass of instructive observations from a greater number of intelligent men for their consideration and guidance.”* The incentive for such men to give up their time and energy was not financial, indeed several of the surveyors worked for free and most claimed only their expenses. Rather, they worked in the name of the public good and in the belief that their undertaking would be of significant value to their nation and its people.

220px-Arthur_Young_(1741-1820)

Arthur Young, 1741-1820

While the stories of many of these contributors are lost to history, a few  were historically notable individuals. The Reverend Dr. Walker, for example, who surveyed the Hebrides was Professor of Natural History at the University of Edinburgh. A distinguished scientist with interests in botany, mineralogy and geology, and a pioneer in the study and teaching of agriculture, he had conducted exploratory tours of the Western Isles on behalf of the Board of Annexed Estates in the 1760s and 70s: a more suitable candidate for surveying these counties for the Board of Agriculture would be hard to imagine.  Where Walker was a pillar of the establishment, Charles Vancouver was a more colourful figure. Like his older brother the explorer George Vancouver (who famously charted the Pacific Coast of North America in the  early 1790s, and after whom the Canadian city Vancouver is named), Charles was a traveller and frontiersman in the American colonies. Of Dutch origin, and originally a farmer, he had spent decades working the land and writing about ‘natural philosophy’ in newly-settled Kentucky, before returning to the UK in the early 1790s. He would later work in the Netherlands, before returning to the Americas, using his ‘practical expertise’ in cultivation and farming to support himself.  Vancouver’s friend and secretary to the Board, Arthur Young, was also an author and completed the survey for Suffolk. Young began his career in a mercantile house, but was more interested in travel, literature and politics than commerce. The author of four novels, pamphlets, magazines, and a number of travelogues, he was also interested in experimental agriculture and in the rights of agricultural workers. Although his experiments did not produce revolutionary results, as an astute as a social and political observer “he remains the greatest of English writers on agriculture.” (Higgs, Dictionary of National Biography, 1885-1900, Vol. 63 p.362 )

In the combined wisdom of such fascinating, experienced and erudite writers, supported by the numerous contributors whose names are lost to posterity, the county surveys offer us insights not just into the agriculture of the time but also into the intellectual milieu and social conditions of Romantic Britain.

*all quotations in this paragraph are from Appendix G of Sinclair’s  1797 Communications to the Board of Agriculture, on Subjects Relative to the Husbandry and Internal Improvement of the Country, Volume 1. p. xlviii-xlix.

 

Taking the Long View: International Perspectives on E-Journal Archiving

We are delighted to announce that registration is now open for  ‘Taking the Long View: International Perspectives on E-Journal Archiving’, an international conference hosted by EDINA and the ISSN IC, as part of the Keepers Extra project.

September 7th 2015, University of Edinburgh

An international conference organised as part of the Jisc-supported Keepers Extra project, ‘Taking the Long View’ brings together international archiving agencies, representatives of national libraries from around the world, and research libraries and consortia to exchange knowledge, share ideas and discuss requirements for potential global collaboration to increase preservation coverage and tackle the ‘long tail’.

The importance of assuring continuing access to e-journal content has long been recognised. Many institutions now have e-first collection policies that require archiving of serial content before e-only or print disposal actions can be taken.  Nations are introducing legal deposit systems for electronic material.  Yet analysis undertaken as part of the Keepers Registry has shown that over 80% of continuing resources assigned an ISSN have yet to be archived. The need for ‘conscious coordination’ of international activity is clear (Lavoie and Malpas, 2015).  It is imperative that we now take the long view and consider if and how we can work together to address the challenge of stewarding the digital scholarly record.

Marking the 40th anniversary of the ISSN International Centre and the 20th anniversary of EDINA, this event presents a networking and briefing opportunity in which librarians can learn about a wide range of international activities, experiences and perspectives on the archiving and preservation of serial content, and gain insight into the operations and ambitions of some of the most important archiving agencies and initiatives from around the world.

More details and instructions on how to register are available on the conference website.

For more on the Keepers Extra project and the Keepers Registry, visit our blog.

issn icedina-logo

jisc-logo

 

Share

Posted in Uncategorized

Working with the Royal Botanic Gardens Edinburgh

Palm_House,_Royal_Botanic_Garden_EdinburghOver the last few weeks we have been working in partnership with the Royal Botanic Gardens Edinburgh, who hold an excellent collection of County Surveys as part of their impressive collections. The RBGE is currently in the process of having their rare books comprehensively catalogued by the Rare Book Cataloguer from the Centre for Research Collections (CRC) at the University of Edinburgh, and we are pleased to be able to contribute to this process by assisting in the cataloguing of the County Survey holdings. Once they are complete, we hope that these new electronic records will from the basis of another data set for our online demonstrator.

The RBGE also has state of the art equipment and digitisation specialists in house: although they are currently involved in an extensive project to digitise specimens from the internationally renowned herbarium, staff have generously shared their knowledge and allowed us to use their equipment to digitise a few of the surveys. We are pleased to report this work is going very well and we should be able to make the digitised copies available soon, so watch this space.

County Surveys Project Launches Online Bibliographic Tool

We are pleased to report that the County Surveys of Great Britain 1793 – 1817 project, which is related to the Statistical Accounts, has now released an online bibliographic search tool. This is a key output of this pilot project and will be of wide interest to historians and researchers in many other fields.

Here we re-post of the County Surveys blog announcement:

We are delighted to announce that our bibliographic search tool is now live and accessible from the ‘Search’ tab in the menu above.

Our demonstrator includes bibliographic data from some of the best collections of the surveys and, where possible, provides links to library catalogue entries and  digital editions. Researchers can search by modern county name, by series, by county and by author. Results are presented in a new tab after each search, so that you can compare multiple search results by toggling between pages. There are also detailed analyses of collections, revealing the extent of holdings and coverage, and indicating which surveys would be needed to complete each collection.

demonstrator2

 

We hope that the demonstrator will be a useful finding aid and discovery tool for those interested in the County Surveys, the history of statistical reporting and British history more broadly. We would welcome any feedback on the tool, and would be very keen to hear about how it is used or whether it could usefully offer other features and information. If you have ideas, please get in touch with us at edina@ed.ac.uk.

Share