Appendix E: MetaArchive-iRODS Transfer

Description:

This is an appendix to the MetaArchive Final Report. This appendix document describes an automation process that involved the sending of a Folger Shakespeare Library collection (246 archival units measuring 1.3TB) hosted on a University of North Texas (UNT) LOCKSS cache to a storage resource hosted by Chronopolis at the San Diego Supercomputer Center (SDSC).

Creator(s):
Creation Date: March 9, 2012
Partner(s):
UNT Libraries
Collection(s):
UNT Scholarly Works
Usage:
Total Uses: 68
Past 30 days: 2
Yesterday: 0
Creator (Author):
Halbert, Martin

Principal Investigator; University of North Texas

Creator (Author):
Hartman, Cathy Nelson

Systems Director; University of North Texas

Creator (Author):
Skinner, Katherine

Outreach Director; Educopia Institute

Creator (Author):
Walters, Tyler

Sustainability Director; Georgia Institute of Technology

Creator (Author):
Schultz, Matt

Project Manager; Educopia Institute

Date(s):
  • Creation: March 9, 2012
Description:

This is an appendix to the MetaArchive Final Report. This appendix document describes an automation process that involved the sending of a Folger Shakespeare Library collection (246 archival units measuring 1.3TB) hosted on a University of North Texas (UNT) LOCKSS cache to a storage resource hosted by Chronopolis at the San Diego Supercomputer Center (SDSC).

Degree:
Department: Libraries
Note:

Abstract: This appendix document describes an automation process that involved the sending of a Folger Shakespeare Library collection (246 archival units measuring 1.3TB) hosted on a University of North Texas (UNT) LOCKSS cache to a storage resource hosted by Chronopolis at the San Diego Supercomputer Center (SDSC). In order to transfer the content to the iRODS-based Chronopolis data grid, each archival unit needed to be extracted from the LOCKSS cache, and sent via use of the iRODS icommands client utilities. In addition, it was imperative to monitor the transfer at all possible points to detect corruption and make certain that the integrity of the data was maintained. Finally, for the purpose of benchmarking and information gathering, time-based statistics needed to be recorded for each stage of the transfer. These processes were managed by a set of Python-based scripts and utilities that are described in this document.

Physical Description:

4 p.

Language(s):
Subject(s):
Keyword(s): LOCKSS | San Diego Supercomputer Center | SDSC | iRODS | Chronopolis
Contributor(s):
Series Title: MetaArchive Cooperative
Partner:
UNT Libraries
Collection:
UNT Scholarly Works
Relation (Is Referenced By): MetaArchive: Final Report, ark:/67531/metadc109710
Identifier:
  • ARK: ark:/67531/metadc109716
Resource Type: Text
Format: Text
Rights:
Access: Public