Date: March 9, 2012
Creator: Halbert, Martin; Hartman, Cathy Nelson; Skinner, Katherine; Walters, Tyler & Schultz, Matt
Description: This is an appendix to the MetaArchive Final Report. This appendix document describes an automation process that involved the sending of a Folger Shakespeare Library collection (246 archival units measuring 1.3TB) hosted on a University of North Texas (UNT) LOCKSS cache to a storage resource hosted by Chronopolis at the San Diego Supercomputer Center (SDSC). In order to transfer the content to the iRODS-based Chronopolis data grid, each archival unit needed to be extracted from the LOCKSS cache, and sent via use of the iRODS icommands client utilities. In addition, it was imperative to monitor the transfer at all possible points to detect corruption and make certain that the integrity of the data was maintained. Finally, for the purpose of benchmarking and information gathering, time-based statistics needed to be recorded for each stage of the transfer. These processes were managed by a set of Python-based scripts and utilities that are described in this document.
Contributing Partner: UNT Libraries