| Description: | This is an appendix to the MetaArchive Final Report. This appendix document describes an automation process that involved the sending of a Folger Shakespeare Library collection (246 archival units measuring 1.3TB) hosted on a University of North Texas (UNT) LOCKSS cache to a storage resource hosted by Chronopolis at the San Diego Supercomputer Center (SDSC). In order to transfer the content to the iRODS-based Chronopolis data grid, each archival unit needed to be extracted from the LOCKSS cache, and sent via use of the iRODS icommands client utilities. In addition, it was imperative to monitor the transfer at all possible points to detect corruption and make certain that the integrity of the data was maintained. Finally, for the purpose of benchmarking and information gathering, time-based statistics needed to be recorded for each stage of the transfer. These processes were managed by a set of Python-based scripts and utilities that are described in this document. |
|---|---|
| Creator(s): | |
| Creation Date: | March 9, 2012 |
| Partner(s): |
UNT Libraries
|
| Collection(s): |
UNT Scholarly Works
|
| Usage: |
Total Uses: 3
Past 30 days: 0
Yesterday: 0
|
| Creator (Author): |
Halbert, Martin
Principal Investigator; University of North Texas |
|
|---|---|---|
| Creator (Author): |
Hartman, Cathy Nelson
Systems Director; University of North Texas |
|
| Creator (Author): |
Skinner, Katherine
Outreach Director; Educopia Institute |
|
| Creator (Author): |
Walters, Tyler
Sustainability Director; Georgia Institute of Technology |
|
| Creator (Author): |
Schultz, Matt
Project Manager; Educopia Institute |
|
| Date(s): |
|
|
| Description: | This is an appendix to the MetaArchive Final Report. This appendix document describes an automation process that involved the sending of a Folger Shakespeare Library collection (246 archival units measuring 1.3TB) hosted on a University of North Texas (UNT) LOCKSS cache to a storage resource hosted by Chronopolis at the San Diego Supercomputer Center (SDSC). In order to transfer the content to the iRODS-based Chronopolis data grid, each archival unit needed to be extracted from the LOCKSS cache, and sent via use of the iRODS icommands client utilities. In addition, it was imperative to monitor the transfer at all possible points to detect corruption and make certain that the integrity of the data was maintained. Finally, for the purpose of benchmarking and information gathering, time-based statistics needed to be recorded for each stage of the transfer. These processes were managed by a set of Python-based scripts and utilities that are described in this document. |
|
| Degree: |
Department:
Libraries
|
|
| Physical Description: |
4 p. |
|
| Language(s): | ||
| Subject(s): |
|
|
| Keyword(s): | LOCKSS | San Diego Supercomputer Center | SDSC | iRODS | Chronopolis | |
| Contributor(s): | ||
| Series Title: | MetaArchive Cooperative | |
| Partner: |
UNT Libraries
|
|
| Collection: |
UNT Scholarly Works
|
|
| Relation (Is Referenced By): | MetaArchive: Final Report, ark:/67531/metadc109710 | |
| Identifier: |
|
|
| Resource Type: | Text | |
| Format: | Text | |
| Rights: |
Access:
Public
|
|
