30 Matching Results

Search Results

Advanced search parameters have been applied.
in subjects: "web crawling"  
Results: 1 - 24 next

Building a Sustainable Quality Assurance Lifecycle at the Library of Congress

Description: Presentation for the IIPC General Assembly and Web Archiving Conference virtually held on May 23-25, 2022. This presentation shares details of the Library of Congress Web Archiving Team's (WAT) comprehensive quality assurance lifecycle, specifically daily workflows and ones under development.
Date: May 24, 2022
Creator: Thomas, Grace & Lyon, Meghan
Item Type: Refine your search to only Presentation
Partner: International Internet Preservation Consortium

Comparing Access Patterns of Robots and Humans in Web Archives

Description: Presentation for the IIPC General Assembly and Web Archiving Conference virtually held on May 23-25, 2022. This presentation is an extension of a previous study and explores the access patterns of humans and robots in the Internet Archive with the goal of determining which accesses are from humans and which are from bots.
Date: May 24, 2022
Creator: Jayanetti, Himarsha R.; Garg, Kritika; Alam, Sawood; Nelson, Michael L. & Weigle, Michele C.
Item Type: Refine your search to only Presentation
Partner: International Internet Preservation Consortium

Web Archiving as Entertainment

Description: Presentation for the IIPC General Assembly and Web Archiving Conference virtually held on May 23-25, 2022. This presentation explores applying gaming concepts to the web archiving process and integration of video games with web archiving.
Date: May 25, 2022
Creator: Reid, Travis; Nelson, Michael L. & Weigle, Michele C.
Item Type: Refine your search to only Presentation
Partner: International Internet Preservation Consortium

Improving the quality of web harvests using Web Curator Tool

Description: Presentation for the IIPC General Assembly and Web Archiving Conference virtually held on June 14-16, 2021. This presentation explores the collaborative improvement of the Web Curator Tool (WCT) in quality management of web archiving.
Date: June 15, 2021
Creator: Hoeven, Jeffrey van der; O'Brien, Ben; Koppelaar, Hanna; Rohrbach, Trienka; Goethals, Andrea & Knight, Steve
Item Type: Refine your search to only Presentation
Partner: International Internet Preservation Consortium

Policies and processes for ingesting WebRecorder WARCS into the UK Web Archive

Description: Presentation for the IIPC General Assembly and Web Archiving Conference virtually held on June 14-16, 2021. This presentation will discuss the policies and procedures for ingest and access that have been developed in order for the British Library to accept WARCs crawled outside the normal workflow into the UK Web Archive collection.
Date: June 16, 2021
Creator: Bingham, Nicola
Item Type: Refine your search to only Presentation
Partner: International Internet Preservation Consortium

Twittervane

Description: Presentation for the 2012 International Internet Preservation Consortium General Assembly. Discusses the status, limitations, and future goals of the Twittervane project.
Date: May 1, 2012
Creator: Hockx-Yu, Helen; Johnson, Stephen & Pennock, Maureen E.
Item Type: Refine your search to only Presentation
Partner: International Internet Preservation Consortium

Leveraging Web Archives Research

Description: Presentation for the 2012 International Internet Preservation Consortium General Assembly. Presentation describes the status and goals of the Lawa (longitudinal analytics of web archive data) project, a web crawling and analytics project by Internet Memory.
Date: May 1, 2012
Creator: Medjkoune, Leïla
Item Type: Refine your search to only Presentation
Partner: International Internet Preservation Consortium

Memento Aggregator Update

Description: Presentation for the 2013 International Internet Preservation Consortium General Assembly. Discusses the current release and development status of the Memento Aggregator, along with future goals and challenges.
Date: April 23, 2013
Creator: Van de Sompel, Herbert; Nelson, Michael L. & Rosenthal, David S. H.
Item Type: Refine your search to only Presentation
Partner: International Internet Preservation Consortium

Web Archiving in 2012 at National Diet Library

Description: Presentation for the 2012 International Internet Preservation Consortium General Assembly. This presentation details ongoing efforts of and current challenges faced by the National Diet Library in Japan, including the archiving of websites pertaining to the 2011 Tohoku Earthquake and Tsunami and the development of new preservation tools.
Date: May 1, 2012
Creator: Shibata, Masaki
Item Type: Refine your search to only Presentation
Partner: International Internet Preservation Consortium

Havel Collection Update

Description: Presentation for the 2012 International Internet Preservation Consortium General Assembly. Discuss the process of and lessons learned in created the topical collection covering the death of Vaclav Havel at the National LIbrary of the Czech Republic.
Date: May 1, 2012
Creator: Coufal, Libor
Item Type: Refine your search to only Presentation
Partner: International Internet Preservation Consortium

Los Alamos National Laboratory

Description: Presentation for the 2012 International Internet Preservation Consortium General Assembly. Discusses the web archiving-related projects at the Los Alamos National Laboratory Research Library.
Date: May 3, 2012
Creator: Van de Sompel, Herbert
Item Type: Refine your search to only Presentation
Partner: International Internet Preservation Consortium

Memento Tracer - An Innovative Approach Towards Balancing Web Archiving at Scale and Quality

Description: Presentation for the IIPC General Assembly and Web Archiving Conference virtually held on June 14-16, 2021. This presentation introduces Memento Tracer, a web archiving framework, that aims at striking a balance between operating at web scale and providing high-quality captures.
Date: June 15, 2021
Creator: Klein, Martin & Van de Sompel, Herbert
Item Type: Refine your search to only Presentation
Partner: International Internet Preservation Consortium

Web Archiving the Olympic & Paralympic Games

Description: Presentation for the IIPC General Assembly and Web Archiving Conference virtually held on May 23-25, 2022. This presentation covers the International Internet Preservation Consortium (IIPC) Olympic/Paralympic collections and reflects on what has been previously collected.
Date: May 24, 2022
Creator: Byrne, Helena
Item Type: Refine your search to only Presentation
Partner: International Internet Preservation Consortium

Hosting the End of Term Web Archive Data in the Cloud

Description: Presentation for the IIPC General Assembly and Web Archiving Conference virtually held on May 23-25, 2022. This presentation discusses the decision to host the web content of the End of Term Web Archive in AWS, the layout used to organize the crawl data, and the tools used for this work. This work can be used as a model for other web archives interested in hosting their data in the cloud for greater access and reuse.
Date: May 24, 2022
Creator: Phillips, Mark Edward & Alam, Sawood
Item Type: Refine your search to only Presentation
Partner: International Internet Preservation Consortium

Common Crawl – Experiences From 10 Years in the Cloud

Description: Presentation for the IIPC General Assembly and Web Archiving Conference virtually held on May 23-25, 2022. This presentation gives an overview of how the Common Crawl web data is used in and outside the cloud over the past ten years that the dataset has been hosted as part of Amazon Web Services’ Open Data Sponsorships program.
Date: May 24, 2022
Creator: Nagel, Sebastian
Item Type: Refine your search to only Presentation
Partner: International Internet Preservation Consortium

Rapid Response Collecting: What Are the New Workflows and Challenges?

Description: Presentation for the IIPC General Assembly and Web Archiving Conference virtually held on May 23-25, 2022. This presentation explores the implementation or elaboration of “rapid response” web archival curation methodologies, including new workflows, challenges, use cases, and also creating program efficiencies in order to better respond to the need to document unplanned and unforeseen, but national historic events.
Date: May 25, 2022
Creator: Smyth, Tom J. & Wertheimer, Melissa
Item Type: Refine your search to only Presentation
Partner: International Internet Preservation Consortium
open access

Sketching and checking quality for web archives: a first stage report from National Library of France (BnF)

Description: Report draft of an overview of web legal deposit issues and organization at the National Library of France (BnF) at the beginning of 2006. It describes where the project stands and where it is going, with a focus on quality issues.
Date: February 8, 2006
Creator: Illien, Gildas
Item Type: Refine your search to only Report
Partner: International Internet Preservation Consortium

Discovering and Archiving the Frisian Web. Preparing for a National Domain Crawl

Description: Presentation for the IIPC General Assembly and Web Archiving Conference held on May 10-12, 2023 in Hilversum, Netherlands. This presentation discusses the "pilot" domain crawl of Friesland, a Dutch province, in preparation for the National Domain Crawl of the Dutch web. It describes lessons learned, specifically the need to maintain integrity and authenticity during a crawl.
Date: May 12, 2023
Creator: van den Eijkel, Susanne; & Geldermans, Iris
Item Type: Refine your search to only Presentation
Partner: International Internet Preservation Consortium

Unsustainability and Retrenchment in American University Web Archives Programs

Description: Presentation for the IIPC General Assembly and Web Archiving Conference held on May 10-12, 2023 in Hilversum, Netherlands. This presentation discusses the issues encountered by the University at Albany when it came to web archive maintenance and development. These issues are reflected in the larger sphere of university web archive programs conducted in the United States.
Date: May 12, 2023
Creator: Wiedeman, Gregory & Greenwood, Amanda
Item Type: Refine your search to only Presentation
Partner: International Internet Preservation Consortium

Laboratory Not Found? Analyzing LANL’s Web Domain Crawl

Description: Presentation for the IIPC General Assembly and Web Archiving Conference held on May 10-12, 2023 in Hilversum, Netherlands. This presentation analyzes the Los Alamos National Laboratory's domain crawl and addresses issues of link rot and content drift, explaining the errors that can occur within a system and efforts to recover URLs.
Date: May 12, 2023
Creator: Klein, Martin & Balakireva, Lyudmila
Item Type: Refine your search to only Presentation
Partner: International Internet Preservation Consortium

Browser-Based Crawling For All: The Story So Far

Description: Presentation for the IIPC General Assembly and Web Archiving Conference held on May 10-12, 2023 in Hilversum, Netherlands. This presentation provides updates on the development of a browser-based crawling system with a user interface, a project that extends the Webrecorder Browsertrix Crawler system.
Date: May 3, 2023
Creator: Klindt Myrvoll, Anders
Item Type: Refine your search to only Presentation
Partner: International Internet Preservation Consortium
Back to Top of Screen