Page tree
Skip to end of metadata
Go to start of metadata

This workflow describes the most obvious steps from a functional point of view. 

See also the schematic view of this workflow. 

Terminology

Ingest archivist: role of persons responsible for offloading the digital archive to the digital archive. See also Voorstel werkplekken en rollen ingest archivist


 PhaseWorkflowstep globalWorkflowstep in detailResponsibility
Pre-Archivematica phase




Contract between the donor and the IISH

Agreement between the archival donor and the IISH on how the archive will be delivered. This is laid down in a submission document. This document is to be included in the AIP?Archival donor and Collection Development

Preparation before transport


The archival donor prepares the files before transport. The IISH advises on the best way to prepare and transfer the archive. See the Checklist for the transfer of archive between the archival donor and the IISH.

Archival donor with help/advise from digital archivist

Delivery of the archive



The archival donor sends the files to the IISH. On the moment there seem to be four possible use cases for delivery:

  1. On HD or USB stick
  2. Via a cloud service (like OwnCloud, Dropbox or Google Drive)
  3. Via filetransfer webservices like Surfnet Filesender or Wetransfer
Archival donor with help/advise from digital archivist/Collection Development

First registration

First registration of the collection (in the Acquisition database/or otherwise) including a high level description of the archive (accession record). The acquisition database generates an accession number (combination of an archive and inventory number - ARCH12345.67890). In case of an addition to an existing archive the archivist in the Acquisition database can select the relevant archive and generate a new inventory number for the addition.

Collection Development/Processing
Creation of hot folderA script creates a folder on the transfer location with the same name (accession/inventory number) as given by the acquisition database. Automatic

Offload of the archive to the sorting area

  • The ingest archivist creates a folder with accession number of the archive on the sorting area.
  • If the files are received on HD - and after a virus scan (see below) - or other carriers like floppies or cd's the carrier is offloaded via one of the specially configured "offload Macs" to the folder in the sorting area.This can be done by copying or - better by using FTP.
  • When the files are delivered via use delivery case 2 - 4 the files via the offload Mac is downloaded directly to the quarantine folder on the sorting area (see below).
Ingest archivist
 Virus check
  • If a HD is offloaded the file on the HD are virus checked the moment HD is attached to the MAC.
  • If the files are delivered via use delivery case 2 - 3 the files are downloaded to the quarantine folder of the sorting area and then checked for viruses. If no virus is found the archive is moved to the folder with the accession number (see step above).
  • If a virus or otherwise harmful content is detected the infected file(s) is/are soon as possible deleted. Also the archival donor is informed of the found virus. In any doubt the digital archivist and/or KNAW ICT services are asked for advise.
Ingest archivist
Quick scan of archive

The goal of the quick scan of an archive that has arrived at the IISH, is to see if we have received that was agreed upon with the archival donor and to check if the archive contains something that might help or hinder further processing. It is a mix of an archival content and a technical check. See the guide First inspection/quick scan of a new archive for the steps involved.

Collection development/ingest archivist
Optional: appraisal and selection

This step is taken if during the quick scan it becomes clear that more work is needed to process the archive.

Collection processing/ingest archivist
Creation of an archival bag

The material is packaged into an archival bag (by use of the Exactly tool on the offload MAC) by the archivist. It is advised to name the bag after the accession and the name of the archive (i.e: ARCH12345.b67_archief_Spaargaren). This will make it easier findable within the Archivematica overview of stored AIP's.

Collection processing/ingest archivist
Move to hot folder/transfer locationThe bag is moved to the hot folder/transfer location. It is important to choose the right folder (the archive and inventory number combination created during the first registration step) . When the move is ready the server creates an MD5 checksum file.Collection Processing and Access
Archivematica phase


























Pre-transfer




 
The MD5 checksum file created by the FTP server is found by the Archivematica automation tool (which checks the transfer location regularly). Automatic
The automation tool checks if the name of bag is correct and checks the fixity of the bag. If the name of bag is incorrect or the bag is damaged (fixity check) further processing is halted and the archivist receives an email. If all checks give a positive result the transfer commences.Automatic
Transfer (list not exhaustive)[1]   Start and approve transferAutomatic
Verification of transfer compliance: verifies that the transfer is properly structured - i.e. with the logs, metadata and objects folders. But also removal of hidden (system) files (like ds store files) an directoriesAutomatic
Assign UUID and checksumsAutomatic
Verify transfer checksum→If one of more checksums don't match the transfer process does stop.Automatic
Virus scan →if the transfer (in unlikely case a virus was not found during the first, pre-Archuvematica check) contains a virus the transfer is halted immediately.Automatic
Clean up names (original file name preserved in METS)Automatic
Indentify file format →Siegfried tool is automatically selectedAutomatic
Validate file formats→By the JHOVE toolAutomatic
Extract packages (for instance ZIP or TAR files)Automatic
Examine contents (for privacy content)Automatic
Create Transfer METS file Automatic
Create SIP →Decision by the archivist if the SIP is first transported to the backlog (for appraisal and selection steps) or that ingest phase and the creation of AIP can start immediately.

Archivist/Automatic

Copy to SIP directoryAutomatic
Backlog/appraisal and selection





Decide if the SIP need appraisal and selection. If not the ingest can be started immediately.

Archivist
Check for identical files. Delete/keep files. → To be realiized in phase 2 of the Archivematica implementation project. Collection Development
In case of a supplement on an existing archive: check if after offload which files are already part of the archival collection (AIP) and which files are new or changed. Delete when possible. → To be realiized in phase 2 of the Archivematica implementation project. Collection Development
Check which files are password protected. Delete/remove password/keep file.  → Uncertain if & when this will be developed phase 2 of the Archivematica implementation project. Collection Development/archivist
Check which files are encrypted. Delete/remove encryption/keep file. → Uncertain if & when this will be developed phase 2 of the Archivematica implementation project. Collection Development/archivist
Ingest (list not exhaustive)[2]






Verify SIP complianceAutomatic
Normalize files following the IISH file format policy (laid down in the File Format Registry) for preservation and accessAutomatic
Indentify file format →Siegfried tool is selected automaticallyAutomatic
Verify checksumsAutomatic

Generate AIP, including:

  • AIP METS file
  • Preservation copy (if necessary according to file format policy)
  • Original file
Automatic

Generate DIP, including:

  • DIP METS file
  • Access copy (if necessary according to file format policy)
  • Original file
  • PID binding for every file and folder.
Automatic

Storage of the AIP

Automatic
Storage of the DIPAutomatic
The bag is removed from the transfer locationAutomatic



---------------------------------------------------------------------------------------------------

[1] Not all steps of Archivematica transfer process are listed. A more comprehensive list can be found here: https://www.archivematica.org/en/docs/archivematica-1.6/user-manual/transfer/transfer/#transfer

[2] Not all steps of Archivematica ingest process are listed. A more comprehensive list can be found here: https://www.archivematica.org/en/docs/archivematica-1.6/user-manual/ingest/ingest/#ingest


  • No labels