Page tree
Skip to end of metadata
Go to start of metadata

This workflow describes the most obvious steps from a functional point of view. 

Related documentation


 PhaseWorkflowstep globalWorkflowstep in detailResponsibilityNet timeDoorlooptijdBottleneck/remarks
Pre-Archivematica phase

Contract between the donor and the IISH

Agreement between the archival donor and the IISH on how the archive will be delivered. This is laid down in a submission document.Archival donor and Collection Development


Preparation before transport


The archival donor prepares the files before transport. The IISH advises on the best way to prepare and transfer the archive. See the Checklist for the transfer of archive between the archival donor and the IISH.

Archival donor with help/advise from digital archivist


Delivery of the archive



The archival donor sends the files to the IISH. On the moment there are three possible use cases for delivery:

  1. On HD or USB stick (or other information carriers)
  2. Via a cloud service (like OwnCloud, Dropbox or Google Drive)
  3. Via filetransfer webservices like Surfnet Filesender or Wetransfer
Archival donor with help/advise from digital archivist/Collection Development

How much time do we want to spend on outdated or hard to read carriers? Do we need more hardware and knowledge? Do we need to invest in Bitcurator? 

First registration

First registration of the collection in the Acquisition database including a high level description of the archive (accession record). It is important that no material is offloaded without a registration in the Acquisition database. Ideally an archive number is already created (ARCH01234) and the material offloaded to a folder of that name. Future step: In Archivespace a acquisition number will be created for these purposes.  

Collection Development/Processing archivist


 Virus check
  • If a HD is offloaded the file on the HD are virus checked the moment HD is attached to the MAC.
  • If the files are delivered via use delivery case 2 - 3 the files are downloaded to the Offload folder and then checked for viruses. If no virus is found the archive is moved to the folder with the accession number (see step above).
  • If a virus or otherwise harmful content is detected the infected file(s) is/are soon as possible deleted. Also the archival donor is informed of the found virus. In any doubt the digital archivist and/or KNAW ICT services are asked for advise.
Collection Development/Processing archivist

Do we need a quarantine room where new archives have to wait a while before being processed further?
Offload of the archive to the offload folder
  • If the files are received on HD - and after a virus scan (see above) - or other carriers like floppies or cd's the carrier are ideally offloaded via one of the specially configured "offload Macs". The files are copied to the Offload folder (on the pre-ingest Workflow share). 
  • After copying check if the amount of files and bytes are the same on the carrier as on the share.
  • When the files are delivered via use delivery case 2 - 3 the files are downloaded via the offload Mac to the offload folder.
Collection development/Processing archivist


Move archive to Inventory folder for a first inspection of the archive
  • The archive is moved from the Offload folder to the Inventory folder. 
  • The goal of the quick scan of an archive that has arrived at the IISH, is to see if we have received that was agreed upon with the archival donor and to check if the archive contains something that might help or hinder further processing. It is a mix of an archival content and a technical check. See the guide First inspection/quick scan of a new archive for the steps involved.
  • If the material is static it can stay for a while in the Inventory folder. Any changes to the material must be done within the sorting / work folder (because here the material has been back-upped). 
Collection development/processing archivist


Move archive to Backup folderIf the archive ready for further processing move the archive to the Backup folder. Collection development/processing archivist


Creation of backup and move to the Sorting folderA backup is made from the archive. When the backup is ready the archive is moved to the Sorting/work folder. The backup is deleted when an AIP has successfully been created. The status of the backup creation can be followed by use of the Workflow Controller (an IISH made pre-ingest monitoring tool) https://workflow-controller.collections.iisg.org/archive?form_archive_name=backup.Automatic


Optional: appraisal and selectionIf any appraisal and selection work is done it must de done within the Sorting work folder. Processing archivist


Optional: divide archive into intellectual entities (inventory numbers)If the archive is to be divided in to more than one intellectual entities (inventory numbers) for each of these entities an accession number has to be created (see step directly below). And the content of these entities then have to be moved to folders of that name.  Processing archivist


Create accession numberWhen the archive is ready for further processing an accession number (combination of an archive and inventory number - ARCH1234.dig1234) must be created (in the Acquisition database). In case of an addition to an existing archive the archivist in the Acquisition database can select the relevant archive and generate a new inventory number for the addition. Without an accession number the archive can not be ingested. An accession number might also be given after sorting but preferably is done before creating the backup (so it can be easily found back). Processing archivist


Create archival descriptionIdeally an archival description is created on this moment in the workflow. Processing archivist


Move to IngestDigitalBorn folder

If the archive is ready (after step 1 or if that is not necessary) to ingest (in Archivematica and for AIP creation) it can be moved to the IngestDigitalBorn folder. 

Collection development/processing archivist


Pre-transfer and validation

In the Workflow Controller the status of the pre-ingest process can be followed: https://workflow-controller.collections.iisg.org/archive?form_archive_name=archivematica_db. If after validation this fails the archive (also later in the ingestion process) will be moved to Work / Failed folder. If this is successful the archive is moved to a hotfolder from which the Archivematica automation tool starts up the ingest process and moves the archive to the Archivematica transfer source.

Automatic


Archivematica phaseTransfer (list not exhaustive)[1]  Start and approve transferAutomatic


Verification of transfer compliance: verifies that the transfer is properly structured - i.e. with the logs, metadata and objects folders. But also removal of hidden (system) files (like ds store files) an directoriesAutomatic


Assign UUID and checksumsAutomatic


Verify transfer checksum→If one of more checksums don't match the transfer process does stop.Automatic


Virus scan →if the transfer (in unlikely case a virus was not found during the first, pre-Archivematica check) contains a virus the transfer is halted immediately.Automatic


Clean up names (original file name preserved in METS)Automatic


Identify file format →Siegfried tool is automatically selectedAutomatic


Validate file formats→By the JHOVE toolAutomatic


Extract packages (for instance ZIP or TAR files)Automatic


Examine contents (for privacy content)Automatic


Create Transfer METS file Automatic


Create SIP 

Automatic




Copy to SIP directoryAutomatic


Ingest (list not exhaustive)[2]Verify SIP complianceAutomatic


Normalize files following the IISH file format policy (laid down in the File Format Registry) for preservation and accessAutomatic


Identify file format →Siegfried tool is selected automaticallyAutomatic


Verify checksumsAutomatic


Generate AIP, including:

  • AIP METS file
  • Preservation copy (if necessary according to file format policy)
  • Original file
Automatic


Storage of the AIP

Automatic


Post-Archivematica phase

Check for succesful ingest

If the ingest went well the Workflow controller will notify that the AIP has been stored. If not the Workflow controller will notify as such and the archive will be placed in the Sorting/Failed folder. The reason why the ingest has failed then has to be found from the Archivematica error log (which is not always easy) and after repair ingested again. Processing archivist


Register If the AIP has been stored register as such in the Acquisition database by not in the remarks field: "RGI (name of processing archivist) 6-6-2021: Verwerkt tot AIP door Archivematica."

Put in depot field: digital ingest depot. Also register on Overzicht verwerking born digital archief under the header 'succesful ingests'. 

If the ingest has failed also note this (including the reason why) in the Acquisition database and Overzicht verwerking born digital archief under the header 'Work needed'. 

Processing archivist


---------------------------------------------------------------------------------------------------

[1] Not all steps of Archivematica transfer process are listed. A more comprehensive list can be found here: https://www.archivematica.org/en/docs/archivematica-1.11/#transfer

[2] Not all steps of Archivematica ingest process are listed. A more comprehensive list can be found here: https://www.archivematica.org/en/docs/archivematica-1.11/user-manual/ingest/ingest/#create-sip


  • No labels