Repository files navigation

Siegfried is a signature-based file format identification tool, implementing:

• the National Archives UK'sPRONOM file format signatures
• freedesktop.org'sMIME-info file format signatures
• the Library of Congress'sFDD file format signatures (beta).
• Wikidata (beta).

1.11.2

sf file.extsf *.extsf DIR

sf -csv file.ext | *.ext | DIR             // Output CSV rather than YAMLsf -json file.ext | *.ext | DIR            // Output JSON rather than YAMLsf -droid file.ext | *.ext | DIR           // Output DROID CSV rather than YAMLsf -nr DIR                                 // Don't scan subdirectoriessf -z file.zip | *.ext | DIR               // Decompress and scan zip, tar, gzip, warc, arcsf -zs gzip,tar file.tar.gz | *.ext | DIR  // Selectively decompress and scan sf -hash md5 file.ext | *.ext | DIR        // Calculate md5, sha1, sha256, sha512, or crc hashsf -sig custom.sig *.ext | DIR             // Use a custom signature filesf -                                       // Scan stream piped to stdinsf -name file.ext -                        // Provide filename when scanning stream sf -f myfiles.txt                          // Scan list of files and directoriessf -v | -version                           // Display version informationsf -home c:\junk -sig custom.sig file.ext  // Use a custom home directorysf -serve hostname:port                    // Server modesf -throttle 10ms DIR                      // Pause for duration (e.g. 1s) between file scanssf -multi 256 DIR                          // Scan multiple (e.g. 256) files in parallel sf -log [comma-sep opts] file.ext          // Log errors etc. to stderr (default) or stdoutsf -log e,w file.ext | *.ext | DIR         // Log errors and warnings to stderrsf -log u,o file.ext | *.ext | DIR         // Log unknowns to stdoutsf -log d,s file.ext | *.ext | DIR         // Log debugging and slow messages to stderrsf -log p,t DIR > results.yaml             // Log progress and time while redirecting resultssf -log fmt/1,c DIR > results.yaml         // Log instances of fmt/1 and chart resultssf -replay -log u -csv results.yaml        // Replay results file, convert to csv, log unknownssf -setconf -multi 32 -hash sha1           // Save flag defaults in a config filesf -setconf -serve :5138 -conf srv.conf    // Save/load named config file with '-conf filename'

By default, siegfried uses the latest PRONOM signatures without buffer limits (i.e. it may do full file scans). To use MIME-info or LOC signatures, or to add buffer limits or other customisations, use theroy tool to build your own signature file.

go install github.com/richardlehane/siegfried/cmd/sf@latestsf -update

Download a pre-built binary from thereleases page. Unzip to a location in your system path. Then run:

sf -update

brew install mistydemeo/digipres/siegfried

Or, for the most recent updates, you can install from this fork:

brew install richardlehane/digipres/siegfried

curl -sL "http://keyserver.ubuntu.com/pks/lookup?op=get&search=0x20F802FE798E6857" | gpg --dearmor | sudo tee /usr/share/keyrings/siegfried-archive-keyring.gpgecho "deb [signed-by=/usr/share/keyrings/siegfried-archive-keyring.gpg] https://www.itforarchivists.com/ buster main" | sudo tee -a /etc/apt/sources.list.d/siegfried.listsudo apt-get update && sudo apt-get install siegfried

pkg install siegfried

git clone https://aur.archlinux.org/siegfried.gitcd siegfriedmakepkg -si

• Filter introduced to improve Wikidata queries and-harvestWikidataSigLen flag sets minimum length of Wikidata signatures. Implemented byRoss Spencer and Andy Jackson
• -noprov flag introduced for Wikidata signatures. Implemented byRoss Spencer
• version command forroy. Implemented byRoss Spencer

• Logged error messages have more context. Implemented byRoss Spencer
• update PRONOM to v120
• Wikidata definitions updated to 4.0.0. ByRoss Spencer
• update LOC to 2025-02-26
• update TIKA to v3.1.0

• Explicitly declared variable offsets are parsed e.g. fmt/1840. Implemented byRijnder Wever

• WASM build. See wasm/README.md for more details. Feature sponsored by Archives New Zealand. Inspired byAndy Jackson
• -sym flag enables following symbolic links to files during scanning. Requested byMax Moser

• XDG_DATA_DIRS checked when determining siegfried home location. Requested byMichał Górny
• Windows 7 build onreleases page (built with go 1.20). Requested byAleksandr Sergeev
• update PRONOM to v118
• update LOC to 2024-06-14

• zips piped into STDIN are decompressed with-z flag. Reported byMax Moser
• panics from OS calls in init functions. Reported byJürgen Enge

• glob-matching for container signatures; seedigital-preservation/pronom#10
• sf -update works for older versions of siegfried; see#231

• default location for siegfried HOME now follows XDG Base Directory Specification; see#216. Implemented byBernhard Hampel-Waffenthal
• siegfried prints version before erroring with failed signature load; requested byRoss Spencer
• update PRONOM to v116
• update LOC to 2023-12-14
• update tika-mimetypes to v3.0.0-BETA
• update freedesktop.org to v2.4

• panic on malformed zip file during container matching; reported byJames Mooney

• update PRONOM to v116
• update LOC to 2023-12-14
• update tika-mimetypes to v3.0.0-BETA
• update freedesktop.org to v2.4

• glob expansion now only on Windows & when no explicit path match. Implemented byBernhard Hampel-Waffenthal
• compression algorithm for debian packages changed back to xz. Implemented byPaul Millar
• -multi droid setting returned empty results when priority lists contained self-references. See#218
• CGO disabled for debian package and linux binaries. See#219

• format classification included as "class" field in PRONOM results. Requested byRobin François. Implemented byRoss Spencer
• -noclass flag added to roy build command. Use this flag to build signatures that omit the new "class" field from results.
• glob paths can be used in place of file or directory paths for identification (e.g.sf *.jpg). Implemented byRoss Spencer
• -multi droid setting for roy build command. Applies priorities after rather than during identification for more DROID-like results. Reported byDavid Clipsham
• /update command for server mode. Requested byLuis Faria

• new algorithm for dynamic multi-sequence matching for improved wildcard performance
• update PRONOM to v111
• update LOC to 2023-01-27
• update tika-mimetypes to v2.7.0
• minimum go version to build siegfried is now 1.18

• archivematica extensions built into wikidata signatures. Reported byRoss Spencer
• trailing slash for folder paths in URI field in droid output. Reported by Philipp Wittwer
• crash when usingsf -replay with droid output

See theCHANGELOG for the full history.

Copyright 2024 Richard Lehane, Ross Spencer

Licensed under theApache License, Version 2.0

Join theGoogle Group for updates, signature releases, and help.

Like siegfried and want to get involved in its development? That'd be wonderful! There are some notes on thewiki to get you started, and please get in touch.

Thanks TNA forhttp://www.nationalarchives.gov.uk/pronom/ andhttp://www.nationalarchives.gov.uk/information-management/projects-and-work/droid.htm

Thanks Ross forhttps://github.com/exponential-decay/skeleton-test-suite-generator andhttp://exponentialdecay.co.uk/sd/index.htm, both are very handy!

Thanks Misty for the brew and ubuntu packaging

Thanks Steffen for the FreeBSD and Arch Linux packaging