doi: 10.1038/s41467-021-21326-w.
Unlocking history through automated virtual unfolding of sealed documents imaged by X-ray microtomography
Affiliations
- PMID:33654094
- PMCID: PMC7925573
- DOI: 10.1038/s41467-021-21326-w
Item in Clipboard
Unlocking history through automated virtual unfolding of sealed documents imaged by X-ray microtomography
Jana Dambrogio et al. Nat Commun..
Display options
Format
Display options
Format
Abstract
Computational flattening algorithms have been successfully applied to X-ray microtomography scans of damaged historical documents, but have so far been limited to scrolls, books, and documents with one or two folds. The challenge tackled here is to reconstruct the intricate folds, tucks, and slits of unopened letters secured shut with "letterlocking," a practice-systematized in this paper-which underpinned global communications security for centuries before modern envelopes. We present a fully automatic computational approach for reconstructing and virtually unfolding volumetric scans of a locked letter with complex internal folding, producing legible images of the letter's contents and crease pattern while preserving letterlocking evidence. We demonstrate our method on four letterpackets from Renaissance Europe, reading the contents of one unopened letter for the first time. Using the results of virtual unfolding, we situate our findings within a novel letterlocking categorization chart based on our study of 250,000 historical letters.
Conflict of interest statement
The authors declare no competing interests.
Figures
For each letterpacket (DB-1538, DB-1627, DB-1976, and DB-2040; “DB” numbers in this article refer to shelfmarks):a outside of letterpacket, front, and back;b transparent view through the volumetric XMT data showing varied distribution of layers across the letterpacket;c crease pattern that would be visible on the paper if fully opened, stylized from the output of virtual unfolding;d proposed step-by-step letterlocking process to make this packet from a flat piece of paper, based on virtual unfolding results;e letterlocking category (LC#) assigned based on virtual unfolding findings, letterlocking categories, and formats chart (see Fig. 7), apart from opened letter DB-2040, assigned based on surviving letterlocking evidence. For videos of the letterlocking process for each letter, see our Supplementary websitebrienne.org/unfolding . Scale bars:a 10 mm.
High-level overview of virtual unfolding showing.a XMT volumetric data and resultingb folded andc,d unfolded reconstructions of unopened letter DB-1538.c XMT scan data is mapped to 2D to produce an image of the sealed letterpacket’s contents.d Curvature is mapped to 2D to generate an image representing the crease pattern of the folded letterpacket.e Ink visible on the surface of the paper shows the presence of double-sided text (running both vertically and horizontally).f A close-up of the crease pattern image depicts sharp creases on the interior of the packet as thin, saturated lines, and rounded creases toward the outside of the packet as diffuse lines, due to their lower curvature spread across a larger region. Surface curvature induced by buckling of the paper near the corner of the letterpacket is also revealed by this analysis (also visible in the 3D reconstruction,f). Scale bars:a 1 mm,c–e 10 mm,f,g 5 mm.
Two sample cross sections of XMT data ataz = 525 andbz = 1611, showing 11 letterpackets. From top to bottom, letter IDs are DB-1465, DB-1818, DB-2004, DB-1538, DB-1976, DB-1627, DB-2199, DB-2156, DB-1888, DB-2040, and DB-1876. Note that DB-1888 is not visible in cross sectionb.c–h Segmentation challenges associated with XMT data.c Paper layer compaction results in ambiguous boundaries between layers. Dense regions ofd ink,e adhesive,f sealing wax, andg pounce sand (used to absorb ink) interfere with neighboring regions of scan.h Paper may be angled relative to a given cross-sectional plane, resulting in a blurred, artificially thickened appearance in a 2D cross section. Scale bars:a,b 10 mm,c–h 1 mm.
a Cropped 3D volume of DB-1538 XMT data showing three sections of folded paper;b–d three cross-sectional slices of the volume;e 3D segmentation result for outermost paper section; andf corresponding flattened state.g 3D mesh after hybrid mesh propagation, with color indicating the order that new vertices were added to fill in segmentation holes, andh corresponding flattened state. 3D mesh reconstruction colored with texturing results showingi greyscale XMT data image andk crease pattern image, and corresponding flattened states (j,l). Black boundary inf,h,j,l shows the bounds of the 3D crop mapped to 2D.m Final flattened reconstruction showing the initial segmentation result in pale blue with hybrid mesh propagation additions shown in a spectrum of colors according to the order in which new vertices were added.n A close-up reveals numerous isolated connected components in pale blue that were merged to form one continuous mesh during this process. Scale bars:a,e,f 1 mm,m 10 mm,n 5 mm.
Colored 2D distance map of thea flattened reconstruction of DB-1538 (same as Fig. 2c), and the same colormap applied tob a cropped section of the 3D folded state demonstrates that nearby vertices in 3D may be very far apart from each other in 2D, if they belong to different layers of the writing substrate. Distance metrics from the 2D and 3D embeddings of the mesh are used during hybrid mesh propagation to enforce global developability of the final result.
Virtual unfolding results for unopened letter DB-1627 (a,b) and opened letter DB-2040, showinga XMT-textured image after virtual unfolding (this image has been inverted and linearly scaled to increase contrast of text and other details, such as a visible watermark in the center and laid and chain lines of the handmade paper);b crease pattern image; andc a close-up view of a portion of the final texturing results for opened letter DB-2040 (inverted and linearly scaled), compared tod a photograph demonstrates low geometric distortion of the virtually flattened result across a crease (indicated by the dashed blue line). Scale bars:a 10 mm,b 50 mm,c,d 10 mm.
Letterlocking categories 1–64, based on manipulations and assigned security score. Filled letterlocking category numbers indicate that historic originals exist; unfilled numbers indicate hypothetical categories. Letterlocking formats show up to 12 edges with indicative examples.
Edge strain map (measured as edge length deviation in the 2D embedding as a percentage of nominal length in 3D embedding) for DB-1627 shows a nearly isometric mapping of the 3D embedding to a plane. Localized strain concentrated around creases (especially where creases intersect each other) and in other areas where poor initial segmentation leads to an accumulation of error during hybrid mesh propagation. Distortions of the 2D embedding can negatively impact automatic merging of connected components as demonstrated by: the large regiona that was not successfully merged to the final result; the regionb that was incorrectly positioned in the final result (should have been located atc); and the regiond containing text that was incorrectly positioned in the final result. Scale bars:a,d 10 mm.
Similar articles
- Virtual Recovery of Content from X-Ray Micro-Tomography Scans of Damaged Historic Scrolls.Rosin PL, Lai YK, Liu C, Davis GR, Mills D, Tuson G, Russell Y.Rosin PL, et al.Sci Rep. 2018 Aug 9;8(1):11901. doi: 10.1038/s41598-018-29037-x.Sci Rep. 2018.PMID:30093680Free PMC article.
- A computational platform for the virtual unfolding of Herculaneum Papyri.Stabile S, Palermo F, Bukreeva I, Mele D, Formoso V, Bartolino R, Cedola A.Stabile S, et al.Sci Rep. 2021 Jan 18;11(1):1695. doi: 10.1038/s41598-020-80458-z.Sci Rep. 2021.PMID:33462265Free PMC article.
- Browsing through sealed historical manuscripts by using 3-D computed tomography with low-brilliance X-ray sources.Stromer D, Christlein V, Martindale C, Zippert P, Haltenberger E, Hausotte T, Maier A.Stromer D, et al.Sci Rep. 2018 Oct 18;8(1):15335. doi: 10.1038/s41598-018-33685-4.Sci Rep. 2018.PMID:30337644Free PMC article.
- Volumetric Colon Wall Unfolding Using Harmonic Differentials.Zeng W, Marino J, Kaufman A, Gu XD.Zeng W, et al.Comput Graph. 2011 Jun 1;35(3):726-732. doi: 10.1016/j.cag.2011.03.008.Comput Graph. 2011.PMID:21765563Free PMC article.
- From damage to discovery via virtual unwrapping: Reading the scroll from En-Gedi.Seales WB, Parker CS, Segal M, Tov E, Shor P, Porath Y.Seales WB, et al.Sci Adv. 2016 Sep 21;2(9):e1601247. doi: 10.1126/sciadv.1601247. eCollection 2016 Sep.Sci Adv. 2016.PMID:27679821Free PMC article.
Cited by
- Enabling 3D CT-scanning of cultural heritage objects using only in-house 2D X-ray equipment in museums.Bossema FG, Palenstijn WJ, Heginbotham A, Corona M, van Leeuwen T, van Liere R, Dorscheid J, O'Flynn D, Dyer J, Hermens E, Batenburg KJ.Bossema FG, et al.Nat Commun. 2024 May 14;15(1):3939. doi: 10.1038/s41467-024-48102-w.Nat Commun. 2024.PMID:38744870Free PMC article.
- A Geometric Feature-Based Algorithm for the Virtual Reading of Closed Historical Manuscripts.Brancaccio R, Albertin F, Seracini M, Bettuzzi M, Morigi MP.Brancaccio R, et al.J Imaging. 2023 Oct 20;9(10):230. doi: 10.3390/jimaging9100230.J Imaging. 2023.PMID:37888337Free PMC article.
References
- Dambrogio, J. Assessment, treatment, and study in the Conservation Laboratory of the Vatican Secret Archives. In Paper Presented at the Book and Paper Group Session, 33rd Annual Meeting of the American Institute for Conservation, Minneapolis, MN (2005).
- Riggi di Numana, G. Il Segreto Epistolare nel Periodo Della Civiltà Della Carta (Libreria Cortina, 2008).
- Dambrogio, J., Smith, D. S. & Unlocking History. The Dictionary of Letterlockinghttp://letterlocking.org/dictionary (2015).
- Ahrendt R, Van der Linden D. The postmasters’ piggy bank: experiencing the accidental archive. Fr. Hist. Stud. 2017;40:189–213.
- Steger C. An unbiased detector of curvilinear structures. IEEE T. Pattern Anal. 1998;20.2:113–125. doi: 10.1109/34.659930. - DOI
Publication types
Related information
LinkOut - more resources
Full Text Sources
Other Literature Sources