Medical imaging applied to heritage

Abstract

The use of imaging has transformed the study of cultural heritage artefacts in the same way that medical imaging has transformed medicine. X-ray-based techniques are common in both medical and heritage imaging. Optical imaging, including scientific photography and spectral imaging techniques, is also common in both domains. Some common medical imaging methods such as ultrasound and MRI have not yet found routine application in heritage, whereas other methods such as imaging with charged and uncharged particles and 3D surface imaging are more common in heritage. Here, we review the field of heritage imaging from the point of view of medical imaging and include some classic challenges of heritage imaging such as reading the text on burnt scrolls, identifying underdrawings in paintings, and CT scanning of mummies, an ancient calculating device and sealed documents. We show how hyperspectral imaging can offer insight into the drawing techniques of Leonardo da Vinci and explain how laparoscopy has identified the method of construction of a 500-year-old pop-up anatomical text book.

Introduction

Readers of the British Journal of Radiology will not need to be persuaded of the transformative impact of imaging in medicine over the last 125 years and beyond. Similarly, heritage science, while less mature, also relies heavily on imaging with some of the more important heritage imaging techniques having been first pioneered in medical imaging. This review examines how imaging has been applied to heritage, including some methods that were adapted from medical imaging and some that were developed independently. Parallels and differences between imaging in medicine and heritage are highlighted.

According to the latest Diagnostic Imaging Dataset Statistical Release from March 2023, ¹ 43.2 million medical imaging tests in the UK were carried out from December 2021 to November 2022. The most common tests were X-ray (21.2M), Ultrasound (9.9M), Computed Tomography (CT; 6.6M), Magnetic Resonance Imaging (MRI; 3.9M), fluoroscopy (0.9M), nuclear medicine (including for this analysis PET/CT and SPECT; 0.6M) and medical photography (54,000). Over the same period (from the interventional tables on the same resource), there were also 60,000 endoscopy tests.

An equivalent analysis for heritage imaging would not be possible. No equivalent tables exist and it is not straightforward to carry out a bibliographic analysis as terms are not always well established. For example, a search for “fluorescence” in the journal Heritage Science identifies terms including “LED-induced fluorescence”, “fluorescence spectroscopy”, “laser-induced fluorescence spectroscopy”, “spectrofluorimetry”, “luminescence imaging” and others, as well as terms linked to X-ray fluorescence and Fourier transform infrared spectroscopy. Perhaps, the lack of an accepted vocabulary is a sign of a developing discipline.

Heritage imaging encompasses a broad range of scales, from visible and electron microscopy for examining pigment crystal structures and small-scale degradation ^2,3 to photogrammetry from drones for surveying vast archaeological sites. ⁴ This review focuses on human-scale heritage imaging that employs familiar medical imaging techniques. Typical subjects include books, manuscripts, paintings and sculptures.

This review forms a non-systematic overview of some of the imaging methods that have had most impact in heritage, concentrating on those that will be most familiar to medical imaging professionals.

Heritage imaging techniques

X-radiography

The use of planar X-radiography in heritage developed more slowly than it did in medicine, with X-rays first being used to examine the structure of paper, papyrus and book bindings between about 1920 and 1950. Now, X-radiography is commonly used to examine works of art, for example, to investigate the canvas, to study any previous conservation interventions or to provide insight into the artist’s painting techniques. Often X-rays are used to visualise underdrawings and can reveal earlier preparatory versions of a painting. Such analysis has had a recent boost with the introduction of machine learning methods for image enhancement and analysis.

One recent example is the work carried out on the Ghent Altarpiece which consists of a series of oil paintings on wood panels attributed to the van Eyck brothers and dated to the 1430s. Multimodal imaging including X-radiography was carried out as part of a major conservation and restoration programme. ⁵ Some panels were painted on both sides, so an X-radiograph consists of a projection image that combines paintings on the front and back as well as the structure of the canvas and support for the painting. The image processing challenge is to unmix these different layers so they can be analysed independently. This was demonstrated successfully ⁶ using a convolutional neural network that combined X-ray images and colour photographs of the front and back of the panels (Figure 1).

Figure 1.

(a) the initial X-ray image of the Ghent alterpiece showing contributions from front and back surfaces; (b) corresponding colour photographs from each side-of the panel (c) reconstructed X-ray images showing separation of the front and back surfaces. Cropped from Figure 5 in Sabetsarvestani et al ⁶ which is licensed under CC BY 4.0 (creativecommons.org/licenses/by/4.0/).

The full range of X-ray methods familiar to medical imaging specialists, including X-ray computed tomography (CT), absorption edge spectroscopy, X-ray fluorescence imaging, X-ray diffraction and phase contrast X-ray have all been used in heritage, often using synchrotron sources. One of the most iconic challenges in heritage imaging is the recovery of text in the Herculaneum scrolls. More than 1800 papyrus scrolls have been discovered in a villa in Herculaneum following the eruption of Vesuvius in 79AD. This represents the only library to have survived from antiquity but the scrolls were carbonised by the heat of the eruption and cannot now be unrolled or read without damaging them. The imaging challenge is to read the inscriptions, written in carbon-based ink, on carbonised papyrus without damaging the intact scroll.

Early attempts to physically unroll the scrolls after softening them with various chemicals had some success but also led to the destruction of some scrolls. More recently, various imaging methods have been used. ⁷ X-ray CT (see section 2.2) is routinely used to reveal the internal structure of the scrolls and has been used to detect carbon ink on a carbon substrate with the contrast possibly provided by impurities but more likely by ink increasing the thickness through which the X-rays travel. X-ray fluorescence analysis has revealed the presence of lead in the ink either as a contaminant or to change the colour or other properties of the ink. ⁸ Perhaps, the most promising X-ray-based method is phase contrast X-ray imaging which detects change in the phase of the X-ray wave caused by variations in the complex refractive index. ⁹ A number of researchers have demonstrated that this method can reveal the presence of writing on the Herculaneum scrolls. When combined with sophisticated image processing to “virtually unroll” the images, ¹⁰ some letters can be discerned ^11,12 as shown in Figure 2.

Figure 2.

Recovered text from an intact Herculaneum scroll, with the Greek letter assigned to each image. Cropped from Figure 3 in Bukreeva et al ¹¹ which is licenced under CC BY 4.0 (creativecommons.org/licenses/by/4.0/).

Low energy X-rays of less than 30 kV, sometimes called Grenz rays, have long been used for superficial treatments especially in dermatological conditions such as eczema and skin cancer. ¹⁴ Heritage materials are frequently either thin (e.g., paper and parchment) or organic (e.g., textiles) and are well suited to imaging and analysis using low energy X-rays. Watermarks are an important source of information about historic paper as the marks change with time and manufacturer. They are created during the paper manufacturing process using a metal stamp to indent a pattern onto the paper as it dries. They may be visible under reflected or transmitted light, but the change in thickness also offers a mechanism for X-ray contrast. Soft X-rays and low energy electrons have been used to image watermarks, for example in a large-scale study of Rembrandt’s etchings ¹⁵ that, when combined with sophisticated image processing, was able to infer information about the chronology of his works and identify reprints. Low energy X-rays have also been successfully used to identify hidden features that demonstrate how North American sandals dating from before 1300 BCE were made. ¹⁶

The X-ray dose is often not considered in heritage applications. Indeed in the paper mentioned above, ¹⁶ X-rays are specifically stated to be “non-destructive”. There is contradictory information in the literature about the potential damaged caused by ionising radiation. The water content of heritage objects is generally low, so they would be expected to be less radiosensitive than tissue. However, at high enough dose, some damage should be expected. There is little consensus on safe dose thresholds across the wide range of different investigation techniques used and indeed dose is rarely measured.

One particularly thorough study used a range of methods to detect potential damage from X-ray micro-computed tomography of parchment ¹⁷ but was unable to “detect a systematic change to the collagen chemistry or structure”. However, higher dose levels from synchrotron or ion-beam sources have been seen to cause damage to objects. ^18–20 This was reviewed in detail by a study ²¹ that proposed to use the term “damage” to refer to visible alterations such as colour change and “radiation-induced side-effect” for non-visible changes to structure or chemistry. Their recommended mitigation strategies include avoidance of unnecessary exposure and optimisation of dose which closely parallel the equivalent strategies in the guidance to the Ionising Radiation Regulations ²² which states that exposures should be justified, optimised and recorded. The safety of ionising and non-ionising radiation when applied to heritage materials is an increasingly active area of study. ²³

Computed Tomography

X-ray CT is now the method of choice for examining intact Egyptian mummies and can reveal details that would otherwise require destructive unwrapping of the mummy such as pathologies, cause of death, methods of mummification and burial traditions. ^24,25

The Antikythera Mechanism is a 2000-year-old ancient Greek device used to predict multiple astronomical events and the four-year cycles of different sets of Olympic Games. It was recovered from a shipwreck in 1900–1901 and consisted of 82 corroded fragments, many of which have been shown to contain gearwheels. Various studies culminated in a thorough X-ray CT analysis in 2005 using a Bladerunner 450 kV microCT scanner made by X-Tek Systems (UK), now Nikon Metrology. The images and their subsequent analysis led to new discoveries about the construction of the device, and revealed inscriptions which act as an instruction manual, ^13,26–28 see Figure 3. It is now clear that the Mechanism is an extraordinary device that offers a mechanical manifestation of many of the Ancient Greek theories of mathematics and astronomy with a complexity that was not surpassed until the Middle Ages. ^28,29

Figure 3.

(a) is a photograph of Fragment A, the largest fragment of the Antikythera Mechanism (from en.wikipedia.org/wiki/Antikythera_mechanism, licensed under CC BY 2.5). (b) shows one X-ray CT projection of the Mechanism; (c) shows one reconstructed gearwheel; (d) shows reconstructed text and its interpretation (b, c and d are cropped from Pakzad et al ¹³ and are licensed under CC BY 4.0 (creativecommons.org/licenses).

X-ray microtomography has also been used with remarkable success to reveal writing in sealed documents. Unlike the Herculaneum scrolls, later manuscripts were usually written in an iron-based ink, making X-ray techniques more feasible. Intact scrolls have been imaged and then digitally unrolled, rendering the contents legible ³⁰ ; envelopes folded such that they cannot be opened – a practice known as ‘letterlocking’ – have been imaged, virtually infolded and read ³¹ ; and badly degraded cellulose acetate film stock has been imaged, unrolled and revealed as a lost episode of Morecambe and Wise. ³²

Multispectral and hyperspectral imaging

A wide range of spectral imaging techniques are used in medicine, for example to monitor brain activity ³³ and to discriminate between healthy and malignant tissue especially in surgery. ³⁴ Similar approaches have been used in heritage to identify and map pigments and to reveal otherwise invisible or illegible features.

Terminology in this field varies. Generally, multispectral imaging uses photography with wavelength-specific lighting and sometimes a filterwheel to exclude the illumination, allowing fluorescence to be detected ³⁵ while hyperspectral imaging uses a white light source and splits the detected light into its component spectra using a grating. ³⁶ Multispectral imaging tends to be used where geometrical accuracy is important, for example to improve legibility, whereas hyperspectral imaging tends to be used where calibrated spectra are required such as for pigment analysis. One of the earliest applications of spectral imaging in heritage was to examine the Dead Sea Scrolls ³⁷ which revealed new characters that were previously illegible and allowed text that had been transferred between sheets to be identified and interpreted.

Multispectral imaging was used to reveal details of drawings by Leonardo da Vinci. ³⁸ His Studies of horses and horses' heads (RCIN 912285 in the Royal Collection) is a c1490 metalpoint drawing. Metalpoint was a technique used in the Renaissance for drawing fine lines using a silver or lead stylus on paper prepared with an abrasive surface. When viewed under normal room light (Figure 4a), some details of the horses at the top of the page are visible. However, when the sheet was illuminated with ultraviolet LED lighting and imaged through a long-pass filter which excluded the illumination light and any wavelengths shorter than red light, much more detail was visible especially of the lower part of the drawing (Figure 4b). The implications of this are still uncertain, but it might suggest that Leonardo used different styluses for the top and bottom halves of the drawing.

Figure 4.

The image on the left shows a silverpoint drawing by Leonardo Da Vinci under room lighting and that on the right is the same drawing, illuminated with ultraviolet light with a red filter. Royal Collection Trust / ^© His Majesty King Charles III 2023 / Cerys Jones.

Hyperspectral imaging is often used for calibrated spectroscopic imaging from which quantitative spectra can be obtained and then analysed further. ³⁹ It is most commonly used for pigment analysis ⁴⁰ but, when combined with multivariate analysis, the spectra can be used to predict other parameters that might not be thought of as naturally associated with spectral changes such as degree of polymerisation, which is a measure of the integrity of cellulose. ⁴¹ Machine learning and similar techniques are now offering new approaches to hyperspectral image analysis. ⁴²

Medical imaging techniques rarely applied to heritage

Ultrasound

The second most common medical imaging method according to the Diagnostic Imaging Dataset Statistical Release is ultrasound. However, it is rarely used in heritage imaging because of the need for a gel to couple the ultrasound source and detector to the medium which understandably is not usually seen as acceptable by the owner of an object. However, there are some applications that are emerging such as the use of ultrasound to examine waterlogged archaeological wood ⁴³ and to examine the strength of stone building materials. ⁴⁴ Photoacoustics – the use of light to generate ultrasound signals – is emerging as a method for monitoring laser cleaning of objects ⁴⁵ as well as to detect underdrawings in artworks. ⁴⁶

Magnetic resonance imaging

MRI is another flagship medical imaging modality which has not been heavily used in heritage. The mechanism of contrast in clinical MRI of course relies on free water which tends not to be commonly found in heritage samples. One exception is waterlogged wood which has been examined with a 3T clinical scanner using sequences used clinically. ⁴⁷ As in clinical imaging, complementary information was obtained when MRI and CT imaging were combined, but the resolution was not yet sufficient to allow the tree rings to be identified for non-invasive dating. MRI has also been use to image mummies using dedicated sequences, coils and gradients that were designed to be sensitive to low water content. ⁴⁸ However, MRI seemed to offer little advantage compared to CT.

Magnetic resonance spectroscopy and high field strength, small bore research magnets have been used in heritage ⁴⁹ but these have less direct relevance to medical imaging. ⁵⁰

Nuclear medicine

Nuclear medicine has not found a place in heritage imaging as there are few opportunities to deliver contrast agents and moreover, the aim of heritage imaging is usually to obtain images of the object’s current condition (analogous to anatomical imaging) rather than its response to a stimulus (analogous to functional imaging).

Endoscopy

Endoscopy is occasionally used in heritage, for example to examine Egyptian mummies if X-ray CT is equivocal. ⁵¹ Laparoscopy was used to examine a printed copy of De humanis corporis fabrica libri septem by Andreas Vesalius (1514–1564) held by UCL Special Collections. An anatomical diagram in the second edition (1555) is printed as a fugitive sheet – it is intended to be removed, cut up and reassembled into a ‘pop-up’ three-dimensional anatomical diagram. The cut-out pop-up fragments appear delicate but actually feel surprisingly substantial. A foetal laparoscope was used to image beneath these flaps to search for signs of additional support. A video was acquired, ⁵² one frame of which is shown in Figure 5. This shows the multilayered structure of the pop-up supports. Black dots can be seen which are hair follicles, showing that this is parchment rather than paper and suggesting that an older manuscript was re-used when the Vesalius was purchased and the pop-up anatomical diagram constructed. ⁵³

Figure 5.

One frame from a laparoscopy video showing parchment support beneath the pop-up flaps of De humanis corporis fabrica libri septem by Vesalius (https://doi.org/10.5522/04/8224085.v1). The video is licenced under CC BY 4.0 (creativecommons.org/licenses/by/4.0/).

Heritage imaging techniques rarely applied to medicine

3D surface imaging

One major area of heritage imaging that has little parallel in medical imaging is that of 3D surface imaging. This could use laser scanning, photogrammetry (where photographs taken using multiple camera positions are combined to produce a 3D surface model) or Reflectance Transformation Imaging (where the camera is held fixed and photographs are taken using multiple flash positions). Often combined with 3D printing, these offer powerful, yet low-cost methods for recording objects at risk of damage and engaging the public with heritage objects. ⁵⁴ Many interactive examples can be seen on the website sketchfab.com. ⁵⁵ The closest analogue in medical imaging is likely to be VisionRT ⁵⁶ and similar camera-based systems for aligning and monitoring patience positioning in radiotherapy.

These methods can record the surface geometry of a static object and offer excellent opportunities for interactive visualisation, but are not usually considered to be quantitative. One exception is a study of the Great Parchment Book, a seventeenth century record of landholding in Northern Ireland, which was damaged by fire in 1786, rendering the parchment sheets fragile, severely distorted and illegible. The book was imaged by photogrammetry and then a computational reconstruction pipeline developed to virtually flatten each sheet. ⁵⁷ The original and flattened images are now available online with a transcription ⁵⁸ and The Great Parchment Book was subsequently inscribed into UNESCO’s Memory of the World Register.

Accelerator and particle-based analysis and imaging

Synchrotrons have long been used in heritage studies where the high flux and tuneable energy allows for high chemical sensitivity using a range of spectroscopic techniques ⁵⁹ with many of the major international facilities offering access to heritage scientists. A series of studies, for example, has used the Diamond Light Source and other synchrotrons to examine objects recovered from the Tudor warship Mary Rose. ⁶⁰ X-ray absorption spectroscopy was used to track the oxidisation of iron and sulphur by bacteria in the ship’s structural timbers, ⁵⁶ X-ray diffraction and fluorescence were used to determine mechanisms for the post-excavation surface corrosion of iron cannonballs, ⁶¹ and X-ray micro CT of small fragments of cannonball have begun to reveal 3D corrosion mechanisms, ⁶² with implications for conservation.

One of the most remarkable secrets of heritage science is that since 1988 there has been a particle accelerator dedicated to the study of cultural heritage installed underneath the Louvre Museum. ^63,64 AGLAE, the Accélérateur Grand Louvre d’analyse élémentaire, can accelerate protons and α particles to 2 MV and has been used for a variety of particle beam examinations including particle induced X-ray emission (PIXE). A recent example of work with AGLAE was a study of the stained glass in the Sainte-Chapelle in Paris, which revealed the elemental composition of different colours of glass and gave some insight into their manufacture. ⁶⁵

The full range of neutron-based analysis methods have been used to study ceramic, metal and organic heritage samples. A study of bronze statuettes from the Rijksmuseum in Amsterdam, for example, was much more successful using neutron radiography, neutron tomography and neutron activation than the equivalent X-ray based techniques. ⁶⁶ Neutrons are even able to detect organic residues in the presence of X-ray dense metals. For example, X-ray, neutron and terahertz tomography have been used to investigate a sealed ancient Egyptian pot, ⁶⁷ in which terahertz imaging identified the presence of unknown content, X-ray CT gave information about the construction and condition of the container and neutron tomography suggested the presence of an organic stopper and seeds.

Another striking application of heritage imaging is the use of cosmic muons to detect and image voids within the massive ancient Egyptian pyramids. ^68,69 In a series of studies, researchers have placed muon detectors inside and around a pyramid and used them to detect variations in the constant flux of muons that result from the interactions of cosmic rays with the atmosphere. They detected the known chambers and also identified void regions that were previously unknown.

Multimodal imaging and image processing

As in medical imaging, ⁷⁰ some of the more recent advances in heritage imaging come from the fusion of different imaging modalities especially when combined with sophisticated image processing and visualisation techniques which increasingly rely on machine learning.

There are many examples of state-of-the-art imaging projects that combine multimodal imaging with image processing. One recent example is Operation Night Watch, a major research project studying Rembrandt’s The Night Watch (1642, oil on canvas, 378× 453 cm) by the Rijksmuseum. It involved extremely high resolution (925,000 by 775,000 pixel) photography, hyperspectral imaging in the visible and near infrared ranges, X-ray fluorescence mapping, X-ray diffraction mapping, optical coherence tomography and 3D imaging as well as a range of non-imaging examinations. ^71–73 Machine learning algorithms were developed to recreate missing parts of the painting. This highly multidisciplinary project included new approaches to public engagement and led to new understanding of the condition of the painting as well as offering insight into pigments, preparatory sketches, changes in composition and the painter’s techniques.

Conclusion

Medical imaging and heritage imaging have similarities in the scales of the objects under examination and the fact that both patients and heritage objects are both fragile and unique. In both cases, imaging forms one important part of the diagnostic information but is supplemented by information and expertise from elsewhere. Both have progressed rapidly with the development of computer power.

There are differences. Medical imaging has a relatively small number of vendors and professional bodies which enables coordination leading to common policies and standards such as DICOM. There has been a lot of effort into creating equivalent paradigms for sharing heritage image data, often under the banner of FAIR data (data which is Findable, Accessible, Interoperable and Reusable ⁷⁴ ) but this remains a significant obstacle. Projects such as Beyond 2022, ⁷⁵ an attempt to create a virtual reconstruction of the Public Record Office of Ireland, which was destroyed during the Irish Civil War in 1922, are made much more challenging given the lack of standard data formats.

It can also be difficult to move objects to a scanner. Different museums and archives have different policies, but some museums will not allow objects off-site, meaning that only portable imaging methods can be deployed. This reduces the knowledge that can be gained from these objects and also reduces the opportunities for collaboration.

Like medical imaging, heritage imaging is inherently highly multidisciplinary. Imaging specialists need to work with curators, who understand the history and context of an object, and conservators who are experts in its condition and preservation. Interpretation might require collaboration with historians, archivists, librarians and other professionals. There is not yet an equivalent tradition to the medical multidisciplinary team meeting.

It is hard to predict how heritage imaging will develop. The rapid growth of machine learning will have its impact in heritage imaging just as much as in the rest of science and society. New technologies, such as bench-top X-ray sources, will mean that it becomes possible to image a wider range of objects in situ. Perhaps the biggest change will result from the ubiquity of mobile phone cameras which will means that heritage imaging becomes democratised – not only are more images being taken, but the diversity of people taking photographs is expanding, perhaps leading to the recognition of new types of heritage that is of interest to communities that have previously been underrepresented.