Pitfalls of defacing whole-head MRI: re-identification risk with diffusion models and compromised research potential

Abstract

Defacing is often applied to head magnetic resonance image (MRI) datasets prior to public release to address privacy concerns. The alteration of facial and nearby voxels has provoked discussions about the true capability of these techniques to ensure privacy as well as their impact on downstream tasks. With advancements in deep generative models, the extent to which defacing can protect privacy is uncertain. Additionally, while the altered voxels are known to contain valuable anatomical information, their potential to support research beyond the anatomical regions directly affected by defacing remains uncertain. To evaluate these considerations, we develop a refacing pipeline that recovers faces in defaced head MRIs using cascaded diffusion probabilistic models (DPMs). The DPMs are trained on images from 180 subjects and tested on images from 484 unseen subjects, 469 of whom are from a different dataset. To assess whether the altered voxels in defacing contain universally useful information, we also predict computed tomography (CT)-derived skeletal muscle radiodensity from facial voxels in both defaced and original MRIs. The results show that DPMs can generate high-fidelity faces that resemble the original faces from defaced images, with surface distances to the original faces significantly smaller than those of a population average face (p < 0.05). This performance also generalizes well to previously unseen datasets. For skeletal muscle radiodensity predictions, using defaced images results in significantly weaker Spearman’s rank correlation coefficients compared to using original images (p ≤ 10⁻⁴). For shin muscle, the correlation is statistically significant (p < 0.05) when using original images but not statistically significant (p > 0.05) when any defacing method is applied, suggesting that defacing might not only fail to protect privacy but also eliminate valuable information. We advocate two solutions for data sharing that comply with privacy: 1) share skull-stripped images along with measurements of facial and cranial features extracted before skull-stripping for public access, while acknowledging that this approach inherently compromises many research potentials; or 2) share the unaltered images with privacy enforced through policy restrictions.

1. Introduction

The practice of defacing whole-head MRI has become increasingly widespread. This trend is driven by multiple factors, including improved image quality, enhanced 3D reconstruction capabilities, more powerful facial recognition techniques, growing efforts in data sharing, and increased emphasis on data privacy due to public awareness and institutional or governmental requirements. High-resolution structural MRIs can reveal facial features that can potentially be used to identify individuals [1,2]. This type of information is protected by regulations such as the Health Insurance Portability and Accountability Act (HIPAA) in the United States and General Data Protection Regulation (GDPR) in Europe. Due to these regulations, and despite variations in specific requirements by each Institutional Review Board (IRB), more entities are applying defacing techniques to MRI datasets before sharing. For instance, the UK Biobank [3] uses FSL_deface [4] for T1-weighted (T1w) and T2-weighted (T2w) MRIs. Similarly, the Human Connectome Project (HCP) [5] datasets use Face_Masking [6] for T1w and T2w MRIs.

There are concerns about defacing MRI. Thorough discussion in the literature has brought three key issues to light. First, defacing itself can fail in two ways: it can be too conservative, failing to remove enough facial features and leaving the MRIs recognizable by facial recognition software, or it can be too aggressive, altering voxel intensities beyond the facial regions or, worse, within the brain [7–9]. Second, defacing can also cause failures in other processing pipelines, including, but not limited to image registration [10], whole-brain segmentation [7,11], white matter lesion segmentation [12], and glioblastoma segmentation [12]. Third, defacing has impact on the reproducibility of analyses [7,8,11–13]. In addition to these known issues, new concerns have emerged with recent progress of deep generative models, e.g., the diffusion probabilistic models (DPMs), as these emerging technologies might be able to recover faces from defaced images. If true, this capability would challenge the rationale of applying defacing in the context of modern AI advancements. Additionally, non-brain regions in whole-head MRIs contain valuable information that could be used for various studies [14–22]. Defacing removes or alters these voxels, thereby eliminating this research potential and limiting the scope of possible scientific inquiries.

In this paper, we delve into the two emerging concerns (Fig. 1). First, we show that DPMs can generate 3D high-resolution MRIs with realistic faces that resemble the original faces from defaced images, demonstrating a potential malicious privacy attack. Second, we investigate a specific example of the valuable information contained within facial voxels that are typically removed by defacing. Specifically, we perform skeletal muscle radiodensity predictions using facial voxels in original and defaced MRIs and compare the discrepancy in performance.

Fig. 1.

There are pitfalls of defacing, a technique used to alter facial voxels in whole-head MRIs to protect privacy. First, with deep generative models such as diffusion probabilistic models, it is possible to synthesize MRIs with realistic faces, which closely resemble the original faces, from defaced MRIs. This capability poses a re-identification risk, thus questioning the efficacy of defacing in protecting privacy. Second, facial and other non-brain voxels in whole-head MRIs contain valuable anatomical information. For instance, this information could be used to study correlations between head and body measurements using paired head MRI and body CT data. The alteration of these voxels results in information loss, thereby compromising such research potentials. The experiments in this paper are designed to showcase these two pitfalls.

1.1. High-resolution 3D refacing of defaced MRI

Refacing can be considered an inpainting task, where a generative model, trained to approximate the probability distribution of the data, imputes missing regions by sampling from the probability distribution based on the observed regions. In medical image analysis, previous studies have demonstrated the possibility to extend imaging boundaries to restore anatomical structures truncated by restricted field-of-view [23,24], adding value to downstream clinically relevant applications as a normalization step [25]. Comparatively, the application of generative models for refacing MRI has been scarce. Abramian et al. used CycleGAN to generate refaced sagittal slices from defaced sagittal slices [26]. Similarly, Xiao et al. used a pixel-constrained convolutional neural network (CNN) for 2D sagittal refacing at a lower resolution [27]. Both of these works performed refacing in 2D, which is insufficient to pose a significant privacy threat. As the authors of these papers acknowledged, 3D generative models and reconstruction are needed for more effective refacing [26,27]. Molchanova et al., on the other hand, used a 3D conditional generative adversarial network for refacing [28]. However, their goal was to provide another defacing (de-identification) tool that replaces the original face with a generic face (similar to what mri_reface [8] does), rather than to demonstrate a potential malicious privacy attack. Consequently, the faces in their refaced images do not resemble the original faces closely enough to be considered a threat.

Our goal is to explore the possibility of recovering faces from defaced images using DPMs, which could potentially be used in malicious privacy attacks, rendering the practice and efforts of defacing ineffective. We input defaced MRI into cascaded DPMs to generate high-resolution 3D images with faces that resemble the original faces prior to defacing. We then measure the similarity between the refaced faces and the original faces to assess the re-identification risk.

1.2. Predicting skeletal muscle radiodensity from original vs. defaced facial voxels

Non-brain regions in whole-head MRI have been shown to contain valuable information in various contexts. For instance, Hitomi et al. demonstrated that gadolinium leakage into ocular structures on post-contrast FLAIR MRI is associated with acute stroke [14]. Wiseman et al. showed that the axial length of the eye can be measured from MRI, providing a proxy for eye size in the absence of biometry [15]. Mi et al. quantified temporalis muscle area from MRI, providing a sarcopenia-related metric associated with oncological outcomes [16]. These examples represent only a small fraction of the anatomical and clinical insights that could be derived from non-brain regions. Other regions, such as the meninges [17], tongue [18], sinuses [19], ear canals [20], marrow [21], and subcutaneous fat [21,22], also hold significant research potential that could be compromised by defacing.

We explore the value of non-brain regions from a new perspective by predicting abdomen, thigh, and shin muscle radiodensities, measured from body CT data, from facial voxels in head MRI. Specifically, we include subjects with paired body CT and T1w MRI and apply four defacing methods to the T1w images. Using the remaining facial voxels from each type of defaced images, we train separate 3D residual neural networks (ResNet) to predict the muscle radiodensity of the abdomen, thigh, and shin, respectively. To examine the impact of information loss caused by defacing, we compare the correlation between predicted and ground truth values derived from defaced images with the correlation derived from original (non-defaced) images.

2. Cascaded diffusion models for refacing defaced MRI

We consider a two-stage refacing pipeline that takes in defaced MRIs and generates high-resolution 3D MRIs with faces intended to resemble the original faces. We measure the similarity between the generated images and the original images from both a face re-identification risk perspective and an image quality perspective.

2.1. Model architecture

Inspired by the cascaded diffusion models described by Ho et al. [29], we use a two-stage approach to achieve high-resolution 3D image generation (Fig. 2). This approach offers advantages over directly training a single 3D model in high resolution, including computational efficiency.

Fig. 2.

Given a defaced T1-weighted (T1w) MRI, our refacing pipeline generates a T1w image with a face. The pipeline consists of two stages. In stage 1, a 3D diffusion model, conditioned on the downsampled defaced image, generates a low-resolution refaced image. In stage 2, a 2.5D diffusion model, conditioned on the high-resolution defaced image and the up-sampled low-resolution refaced image from stage 1, generates a high-resolution refaced image. This is done in a slab-by-slab manner, where each slab consists of a stack of axial slices. To mitigate border effects, adjacent slabs have overlapping slices to share anatomical context. Finally, the high-resolution slabs produced by stage 2 are merged to form the complete high-resolution 3D refaced image.

Our pipeline consists of two diffusion models, one for each stage, both of which are Denoising Diffusion Implicit Models (DDIM) [30] (Fig. 2). The stage-1 model is 3D and imputes the missing facial voxels, conditioned on the downsampled defaced image, to generate a low-resolution refaced image. The stage-2 model is 2.5D and performs super-resolution enhancement, conditioned on the up-sampled, low-resolution refaced image from stage-1 and the defaced image. This is done in a slab-wise manner, where each slab consists of a stack of 8 axial slices. To mitigate border effects commonly observed in 2.5D approaches, adjacent slabs have 4 overlapping slices, allowing them to share anatomical context. The high-resolution slabs from stage-2 are then merged to form the complete high-resolution 3D refaced image. Our implementation of the diffusion models is adapted from Song et al.’s implementation [30], available at https://github.com/ermongroup/ddim. We slightly modify the DDIM denoising scheme to let the model to predict the ground-truth image $x_0$ at each denoising step $t$ instead of predicting the noise ${ϵ}_{\theta }^{\left(t\right)}\cdot {ϵ}_{\theta }^{\left(t\right)}$ is estimated using the predicted $x_0$ following the first section of formula (12) in Song et al. (2021) [30]. Then, we follow formula (12) to infer the less noisy image $x_{t-1}$ for the next denoising step. This modification avoids the challenges in converging during the denoising procedure, which is observed when directly applying DDIM denoising scheme.

2.2. Data

Since most MRI defacing tools work for T1w MRI, we use T1w images for our experiments. We include 21, 179, and 469 subjects from the Kirby-21 [31], OASIS3 [32], and BLSA datasets [33], respectively, with one T1w image per subject. Of the 200 subjects from Kirby-21 and OASIS3, 180 were used to train the diffusion models, while 20 were reserved for internal testing. Five of these internal testing subjects were excluded because their faces were already not visible even before defacing. The 469 BLSA subjects, whose faces were visible, serve as an external testing set. We apply FSL_deface [4], MRI_Deface [34], Pydeface [35], and Quickshear [36] to the T1w images. In total, there are 720 (180 subjects * 4 defacing methods) pairs of training data.

To examine whether the generated faces are any better than a population average face, regarding the similarity with the original faces, we run mri_reface [8], which replaces the original face with a population average face linearly aligned to the original face.

IRB of Vanderbilt University waived ethical approval for de-identified access of the human subject data.

2.3. Similarity metrics

To quantitatively measure the similarity between faces in two images, we use the mean absolute surface distance (MASD). First, we generate a binary mask of the head in each image using Otsu’s thresholding [37] followed by morphological operations. To focus the comparison on the face, we crop out 10 axial slices at the inferior to exclude the neck and the posterior half of the head to exclude the back of the head. Using these binary masks, we apply the marching cubes algorithm to extract a mesh of vertices representing the surface of each face. We then use a K-dimensional tree (KDTree) [38,39] to query the nearest neighbor on the surface of one face for every vertex on the other face and compute the absolute distance. The MASD between the two faces is obtained by averaging these distances.

To measure the similarity of two images from an image quality perspective, we report the structural similarity index measure (SSIM) and peak signal-to-noise ratio (PSNR). SSIM and PSNR are computed for two regions: 1) the whole head, defined by the binary mask of the head in the original image; and 2) the area that has been changed (Supplementary Material), specifically the intersection of areas removed by defacing, altered by mri_reface, and imputed by our refacing. The masks used for calculating these metrics were visually inspected for quality assurance.

3. Estimation of skeletal muscle radiodensity from facial voxels

We estimate CT-derived abdomen, thigh, and shin skeletal muscle radiodensities from facial voxels in T1w MRI. We perform these estimations using either original or defaced T1w images. By comparing the results, we evaluate the impact of defacing on the accuracy of estimating body measurements from head MRI.

3.1. Paired MRI-CT data

We include subjects from BLSA [33] who have paired whole-head T1w MRI and single-slice CT data of the abdomen, thigh, and shin. For the T1w images, we apply defacing methods including FSL_deface [4], MRI_Deface [34], Pydeface [35], and Quickshear [36] to generate four types of defaced images from the original images. As a baseline reference, representing the most aggressive defacing where all facial voxels are removed, we apply skull-stripping using segmentation masks produced by SLANT-TICV [40] to obtain skull-stripped images from the original images. Each original T1w image is affinely registered to the NMRI225 template [41], a T1w MRI template with a large field of view that covers the whole head. The same transformation is applied to the corresponding defaced and skull-stripped images to align them with the registered original image. To focus solely on facial voxels, we crop out the posterior two-thirds of the head to exclude the back of the head and mask out the brain to further isolate the facial voxels. Examples are available in the Supplementary Materials. The intensities of the remaining voxels in the T1w image are normalized by subtracting the mean and dividing by the standard deviation. For the single-slice CT images of the abdomen, thigh, and shin, we use deep learning-based methods to segment the muscle in each body part [42–44]. We then calculate the mean voxel intensity within the segmented muscle mask for each body part to quantify the muscle radiodensity (in Hounsfield Units) of the abdomen, thigh, and shin, respectively.

We perform quality assurance by visually inspecting the processed data and exclude MRI-CT pairs where any processing step failed, ensuring that all groups (image types) have the same number of valid data points for the regression experiments. In the end, we have 948 BLSA subjects with paired MRI and abdomen CT data. These subjects are split into 616 for training, 142 for validation, and 190 for testing. Additionally, we have 990 BLSA subjects with paired MRI and thigh and shin CT data, split into 643 for training, 148 for validation, and 199 for testing.

IRB of Vanderbilt University waived ethical approval for de-identified access of the human subject data.

3.2. Regressing out age and sex

Age and sex are two confounding factors that are strongly correlated with skeletal muscle radiodensity. Since our goal is to investigate whether facial voxels in T1w images contain information specific to abdomen, thigh, and shin muscle radiodensity, we regress these two confounding factors out of the muscle radiodensity measurements. Specifically, we fit three linear mixed-effects models [45], one for each body part (abdomen, thigh, or shin), using the formula:

$$y_{i,j}={\beta }_0+{\beta }_1\times {\mathrm{A}\mathrm{g}\mathrm{e}}_{i,j}+{\beta }_2\times {\mathrm{S}\mathrm{e}\mathrm{x}}_i+r_i+{\epsilon }_{i,j}$$ (1)

where $y_{i,j}$ is the muscle radiodensity of subject $i$ at visit $j,{\mathrm{A}\mathrm{g}\mathrm{e}}_{i,j}$ is the age of subject $i$ at visit $j$, ${\mathrm{S}\mathrm{e}\mathrm{x}}_i$ is the sex of subject $i$, $r_i$ represents the random intercept for each subject, and ${\epsilon }_{i,j}$ is the residual error of the linear mixed-effects model. We then subtract the predicted muscle radiodensity ${\widehat{y}}_{i,j}$ by age and sex from the observed muscle radiodensity to obtain the residuals, using the equation:

$$y_{i,j}-{\widehat{y}}_{i,j}=y_{i,j}-{\beta }_0-{\beta }_1\times {\mathrm{A}\mathrm{g}\mathrm{e}}_{i,j}-{\beta }_2\times {\mathrm{S}\mathrm{e}\mathrm{x}}_i$$ (2)

These residuals will be the target of our regression models, which are introduced in the next section.

3.3. Regression model

We use 3D residual neural networks (ResNet) [46], specifically ResNet10, initialized with weights pretrained on large medical image datasets [47]. The input to the ResNet is one of six types of cropped 3D images containing only facial voxels. Each image is intensity-normalized before being fed into the ResNet. The target of the ResNet is the muscle radiodensity in one of the body parts (abdomen, thigh, or shin) with age and sex regressed out using equation (2). The values are normalized using the mean and standard deviation computed from the training set. For each input-target pair, we train a separate ResNet. In total, there are 18 (6 types of images * 3 body parts) ResNets. All ResNets are trained using the same hyperparameters. To mitigate the models’ bias towards subjects with more samples, the probability of selecting each sample is normalized based on the number of samples available for each subject.

4. Results

Fig. 3 shows a testing subject from Kirby-21 whose face was removed by MRI_Deface and subsequently recovered by our DPMs. The DPMs-refaced face closely resembles the original face, based on visual inspection, whereas the linearly aligned population average face appears less similar to the original face. This visual observation is consistent with the MASD measurements: for DPMs-refaced face, the MASD to the original face is 0.363 mm, compared to 0.792 mm for the linearly aligned population average face. White patches were placed over the eyes in the screenshots of the renderings to obscure subject identity, but all computations were performed without such obscuring. For the same subject, we present 3D renderings of the images refaced from each type of defaced image by our DPMs, ordered by ascending MASD to the original face (Fig. 4). The MASD generally aligns with the perceptual dissimilarity between the two faces. On skull-stripped images, the DPMs (retrained on skull-stripped images) fail to reconstruct faces that resemble the original ones (Figs. 5 and 6).

Fig. 3.

For an example subject, 3D renderings of the faces in the original image, image defaced by MRI_Deface, image refaced from the defaced image by our DPMs, and image processed with mri_reface (which replaces the original face with a population average face) are presented in each row. The MASD is computed between the original face and the face in each image type. A smaller distance corresponds to a higher similarity to the original face.

Fig. 4.

For the same subject in Fig. 3, 3D renderings of the faces in the DPMs-refaced images generated from each type of defaced image are presented alongside the original face. The images are ordered by the MASD to the original face. As the distance increases, the face appears perceptually more different from the original face.

Fig. 5.

From skull-stripped images, the diffusion probabilistic models (DPMs) failed to generate faces that resemble the original faces.

Fig. 6.

On the external testing set (N = 469), faces generated by our DPMs from images defaced using Pydeface, FSL_deface, Quickshear, and MRI_Deface show significantly lower mean absolute surface distances (MASDs) to the original faces, compared to the linearly aligned population average face produced by mri_reface, indicating higher facial similarity. In contrast, faces generated by DPMs from skull-stripped images show significantly higher MASDs, indicating lower facial similarity. Statistical significance was assessed using the Wilcoxon signed-rank test; “****” indicates p ≤ 0.0001.

On the internal testing set of 15 subjects, MASDs between DPMs-refaced faces and original faces are significantly smaller compared to MASDs between population average faces and original faces (Table 1). Among the DPMs-refaced faces, those generated from MRI_Deface-defaced images have the smallest MASD (0.34 mm), while those generated from Quickshear-defaced images have the largest MASD (0.63 mm). Similarly, on the external testing set of 469 subjects, MASDs between DPMs-refaced faces and original faces are significantly smaller compared to MASDs between population average faces and original faces. Among the DPMs-refaced faces, those generated from MRI_Deface-defaced images have the smallest MASD (0.38 mm), while those generated from Pydeface-defaced images have the largest MASD (0.67 mm). In general, DPMs-refaced images have higher PSNR and SSIM in the whole head area, but lower PSNR and SSIM in the facial area removed by defacing, compared to mri_reface processed images (Tables 2 and 3).

Table 1.

Mean absolute surface distance between the original face and the altered (or synthetic) face.

		Mean absolute surface distance to the original face (mm)b			p-valuec
Testing set	Defaced by	Defaced	Refaced with a population averagea,c	Refaced with DPMsc
Internal (N = 15)	FSL_deface	3.65 [3.27, 4.05]	0.81 [0.69, 0.93]	0.60 [0.53, 0.67]	0.026
	MRI_Deface	2.35 [1.91, 2.83]	0.81 [0.69, 0.93]	0.34 [0.28, 0.40]	0.001
	Pydeface	6.01 [5.69, 6.33]	0.81 [0.69, 0.93]	0.61 [0.53, 0.68]	0.018
	Quickshear	5.54 [4.89, 6.20]	0.81 [0.69, 0.93]	0.63 [0.55, 0.71]	0.030
External (N = 469)	FSL_deface	2.97 [2.91, 3.03]	0.71 [0.68, 0.73]	0.63 [0.61, 0.64]	≪0.001
	MRI_Deface	2.19 [2.16, 2.22]	0.71 [0.68, 0.73]	0.38 [0.36, 0.39]	≪0.001
	Pydeface	6.01 [5.95, 6.06]	0.71 [0.68, 0.73]	0.67 [0.65, 0.69]	≪0.001
	Quickshear	5.01 [4.85, 5.18]	0.71 [0.68, 0.73]	0.62 [0.60, 0.64]	≪0.001

^aThe population average face (linearly aligned with the original face) is applied by mri_reface.

^bMean value and 95 % confidence intervals (in the square bracket) are estimated with bootstrapping (n = 1000).

^cWilcoxon signed-rank test is performed between population average derived distances and DPMs derived distances.

Table 2.

Peak signal-to-noise ratio (PSNR) of defaced, mri_reface, and DPMs-refaced images.

		Peak signal-to-noise ratio (PSNR)a
		whole head			face (area removed by defacing)
Testing set	Defaced by	defaced	mri_reface	DPMs-refaced	defaced	mri_reface	DPMs-refaced
Internal (N = 15)	FSL_deface	15.40 [14.87, 15.91]	20.23 [19.78, 20.68]	22.44 [21.80, 23.05]	4.69 [4.48, 4.90]	12.09 [11.76, 12.43]	11.81 [11.57, 12.06]
	MRI_Deface	13.96 [13.21, 14.63]	20.23 [19.78, 20.68]	21.11 [20.49, 21.65]	4.63 [4.28, 4.94]	12.90 [12.48, 13.36]	11.29 [11.01, 11.60]
	Pydeface	14.34 [13.59, 15.08]	20.23 [19.78, 20.68]	20.89 [20.18, 21.55]	5.33 [4.99, 5.68]	12.38 [12.01, 12.79]	12.01 [11.69, 12.35]
	Quickshear	17.05 [16.34, 17.73]	20.23 [19.78, 20.68]	23.07 [22.44, 23.71]	5.58 [5.25, 5.91]	11.94 [11.60, 12.28]	11.96 [11.67, 12.25]
External (N = 469)	FSL_deface	16.07 [15.97, 16.17]	20.39 [20.30, 20.47]	23.02 [22.91, 23.13]	4.51 [4.47, 4.54]	11.43 [11.37, 11.49]	11.34 [11.29, 11.38]
	MRI_Deface	13.48 [13.44, 13.52]	20.39 [20.30, 20.47]	20.73 [20.68, 20.78]	4.21 [4.17, 4.24]	12.62 [12.55, 12.68]	11.25 [11.20, 11.29]
	Pydeface	13.38 [13.28, 13.49]	20.39 [20.30, 20.47]	19.16 [19.02, 19.30]	4.88 [4.84, 4.92]	12.29 [12.22, 12.35]	11.54 [11.49, 11.59]
	Quickshear	16.82 [16.67, 16.97]	20.39 [20.30, 20.47]	22.96 [22.79, 23.13]	4.92 [4.88, 4.97]	11.67 [11.60, 11.74]	11.41 [11.35, 11.47]

^aMean value and 95 % confidence intervals (in the square bracket) are estimated with bootstrapping (n = 1000).

Table 3.

Structural similarity index measure (SSIM) of defaced, mri_reface, DPMs-refaced images.

		Structural similarity index measure (SSIM)a
		whole head			face (area removed by defacing)
Testing set	Defaced by	defaced	mri_reface	DPMs-refaced	defaced	mri_reface	DPMs-refaced
Internal (N = 15)	FSL_deface	0.92 [0.91, 0.93]	0.90 [0.89, 0.90]	0.93 [0.92, 0.94]	0.00 [0.00, 0.00]	0.28 [0.27, 0.30]	0.17 [0.16, 0.18]
	MRI_Deface	0.87 [0.86, 0.88]	0.90 [0.89, 0.90]	0.90 [0.89, 0.91]	0.00 [0.00, 0.00]	0.38 [0.36, 0.39]	0.20 [0.19, 0.21]
	Pydeface	0.87 [0.86, 0.88]	0.90 [0.89, 0.90]	0.90 [0.89, 0.91]	0.02 [0.02, 0.02]	0.24 [0.21, 0.26]	0.19 [0.18, 0.20]
	Quickshear	0.92 [0.92, 0.93]	0.90 [0.89, 0.90]	0.93 [0.93, 0.94]	0.00 [0.00, 0.00]	0.20 [0.17, 0.22]	0.16 [0.15, 0.17]
External (N = 469)	FSL_deface	0.93 [0.93, 0.93]	0.91 [0.90, 0.91]	0.94 [0.94, 0.94]	0.00 [0.00, 0.00]	0.30 [0.30, 0.31]	0.19 [0.19, 0.19]
	MRI_Deface	0.87 [0.87, 0.87]	0.91 [0.90, 0.91]	0.90 [0.90, 0.90]	0.01 [0.01, 0.01]	0.40 [0.40, 0.41]	0.24 [0.24, 0.24]
	Pydeface	0.84 [0.84, 0.84]	0.91 [0.90, 0.91]	0.86 [0.86, 0.87]	0.02 [0.02, 0.02]	0.30 [0.29, 0.30]	0.21 [0.21, 0.21]
	Quickshear	0.93 [0.93, 0.93]	0.91 [0.90, 0.91]	0.94 [0.93, 0.94]	0.00 [0.00, 0.00]	0.23 [0.23, 0.24]	0.18 [0.17, 0.18]

^aMean value and 95 % confidence intervals (in the square bracket) are estimated with bootstrapping (n = 1000).

For all three body parts, defacing results in significantly lower Spearman’s rank correlation coefficients between the predicted and ground truth values of muscle radiodensity (p ≤ 10⁻⁴) (Fig. 7). When predicting abdomen muscle radiodensity, the correlation is significant (p < 0.05, as shown by the 95 % confidence intervals) when using original images, but not significant (p > 0.05) when using defaced images by FSL_deface, MRI_Deface, and Pydeface. When predicting shin muscle radiodensity, the correlation is significant only when using original images (p < 0.05), while the use of any defacing method results in nonsignificant correlations (p > 0.05).

Fig. 7.

The correlation between predicted residuals of abdomen, thigh, and shin muscle radiodensity (measured from segmented ROI in CT data, in Hounsfield units, with age and sex regressed out using linear models) and ground truth values is stronger using original images compared to defaced or skull-stripped images. Examples of each image type are provided in the Supplementary Materials. The image types are arranged such that those with more facial voxels removed are placed on the right (e.g., skull-stripping), while those with fewer facial voxels removed are placed on the left (e.g., original). As more voxels are removed, the correlation between head and body becomes weaker and more difficult to capture. For instance, the correlations between predicted abdomen muscle radiodensity from FSL_deface, MRI_Deface, Pydeface, and skull-stripped images and the ground truth values are not statistically significant, as indicated by 95 % confidence intervals (CI) overlapping with 0. For shin muscle, the head-body correlation is statistically significant only when using original images. Bootstrapping (n = 1000) is used to estimate the mean and 95 % confidence intervals of the Spearman’s rank correlation coefficients. Statistical significance is indicated by “****” for p-value ≤10⁻⁴, based on the Wilcoxon signed-rank test.

5. Discussion

5.1. The cascaded DPMs can generate MRIs with high-fidelity faces from defaced MRIs

Our refacing pipeline based on DPMs can generate high-resolution 3D T1w MRIs with faces that resemble the original faces, from defaced images. We used MASD as a metric to quantify the similarity between two faces. This metric generally aligns with our perceptual impression of whether two faces belong to the same person (Fig. 4). The DPMs-refaced faces have a smaller MASD to the original faces compared to the linearly aligned population average face. The mechanism behind this capability is twofold. First, the DPMs learn a statistical prior of craniofacial anatomy from the training set. This process is related to the known ability of generative models to “memorize” and replicate features from their training data, a significant privacy concern in itself [48]. Second, and more importantly, the model uses the remaining, non-defaced portions of the skull and surrounding tissue as a strong condition to guide the generative process. This transforms the task into a conditional inpainting problem, where the learned anatomical prior is constrained by the unique anatomy of the individual subject. Therefore, the DPMs leverage the underlying anatomical information present in the MRIs, allowing the model to reconstruct the defaced regions in a way that is more personalized and accurate, reflecting individual anatomical features rather than just an averaged representation. Additionally, we found that the more aggressive the defacing, the more challenging the refacing becomes, as indicated by a larger MASD to the original faces (Fig. 4 and Table 1). On the skull-stripped images, the DPMs fail to reconstruct the faces (Figs. 5 and 6), providing strong evidence for this conditional mechanism, as the necessary anatomical context is absent. However, this does not necessarily make skull-stripping an ideal solution, because skull-stripping removes non-brain voxels that contain valuable information, as we will discuss in the next section.

On the other hand, the DPMs-refaced faces do not achieve higher (and often even lower) SSIM and PSNR compared to the population average face within the facial voxels (Tables 2 and 3). We attribute this to two main factors. First, the population average face applied by mri_reface was registered to the original faces, whereas the DPMs generate the face without any access to the original faces during inference. Consequently, the linearly aligned population average face aligns the internal structure of the face better compared to the DPMs-refaced faces. Second, the DPMs were trained on a relatively small dataset of only 180 subjects. It is reasonable to expect that using larger datasets that encompass a diverse range of cohorts for training would lead to higher SSIM and PSNR values for the refaced images. However, we refrained from using large-scale, multi-site datasets in this refacing-related project to avoid potential violations of the data use agreements.

5.2. Facial voxels removed by defacing contain valuable information

As mentioned in Section 1.2, studies have shown the value of non-brain regions in whole-head MRIs for various applications. These studies provide examples where defacing may compromise the feasibility of certain research, as the regions of interest are corrupted by defacing. To further illustrate the impact of defacing on studies beyond the regions directly altered, we explore the connections between MRI facial features and overall body compositions. We specifically focus on muscle radiodensity derived from CT as a metric of interest, as previous studies have shown its association with muscle fat infiltration, which is predictive of various health outcomes, including metabolic diseases and overall mortality [25]. Our findings indicate that predicting abdomen, thigh, and shin muscle radiodensities from facial voxels is feasible using original (pre-defaced) MRIs, as evidenced by significant correlations between the predicted and ground truth values for all body parts. However, when using defaced images, the correlations captured by the prediction models are significantly weaker and, in many cases, not statistically significant.

5.3. Future studies

First, to assess whether a lower MASD corresponds to a higher success rate of facial re-identification, a future study involving face recognition is needed, for example, using a standard face recognition algorithm (e.g., ArcFace [49]). Such a study will require approval from the IRB and informed consent from the human subjects to ensure ethical compliance. Second, we used DPMs as an example to demonstrate a potential malicious privacy attack through refacing because of their performance in high-fidelity image generation [29,30]. Other generative models may also be capable for similar purposes; however, comparing their performances is beyond the scope of this paper. Third, de-identification through defacing is just one of the privacy-preserving approaches for neuroimages. There are other techniques, such as encryption [50], summary statistics [51], decentralized computation [52], and federated learning [53], that could potentially be integrated to provide a more comprehensive privacy protection solution. Importantly, we note that there is no “once and for all” solution, and strategies should adapt as (both protectors’ and attackers’) techniques evolve [54].

6. Conclusion

These concerning results indicate that defacing might not only fail to protect privacy in the face of DPMs but also eliminate valuable information, thereby compromising research potential. We advocate two solutions for data sharing in compliance with privacy: 1) share skull-stripped images along with measurements of facial and cranial features extracted before skull-stripping (e.g., intracranial volume) for public access, while acknowledging that this approach inherently compromises many research potentials; or 2) share the unaltered images with privacy enforced through the use of policy restrictions.

Appendix A. Supplementary data

Supplementary data to this article can be found online at https://doi.org/10.1016/j.compbiomed.2025.111112.

Data sharing statement

Our models will not be released. Code is available upon request and agreement not to use or disseminate it for re-identification purposes.