The counterfactual framework in Jarmin et al. is not a measure of disclosure risk of respondents

Jarmin et al. (1) suggest assessing disclosure risk by using a counterfactual method to compare the posterior-to-posterior probability of an inference with and without the target record. They argue that this methodology is superior to the absolute and relative risk assessment methodologies. The counterfactual method, as originally proposed in ref. 2, explicitly rejects the “with and without” target record comparison. Since the counterfactual formulation in ref. 1 uses this inappropriate comparison, their methodology is inextricably linked to differential privacy (DP) but without rigorous formalization, making it impossible to establish a privacy guarantee or prove it satisfies the desiderata.

What is clear is that, unlike the other two methodologies being considered in ref. 1, the counterfactual method does not measure risk to individual respondents. Rather, it assesses whether a protection algorithm satisfies a DP-like requirement. If it does, no one is at risk; if it does not, everyone is at risk. To illustrate this fallacy, consider a population where every respondent is identical and therefore protected against disclosure. The counterfactual methodology in ref. 1 would deem every respondent to be at risk (whereas the preferred option in ref. 2 would not). Having already adopted this measure for the controversial 2020 US Decennial Census (3), Jarmin et al. are attempting to impose a questionable standard by diktat.

The appendix in ref. 1 criticizes four studies based on methodological issues. We now address this criticism.

• (1)Ruggles and Van Riper (4) used a simple Monte Carlo simulation to estimate a baseline for evaluating the effectiveness of the Census Bureau’s database reconstruction experiment. Jarmin et al. argue that the simulation is invalid because the Census experiment included a previously undocumented rule that “a record in the reconstructed data can be assigned to at most one record in the confidential data.” However, it does not make much difference; if the simulation is modified to use each record no more than once, it remains the case that most of the reconstructed individuals have no match in the real population, and most of the matches that do occur would be expected purely by chance.
• (2)Muralidhar (5) was trying to show that the reconstruction approach used by the Census was unnecessarily complicated. Criticizing him for using a simpler schema than the one used by the Census is missing the very point of the paper.
• (3)Criticizing Francis (6) for not accurately predicting non-modal race/ethnicity is also missing the point. The very purpose of his paper was to show that race/ethnicity for individuals can be predicted accurately with knowledge only of the modal block value.
• (4)The criticism of Muralidhar and Domingo-Ferrer (7) rests on the false claim that they assume that suppression methods were used in the Summary File 1 in 2010 tabular data release. Nowhere in their paper do they make that assumption.

In summary, at no point does ref. 1 refute the key conclusion of these studies that in the 2010 Census reconstruction a) most of the matches were random and b) that the reconstruction is primarily due to generalizable inference rather than privacy-violating inference.