The key role of absolute risk in the disclosure risk assessment of public data releases

We welcome Jarmin et al.’s (1) desire to engage with our article (2). We argued that absolute disclosure risk should be used to assess privacy risk. Jarmin et al. disagree and argue that differential privacy (DP) is superior. We find their arguments unpersuasive; our views are unaltered by their article. In addition, Jarmin et al. often muddled our article’s arguments. Interested parties should read our article to understand our arguments.

A major point unmentioned by Jarmin et al. is that individuals care about the absolute disclosure risk with the public release of data, which is needed to assess the benefits and costs of such a release. We illustrated this with a public health example. Jarmin et al. mention this example but do not make its point clear. In the context of data privacy, individuals may care about how much their privacy is reduced if they participate in a publicly released data set in the sense of relative disclosure risk. However, it should matter a great deal what the level of absolute disclosure risk is. That is, does the absolute disclosure risk change from 1 to 2 in a million or from 1 to 2 in a 100? Jarmin et al. fail to acknowledge what we noted ref. 3 established, namely that the DP criterion bounds relative disclosure risk, which is just the ratio of absolute disclosure risk to an intruder’s prior. As we note in our article, the latter result complicates the way the DP criterion controls absolute disclosure risk.

To bolster this point, we referenced an important article (4) which demonstrated it dramatically with an illustrative simulation exercise. The article showed that absolute disclosure risk “can be small even when [DP bounding] is relatively large” (p. 536) and provided an example demonstrating that “decreasing [DP’s] epsilon from ten to one did not noticeably reduce [absolute disclosure risk]” (p. 536). Our article concurred with the conclusion of ref. 4 that data stewards need to consider absolute disclosure risk directly when considering data release.

Jarmin et al. argue that DP’s management of relative disclosure risk is superior to focusing on controlling absolute risk because DP has convenient mathematical properties. One is composability (additivity of risk bounds across multiple queries); using absolute risk would require more difficult calculations. Another is that DP is independent of intruder knowledge, whereas absolute disclosure risk requires an assessment of that knowledge, which can be difficult. The technical points are correct. However, one should not choose a risk measure for mathematical convenience or ease of computation if it ignores the absolute risk of people’s identities being disclosed. Thus, we argue for using a risk measure that is calibrated to how much intruder knowledge exists and how it may change over time.

Jarmin et al. list 13 criteria against which risk measures should be assessed. Many seem vague and marginal to the issues at the heart of these decisions, chosen to highlight an advantage of DP not based on substantive importance.

We welcome continued discussion of these important issues for American society.