Increase of Uncertainty in Summed-Score-Based Scoring in Non-Rasch IRT

Abstract

Summed-score (SS)-based scoring in non-Rasch IRT allows for pencil-and-paper administration and is used in the Patient-Reported Outcomes Measurement Information System (PROMIS) alongside response-pattern-based scoring. However, this convenience comes with an increase in uncertainty (the increase) associated with SS scoring. The increase can be quantified through the relationship between Bayesian SS and RP scoring. Given an SS of s, the SS posterior is a weighted sum of RP posteriors, with weights representing the marginal probabilities of RPs. From this mixture, the SS score (SS posterior mean) is a weighted sum of RP posterior means, and its uncertainty (variance of the SS posterior) is decomposed into the uncertainty of RP scoring (the weighted sum of RP posterior variances) and the increase (variance of RP posterior means). Without quantifying the increase, PROMIS recommends RP scoring for greater accuracy, suggesting SS scoring as a second option. Using variance decomposition, we quantified the increases for two short forms (SFs). In one, the increase is very small, making SS scoring as accurate as RP scoring, while in the other, the increase is large, indicating SS scoring may not be a viable second option. The increase varies widely, influencing scoring decisions, and should be reported for each SF when SS scoring is used.

Introduction

Summed-score-based (SS) scoring is an attractive administrative method in Health-Related Patient-Reported Outcome (HR-PRO) studies within the context of non-Rasch item response theory (IRT) because it allows for paper-and-pencil administration, which is a widely used approach in clinical settings. Recognizing this importance, the Patient-Reported Outcomes Measurement Information System (PROMIS), which offers over 200 HR-PRO short forms (SFs) (126, 27, 16, and 39 SFs for Adult, Pediatrics, Early Childhood Parent-Report [ages 1–5], and Parent Proxy for Pediatric Patients [ages 5–17], respectively, as of 5/16/2024), has integrated SS scoring along with response-pattern-based (RP) scoring (Cella et al., 2007, 2010; “HealthMeasures,” 2024). Three administrative methods are available in PROMIS: Computer Adaptive Testing with RP scoring, fixed-length testing with RP scoring, and fixed-length testing with SS scoring. Earlier research and this manuscript compare the first two methods and the latter two methods, respectively (Segawa et al., 2020).

Users of SS scoring experience greater uncertainty in scores compared to RP scoring, as SS scoring only uses partial information from response patterns (RPs). Prior studies have shown that this increase in uncertainty (the increase) is small—no more than 10% of the standard deviation (SD)—as illustrated through tables and figures in several works (Steinberg & Thissen, 2013; Thissen et al., 1995, 2001; Thissen and Orlando 2001).

Based on these studies, PROMIS guidance recommends RP scoring as the first option due to its higher accuracy, with SS scoring as a good second option (Appendix A. All appendices are available as online supplements). Following these studies, we employ Bayesian scoring, where posterior means and variances represent scores and their associated uncertainties (Ayala, 2009). The increase is calculated by subtracting RP posterior variance from the corresponding SS posterior variance. We use posterior variance instead of posterior standard deviation (SD) due to the mathematical relationship between the two variances, which does not hold for SDs. The percent increase in SD is roughly half of that in variance (Appendix B).

We computed the increase of 34 PROMIS SFs using a different method than what is presented in this paper. Item parameters of 34 PROMIS SFs were available to us. Most were publicly available, with a few sourced from a previous project (Segawa et al., 2020). The PROMIS guidance may mislead users for certain SFs. The increases in 6 SFs were very small (less than 2% in SD), while one SF exhibited a large increase (more than 20% in SD). For SFs with very small (and negligible) increases, framing SS scoring as “a good second option” might lead increase-sensitive users to prioritize accuracy over convenience, potentially causing unnecessary shifts from paper-and-pencil scoring to avoid the negligible increase. For the SF with a large increase, presenting SS scoring as “a good second option” may mislead users into believing their increases are small, even though they are not.

SFs with neither very small nor large increases must be identified earlier. Previous studies examined only a few SFs, and their findings were applied solely to those SFs. However, PROMIS generalized these findings to more than 200 SFs, which we see as a generalization fallacy. To avoid this, the guidance should include the increase for each SF. However, the previous approach cannot accurately measure this increase because it obtained the maximum uncertainty difference between SS and RP scoring but not the average difference. To address this limitation, we propose a new method. We calculate the increase by considering the nested structure of the data: RPs are nested within SSs, and SSs are nested within SFs. Since scoring decisions occur at the SF-level, the increase must be aggregated accordingly (SF-level increase). In the first nesting, we apply a mathematical relationship: SS posterior is a weighted sum of RP posteriors with weights being the marginal probabilities of RPs (SS posterior is a mixture of RP posteriors, proof is presented later).

SF-level RP posterior variance is computed in two steps. First, SS-level RP posterior variances are obtained for all SSs by computing weighted sums of RP posterior variances, with weights being the marginal probabilities of RPs (RP WEIGHTS). Second, SF-level RP posterior variance is obtained by computing a weighted sum of SS-level RP posterior variances. The weights (SS WEIGHTS) reflect the expected distribution of latent symptoms among patients in the target population. SF-level SS posterior variance is a weighted sum of SS posterior variances using SS WEIGHTS. Finally, the SF-level increase is computed by subtracting SF-level RP posterior variance from SF-level SS posterior variance.

To better contextualize the evaluation of the increase, we introduce the increase in posterior uncertainty due to shortening SFs, a well-known consideration in HR-PRO research. Shortening is particularly relevant because reducing patient burden is a key objective in SF development. PROMIS provides multiple versions of SFs—8-item, 6-item, and 4-item—allowing users to balance measurement precision and respondent burden. Segawa et al. (2020) examined the relationship between SF length and the increase in uncertainty, highlighting its practical implications. For example, if shortening an 8-item SF by one and two items results in average increases of 10% and 20%, respectively, PROMIS users can better interpret the magnitude of the increase. A 5% increase in SS scoring would be considered minor, as it is half the effect of a 1-item reduction, whereas a 40% increase would be substantial, doubling the impact of a 2-item reduction.

In the following sections, we describe SS and RP scoring, establish the mathematical relationship between SS and RP posteriors, validate this relationship, present our approach to measuring the increase, and examine the variation in RP scores that induces the increase in SS scoring. These analyses are conducted using two SFs, one with a very small increase (one of the 34 SFs with very small increase) and another with a large increase (one with the highest increase among the 34 SFs). To ease the interpretation of the increase, we reference the increase in uncertainty associated with shortening SFs.

Bayesian RP Scoring and SS Scoring

Let there be a total of $i=1,\dots ,I$ ordinal items. Let $T_i\left(k\mid \theta \right)$ be the $i$ -th item’s trace line for category $k=1,\dots ,K$ . Due to the assumption of independence of item responses, conditional on the latent $\theta$ , the likelihood for response pattern $\mathbf{y}=\left(y_1,y_2,\dots ,y_I\right)$ is given by:

$$L\left(\mathbf{y}\mid \theta \right)={\displaystyle \prod }_{i=1}^I{T}_i\left(y_i\mid \theta \right).$$ (1)

Denoting ${\mathcal{Y}}_s=\left\{\mathbf{y}\mid {\displaystyle \sum }_{i=1}^I{y}_i=s\right\}$ as the set of RPs that yield a sum score of $s$ , the likelihood of sum score $s=1,2,\dots ,S$ is:

$$L\left(s\mid \theta \right)={\displaystyle \sum }_{\mathbf{y}\in {\mathcal{Y}}_s}L\left(\mathbf{y}\mid \theta \right)={\displaystyle \sum }_{\mathbf{y}\in {\mathcal{Y}}_s}{\displaystyle \prod }_{i=1}^I{T}_i\left(y_i\mid \theta \right).$$ (2)

Given a prior distribution $g\left(\theta \right)$ , the posterior distribution is:

$$p\left(\theta \mid s\right)=\frac{L\left(s\mid \theta \right)g\left(\theta \right)}{p\left(s\right)}$$ (3)

where $p\left(s\right)$ is the marginal probability:

$$p\left(s\right)=\int L\left(s\mid \theta \right)g\left(\theta \right)\hspace{0.17em}d\theta$$ (4)

Therefore, the posterior mean and posterior variance are:

$$E\left(\theta \mid s\right)=\frac{1}{p\left(s\right)}\int \theta \hspace{0.17em}L\left(s\mid \theta \right)g\left(\theta \right)\hspace{0.17em}d\theta$$ (5)

and

$$V\left(\theta \mid s\right)=E\left({\theta }^2\mid s\right)-E^2\left(\theta \mid s\right)=\frac{1}{p\left(s\right)}\int {\theta }^2\hspace{0.17em}L\left(s\mid \theta \right)g\left(\theta \right)\hspace{0.17em}d\theta -E^2\left(\theta \mid s\right)$$ (6)

The posterior mean and variance are a point estimate for $\theta$ (score) and a score uncertainty, respectively. The RP posterior $p\left(\theta \mid \mathbf{y}\right)$ , marginal probability $p\left(\mathbf{y}\right)$ , posterior mean $E\left(\theta \mid \mathbf{y}\right)$ , and posterior variance $V\left(\theta \mid \mathbf{y}\right)$ are obtained by replacing the SS likelihood in equation (2) with the RP likelihood in equation (1).

Calculating the SS likelihood involves evaluating all $K^I$ possible RP likelihoods, which becomes computationally intractable for large $I$ . To address this, Lord and Wingersky (1984) proposed a recursive algorithm for computing the SS likelihoods. Subsequent work by Thissen et al. (1995) and Cai (2015) extended the algorithm to accommodate ordinal items and multidimensional IRT, enabling practical computation for large $I$ .

1PL Model in SS Scoring

This section demonstrates that SS and RP scoring are identical under the 1PL model. To illustrate this, we demonstrate that the likelihoods of SS and RP scoring are equivalent up to a constant factor, ensuring identical posterior distributions under Bayesian scoring. Consider the likelihood of a response pattern ${\mathbf{y}}^{*}$ with a summed score $s$ , denoted as $L\left({\mathbf{y}}^{*}\mid \theta \right)$ . Due to the sufficiency property of the 1PL model—where all information about a person’s ability is fully encapsulated in their SS—the likelihood of any response pattern $\mathbf{y}$ with the same SS is identical up to a constant:

$$L\left({\mathbf{y}}^{*}\mid \theta \right)=aL\left(\mathbf{y}\mid \theta \right)$$ (7)

for all ${\mathbf{y}}^{*},\mathbf{y}\in {\mathcal{Y}}_s$ (Appendix C). Applying equation (7) to the SS likelihood in equation (2), we obtain:

$$L\left(s\mid \theta \right)={\displaystyle \sum }_{\mathbf{y}\in {\mathcal{Y}}_s}L\left(\mathbf{y}\mid \theta \right)={\displaystyle \sum }_{i=1}^m{a}_{i}L\left({\mathbf{y}}^{*}\mid \theta \right)=\left({\displaystyle \sum }_{i=1}^m{a}_i\right)L\left({\mathbf{y}}^{*}\mid \theta \right)$$ (8)

where $m$ is the number of response patterns that yield $s$ , and $a_i$ is a constant specific to each pattern. Equation (8) shows that SS likelihood is equivalent to RP likelihood up to a constant ${\displaystyle \sum }_{i=1}^m{a}_i$ . Consequently, when Bayesian scoring is applied with identical priors (e.g., standard normal), the resulting posterior distributions—and thus the scores—are the same for both SS and RP scoring.

SS Posterior as a Weighted Sum of RP Posteriors

By substituting equation (2) into equation (3) and using $L\left(\mathbf{y}\mid \theta \right)g\left(\theta \right)=p\left(\theta \mid \mathbf{y}\right)p\left(\mathbf{y}\right)$ , the SS posterior can be expressed as a weighted sum of RP posteriors for SSs equal to $s$ :

$$p\left(\theta \mid s\right)=\frac{{\displaystyle \sum }_{\mathbf{y}\in {\mathcal{Y}}_s}L\left(\mathbf{y}\mid \theta \right)g\left(\theta \right)}{p\left(s\right)}=\frac{{\displaystyle \sum }_{\mathbf{y}\in {\mathcal{Y}}_s}p\left(\theta \mid \mathbf{y}\right)p\left(\mathbf{y}\right)}{p\left(s\right)}={\displaystyle \sum }_{\mathbf{y}\in {\mathcal{Y}}_s}p\left(\theta \mid \mathbf{y}\right)\frac{p\left(\mathbf{y}\right)}{p\left(s\right)}.$$ (9)

Since $p\left(s\right)$ , the marginal probability of SS being $s$ , is the sum of marginal probabilities of RPs $\mathbf{y}$ that yield $s$ :

$$p\left(s\right)=\int L\left(s\mid \theta \right)g\left(\theta \right)\hspace{0.17em}d\theta =\int {\displaystyle \sum }_{\mathbf{y}\in {\mathcal{Y}}_s}L\left(\mathbf{y}\mid \theta \right)g\left(\theta \right)\hspace{0.17em}d\theta ={\displaystyle \sum }_{\mathbf{y}\in {\mathcal{Y}}_s}\int L\left(\mathbf{y}\mid \theta \right)g\left(\theta \right)\hspace{0.17em}d\theta ={\displaystyle \sum }_{\mathbf{y}\in {\mathcal{Y}}_s}p\left(\mathbf{y}\right),$$

Equation (9) shows that the SS posterior is a weighted sum of RP posteriors with RP WEIGHTS. Using the properties of the mean and variance of mixture, the mean and variance of the SS posterior are:

$$E\left(\theta \mid s\right)=E_y\left(E\left(\theta \mid \mathbf{y}\right)\right)$$ (10)

$$V\left(\theta \mid s\right)=E_y\left(V\left(\theta \mid \mathbf{y}\right)\right)+V_y\left(E\left(\theta \mid \mathbf{y}\right)\right)$$ (11)

where the outer expectation and variance are on the probability space of RPs $\mathbf{y}$ (Frühwirth-Schnatter, 2006). Equation (10) indicates that the SS posterior mean is a weighted sum of RP posterior means, and equation (11) indicates that the variance of the SS posterior is composed of two parts: the expected (SS-level) RP posterior variance and the variance of RP posterior means. Details of the derivations of equations (10) and (11) are found in Appendix D.

The Increase of Uncertainty in SS Scoring

The first and second terms in equation (11) are the SS-level RP posterior variance and the increase, respectively. Weighted sums of SS-level RP posterior variances and SS posterior variances, using SS WEIGHTS, provide SF-level RP and SS posterior variances, respectively. The increase is the difference between SF-level SS and RP posterior variances.

Methods

Following PROMIS, the latent variable $\theta$ is re-scaled by multiplying by 10 and adding by 50. We present methods for four analyses: verification of the above mathematical relations numerically (VERIFICATION analysis), an example analysis of our approach to examining the increase in SS scoring (INCREASE analysis), an examination of variation of RP scores within SS which induces the increase (variance of RP scores analysis), and comparisons of the increase of uncertainties in SS scoring with shortened SF (SHORTENED SF analysis). The analyses were conducted using the R software and R-packages of mirt, tidyverse, and kableExtra (Chalmers, 2012; R Core Team, 2021; Wickham et al., 2019; Zhu, 2024).

Instruments

Samejima’s graded model was used in all analyses (Samejima, 1969). For the VERIFICATION analysis, we use the Science SF, which includes four 4-category ordinal items, and Pediatric Life Satisfaction 8b, which consists of eight 5-category items (Ayala, 2009; Forrest et al., 2018). For the INCREASE analysis and variance of posterior scores analysis, we used Pediatric Life Satisfaction 8b, which exhibits a large increase, and the Adult Anxiety 8a, which has a very small increase (Forrest et al., 2018; Pilkonis et al., 2011). For the SHORTENED SF analysis, we used the PROMIS Adult Profile, which measures seven domains of basic health-related quality of life using 7 SFs. For each domain, we used two versions: 8-item and 6-item version. The 6-item version was created by excluding the last two items from the 8-item version. In addition, we create a 7-item version, specific to this analysis, by excluding the last item from the 8-item version. Thus, three versions—8-item, 7-item, and 6-item—were used to measure each domain. Each SF consisted of 5-category ordinal items. The item parameters for the SCIENCE, Anxiety 8a, and Pediatric Life Satisfaction 8b are publicly available (Ayala, 2009; Forrest et al., 2018; Pilkonis et al., 2011).

Procedures

VERIFICATION Analysis

Using the Science SF, we plotted the SS posterior when SS equals 5, using both the original definition in equation (3) and the relation in equation (9). There are four RPs: $\left[1112\right],\left[1121\right],\left[1211\right],$ and $\left[2111\right]$ , which yield a score of 5. Using PROMIS Pediatric Life Satisfaction 8b, we verified equations (10) and (11).

We employed a simulation-based approach instead of full enumeration for verification. Initially, we considered full enumeration to be computationally demanding and its implementation complex. While full enumeration is feasible for short forms with a small number of items (e.g., $5^8$ = 390,625 possible RPs), it does not scale to longer PROMIS scales with 30 items, where the number of possible RPs grows exponentially. In contrast, our simulation-based approach remains practical for larger questionnaires. Importantly, our approach successfully verified equations (10) and (11), as demonstrated in the results section, making full enumeration unnecessary for our purposes.

Ten million latent values were sampled from the standard normal distribution. These latent values were converted into response patterns using known item parameters, and their RP posterior means and variances were computed, which were then grouped by SSs. The weighted sums of RP posterior means by SS served as approximations of SS posterior means (equation (10)). The sum of the weighted RP posterior variances (approximating SS-level RP posterior variance) and the weighted sum of squared differences between RP posterior means and SS posterior mean (approximating the increase in SS scoring) gave an approximation of the SS posterior variance (equation (11)). Alternatively, we directly computed SS posterior means and variances using the recursive method (Lord & Wingersky, 1984). We compute the mean and maximum of the absolute errors. This verification of equations (10) and (11) also serves as an assessment of the accuracy of the computations used in later analyses.

INCREASE Analysis

We computed SS posterior variances using the recursive method and SS-level RP posterior variances for Anxiety 8a, just as we did for Life Satisfaction 8b in the second part of the VERIFICATION analysis. We then plotted SS posterior variances and SS-level RP posterior variances for both SFs, producing posterior variance curves.

SF-level statistics are computed as weighted sums of SS-level counterparts, with weights determined by the proportions of SSs in the simulated data. However, when latent values are sampled from a standard normal distribution (mean = 0), as in the VERIFICATION analysis, healthier SSs tend to be oversampled. This occurs because PROMIS SFs are designed so that the mean of the general population, which is healthier than the target population, aligns with the mean of the standard normal distribution. As a result, using a standard normal distribution oversamples the healthiest SS (all item responses are the healthiest) and undersamples the unhealthiest SS (all item responses are the least healthy). While both extreme SSs yield the increase of 0, the sum of their proportions is greater when the healthiest SS is oversampled than when the two extremes are more balanced. Consequently, using a standard normal distribution skews the weighting in favor of SS scoring by increasing the proportion of SSs with the increase of 0.

To mitigate this issue, we shift the mean of the standard normal distribution by $m$ to reduce the overrepresentation of the healthiest SS. Specifically, we compute the proportions of the smallest and largest SSs using one million latent values sampled from a normal distribution with its mean shifted by 1 in the unhealthy direction. We then iteratively adjust the mean in increments of 0.1 in the direction that decreases the sum of extreme SS proportions. This process continues until we determine the optimal $m$ , which minimizes the total proportion of extreme SSs. Using the resulting SS proportions as weights, we compute SF-level SS posterior variance, RP posterior variance, the increase, and the percent increase.

Variance of RP Scores Analysis

We examined the variation of RP scores, which drives the increase in SS scoring, by plotting RP and SS scores for Anxiety 8a and Life Satisfaction 8b (equation (11)). To prevent overcrowding due to the large number of RPs ( $5^8$ ), we selected one SS value with a relatively small number of RPs for each SF. Further, we randomly selected one-tenth of the RP scores to ensure clarity.

SHORTENED SF Analysis

Two factors influence the increases in 1- and 2-item shortenings. To ensure reliable reference values, we controlled for these factors. The first is which items within an SF are excluded, and the second is the selection of SFs. To control the first factor, as described in the Instrument section, we excluded the last two items (present in the 8-item but not the 6-item version) for the 2-item shortening. The last item was excluded for the 1-item shortening. To control the second factor, we could have computed weighted averages of the increases in 1- and 2-item shortenings across all PROMIS SFs, using SF usage frequencies as weights. However, due to the unavailability of item parameters and usage frequencies, we instead used the average across the seven SFs in the PROMIS Profile, which spans a broad range of HR-PRO domains and is frequently used—a reasonable representative of PROMIS SFs.

We randomly sampled one million latent values from a normal distribution with a mean $m$ (determined as described in the INCREASE analysis) and a standard deviation of 1. Using these samples, we computed the increases in SS scoring across the 7 domains as in the INCREASE analysis. Since SSs for the 8-item SF are not available for the 6- and 7-item SFs, we computed the SF-level RP posterior variances for the 7- and 6-item SFs by averaging their respective RP posterior variances. Using the SF-level posterior variances for the 8-, 7-, and 6-item SFs, we calculated the increases in posterior variances for the 7- and 6-item SFs and then computed the percent increases.

Results

VERIFICATION Analysis

We confirmed equation (9) by showing that the weighted mean of the four RP posteriors perfectly overlapped with the directly computed SS posterior (Figure 1). Equations (10) and (11) were also validated, as the approximated posterior means and variances closely matched those obtained through the recursive method. The means and maximums of absolute errors were 0.011% and 0.01% for the SS posterior means and 0.14% and 0.67% for the SS posterior variances.

Figure 1.

SS posterior when summed score is 5, four weighted RP posteriors which yield 5, and a weighted sum of the four RP posteriors for Science SF.

INCREASE Analysis

Excluding the endpoints where SS and RP scoring are identical and SS posterior variances (circles) completely overlap with SS-level RP posterior variances (Xs), we observed that SS posterior variances (circles) were slightly larger in Anxiety 8a and clearly larger in Life Satisfaction 8b compared to their corresponding SS-level RP posterior variances (Xs) (Figure 2). Consequently, the SF-level increases were 2.55% for Anxiety 8a and 39.56% for Life Satisfaction 8b.

Figure 2.

Comparison of SS posterior variances and SS-level RP posterior variances in Anxiety 8a and Life Satisfaction 8b.

Variance of RP Scores Analysis

SS and SS-level RP posterior variances for the selected SS values (12 and 35 for Anxiety 8a and Life Satisfaction 8b, respectively) were highlighted with arrows in Figure 2. The rectangle in Life Satisfaction 8b represents the smallest region containing all RP scores, and the same-sized rectangle was applied to Anxiety 8a. These rectangles are expanded in Figure 3, showing 58,868 and 54,441 RP scores for Anxiety 8a and Life Satisfaction 8b, respectively.

Figure 3.

Individual RP points for Anxiety 8a and Life Satisfaction 8b.

RP-level increases, defined as (SS posterior variance) - (RP posterior variance), varied across RPs for both SFs. In some cases, RP scores were less precise (i.e., had negative increases) compared to SS scores. The proportion of such cases was larger for Anxiety 8a than for Life Satisfaction 8b. Additionally, the variance of RP posterior means, corresponding to the increase (equation (11)), was much smaller in Anxiety 8a than in Life Satisfaction 8b.

SHORTENED SF Analysis

The averages of 1- and 2-item shortening (reference points) were 11.87% and 25.29%. The percent increase for Anxiety 8a (2.55%) was less than a quarter of the mean increase for 1-item shortening, while the percent increase for Life Satisfaction 8b (39.56%) was more than 1.5 times the average percent increase for 2-item shortening.

Conclusions and Discussions

We developed a method that accurately measures the increase by incorporating the mathematical relationship that SS posterior is the mixture of RP posteriors. Using this method, we demonstrated that the increase can be either inconsequentially small or large, findings not observed in previous studies. Because the current guidance might mislead users when SFs exhibit such increases, we recommend including the increase of each SF in the guidance so that users can make informed decisions when choosing a scoring method.

Distribution of the Increases

The next step would involve computing the increases for more than 200 PROMIS SFs using our method to obtain a distribution of these increases, including the proportions of SFs with inconsequential and large increases. This would reveal the average increase as well as the proportions of those extreme increases in SFs within PROMIS. That knowledge would contribute to our understanding of SS scoring in PROMIS and help in writing appropriate scoring guidance.

Cause of the Increase

Understanding the cause of the increase is important, as it may provide insights into how to mitigate it. Our preliminary analysis indicated that a larger variance in slope parameters was associated with a larger increase. When constructing SFs, higher-slope items are generally selected for their superior discrimination, while lower-slope items are sometimes included for substantive reasons. This suggests that incorporating lower-slope items not only reduces overall discrimination but also contributes to the increase. While selecting lower-slope items may decrease the variance in slopes and potentially reduce the increase, it would come at the cost of lower discrimination, which we do not recommend.

The Increase in Target Population

In this manuscript, we specified SS WEIGHTS to reflect the target population of each SF. However, a user could obtain a more accurate expected increase by specifying SS WEIGHTS to reflect her study. For example, suppose she uses Anxiety 8a for screening anxiety and expects about 30% of samples to have the minimum score, at which the SS-level increase is 0. The larger weight of 30% on 0 and smaller weights on others would even lower the SF-level increase.

The Increase in SD

Using the formula that converts the increase in variance to one in SD, the increases in standard deviation in Anxiety 8a and Life Satisfaction 8b are 1.27% and 18.13%, respectively (Appendix B). Although the variance should be used in the computation stage to reflect the mathematical relationship between SS and RP posterior variances as found in this manuscript, the standard deviation might be more appropriate for improving interpretation in the guidance.

ORCID iD

Eisuke Segawa https://orcid.org/0000-0003-1103-9448