. Author manuscript; available in PMC: 2017 Apr 1.

Published in final edited form as: Acad Radiol. 2016 Feb 18;23(4):496–506. doi: 10.1016/j.acra.2015.12.020

Table 4.

Testing Bias and Linearity using CT Volumetry for Illustration^*

STEP	DESCRIPTION
7: Make measurements from N cases	For each case calculate the tumor volume (denoted Y_i), where i denotes the i-th case (i=1, 2, …N).
8. Calculate individual bias	For each case calculate the bias or % bias: b_i = (Y_i − X_i) and %b_i = [(Y_i − X_i)/X_i] × 100, where X_i is the measurand value (i.e. true value). For CT volumetry, %b is used.
9. Estimate overall bias and its variance^*	Over N cases, estimate the bias: $\hat{b} = \sqrt{\sum_{i = 1}^{N} % b_{i} / N}$ . The estimate of the variance of the bias (i.e. between-case variance) is ${\hat{Var}}_{b} = \sum_{i = 1}^{N} {(% b_{i} - \hat{b})}^{2} / (N - 1)$ .
10. Construct 95% CI for b̂	The 95% CI for the bias is $\hat{b} \pm t_{α = 0.025, (N - 1) d f} \times \sqrt{{\hat{Var}}_{b}}$ , where t_{α=0.025,(N−1)df} is from the Student’s t-distribution^** with α=0.025 and (N−1) degrees of freedom. To test whether the actor’s bias satisfies the performance requirement in the Profile, the smallest and largest values in the 95% CI are examined. If the smallest value is greater than the minimum requirement stated in the Profile and the largest value is less than the maximum requirement stated in the Profile, then the performance requirement for the overall bias is met.
11. Bias Profile	Separate the cases into strata based on covariates known to affect bias (tumor size and density). For each stratum estimate the bias.
12. Perform OLS regression	Fit an ordinary least squares (OLS) regression of the Y_i’s on X_i’s. A quadratic term is first included in the model to rule out non-linear relationships: Y = β_o + β₁X + β₂X². Then a linear model should be fit: Y = β_o + β₁X where R-squared (R²) >0.90.
13. Construct 95% CI for slope	Let $\hat{β_{1}}$ denote the estimated slope from step 12 (assuming β₂ = 0). Calculate its variance as ${\hat{Var}}_{β_{1}} = {\sum_{i = 1}^{N} {(Y_{i} - Ŷ_{ι})}^{2} / (N - 2)} / \sum_{i = 1}^{N} {(X_{i} - \bar{X})}^{2}$ , where Ŷ_ι is the fitted value of Y_i from the regression line and X̄ is the mean of the true values. The 95% CI is $\hat{β_{1}} \pm t_{α = 0.025, (N - 2) d f} \sqrt{{\hat{Var}}_{β_{1}}}$

As in Table 3, if multiple readers are studied, then the bias of the average reader must be compared to the performance requirement in the Profile. A generalized linear model can be built for the bias, treating readers as a random effect nested in cases [5]. From the model, a 95% CI for the readers’ mean bias is constructed and used to evaluate the actor’s performance relative to the requirements in the Profile.

^**

Student’s t-distribution is a commonly used probability distribution that is used when a statistic, like the bias estimator, is normally distributed but the study sample size is small (sample size of N) and the population standard deviation is unknown and must be estimated from the data.