Table 3.

Overview of the analytic process for evaluating different aspects of validity

Step	Aspect of validity measured	Statistical Approach and Criteria	Fatigue Scale Results	Energy Scale Results
1	Rating scale functioning: Does the rating scale function consistently across items? (substantive validity)	Each category should have a minimum of 10 observations. Average measures for each category on each item should advance monotonically z-values < 2.0 for outfit mean square (MnSq) values in step category calibrationsa	Each category was observed ≥10 times All items met criteria	Categories 8, 9, and 10 were ordered illogically (10, 8 , 9) When 8, 9, 10 were collapsed into a single category, all items met criteria.
2	Internal scale validity: Do item responses match expected responses from the Rasch model? (content validity)	Item goodness-of-fit statistics MnSq values between 0.7 and 1.3b	All items met criterion	All items met criterion
3	Internal scale validity: Is the scale unidimensional? (structural validity)	Principal component analysis ≥ 50% of total variance explained by first component (fatigue/energy)c	First component explained 52.1% of total variance	First component explained 63.5% of total variance
4	Differential item functioning (DIF): Are item difficulty calibrations stable in relation to relevant variables?d (generalizability validity)	Mantel-Haenszel statistics p < .01 with Bonferroni correctione	Education: #13 Time of day: #14, 18	Education: #8
5	Time of day effects: Are morning and evening ratings correlated but different? (generalizability validity)	Pearson correlations and paired t-test r > .30 Evening > Morning (fatigue) Morning > Evening (energy)	r = .38 Evening > Morning (p<.001)	r = .22 Morning > Evening (p<.05)
6	Subscale correlations: Are the Fatigue and Energy subscales correlated? (structural validity)	Pearson correlations r > .30	Fatigue and Energy Scale Results
			Morning r = −0.31, p<.01 Evening r = 0.05, not significant

^aLinacre 2002;

^bSmith et al. 1998;

^cLinacre 2010;

^dDIF variables evaluated: age, education, employment, income, partner, children, years since HIV diagnosis, CD4 count, and time of day;

^eMantel 1963