The Validity of Disease-specific Quality of Life Attributions Among Adults with Multiple Chronic Conditions

Abstract

Background

A crucial assumption underlying all disease-specific quality of life (QOL) measures, that patients can validly differentiate a specific disease in the presence of multiple chronic conditions, has not been tested using multiple methods. Our objective was to evaluate the convergent and discriminant validity of QOL attributions to specific diseases among adults with multiple chronic conditions (MCC).

Methods

Adults age 18 and older (N=4,480) sampled from eight pre-identified condition groups (asthma, COPD, angina/MI with angina, congestive heart failure, diabetes, chronic kidney disease, osteoarthritis, rheumatoid arthritis) completed an Internet survey. Comorbid conditions were determined using a 35-condition checklist. Product-moment correlations were analyzed separately by pre-identified condition group using the multitrait-multimethod of construct validation, where traits were defined by 9–26 conditions and each condition was measured by two methods: disease severity rating and Disease-specific Quality of Life Impact Scale (QDIS) global rating. A third method (symptom or clinical marker) was available for the eight pre-identified conditions. Convergent validity was supported when correlations among different methods of measuring the same condition (trait) were substantial (r≥0.40). Discriminant validity was supported when correlations between the same and different methods of measuring different conditions were significantly lower than corresponding convergent correlations.

Results

In support of convergent validity, 22 of 24 convergent correlations were substantial (r=0.38–0.84, median=0.53). In support of discriminant validity, 833 of 924 tests (90.2%) yielded significantly higher convergent than discriminant correlations across the eight pre-identified conditions. Exceptions to this pattern of results were most often observed for comorbid conditions within the same clinical area.

Conclusions

Collectively, convergent and discriminant test results support the construct validity of disease-specific QOL impact attributions across MCC within the eight pre-identified conditions. Noteworthy exceptions should be considered when interpreting some specific QOL impact attributions and warrant further study. Pursuit of a summary disease-specific QOL impact score standardized across MCC is recommended.

Introduction

Multiple chronic conditions (MCC) have been defined as two or more concurrent chronic conditions that have an “adverse effect on health status, function, or quality of life” [1]. Presence of MCC is predictive of mortality, treatment outcomes, health care utilization and expenditures, disability, and quality of life (QOL) [2–4]. In addition to their use in prediction and case mix control in outcomes research, knowledge of MCC contributes to the understanding of patient complexity [5] and is a major consideration in establishing guidelines for quality of care [6]. Accordingly, estimates of MCC impact are an essential element in observational approaches to comparative effectiveness research [7] and a priority of information systems used in improving health care quality.

For purposes of collective assessment, MCC have been simply counted [8,9] or weighted according to a particular criterion, such as mortality [10] or health care utilization [11]. However, the increased emphasis on function and QOL portends a growing need for patient-centered measures of overall MCC burden [12]. For observational studies of QOL outcomes, it makes the most sense to adjust for MCC in terms of their patient-reported impact on QOL, as opposed to simple counts or weights based on survival or cost.

QOL measures are conceptualized as generic, applying to all conditions and treatments, or disease-specific, pertaining to a single disease or a disease cluster within a clinical area. What makes a measure disease-specific? A number of approaches have been employed with the goal of achieving disease-specific measures that are more clinically useful, actionable, and responsive in comparison with generic QOL measures. FDA guidelines for industry-sponsored clinical trials using patient-reported outcomes (PRO) [13] recommend assessment of symptoms that are specific, such as joint pain in arthritis [14] or dyspnea in respiratory disease [15]. Widely-accepted conceptual frameworks also require the representation of symptoms to achieve comprehensiveness in a PRO endpoint model [16]. However, symptoms may not be as specific as their inclusion in disease-specific measures implies; dyspnea measures for example are included in disease-specific measures for asthma [17], congestive heart failure [18] and kidney disease [19].To the extent that symptoms are specific, they are useful in evaluating the convergent validity of disease-specific QOL measures. Conceptually speaking, however, symptoms alone do not reflect QOL [20]. The content validity of disease-specific measures can be improved by better representing QOL outcomes known to be most affected by a disease; for example, physical functioning in osteoarthritis. Another measurement approach uses anatomic specificity, which may or may not achieve the desired disease specificity; for example, chest pain may be due to angina or due to chronic cough with COPD. Finally, simply asking patients to attribute the QOL impact of a specific disease is a patient-centered approach. In pursuit of this approach, content representation has been improved to the point that a primary difference between generic and disease-specific QOL measures is whether attributions are made to health in general or to a specific disease [21].

While generic QOL measures such as the EQ-5D [22], PROMIS [23], and the SF-36 Health Survey [24] allow illness burden and treatment effects to be compared across diseases, generic measures also have been characterized as being less responsive [25] and less useful in clinical research [16] than disease-specific measures [13]. However, the lack of disease-specific measures that are standardized across diseases has, up until now, precluded their use in meaningfully comparing disease-specific QOL impact or summarizing total MCC impact across diseases. There is a need for disease-specific QOL impact measures that fill the gap between symptom measures that conceptually are not QOL and generic QOL measures that are not disease-specific. One such effort, which had the goals of standardizing the content and scoring of measures of QOL impact attributed to specific conditions, was the Computerized Adaptive Assessment of Disease Impact (DICAT) project [26,27]. DICAT showed that disease-specific QOL measures could be constructed with standardized item content and scoring across diseases, thereby allowing disease-specific QOL impact to be meaningfully compared across conditions. These disease-specific measures also had greater discriminant validity than generic QOL measures in distinguishing among severity levels, in studies of individual diseases [21]. Whether this approach works in the context of MCC is a question that is best addressed empirically and comprehensively.

In this paper, we use DICAT data to expand psychometric evaluations of disease-specific QOL measures beyond single-disease comparisons that ignore comorbidity. We use a multitrait(multi-disease)-multimethod analytic approach to evaluate multiple methods of measurement for each disease across samples with multiple chronic conditions. This approach, which has advantages in testing construct validity as demonstrated in the fields of educational and psychological measurement [28], allows us to address a fundamental question underlying all uses of disease-specific attributions in the measurement of QOL: Are disease-specific QOL attributions valid when multiple comorbid conditions are present? While the focus of this paper is on testing the convergent and discriminant validity of disease-specific QOL impact ratings, results may have implications for all efforts to aggregate disease-specific measures for purposes of patient-centered summary measurement of total MCC burden.

Methods

Sampling and data collection

As described in detail elsewhere [21,29], data were collected from a probability-based sample aged 18 and older (N=5,418) recruited from the GfK (formerly Knowledge Networks (KN)) research panel of approximately 50,000 adults [30] during the DICAT project. Unlike convenience (“opt-in”) panels, the KN panel is a probability-based sample, as recommended for Internet surveys [31]. It includes cell-phone only households, and panelists are provided with a computer and Internet connection if needed. Panelists had previously reported being diagnosed (Pre-ID) with any of nine conditions: respiratory disease (asthma, chronic obstructive pulmonary disease), cardiovascular disease (angina, myocardial infarction in past year (limited to those reporting comorbid angina in this analysis), congestive heart failure), diabetes, chronic kidney disease (CKD), and arthritis (osteoarthritis, rheumatoid arthritis). They were sampled to achieve at least 1,000 respondents within three (arthritis, diabetes and respiratory) groups in order to increase the number of sufficiently prevalent comorbid conditions, with smaller targets for less prevalent diagnoses (cardiovascular, CKD). Pre-ID conditions were confirmed at the start of the DICAT survey.

Panelists were recruited by KN via a routine invitation, which was followed by e-mail and IVR telephone reminders if necessary. Invitations were received by 9,160 pre-ID panelists, of whom 6,828 (74.5%) opened an informed consent screen; 5,585 consented, and 5,418 (97%) completed surveys. Survey protocols were designed to limit respondent burden (median ≤25 minutes). For all protocols, modules were administered in the following order: generic QOL measures (not analyzed here), 35-item chronic condition checklist, global disease severity and QOL impact ratings for all conditions reported, and other previously validated (legacy) disease-specific measures. The chronic condition checklist and, therefore, estimates of comorbid conditions were limited to 35 conditions. Represented were all but two of the conditions (all but dementia and peripheral vascular disease) in the Charlson index [10], along with other conditions selected from large-scale health surveys such as the National Health Interview Survey (see Table 3). This analysis was limited to the 4,480 respondents who reported a pre-ID condition and completed the corresponding legacy measure used in convergent validation.

Measures

Development and standardization of QOL impact metrics across conditions required large pre-ID disease-specific item banks along with modules containing legacy measures specific to each pre-ID condition. Limits on respondent burden therefore precluded comprehensive disease-specific measurement for comorbid conditions on a par with that for pre-ID conditions. Each pre-identified condition was measured using three methods: (1) a previously-validated symptom or clinical marker specific to each condition (see Table 1); (2) a standardized disease severity rating in response to “How would you rate the severity of your <condition> in the past 4 weeks?” with five response choices ranging from None to Very Severe; and (3) a standardized global rating of disease-specific QOL impact from the Disease-specific Quality of Life Impact Scale (QDIS) asking “In the past 4 weeks, how much did your <condition> limit your everyday activities or your quality of life?”, with five response choices ranging from Not at all to Extremely [21,29]. Methods (2) and (3) were also measured for each comorbid condition reported. For comorbid obesity, a clinical marker was also measured (see Table 1). In support of its reliability and validity, the global disease-specific QOL impact item has been shown to be very highly correlated (r=0.96) with the total score from the full 49-item QDIS bank which measures 10 disease-specific QOL content areas, across the eight pre-ID groups [21]. Item-level missing data rates for the disease severity and disease-specific QOL impact ratings were low (median=0.67%, range 0.00-2.01%). Test-retest (intraclass correlation coefficient, 5–14 days) reliability was satisfactory (ICC_(3,1)=0.74 for severity ratings, ICC_(3,1)=0.79 for QDIS global impact ratings). For all methods reported here, higher scores indicate worse symptoms, severity or QOL impact, with the exception of eGFR, as noted in Table 1. Accordingly, all correlations were hypothesized to be positive, except those for eGFR.

Table 1.

Variable definitions

Symptom and Clinical Markers by Pre-ID Condition
Asthma (Asthma Control Test™)
During the past 4 weeks, how often have you had shortness of breath? More than once a day, Once a day, 3 to 6 times a week, Once or twice a week, Not at all (Scored so higher score=greater shortness of breath) [17]
COPD (St. George’s Respiratory Questionnaire)
Over the last year, I have had shortness of breath: Most days a week, Several days a week, A few days a week, Only with chest infections, Not at all (Scored so higher score=greater shortness of breath) [15]
Angina (Seattle Angina Questionnaire)
Over the past 4 weeks, on average, how many times have you had chest pain, chest tightness, or angina? 4 or more times per day, 1–3 times per day, 3 or more times per week but not every day, 1–2 times per week, Less than once a week, None over the past 4 weeks (Scored so higher score=more chest pain) [38]
Congestive Heart Failure (Minnesota Living with Heart Failure® Questionnaire)
Did your heart failure prevent you from living as you wanted during the past month (4 weeks) by making you short of breath? 0=No, 1=Very Little, 2, 3, 4, 5=Very much (Scored so higher score=greater shortness of breath) [18]
Diabetes (Medical Outcomes Study)
Count of number of diabetes complications (0,1,2,3,4+): In the past 12 months have you (yes/no): (1) had episodes of sudden weakness, dizziness, heart pounding, sweating, or tremors, relieved by eating or drinking orange juice; (2) had ulcers or infections on your toes or feet; (3) gone to an ER or been hospitalized for diabetic ketoacidosis or diabetic coma; (4) had episodes of excessive thirst and urinating much more than usual; (5) had eye problems for which your doctor referred you to an ophthalmologist (eye specialist); (6) had gangrene of any of your toes; (7) been treated for other complications of diabetes (e.g., leg or foot infections)
Chronic Kidney Disease (National Kidney Foundation, 2009 CKD-EPI Creatinine Equation)
www.kidney.org/professionals/KDOQI/gfr_calculator. Formula uses response to item “What was the result of your last Serum Creatinine test?” (Lower eGFR=worse kidney function)
Osteoarthritis
Two-part question: “In the past 12 months, have you had pain, stiffness, or aching in your joints lasting at least one month?” Yes/ (1) No. If Yes: “How frequently did you have pain flare-ups that lasted more than one week in the past 12 months?” (2) Less than once a month; (3) About once a month; (4) About twice a month; (5) More than twice a month (Scored so higher score=worse pain)
Rheumatoid arthritis
Two-part question: “In the past 12 months, have you had pain, stiffness, or aching in your joints lasting at least one month?” Yes/ (1) No. If Yes: “How frequently did you have pain flare-ups that lasted more than one week in the past 12 months?” (2) Less than once a month; (3) About once a month; (4) About twice a month; (5) More than twice a month (Scored so higher score=worse pain)
Obesity
Body mass index calculated based on self-reported weight and height
Disease Severity
How would you rate the severity of your <condition> in the past 4 weeks? None, Mild, Moderate, Severe, Very severe [21]
Disease QOL impact
In the past 4 weeks, how much did your <condition> limit your everyday activities or your quality of life? Not at all, A little, Some, A lot, Extremely [21]

Analysis

Data were analyzed separately for each pre-ID condition using the multitrait-multimethod (MTMM) approach to construct validation [32]. In contrast to common measurement evaluations based on a single method applied to a single trait (disease), the MTMM approach is ideal for studies of multiple comorbid conditions. Its strength is the production of a matrix of correlations among multiple traits, each measured by multiple methods sufficient to more rigorously evaluate construct validity. Traits were defined by chronic conditions and methods included up to three operational definitions (see above).

To evaluate convergent validity, a matrix of product-moment correlations among different methods of measuring the same condition (trait) was estimated for identical samples across three methods, within each pre-ID sample. These monotrait-heteromethod correlations are indicative of convergent validity when they are significantly different from zero and substantial in magnitude. Within each pre-ID condition, the stationarity of convergent correlations across comorbid pairs was evaluated to determine whether one convergent matrix could be used for all comparisons. For this decision, the matrix of convergent correlations based on all respondents for a pre-ID condition was compared with the same matrix of convergent correlation estimates for each smaller comorbid sub-sample. Because stationarity was confirmed collectively across pre-ID conditions (information available upon request), the convergent matrix for each full pre-ID sample is the basis for comparisons and the significance testing reported here.

To evaluate discriminant validity across traits (conditions), matrices of product-moment correlations among the same (monomethod-heterotrait) and different (heteromethod-heterotrait) methods of measuring different conditions were estimated. Discriminant validity was supported to the extent that correlations between different conditions (heterotraits) were low in terms of absolute magnitude and significantly lower whether measured by the same or by different methods, in comparison with convergent correlations in the same matrix. In other words, a trait should correlate more highly with itself than with something else. Minimum sample sizes (N≥50) were established for all tests of convergent and discriminant validity.

Because 24 convergent correlations (3 for each condition) were compared with 924 discriminant correlations (51–153 across comorbid conditions), pre-specified standards were applied to categorize results from convergent and discriminant tests. Convergent validity was supported by substantial (r≥0.40) monotrait-heteromethod correlations. Discriminant validity evaluation was a 2-step process. First, discriminant validity was supported by low (r≤0.30, absolute magnitude) monomethod-heterotrait and heteromethod-heterotrait correlations. Second, discriminant validity was supported when discriminant (heterotrait) correlations were significantly lower (p<0.05, one-tailed test as documented in appended tables) in comparison with the corresponding convergent correlations. This second test was important because in most cases in which heterotrait correlations did not meet the r≤0.30 standard, they were still significantly lower than their corresponding convergent correlations, which is a pattern of results that supports construct validity.

Method effects, a possible contributor to discriminant failures, were examined by comparing correlations between two conditions measured by the same method (heterotrait-monomethod) relative to the same two conditions measured by two different methods (heterotrait-heteromethod). A method effect was indicated to the extent that heterotrait-monomethod correlations were greater than heterotrait-heteromethod correlations for the same two conditions.

Results

Sample characteristics

Characteristics of pre-identified disease groups are documented in Table 2. Overall, age ranged from 18–93 (median=61), with 56.6% female and 80.1% non-Hispanic white; 23.4% had an education level of high school graduate or less.

Table 2.

Sample characteristics

N	Asthma 742	COPD 274	Angina 221	CHF 271	Diabetes 1306	CKD 147	OA 1026	RA 493
Mean Age (SD)	48.3 (16.3)	63.9 (10.6)	64.9 (10.3)	66.9 (10.7)	60.2 (11.9)	64.7 (12.5)	62.7 (10.6)	60.3 (12.2)
% Male	34.0	43.8	58.8	53.1	51.8	53.1	35.9	35.1
Race/Ethnicity
White non-Hispanic	73.5	86.1	85.5	84.9	77.5	84.3	85.1	77.3
Black non-Hispanic	7.4	5.1	3.2	5.5	11.2	7.5	5.6	9.3
Hispanic	11.7	3.3	2.3	2.6	6.2	3.4	4.6	6.9
Other non-Hispanic	7.4	5.5	9.0	7.0	5.1	4.8	4.7	6.5
Education
< High school grad.	3.5	6.9	4.1	5.1	2.6	4.8	2.1	3.6
High school graduate	19.8	33.2	14.9	17.0	19.2	12.9	20.7	20.5
Some college	34.6	35.8	43.0	43.2	39.6	42.9	30.1	42.8
College graduate	42.1	24.1	38.0	34.7	38.6	39.4	47.1	33.1
Income
< $20,000	12.4	23.7	12.2	15.1	10.7	9.5	9.6	16.6
$20,000–39,999	17.5	26.3	28.0	25.8	21.8	26.5	20.1	25.2
$40,000–99,999	48.3	40.2	48.0	48.0	52.2	50.4	48.7	43.8
$100,000+	21.8	9.8	11.8	11.1	15.3	13.6	21.6	14.4
Employment Status
Employed	52.4	21.2	24.4	15.5	43.6	21.1	37.9	32.9
Unemployed	6.2	4.0	4.1	2.6	4.9	1.4	3.3	5.9
Retired due to age	15.1	38.0	37.6	46.9	32.9	46.9	39.7	31.9
Disableda	7.8	28.8	27.6	26.2	10.3	22.4	10.2	18.4
Otherb	18.2	6.9	5.9	7.7	7.6	6.8	8.5	10.1
Missing	0.3	1.1	0.4	1.1	0.7	1.4	0.4	0.8
Mean PCS (SD)	47.5 (9.9)	39.6 (11.4)	40.1 (11.0)	39.2 (10.0)	47.0 (9.5)	40.8 (10.8)	43.5 (9.8)	41.7 (10.4)
Mean MCS (SD)	48.1 (10.2)	46.7 (11.5)	48.1 (10.5)	49.5 (9.6)	50.6 (9.0)	49.4 (9.8)	49.9 (9.7)	48.7 (9.8)

Note: Angina is with or without myocardial infarction in past year. CKD is limited to Stages 3–5 using self-reported estimated glomerular filtration rate (eGFR). PCS = SF-36 Physical Component Summary. MCS = SF-36 Mental Component Summary.

^aRetired due to disability.

^bHomemaker, student, other.

Comorbid conditions

Table 3 summarizes the sample sizes for pairs of comorbid conditions for each pre-ID condition (columns) along with the frequency of each comorbid condition (rows); pairs with N≥50 were included in the MTMM analyses. The number of comorbid conditions that could be tested varied according to the sample size for each pre-ID group (N=147-1,306) and group differences in comorbidity prevalence. Accordingly, the number of comorbidity tests within each pre-ID condition ranged from 14 to 25, with the exception of CKD with only eight comorbid conditions. As documented in the right-most column of Table 3, five comorbidities were tested in every pre-ID group and 23 of the 35 were tested at least four times. Only six could not be tested.

Table 3.

Frequency of pairs of comorbid chronic conditions

Conditiona	Asthma (N=742)	COPD (N=274)	Angina (N=221)	CHF (N=271)	Diabetes (N=1306)	CKD (N=147)	OA (N=1026)	RA (N=493)	Freq. in MTMM Analyses
Asthma	—	138	47	59	63	24	54	29	4
COPD	***	—	41	61	26	22	17	21	1
Angina	12	7	—	68	18	13	15	11	1
Myocardial infarction	1	1	32	0	5	5	3	4	0
Congestive heart failure	8	16	10	—	26	22	12	8	0
Diabetes	124	59	73	92	—	56	202	90	7
Kidney disease	5	4	13	10	23	—	12	8	0
Osteoarthritis	168	82	74	101	128	57	—	177	7
Rheumatoid arthritis	74	47	39	32	84	26	***	—	2
Allergies, seasonal	530	146	103	123	520	60	469	220	8
Chronic back problems	207	115	111	95	292	51	425	200	8
Joint problems, hip/knee	215	94	100	122	331	59	638	275	8
Hypertension	260	168	161	200	900	121	566	269	8
Obesity	211	78	75	101	548	50	337	154	8
Allergies, chronic	369	126	84	95	252	46	300	156	7
Hearing, trouble	96	81	75	88	239	38	216	77	7
Vision, trouble	94	61	50	54	160	25	129	84	7
Anemia	135	31	48	67	143	56	149	91	6
Depression	164	66	52	48	172	39	202	99	6
Dermatitis/skin conditions	130	36	50	58	123	33	146	85	6
Joint problems, foot/ankle	114	46	51	62	171	33	251	167	6
Hypothyroidism	107	52	38	52	183	33	181	90	6
Erectile dysfunction	42	31	56	59	213	34	92	50	5
Limb, limitations in use	61	37	36	51	128	29	150	75	5
Osteoporosis	56	52	33	35	65	22	146	76	5
Cancer (non-skin)	50	36	38	48	134	33	134	56	4
Irritable bowel syndrome	95	35	37	27	114	16	162	74	4
Migraine headaches	179	46	46	38	139	22	216	108	4
Ulcer/stomach disease	97	42	48	44	113	25	154	76	4
Enlarged prostate (BPH)	37	39	44	55	145	28	119	37	3
Fibromyalgia	44	24	24	16	34	8	93	72	2
Stroke	21	10	33	37	52	19	33	17	1
Chronic fatigue syndrome	32	11	21	9	24	7	40	40	0
HIV or AIDS	3	2	0	4	7	2	0	2	0
Liver disease	19	10	5	13	40	10	24	20	0
Number of comorbid cond. in MTMM analysis	22	14	14	20	25	8	24	23

Note: Angina is with or without myocardial infarction in past year. CKD is limited to Stages 3–5 using self-reported eGFR.

^***Comorbid condition not included for this pre-ID group.

^aFor most conditions, the survey item included words in addition to the column labels; full item content available upon request.

Construct validity

To illustrate how analyses were performed and results were interpreted, Table 4 presents results for the pre-ID asthma group and the first two comorbid conditions (obesity and diabetes). This table includes sample sizes for asthma and for subsamples with asthma and comorbid obesity or diabetes (CC). The rows present results for the three different methods of measuring asthma (shortness of breath (As1), severity (As2) and QOL impact (As3)) along with three methods for comorbid obesity and two methods for comorbid diabetes, as documented (see Methods, Table 1 and Appendix Table A-1). The first three columns present results for the three methods of measuring asthma (As1, As2, As3). Entries for the rightmost two columns vary by comorbid condition; for example, for the obesity rows, CC1 is Ob1 (Body mass index) and CC2 is Ob2 (Obesity severity). For the diabetes rows, CC2 is Di2 (Diabetes severity); no symptom measure or clinical marker (CC1) was available for comorbid diabetes when the pre-ID condition was asthma. No CC1 correlations are presented for all comorbid conditions other than obesity, by design.

Table 4.

Sample of Results for Methods of Measuring Asthma and Prevalent Comorbid Conditions

CC1 = Symptom measure for comorbid condition; CC2 = Severity measure for comorbid condition. Higher scores on all measures indicate greater disease severity or impact.

^aShortness of breath defined in Table 1. Scored so higher score=greater shortness of breath.

All correlations used to test convergent validity are bolded. Correlations within the triangle are for the pre-ID condition.

Heterotrait correlations greater than 0.30 in boxed discriminant tests are underlined.

Heterotrait correlations not significantly (p<0.05) lower than their referent convergent correlation are starred (*). Heterotrait correlations r>0.46 failed convergent-discriminant 1-tailed test (p<0.05). Median (r=0.53) correlation from convergent triangle selected for convergent-discriminant comparisons.

Table A-1.

Asthma: Correlations with Prevalent Comorbid Conditions

	Asthma			Comorbid Condition
	As1	As2	As3	CC1	CC2
Asthma (N=742)
As1 Shortness of breatha
As2 Asthma severity	0.52
As3 Asthma QOL impact	0.53	0.70
Obesity (N=211)
Ob1 Body mass index	0.17	0.04	0.03
Ob2 Obesity severity	0.22	0.11	0.16	0.46
Ob3 Obesity QOL impact	0.34	0.13	0.20	0.37	0.66
Diabetes (N=124)
Di2 Diabetes severity	0.05	0.07	0.10	---
Di3 Diabetes QOL impact	0.21	0.09	0.13	---	0.52
Osteoarthritis (N=168)
OA2 OA severity	0.24	0.23	0.22	---
OA3 OA QOL impact	0.31	0.25	0.30	---	0.73
Rheumatoid arthritis (N=74)
RA2 RA severity	0.41	0.42	0.38	---
RA3 RA QOL impact	0.37	0.39	0.38	---	0.84
Allergies-chronic (N=369)
Allc2 Allergy-chronic severity	0.34	0.32	0.35	---
Allc3 Allergy-chronic QOL impact	0.32	0.32	0.43	---	0.73
Allergies-seasonal (N=530)
Alls2 Allergy-seasonal severity	0.26	0.27	0.32	---
Alls3 Allergy-seasonal QOL impact	0.28	0.27	0.36	---	0.76
Anemia (N=135)
An2 Anemia severity	0.07	0.12	0.20	---
An3 Anemia QOL impact	0.14	0.24	0.31	---	0.65
Cancer, non-skin (N=50)
Ca2 Cancer severity	0.07	0.18	0.20	---
Ca3 Cancer QOL impact	0.00	0.13	0.21	---	0.67
Chronic back problems (N=207)
Bp2 Back severity	0.26	0.24	0.30	---
Bp3 Back QOL impact	0.26	0.24	0.28	---	0.82
Depression (N=164)
De2 Depression severity	0.26	0.12	0.20	---
De3 Depression QOL impact	0.23	0.10	0.16	---	0.87
Dermatitis/skin conditions (N=130)
Sk2 Dermatitis severity	0.29	0.03	0.05	---
Sk3 Dermatitis QOL impact	0.26	0.08	0.14	---	0.72
Hearing, trouble (N=96)
He2 Hearing severity	0.25	0.14	0.21	---
He3 Hearing QOL impact	0.16	0.14	0.21	---	0.61
Hypertension (N=260)
Hy2 Hypertension severity	0.27	0.38	0.40	---
Hy3 Hypertension QOL impact	0.29	0.34	0.41	---	0.55
Hypothyroidism (N=107)
Th2 Hypothyroidism severity	0.16	0.17	0.13	---
Th3 Hypothyroidism QOL impact	0.25	0.23	0.23	---	0.67
Irritable bowel syndrome (N=95)
Ib2 IBS severity	0.21	0.32	0.16	---
Ib3 IBS QOL impact	0.24	0.37	0.25	---	0.86
Joint problems, foot/ankle (N=114)
Fa2 Foot/ankle severity	0.24	0.17	0.12	---
Fa3 Foot/ankle QOL impact	0.21	0.11	0.11	---	0.83
Joint problems, hip/knee (N=215)
Hk2 Hip/knee severity	0.24	0.23	0.17	---
Hk3 Hip/knee QOL impact	0.24	0.19	0.18	---	0.81
Limb, limitations in use (N=61)
Lm2 Limb severity	0.22	0.21	0.07	---
Lm3 Limb QOL impact	0.21	0.30	0.15	---	0.88
Migraine headaches (N=179)
Mi2 Migraine severity	0.21	0.19	0.20	---
Mi3 Migraine QOL impact	0.26	0.27	0.29	---	0.90
Osteoporosis (N=56)
Os2 Osteoporosis severity	0.18	0.24	0.17	---
Os3 Osteoporosis QOL impact	0.04	0.28	0.17	---	0.67
Ulcer/stomach disease (N=97)
Ul2 Ulcer/stomach severity	0.26	0.24	0.11	---
Ul3 Ulcer/stomach QOL impact	0.29	0.26	0.15	---	0.76
Vision, trouble (N=94)
Vi2 Vision severity	0.16	0.19	0.09	---
Vi3 Vision QOL impact	0.28	0.31	0.25	---	0.60

CC1 = Symptom measure for comorbid condition; CC2 = Severity measure for comorbid condition.

Higher scores on severity and impact items indicate greater disease severity or greater disease impact.

^aShortness of breath defined in Table 1. Higher scores indicate greater shortness of breath.

^*Heterotrait correlations r>0.46 (absolute value) failed convergent-discriminant 1-tailed test. Median (r=0.53) correlation from convergent triangle selected for convergent-discriminant comparisons.

Monotrait-heteromethod correlations among the three asthma measurement methods, which are contained in the triangle, support the convergent validity of asthma-specific attributions because all convergent correlations for asthma in the triangle were: (a) statistically significant (bolded) and (b) substantial in magnitude (r=0.52–0.70, median=0.53). In every matrix, the median convergent correlation serves as the referent for purposes of significance testing. Further, in support of discriminant validity: (c) heterotrait correlations that should be low within the box were low for 14 of 15 estimates (ranging from 0.04–0.34, median=0.13); the one exception is underlined; and (d) all boxed estimates are significantly (p<0.05) lower than the referent (median) convergent correlation in the triangle. If any of the latter had failed this second test, they would have been starred (*). Taken together, these results support the construct validity of these disease-specific attributions. Convergent correlations for comorbid conditions are presented in the right-most two columns, including three for obesity (the only comorbid condition measured by three methods) and one for diabetes (the correlation between its two methods). Complete results from convergent and discriminant tests for asthma, which are consistent with results in this example, are documented in Appendix Table A-1. Complete results for the other seven pre-ID conditions are documented in Appendix Tables A-2 to A-8.

Table A-2.

COPD: Correlations with Prevalent Comorbid Conditions

	COPD			Comorbid Condition
	Co1	Co2	Co3	CC1	CC2
COPD (N=274)
Co1 Shortness of breatha
Co2 COPD severity	0.60
Co3 COPD QOL impact	0.62	0.84
Obesity (N=78)
Ob1 Body mass index	0.12	0.10	0.15
Ob2 Obesity severity	0.13	0.31	0.32	0.67
Ob3 Obesity QOL impact	0.25	0.38	0.45	0.68	0.82
Asthma (N=138)
As2 Asthma severity	0.44	0.67*	0.67*	---
As3 Asthma QOL impact	0.52*	0.72*	0.79*	---	0.86
Diabetes (N=59)
Di2 Diabetes severity	0.08	0.32	0.31	---
Di3 Diabetes QOL impact	0.25	0.45	0.50	---	0.76
Osteoarthritis (N=82)
OA2 OA severity	0.08	0.21	0.27	---
OA3 OA QOL impact	0.19	0.24	0.36	---	0.80
Allergies-chronic (N=126)
Allc2 Allergy-chronic severity	0.18	0.41	0.41	---
Allc3 Allergy-chronic QOL impact	0.20	0.41	0.52*	---	0.75
Allergies-seasonal (N=146)
Alls2 Allergy-seasonal severity	0.24	0.43	0.48	---
Alls3 Allergy-seasonal QOL impact	0.19	0.38	0.47	---	0.81
Chronic back problems (N=115)
Bp2 Back severity	0.38	0.45	0.46	---
Bp3 Back QOL impact	0.37	0.46	0.52*	---	0.86
Depression (N=66)
De2 Depression severity	0.32	0.58*	0.56*	---
De3 Depression QOL impact	0.27	0.55*	0.51	---	0.87
Hearing, trouble (N=81)
He2 Hearing severity	0.10	0.19	0.15	---
He3 Hearing QOL impact	0.13	0.22	0.28	---	0.52
Hypertension (N=168)
Hy2 Hypertension severity	0.16	0.39	0.35	---
Hy3 Hypertension QOL impact	0.30	0.51	0.51	---	0.59
Hypothyroidism (N=52)
Th2 Hypothyroidism severity	0.02	0.41	0.42	---
Th3 Hypothyroidism QOL impact	0.14	0.42	0.41	---	0.66
Joint problems, hip/knee (N=94)
Hk2 Hip/knee severity	0.25	0.25	0.26	---
Hk3 Hip/knee QOL impact	0.26	0.25	0.29	---	0.87
Osteoporosis (N=52)
Os2 Osteoporosis severity	0.06	0.22	0.17	---
Os3 Osteoporosis QOL impact	0.16	0.24	0.24	---	0.74
Vision, trouble (N=61)
Vi2 Vision severity	0.19	0.29	0.28	---
Vi3 Vision QOL impact	0.20	0.31	0.32	---	0.76

CC1 = Symptom measure for comorbid condition; CC2 = Severity measure for comorbid condition.

Higher scores on severity and impact items indicate greater disease severity or greater disease impact.

^aShortness of breath defined in Table 1. Higher scores indicate greater shortness of breath.

^*Heterotrait correlations r>0.51 (absolute value) failed convergent-discriminant 1-tailed test. Median (r=0.62) correlation from convergent triangle selected for convergent-discriminant comparisons.

Table A-8.

Rheumatoid Arthritis: Correlations with Prevalent Comorbid Conditions

	Rheumatoid Arthritis			Comorbid Condition
	RA1	RA2	RA3	CC1	CC2
Rheumatoid arthritis (N=493)
RA1 Joint paina
RA2 RA severity	0.53
RA3 RA QOL impact	0.49	0.78
Obesity (N=154)
Ob1 Body mass index	0.21	0.24	0.25
Ob2 Obesity severity	0.31	0.33	0.31	0.58
Ob3 Obesity QOL impact	0.23	0.36	0.37	0.45	0.72
Diabetes (N=90)
Di2 Diabetes severity	0.00	0.11	0.07	---
Di3 Diabetes QOL impact	0.11	0.32	0.36	---	0.52
Osteoarthritis (N=177)
OA2 OA severity	0.29	0.29	0.31	---
OA3 OA QOL impact	0.33	0.32	0.45	---	0.83
Allergies-chronic (N=156)
Allc2 Allergy-chronic severity	0.31	0.28	0.24	---
Allc3 Allergy-chronic QOL impact	0.26	0.25	0.33	---	0.68
Allergies-seasonal (N=220)
Alls2 Allergy-seasonal severity	0.19	0.30	0.22	---
Alls3 Allergy-seasonal QOL impact	0.28	0.32	0.31	---	0.76
Anemia (N=91)
An2 Anemia severity	0.31	0.28	0.23	---
An3 Anemia QOL impact	0.27	0.14	0.09	---	0.80
Cancer, non-skin (N=56)
Ca2 Cancer severity	0.23	0.03	0.08	---
Ca3 Cancer QOL impact	0.29	−0.01	−0.11	---	0.49
Chronic back problems (N=200)
Bp2 Back severity	0.47*	0.55*	0.54*	---
Bp3 Back QOL impact	0.45	0.48*	0.55*	---	0.84
Depression (N=99)
De2 Depression severity	0.26	0.42	0.41	---
De3 Depression QOL impact	0.29	0.34	0.42	---	0.87
Dermatitis/skin conditions (N=85)
Sk2 Dermatitis severity	0.10	0.21	0.17	---
Sk3 Dermatitis QOL impact	0.27	0.30	0.27	---	0.77
Erectile dysfunction (N=50)
Ed2 Erectile dysfunction severity	0.23	0.23	0.17	---
Ed3 Erectile dysfunction QOL impact	0.47*	0.40	0.34	---	0.51
Fibromyalgia (N=72)
FM2 Fibromyalgia severity	0.49*	0.49*	0.47*	---
FM3 Fibromyalgia QOL impact	0.49*	0.49*	0.51*	---	0.91
Hearing, trouble (N=77)
He2 Hearing severity	0.03	0.19	0.13	---
He3 Hearing QOL impact	0.13	0.29	0.37	---	0.53
Hypertension (N=269)
Hy2 Hypertension severity	0.26	0.36	0.34	---
Hy3 Hypertension QOL impact	0.32	0.41	0.45	---	0.58
Hypothyroidism (N=90)
Th2 Hypothyroidism severity	0.36	0.28	0.32	---
Th3 Hypothyroidism QOL impact	0.24	0.21	0.30	---	0.75
Irritable bowel syndrome (N=74)
Ib2 IBS severity	0.39	0.15	0.05	---
Ib3 IBS QOL impact	0.45	0.22	0.22	---	0.83
Joint problems, foot/ankle (N=167)
Fa2 Foot/ankle severity	0.33	0.49*	0.46*	---
Fa3 Foot/ankle QOL impact	0.31	0.44	0.49*	---	0.80
Joint problems, hip/knee (N=275)
Hk2 Hip/knee severity	0.40	0.38	0.43	---
Hk3 Hip/knee QOL impact	0.42	0.41	0.55*	---	0.81
Limb, limitations in use (N=75)
Lm2 Limb severity	0.33	0.24	0.25	---
Lm3 Limb QOL impact	0.25	0.25	0.43	---	0.70
Migraine headaches (N=108)
Mi2 Migraine severity	0.15	0.12	0.12	---
Mi3 Migraine QOL impact	0.29	0.20	0.19	---	0.84
Osteoporosis (N=76)
Os2 Osteoporosis severity	0.38	0.52*	0.46*	---
Os3 Osteoporosis QOL impact	0.38	0.42	0.46*	---	0.72
Ulcer/stomach disease (N=76)
Ul2 Ulcer/stomach severity	0.32	0.46*	0.44	---
Ul3 Ulcer/stomach QOL impact	0.27	0.42	0.48*	---	0.78
Vision, trouble (N=84)
Vi2 Vision severity	0.12	0.20	0.26	---
Vi3 Vision QOL impact	0.24	0.41	0.41	---	0.63

CC1 = Symptom measure for comorbid condition; CC2 = Severity measure for comorbid condition.

Higher scores on severity and impact items indicate greater disease severity or greater disease impact.

^aJoint pain defined in Table 1. Higher scores indicate greater joint pain.

^*Heterotrait correlations r>0.45 (absolute value) failed convergent-discriminant 1-tailed test. Median (r=0.53) correlation from convergent triangle selected for convergent-discriminant comparisons.

Applying MTMM logic and specific standards illustrated above, Table 5 summarizes appended results across all eight pre-ID conditions. Collectively, results support the construct validity of condition-specific QOL impact attributions, although there were noteworthy exceptions. First, with respect to convergent validity: (a) the eight sets of three monotrait-heteromethod correlations for pre-ID groups were all statistically significant; and (b) 22 of 24 estimates were substantial in magnitude (r=0.38–0.84, median=0.53). The two exceptions were observed for diabetes and CKD. In support of discriminant validity: (a) across 34–102 heterotrait-heteromethod tests per group, most were below the standard required for discriminant validity (r≤0.30) with medians of 0.18–0.29; and (b) across 17–51 heterotrait-monomethod tests per group, most correlations were below the standard of r≤0.30, with medians ranging 0.17–0.41, as hypothesized. Finally, in support of convergent-discriminant validity, heterotrait (heteromethod and monomethod) correlations were significantly lower than referent convergent correlations in 71–100% (median=90%) of tests across the eight groups (see appended tables for detailed results summarized in Table 5). In other words, in 9 out of 10 cases in which correlations between different conditions (heterotraits) were hypothesized to be low, they were significantly lower than their corresponding convergent correlations. This pattern of results supports construct (convergent-discriminant) validity.

Table 5.

Summary results and success rates for multitrait-multimethod correlational tests of convergent and discriminant validity, eight pre-identified disease groups

						Discriminant validity (Heterotrait)
			Convergent validitya			Heteromethodb			Monomethodc			Successd
Disease	N	CC	#	Mede	Rangee	#	Med	Range	#	Med	Range	Ratio	%
Asthma	742	22	3	0.53	.52–.70	90	0.23	.00–.41	45	0.21	.03–.43	135/135	100
COPD	274	14	3	0.62	.60–.84	58	0.26	.02–.72	29	0.41	.12–.79	77/87	89
Angina	221	14	3	0.60	.56–.77	58	0.25	.00–.49	29	0.29	.05–.65	83/87	95
CHF	271	20	3	0.53	.50–.77	82	0.22	.00–.54	41	0.17	.01–.55	112/123	91
Diabetes	1306	25	3	0.42	.38–.52	102	0.18	.01–.37	51	0.25	.02–.51	144/153	94
CKD	147	8	3	0.45	.39–.72	34	0.20	.02–.48	17	0.26	.00–.49	36/51	71
OA	1026	24	3	0.51	.49–.75	98	0.21	.04–.60	49	0.26	.05–.71	126/147	86
RA	493	23	3	0.53	.49–.78	94	0.29	.00–.54	47	0.33	.03–.55	120/141	85

Note: CC=Number of comorbid conditions with N≥50; #=Number of tests of each type; COPD=chronic obstructive pulmonary disease; CHF=congestive heart failure; CKD=chronic kidney disease; OA=osteoarthritis; RA=rheumatoid arthritis.

^aMTHM=Monotrait-heteromethod, same disease-different methods (convergent validity)

^bHTHM=Heterotrait-heteromethod, different diseases-different methods (discriminant validity)

^cHTMM=Heterotrait-monomethod, different diseases-same method (method effect)

^dConvergent/discriminant success ratio is the number of heterotrait correlations that were significantly lower than the referent (median) convergent correlation divided by the total number of discriminant tests in each disease group. The overall success ratio was 833/924=90.2%.

^eMedian and range (lowest to highest) correlations, absolute value.

As summarized in in the rightmost column of Table 5, the lowest convergent-discriminant success rates were observed for COPD (89%), CKD (71%), OA (86%), and RA (85%). Other clues apparent for COPD include one of the only two median heterotrait-monomethod correlations that was substantial in magnitude (see third set of columns) and also the very high correlations at the top of the ranges (r=0.72 and 0.79) for heterotrait-heteromethod and heterotrait-monomethod correlations, all of which should be low for discriminating measures. Inspection of appended Table A-2 reveals that two comorbid conditions (asthma and depression) accounted for most significant failures to discriminate observed for severity and QOL impact rating methods of measuring COPD. The shortness of breath method produced mixed results with the two asthma ratings. For osteoarthritis (OA), appended Table A-7 reveals that comorbid back problems, fibromyalgia, joint problems (foot/ankle, hip/knee), limitations in limb usage and osteoporosis accounted for all exceptions for OA severity and QOL impact rating methods. A cross-tabulation (not reported) of self-reports of OA versus hip/knee joint problems and limb limitations suggested that the latter two conditions were perceived as subsets of OA. The same pattern of results within the RA group was observed for comorbid back problems, fibromyalgia, joint problems (foot/ankle, hip/knee) and osteoporosis along with other comorbid conditions (see appended Table A-8).

Table A-7.

Osteoarthritis: Correlations with Prevalent Comorbid Conditions

	Osteoarthritis			Comorbid Condition
	OA1	OA2	OA3	CC1	CC2
Osteoarthritis (N=1026)
OA1 Joint paina
OA2 OA severity	0.51
OA3 OA QOL impact	0.49	0.75
Obesity (N=337)
Ob1 Body mass index	0.05	0.15	0.21
Ob2 Obesity severity	0.14	0.22	0.25	0.55
Ob3 Obesity QOL impact	0.19	0.24	0.32	0.39	0.60
Asthma (N=54)
As2 Asthma severity	−0.06	0.28	0.21	---
As3 Asthma QOL impact	0.07	0.37	0.45	---	0.80
Diabetes (N=202)
Di2 Diabetes severity	0.12	0.12	0.09	---
Di3 Diabetes QOL impact	0.17	0.12	0.18	---	0.53
Allergies-chronic (N=300)
Allc2 Allergy-chronic severity	0.10	0.07	0.09	---
Allc3 Allergy-chronic QOL impact	0.17	0.04	0.14	---	0.62
Allergies-seasonal (N=469)
Alls2 Allergy-seasonal severity	0.20	0.19	0.17	---
Alls3 Allergy-seasonal QOL impact	0.17	0.17	0.23	---	0.61
Anemia (N=149)
An2 Anemia severity	0.16	0.18	0.16	---
An3 Anemia QOL impact	0.25	0.25	0.29	---	0.70
Benign prostatic hyperplasia (N=119)
BPH2 BPH severity	0.16	0.19	0.14	---
BPH3 BPH QOL impact	0.25	0.22	0.22	---	0.46
Cancer, non-skin (N=134)
Ca2 Cancer severity	−0.05	0.05	0.11	---
Ca3 Cancer QOL impact	−0.05	0.07	0.17	---	0.85
Chronic back problems (N=425)
Bp2 Back severity	0.39	0.52*	0.48*	---
Bp3 Back QOL impact	0.38	0.48*	0.57*	---	0.81
Depression (N=202)
De2 Depression severity	0.15	0.26	0.29	---
De3 Depression QOL impact	0.12	0.27	0.31	---	0.83
Dermatitis/skin conditions (N=146)
Sk2 Dermatitis severity	0.07	0.15	0.21	---
Sk3 Dermatitis QOL impact	0.11	0.10	0.23	---	0.54
Erectile dysfunction (N=92)
Ed2 Erectile dysfunction severity	0.26	0.20	0.27	---
Ed3 Erectile dysfunction QOL impact	0.19	0.30	0.38	---	0.51
Fibromyalgia (N=93)
FM2 Fibromyalgia severity	0.48*	0.53*	0.50*	---
FM3 Fibromyalgia QOL impact	0.57*	0.50*	0.53*	---	0.86
Hearing, trouble (N=216)
He2 Hearing severity	0.20	0.16	0.11	---
He3 Hearing QOL impact	0.18	0.19	0.24	---	0.58
Hypertension (N=566)
Hy2 Hypertension severity	0.21	0.28	0.27	---
Hy3 Hypertension QOL impact	0.19	0.29	0.32	---	0.47
Hypothyroidism (N=181)
Th2 Hypothyroidism severity	0.24	0.26	0.30	---
Th3 Hypothyroidism QOL impact	0.25	0.27	0.29	---	0.62
Irritable bowel syndrome (N=162)
Ib2 IBS severity	0.19	0.13	0.12	---
Ib3 IBS QOL impact	0.24	0.21	0.26	---	0.74
Joint problems, foot/ankle (N=251)
Fa2 Foot/ankle severity	0.26	0.31	0.31	---
Fa3 Foot/ankle QOL impact	0.33	0.38	0.47*	---	0.78
Joint problems, hip/knee (N=638)
Hk2 Hip/knee severity	0.40	0.61*	0.60*	---
Hk3 Hip/knee QOL impact	0.42	0.58*	0.71*	---	0.79
Limb, limitations in use (N=150)
Lm2 Limb severity	0.32	0.48*	0.45	---
Lm3 Limb QOL impact	0.36	0.53*	0.60*	---	0.78
Migraine headaches (N=216)
Mi2 Migraine severity	0.22	0.21	0.21	---
Mi3 Migraine QOL impact	0.21	0.17	0.22	---	0.86
Osteoporosis (N=146)
Os2 Osteoporosis severity	0.44	0.49*	0.47*	---
Os3 Osteoporosis QOL impact	0.36	0.42	0.54*	---	0.70
Ulcer/stomach disease (N=154)
Ul2 Ulcer/stomach severity	0.25	0.27	0.30	---
Ul3 Ulcer/stomach QOL impact	0.22	0.19	0.26	---	0.78
Vision, trouble (N=129)
Vi2 Vision severity	0.20	0.14	0.08	---
Vi3 Vision QOL impact	0.23	0.20	0.28	---	0.64

CC1 = Symptom measure for comorbid condition; CC2 = Severity measure for comorbid condition.

Higher scores on severity and impact items indicate greater disease severity or greater disease impact.

^aJoint pain defined in Table 1. Higher scores indicate greater joint pain.

^*Heterotrait correlations r>0.45 (absolute value) failed convergent-discriminant 1-tailed test. Median (r=0.51) correlation from convergent triangle selected for convergent-discriminant comparisons.

Convergent correlations for the three methods of measuring CKD were among the lowest observed and this group had the smallest sample. Although the convergent correlations for eGFR were substantial in magnitude, they were not significantly larger than eGFR correlations with ratings of back problems, and CKD severity and QOL impact ratings failed discriminant tests in relation to ratings of comorbid anemia, back problems, and other conditions (see appended Table A-6).

Table A-6.

Chronic Kidney Disease (Stages 3-5): Correlations with Prevalent Comorbid Conditions

	Chronic Kidney Disease			Comorbid Condition
	Kd1	Kd2	Kd3	CC1	CC2
CKD (N=147)
Kd1 Glomerular filtration ratea
Kd2 CKD severity	−0.45
Kd3 CKD QOL impact	−0.39	0.72
Obesity (N=50)
Ob1 Body mass index	0.02	0.20	0.23
Ob2 Obesity severity	−0.05	0.26	0.40*	0.57
Ob3 Obesity QOL impact	0.02	0.22	0.28	0.47	0.76
Diabetes (N=56)
Di2 Diabetes severity	−0.13	0.24	0.20	---
Di3 Diabetes QOL impact	−0.23	0.24	0.34*	---	0.74
Osteoarthritis (N=57)
OA2 OA severity	0.09	−0.01	0.13	---
OA3 OA QOL impact	0.05	−0.06	0.08	---	0.81
Allergies-seasonal (N=60)
Alls2 Allergy-seasonal severity	−0.18	0.21	0.32*	---
Alls3 Allergy-seasonal QOL impact	−0.24	0.32*	0.39*	---	0.72
Anemia (N=56)
An2 Anemia severity	−0.30	0.49*	0.48*	---
An3 Anemia QOL impact	−0.22	0.37*	0.46*	---	0.82
Chronic back problems (N=51)
Bp2 Back severity	−0.37*	0.37*	0.37*	---
Bp3 Back QOL impact	−0.33*	0.41*	0.48*	---	0.88
Hypertension (N=121)
Hy2 Hypertension severity	−0.07	0.25	0.12	---
Hy3 Hypertension QOL impact	−0.02	0.17	0.20	---	0.58
Joint problems, hip/knee (N=59)
Hk2 Hip/knee severity	−0.04	0.00	0.18	---
Hk3 Hip/knee QOL impact	0.08	0.02	0.27	---	0.79

CC1 = Symptom measure for comorbid condition; CC2 = Severity measure for comorbid condition.

Higher scores on severity and impact items indicate greater disease severity or greater disease impact.

^aEstimated glomerular filtration rate defined in Table 1. Lower eGFR indicates worse kidney function.

^*Heterotrait correlations r<−0.31 (Kd1 column) or r>0.31 (Kd2 and Kd3 columns) failed convergent-discriminant 1-tailed test. Median (r=−0.45 (Kd1 tests) or r=0.45 (Kd2 and Kd3 columns) correlation from convergent triangle selected for convergent-discriminant comparisons.

The third set of comparisons in Table 5 (heterotrait-monomethod) distinguishes 17–51 correlations per disease from corresponding heterotrait-heteromethod correlations (second set of comparisons) for purposes of testing for any method effects. To the extent that the third set was larger than the second set, methods effects are supported. The most noteworthy method effect, observed for COPD, is documented above.

Discussion and Conclusions

Results from 22 of 24 convergent (multiple methods for same condition) tests supported the construct validity of disease-specific QOL impact attributions across eight disease groups, for which three-fourths reported three or more MCC. Further, across more than 900 discriminant tests, 9 out of 10 results supported construct validity. With some exceptions, these results support the validity of a global disease-specific QOL impact score based on the attribution to each condition for purposes of monitoring QOL outcomes, as well as the aggregation of impact ratings to estimate total MCC burden in QOL terms. We discuss these issues along with limitations of the study and summarize our conclusions below.

Lessons from noteworthy exceptions

Variability and exceptions in patterns of discriminant validity findings provide clues for further studies of the validity of QOL impact attributions. Among the conditions with the lowest convergent-discriminant success rates – CKD (71%), RA (85%), OA (86%) and COPD (89%) – several patterns stand out as reasons for caution in interpretations and potentially fruitful targets of additional research. For example, among the weakest evidence of discriminant validity were correlations between multiple methods studied among patients with both COPD and asthma. One clue warranting more in-depth analysis is that COPD and asthma have a specific symptom in common – shortness of breath – that correlated substantially with measures of the quality of life impact attributed to both conditions. The practical implication is very high correlations (r=0.67–0.79) between ratings of different conditions that should be low (r≤0.30) according to the discriminant validity standard. Distinctions between these conditions have been debated, which suggests that there may be clinical explanations for any lack of distinction; regardless, results reported here suggest that patient ratings of severity and QOL impact should be interpreted with caution when both of these specific conditions are present, which was a common occurrence in the current study.

Another example is congestive heart failure and comorbid angina. Although convergent correlations for CHF were more than substantial (r=0.50–0.77) and the overall success rate for CHF was high (112/123 tests, 91%), all six of the heterotrait estimates for comorbid angina that should be low (r≤0.30) according to the discriminant validity standard were substantial (r=0.47–0.55) and none were significantly lower than the corresponding convergent correlations. Analogous to the COPD-asthma example above, CHF and angina ratings consistently failed in discriminant tests of validity and should be interpreted with caution when both conditions are present. These patterns of results underscore the importance of comprehensive multitrait-multimethod validations of tools used in studies of MCC and also call into question how specific some widely-used disease-specific measures actually are. The addition of correlation estimates using more objective clinical measures for comorbid conditions in future studies would make it possible to better interpret results and evaluate alternate explanations.

Another pattern of exceptions may be the result of asking patients to evaluate conditions that are a subset of another, for example, joint problems (hip/knee) and limitations in limb usage as conditions within osteoarthritis (OA). However, this does not explain high correlations observed between QOL impact for OA and other conditions such as fibromyalgia. A similar pattern of results was observed within the RA group for comorbid back problems, fibromyalgia, joint problems (foot/ankle and hip/knee) and osteoporosis, among others. These exceptions also suggest the need for caution in interpreting patient attributions of the QOL impact of each condition in the presence of these specific MCC.

Convergent correlations for the three methods of measuring CKD were among the lowest observed. To the extent that convergence is weak, success in tests of discriminant validity is less likely. Although the convergent correlations for eGFR were substantial in magnitude, they were not significantly larger than the discriminant correlations for ratings of back problems; also, both CKD severity and QOL impact ratings failed discriminant tests in relations to ratings of comorbid anemia and back problems. Are fatigue or joint pain underlying problems common to CKD and these comorbid conditions? Suggesting that CKD-specific attributions are CKD-specific, a study comparing them with generic measures across clinically-determined eGFR groups found that the CKD QDIS measures were the most valid of those studied [33]. There also is strong evidence that CKD-specific QOL ratings are more valid in discriminating across severity levels, in comparison with generic physical and mental measures with attributions to health in general [21,34].

The value of replications across multiple conditions and multiple methods in interpreting results is also illustrated in the appended tables. For example, the failure of ratings attributed to comorbid depression in three of six discriminant tests within the COPD group might have been interpreted as evidence that depressed patients make less valid attributions. However, among the five other pre-ID groups (angina, asthma, diabetes, OA and RA) with samples sufficient to analyze comorbid depression, 28 of 30 (93.3%) tests met the criteria (corresponding convergent significantly higher than discriminant (heterotrait) correlation). An alternative explanation for the COPD findings, consistent with some prior research [35], is a substantial association between traits (i.e., COPD and depression) as experienced by patients.

Methodological issues

Multiple standards of convergent-discriminant validity were applied because of the complexity of evaluating construct validity. In a correlated world, valid methods of measuring each of two different traits (i.e., pairs of chronic conditions) should be correlated to the extent that the traits are correlated. If so, the first standard requiring low (r≤0.30) correlations between discriminating measures of different traits will not be met. For this reason, the second standard is based on formal tests of whether corresponding convergent correlations across different methods of measuring the same condition are significantly higher than those for comorbid conditions. For example, within the asthma group, although many of the discriminant (heterotrait) correlations with comorbid conditions exceeded the r≤0.30 standard, tests of the second standard revealed that 100% were significantly lower than the corresponding convergent correlation.

The MTMM approach applied here is in contrast to typical validation paradigms that yield construct evidence of validity in the form of correlations between different methods in single-disease studies or that link different conditions to a generic measure. The most common approaches have been single-disease comparisons of groups differing in severity within that disease, without regard to the presence or influence of MCC. Better ways to test measurement specificity (discriminant validity) are to compare convergent and discriminant tests, or to compare disease-specific and generic measures in relation to a specific criterion. Otherwise, results say little or nothing about specificity of measurement or discriminant validity. Despite their advantages, validations of disease-specific PRO measures employing multiple methods across samples with multiple chronic conditions are rare. To the best of our knowledge, this is the first attempt at replicating multiple methods within each disease across multiple chronic conditions.

Limitations

A limitation of this study is reliance entirely on self-report for disease severity and impact as well as legacy disease-specific measures. Although self-report methods have been useful in validation, results may have been different if “criteria” had been based on independent clinical judgement. However, it is reasonable to expect that mere response bias would not account for the superior discriminant validity of a self-report for one disease in comparison with the same report for another comorbid condition, and the data consistently support this expectation. Examples of method tests that were more distinct than the two rating methods were the body mass index method for obesity and eGFR for CKD. Results for obesity were strong in discriminating across all pre-identified conditions studied. Regardless, future validity tests would be strengthened by using clinician-reported methods and other more independent and objective methods, which are also likely to make study findings more useful in establishing interpretation guidelines. An example is a recent study of the validity of QOL attributions to smoking that used biomarkers of smoking exposure as a method of construct validation [36].

Another limitation is the lack of specific symptoms and more objective markers as methods of measuring comorbid conditions. Because of constraints on respondent burden during the DICAT study, which required large disease-specific QOL impact item banks and other disease-specific surveys, only two ratings (severity, QOL impact) were possible for almost all comorbid conditions. High priority should be placed on achieving at least three methods for each chronic condition and on making those methods as distinctly different as possible, including clinical metrics of interest to clinical researchers and stronger convergent tests. Another advantage of three or more methods per trait (condition) is the identification of confirmatory multitrait-multimethod models that enable formal tests of model fit indicative of convergent and discriminant validity [28].

Suggestions for future research

Regardless of whether specificity is only moderate, in the aggregate a summary MCC QOL impact estimate including moderately discriminating measures may still provide a very useful QOL-based comorbidity index for use in health outcomes research. Its advantages include a QOL metric that is standardized (across diseases) and calibrated in terms of QOL impact, as opposed to a mortality weight such as that used in the Charlson index. The standardized global QOL impact measures analyzed here may also enable a pathway to more directly comparable disease-specific QOL impact measures that are practical enough to enable an individualized summary measure of QOL impact across multiple comorbid conditions. However, this advance presumes that disease-specific attributions are valid in the presence of multiple comorbid conditions. They appear to be sufficiently so to warrant further research addressing the issues discussed above as well as addressing different aggregation approaches and we recommend such further investigation.

Screening to meet sample quotas sufficient to analyze more permutations of MCC beyond the samples of pairs of conditions that were possible within the DICAT database should be a priority for future research. For example, no tests could be performed for six comorbid conditions (see Table 3). Further, 3-disease models were beyond the scope of the current paper, although three-fourths of the pre-ID sample reported three or more conditions. Inspection of sample sizes for the most prevalent MCC combinations indicated that many are large enough for 3-disease MTMM correlation matrices by adding a third condition to a pre-ID condition/comorbid pair reported here. Exploring these MCC triads would be very useful in planning future studies using the MTMM approach.

Some of the advantages of standardization of content and scoring of MCC using QOL-based weights [37] have been realized using generic QOL measures, because the generic physical component of QOL is related to counts of conditions and conditions weighted for QOL impact. The theoretical advantage of disease-specific QOL weighting is that the estimates are more specific to each index condition and, therefore, more attributable and actionable in relation to that specific condition. Carrying this logic another step forward, advantages of individualized over population weights for each MCC may be a substantial further gain in precision. The rating methods evaluated here make individualized MCC QOL impact estimates practical and the overall pattern of results suggests they may have sufficient construct validity.

Conclusions

Although there are noteworthy exceptions, collective results reported here indicate very high rates of success in convergent and divergent tests, in support of the construct validity of disease-specific QOL impact attributions across multiple chronic conditions. Hence, a global QOL impact measure that uses disease-specific attribution is a valid addition to the toolkit for monitoring patient-reported outcomes for each of the conditions studied here. Further, results support aggregating disease-specific QOL impact ratings across conditions to estimate total MCC burden in QOL terms. For the pairs of MCC yielding noteworthy exceptions, the interpretation of some specific QOL impact attributions should be pursued with caution and those exceptions should be the focus of further studies using the multitrait-multimethod approach, strengthened with the inclusion of clinical markers and other objective methods to supplement the QOL ratings studied here. With respect to the overall objective of this study, we conclude that the collective pattern of results supports the validity of disease-specific quality of life attributions sufficiently to warrant their use among adults with multiple chronic conditions.

Table A-3.

Angina: Correlations with Prevalent Comorbid Conditions

	Angina			Comorbid Condition
	Ang1	Ang2	Ang3	CC1	CC2
Angina (N=221)
Ang1 Chest pain/tightnessa
Ang2 Angina severity	0.56
Ang3 Angina QOL impact	0.60	0.77
Obesity (N=75)
Ob1 Body mass index	0.08	0.11	0.13
Ob2 Obesity severity	0.24	0.14	0.20	0.64
Ob3 Obesity QOL impact	0.25	0.18	0.29	0.55	0.71
Diabetes (N=73)
Di2 Diabetes severity	0.16	0.39	0.38	---
Di3 Diabetes QOL impact	0.25	0.44	0.48	---	0.73
Osteoarthritis (N=74)
OA2 OA severity	0.25	0.20	0.15	---
OA3 OA QOL impact	0.36	0.30	0.27	---	0.83
Allergies-chronic (N=84)
Allc2 Allergy-chronic severity	-0.03	0.14	0.12	---
Allc3 Allergy-chronic QOL impact	0.08	0.21	0.20	---	0.68
Allergies-seasonal (N=103)
Alls2 Allergy-seasonal severity	0.09	0.21	0.29	---
Alls3 Allergy-seasonal QOL impact	0.29	0.39	0.41	---	0.69
Chronic back problems (N=111)
Bp2 Back severity	0.11	0.27	0.24	---
Bp3 Back QOL impact	0.17	0.40	0.35	---	0.81
Depression (N=52)
De2 Depression severity	0.46	0.30	0.35	---
De3 Depression QOL impact	0.48	0.44	0.54*	---	0.83
Dermatitis/skin conditions (N=50)
Sk2 Dermatitis severity	0.28	0.19	0.31	---
Sk3 Dermatitis QOL impact	0.45	0.40	0.39	---	0.72
Erectile dysfunction (N=56)
Ed2 Erectile dysfunction severity	0.00	0.13	0.14	---
Ed3 Erectile dysfunction QOL impact	0.43	0.40	0.52*	---	0.39
Hearing, trouble (N=75)
He2 Hearing severity	-0.11	-0.05	-0.03	---
He3 Hearing QOL impact	-0.01	0.07	0.18	---	0.54
Hypertension (N=161)
Hy2 Hypertension severity	0.40	0.46	0.48	---
Hy3 Hypertension QOL impact	0.46	0.49*	0.65*	---	0.61
Joint problems, foot/ankle (N=51)
Fa2 Foot/ankle severity	0.25	0.26	0.25	---
Fa3 Foot/ankle QOL impact	0.28	0.29	0.29	---	0.88
Joint problems, hip/knee (N=100)
Hk2 Hip/knee severity	0.21	0.20	0.17	---
Hk3 Hip/knee QOL impact	0.33	0.33	0.30	---	0.77
Vision, trouble (N=50)
Vi2 Vision severity	0.18	0.32	0.27	---
Vi3 Vision QOL impact	0.23	0.32	0.31	---	0.85

CC1 = Symptom measure for comorbid condition; CC2 = Severity measure for comorbid condition.

Higher scores on severity and impact items indicate greater disease severity or greater disease impact.

^aChest pain defined in Table 1. Higher scores indicate greater chest pain.

^*Heterotrait correlations r>0.48 (absolute value) failed convergent-discriminant 1-tailed test. Median (r=0.60) correlation from convergent triangle selected for convergent-discriminant comparisons.

Table A-4.

Congestive Heart Failure: Correlations with Prevalent Comorbid Conditions

	Congestive Heart Failure			Comorbid Condition
	Chf1	Chf2	Chf3	CC1	CC2
Congestive Heart Failure (N=271)
Chf1 Shortness of breatha
Chf2 CHF severity	0.50
Chf3 CHF QOL impact	0.53	0.77
Obesity (N=101)
Ob1 Body mass index	0.28	0.01	0.01
Ob2 Obesity severity	0.29	0.08	0.03	0.62
Ob3 Obesity QOL impact	0.42	0.16	0.16	0.52	0.65
Angina (N=68)
Ang2 Angina severity	0.47*	0.55*	0.48*	---
Ang3 Angina QOL impact	0.53*	0.54*	0.50*	---	0.76
Asthma (N=59)
As2 Asthma severity	0.32	0.17	0.13	---
As3 Asthma QOL impact	0.35	0.09	0.09	---	0.81
COPD (N=61)
Co2 COPD severity	0.31	0.05	0.22	---
Co3 COPD QOL impact	0.44*	0.11	0.24	---	0.84
Diabetes (N=92)
Di2 Diabetes severity	0.17	0.25	0.20	---
Di3 Diabetes QOL impact	0.16	0.13	0.13	---	0.61
Osteoarthritis (N=101)
OA2 OA severity	-0.05	-0.08	-0.13	---
OA3 OA QOL impact	0.15	0.07	0.01	---	0.72
Allergies-chronic (N=95)
Allc2 Allergy-chronic severity	0.22	0.09	0.11	---
Allc3 Allergy-chronic QOL impact	0.27	0.00	0.14	---	0.65
Allergies-seasonal (N=123)
Alls2 Allergy-seasonal severity	0.19	−0.01	−0.02	---
Alls3 Allergy-seasonal QOL impact	0.30	0.10	0.06	---	0.78
Anemia (N=67)
An2 Anemia severity	0.15	0.04	0.03	---
An3 Anemia QOL impact	0.28	0.11	0.09	---	0.78
Benign prostatic hyperplasia (N=55)
BPH2 BPH severity	0.40	0.23	0.23	---
BPH3 BPH QOL impact	0.41	0.28	0.26	---	0.60
Chronic back problems (N=95)
Bp2 Back severity	0.03	0.18	0.05	---
Bp3 Back QOL impact	0.15	0.15	0.08	---	0.83
Dermatitis/skin conditions (N=58)
Sk2 Dermatitis severity	0.36	0.26	0.36	---
Sk3 Dermatitis QOL impact	0.28	0.08	0.23	---	0.61
Erectile dysfunction (N=59)
Ed2 Erectile dysfunction severity	0.19	0.23	0.26	---
Ed3 Erectile dysfunction QOL impact	0.25	0.34	0.33	---	0.64
Hearing, trouble (N=88)
He2 Hearing severity	-0.01	-0.05	-0.10	---
He3 Hearing QOL impact	0.27	-0.03	-0.07	---	0.53
Hypertension (N=200)
Hy2 Hypertension severity	0.31	0.27	0.23	---
Hy3 Hypertension QOL impact	0.41	0.38	0.45*	---	0.57
Hypothyroidism (N=52)
Th2 Hypothyroidism severity	0.42	0.51*	0.37	---
Th3 Hypothyroidism QOL impact	0.34	0.47*	0.48*	---	0.72
Joint problems, foot/ankle (N=62)
Fa2 Foot/ankle severity	0.28	0.07	0.08	---
Fa3 Foot/ankle QOL impact	0.40	0.18	0.15	---	0.64
Joint problems, hip/knee (N=122)
Hk2 Hip/knee severity	0.37	0.22	0.16	---
Hk3 Hip/knee QOL impact	0.36	0.23	0.15	---	0.81
Limb, limitations in use (N=51)
Lm2 Limb severity	0.31	0.40	0.28	---
Lm3 Limb QOL impact	0.30	0.32	0.20	---	0.84
Vision, trouble (N=54)
Vi2 Vision severity	0.03	−0.19	−0.22	---
Vi3 Vision QOL impact	0.01	−0.16	−0.14	---	0.62

CC1 = Symptom measure for comorbid condition; CC2 = Severity measure for comorbid condition.

Higher scores on severity and impact items indicate greater disease severity or greater disease impact.

^aShortness of breath defined in Table 1. Higher scores indicate greater shortness of breath.

^*Heterotrait correlations r>0.42 (absolute value) failed convergent-discriminant 1-tailed test. Median (r=0.53) correlation from convergent triangle selected for convergent-discriminant comparisons.

Table A-5.

Diabetes: Correlations with Prevalent Comorbid Conditions

	Diabetes			Comorbid Condition
	Di1	Di2	Di3	CC1	CC2
Diabetes (N=1306)
Di1 Diabetes complicationsa
Di2 Diabetes severity	0.38
Di3 Diabetes QOL impact	0.42	0.52
Obesity (N=548)
Ob1 Body mass index	0.03	0.10	0.01
Ob2 Obesity severity	0.17	0.28	0.19	0.47
Ob3 Obesity QOL impact	0.20	0.30	0.37*	0.43	0.62
Asthma (N=63)
As2 Asthma severity	0.16	−0.02	0.03	---
As3 Asthma QOL impact	0.19	0.17	0.15	---	0.75
Osteoarthritis (N=128)
OA2 OA severity	0.18	0.25	0.19	---
OA3 OA QOL impact	0.26	0.28	0.24	---	0.84
Rheumatoid arthritis (N=84)
RA2 OA severity	−0.01	0.28	0.29	---
RA3 OA QOL impact	0.03	0.32	0.34	---	0.81
Allergies-chronic (N=252)
Allc2 Allergy-chronic severity	0.05	0.12	0.12	---
Allc3 Allergy-chronic QOL impact	0.17	0.22	0.32	---	0.68
Allergies-seasonal (N=520)
Alls2 Allergy-seasonal severity	0.14	0.18	0.22	---
Alls3 Allergy-seasonal QOL impact	0.21	0.22	0.29	---	0.63
Anemia (N=143)
An2 Anemia severity	0.02	0.04	0.06	---
An3 Anemia QOL impact	0.14	0.11	0.10	---	0.73
Benign prostatic hyperplasia (N=145)
BPH2 BPH severity	0.25	0.31	0.21	---
BPH3 BPH QOL impact	0.37*	0.35	0.44*	---	0.56
Cancer, non-skin (N=134)
Ca2 Cancer severity	0.02	0.14	0.03	---
Ca3 Cancer QOL impact	0.08	0.13	0.02	---	0.71
Chronic back problems (N=292)
Bp2 Back severity	0.21	0.26	0.26	---
Bp3 Back QOL impact	0.24	0.25	0.31	---	0.77
Depression (N=172)
De2 Depression severity	0.29	0.36	0.35	---
De3 Depression QOL impact	0.29	0.31	0.37*	---	0.85
Dermatitis/skin conditions (N=123)
Sk2 Dermatitis severity	0.14	0.19	0.26	---
Sk3 Dermatitis QOL impact	0.09	0.18	0.41*	---	0.54
Erectile dysfunction (N=213)
Ed2 Erectile dysfunction severity	0.21	0.18	0.19	---
Ed3 Erectile dysfunction QOL impact	0.24	0.20	0.29	---	0.48
Hearing, trouble (N=239)
He2 Hearing severity	−0.01	0.05	0.03	---
He3 Hearing QOL impact	0.22	0.24	0.29	---	0.48
Hypertension (N=900)
Hy2 Hypertension severity	0.17	0.38*	0.30	---
Hy3 Hypertension QOL impact	0.21	0.28	0.51*	---	0.45
Hypothyroidism (N=183)
Th2 Hypothyroidism severity	0.14	0.24	0.26	---
Th3 Hypothyroidism QOL impact	0.14	0.12	0.30	---	0.65
Irritable bowel syndrome (N=114)
Ib2 IBS severity	0.21	0.22	0.34	---
Ib3 IBS QOL impact	0.16	0.14	0.36	---	0.79
Joint problems, foot/ankle (N=171)
Fa2 Foot/ankle severity	0.12	0.27	0.25	---
Fa3 Foot/ankle QOL impact	0.10	0.27	0.26	---	0.76
Joint problems, hip/knee (N=331)
Hk2 Hip/knee severity	0.15	0.17	0.17	---
Hk3 Hip/knee QOL impact	0.17	0.20	0.23	---	0.76
Limb, limitations in use (N=128)
Lm2 Limb severity	0.06	0.02	0.01	---
Lm3 Limb QOL impact	0.11	0.11	0.11	---	0.77
Migraine headaches (N=139)
Mi2 Migraine severity	0.22	0.10	0.16	---
Mi3 Migraine QOL impact	0.17	0.15	0.18	---	0.88
Osteoporosis (N=65)
Os2 Osteoporosis severity	0.10	0.39*	0.26	---
Os3 Osteoporosis QOL impact	−0.09	0.24	0.25	---	0.63
Stroke (N=52)
St2 Stroke severity	0.14	0.20	0.34	---
St3 Stroke QOL impact	0.03	0.02	0.31	---	0.74
Ulcer/stomach disease (N=113)
Ul2 Ulcer/stomach severity	0.13	0.23	0.25	---
Ul3 Ulcer/stomach QOL impact	0.21	0.31	0.40*	---	0.73
Vision, trouble (N=160)
Vi2 Vision severity	0.18	0.05	0.18	---
Vi3 Vision QOL impact	0.26	0.11	0.32	---	0.66

CC1 = Symptom measure for comorbid condition; CC2 = Severity measure for comorbid condition.

Higher scores on severity and impact items indicate greater disease severity or greater disease impact.

^aNumber of diabetes complications defined in Table 1.

^*Heterotrait correlations r>0.36 (absolute value) failed convergent-discriminant 1-tailed test. Median (r=0.42) correlation from convergent triangle selected for convergent-discriminant comparisons.