Abstract
Objective
Gout is often defined by self-report in epidemiologic studies. Yet the validity of self-reported gout is uncertain. We evaluated the reliability and sensitivity of the self-report of physician-diagnosed gout in the Campaign Against Cancer and Heart Disease (CLUE II) and the Atherosclerosis Risk in the Community (ARIC) cohorts.
Methods
The CLUE II cohort comprises 12,912 individuals who self-reported gout status on either the 2000, 2003, or 2007 questionnaires. We calculated reliability as the percentage of participants reporting having gout on more than 1 questionnaire using Cohen’s κ statistic. The ARIC cohort comprises 11,506 individuals who self-reported gout status at visit 4. We considered a hospital discharge diagnosis of gout or use of a gout-specific medication as the standard against which to calculate the sensitivity of self-reported, physician-diagnosed gout.
Results
Of the 437 CLUE II participants who self-reported physician-diagnosed gout in 2000, and subsequently answered the 2003 questionnaire, 75% reported gout in 2003 (κ = 0.73). Of the 271 participants who reported gout in 2000, 73% again reported gout at the 2007 followup questionnaire (κ = 0.63). In ARIC, 196 participants met the definition for gout prior to visit 4 and self-reported their gout status at visit 4. The sensitivity of a self-report of physician-diagnosed gout was 84%. Accuracy was similar across sex and race subgroups, but differed across hyperuricemia and education strata.
Conclusion
These 2 population-based US cohorts suggest that self-report of physician-diagnosed gout has good reliability and sensitivity. Thus, self-report of a physician diagnosis of gout is appropriate for epidemiologic studies.
Key Indexing Terms: GOUT, SENSITIVITY, RELIABILITY, EPIDEMIOLOGY
Gout is a major cause of inflammatory arthritis in the United States1. In the clinical setting, the diagnostic “gold standard” for gouty arthritis is the demonstration of monosodium urate crystals in synovial fluid or in a tophus. Alternatively, the classification of primary gout can be achieved with high specificity using the American College of Rheumatology (ACR) criteria2, which require the presence of 6 of 12 specific clinical, laboratory, or radiographic features2. While these diagnostic methods may be appropriate in clinical trials and in the context of routine patient care, they are quite difficult and often impractical to apply to large population-based investigations. Yet large epidemiological studies are needed to establish clinical and genetic risk factors for gout in the community setting. Such large-scale cohorts provide sufficient numbers of participants to identify both incident and prevalent cases of gout. Thus, an alternative means of case ascertainment is necessary to study gout at the population level when diagnostic arthrocentesis —used to confirm the presence of urate crystals within synovial fluid — is either not available or not feasible.
We did not identify a study in which the reliability and sensitivity of a self-report of physician-diagnosed gout in a US population-based or community-based setting was assessed. Studies of physicians3 and other health professionals4 with gout have confirmed self-reported diagnoses of gout. However, this approach is susceptible to misclassification in that participants who do not self-report may in fact have gout, but would never be confirmed because the confirmation process is only applied to the self-reported cases. The reliability of self-reported gout has also been examined in European population-based studies over a short-term period not exceeding 2 years of followup5,6. Gout claims in an administrative database were validated in 1 study, but the validity of self-reported gout was not determined7. Importantly, the sensitivity of a self-reported, physician diagnosis of gout has not been assessed against hospital discharge diagnoses of gout or prescription data for gout-specific medication.
Thus, an important step toward future gout-related epidemiological research is the determination of the longterm reliability and sensitivity of self-reported gout from longitudinal cohort studies. Reliability refers to the ability of participants to consistently self-report the same gout status over time on multiple questionnaires or at multiple visits. Sensitivity is the ability of participants to correctly self-report their gout status among those who truly have gout. Our objective was to evaluate the reliability and sensitivity of a self-report of physician-diagnosed gout in the Campaign Against Cancer and Heart Disease (CLUE II) cohort and the Atherosclerosis Risk in the Community (ARIC) cohort. First, we evaluated the 3-year and 7-year reliability of a self-report of physician-diagnosed gout in CLUE II and further examined the reliability of reported age of gout onset in this cohort. Next, utilizing the ARIC cohort, we assessed the sensitivity of a self-report of physician-diagnosed gout in participants with a hospitalization in which gout was listed as a discharge diagnosis or among those in possession of a prescription for a gout-specific medication.
MATERIALS AND METHODS
CLUE II materials and methods
CLUE II is a community-based cohort of 32,894 individuals residing within or surrounding Washington County, Maryland, USA. The cohort was established in 1989 (Figure 1). The Institutional Review Board of Johns Hopkins University approved the CLUE study protocol and study participants provided written informed consent. Followup questionnaires were administered in 1996, 1998, 2000, 2003, and 2007. All CLUE II participants who answered the 2000 questionnaire were included in the study population because this was the first questionnaire in which participants were queried about their gout status. The 2000, 2003, and 2007 questionnaires included a question asking, “Have you ever been told by a doctor or other health professional that you have gout?”. To those with an affirmative response, the 2000 questionnaire included additional questions regarding the year of gout onset. In contrast, the 2003 and 2007 questionnaires included questions about the age (rather than calendar year) of gout onset.
We assessed the reliability of a self-report of physician-diagnosed gout by calculating the percentage of participants who reported gout to be present on more than one questionnaire, among those who answered the previous questionnaire. Additionally, Cohen’s κ statistic was calculated to estimate the strength of agreement. A κ of 0.8–1.0 was considered to be very good agreement; 0.6–0.8 good agreement; 0.4–0.6 moderate agreement; and < 0.4 was low agreement. This measure was calculated for participants who responded to the 2000 and 2003, the 2003 and 2007, the 2000 and 2007 questionnaires and for those who responded to all 3 questionnaires. In addition, for those participants who reported physician-diagnosed gout on the 2003 and 2007 questionnaires, we calculated the Spearman correlation coefficient between the 2 self-reported ages of gout onset; a nonparametric measure was used because the 2 measures of age were not normally distributed. Additionally, we calculated the mean and median difference in self-reported ages of gout onset from the data furnished in these 2 different questionnaires. To assess agreement, we plotted the average of self-reported age of gout onset against the difference of the self-reported age of gout onset (Bland-Altman plot)8.
ARIC materials and methods
ARIC is a population-based cohort study of 15,792 individuals recruited in 1987–89 from 4 US communities (Washington County, Maryland; Forsyth County, North Carolina; Jackson, Mississippi; Minneapolis, Minnesota; Figure 2). The institutional review board of the participating institutions (Johns Hopkins University, University of Mississippi, Wake Forest University, University of Minnesota, Baylor University, University of Texas, and University of North Carolina) approved the ARIC study protocol and study participants provided written informed consent. This study consisted of 1 baseline visit (visit 1) and 3 followup visits (visits 2, 3, and 4) administered 3 years apart. Importantly, there was a surveillance component to ARIC throughout the followup period. When a participant was hospitalized within the catchment area of the participating institutions, all of the corresponding discharge diagnoses were recorded.
Our study population consisted of men and women who self-reported their gout status at visit 4. Notably, at ARIC visit 4 each participant was asked, “Has a doctor ever told you that you had gout?”. Participants who answered “Yes” were considered to have a self-reported, physician-diagnosed case of gout. If a participant was recorded through surveillance as having a hospital discharge summary that listed an International Classification of Diseases (ICD)-9 code for gout (274.0, 274.1, 274.8, or 274.9), then they were considered to have gout based on this assignment of a gout discharge diagnosis. Therefore, if a participant attended visit 4 and the date of the gout-related hospitalization was prior to the visit 4 date, they were considered to be a gout case for the assessment of sensitivity. In addition, at each of the 4 ARIC visits, all medications used within the preceding month were recorded. We defined gout medications as colchicine, probenecid, and allopurinol. If a participant reported the use of any of these 3 medications at any study visit, they were considered to be a gout case. In our study, the gold standard for a diagnosis of gout was defined as either a hospital discharge diagnosis of gout or use of gout medication at any cohort visit. Although a prescription for these medications does not mean with absolute certainty that the ARIC participant has gout, in a random sample of 4 US communities, a prescription for colchicine, allopurinol, or probenecid is most likely issued to treat gout.
The gold standard gout definition was applied to all participants who attended visit 4. We calculated the sensitivity of a self-report of physician-diagnosed gout. Sensitivity was defined as the percentage of gold standard gout cases with a corresponding affirmative self-report of gout on the visit 4 questionnaire. Next, we conducted a stratified analysis for the sensitivity of a self-report of physician-diagnosed gout by sex, race, education, and hyperuricemia (serum urate level > 7.0 mg/dl at either visit 1 or 2) categories.
Additionally, we performed a sensitivity analysis to assess whether the sensitivity of a report of gout at visit 4 depended upon the definition of the gold standard. Specifically, we calculated sensitivity, separately, for participants with a hospital discharge diagnosis of gout as well as for those using gout medications. All analyses were performed in SAS, version 9.1 (SAS Institute, Cary, NC, USA).
RESULTS
Reliability of self-reported, physician-diagnosed gout in CLUE II
Figure 1 displays the number of CLUE II participants who self-reported gout and were included in the reliability study. We first examined the reliability of the self-report of physician-diagnosed gout and found good reliability based on the κ statistic (Table 1). Of the 437 participants who reported physician-diagnosed gout in 2000 and thereafter answered the 2003 questionnaire, 75% again reported physician-diagnosed gout in 2003 (κ = 0.73). In addition, of the 271 participants who reported gout in 2000 and subsequently answered the 2007 questionnaire, 73% (κ = 0.63) also reported physician-diagnosed gout on this followup questionnaire. Of the 277 participants who reported physician-diagnosed gout on the 2003 questionnaire, 81% (κ = 0.70) reported physician-diagnosed gout on the 2007 questionnaire. Of the 247 participants who reported gout in 2000, 160 participants (65%) reported physician-diagnosed gout on all 3 questionnaires.
Table 1.
Characteristics | Questionnaires | ||
---|---|---|---|
2000 | 2003 | 2007 | |
No. participants | 12,797 | 9296 | 6873 |
Participants reporting gout (%) | 609 (4.8) | 523 (5.6) | 393 (5.7) |
Participants reporting their gout status in 2000 and on the subsequent questionnaire | 437 | 271 | |
Participants reporting gout in 2000 and on the subsequent questionnaire | — | 326 | 197 |
Reliability, % (κ) | — | 75 (0.73)* | 73 (0.63)* |
Participants reporting their gout status in 2003 and on the 2007 questionnaire | — | — | 277 |
Participants reporting gout in 2003 and on the 2007 questionnaire | — | — | 225 |
Reliability, % (κ) | — | — | 81 (0.70)* |
p < 0.001.
Among the 190 participants who reported the age of gout onset in 2003 and 2007, we found high agreement among 3 different measures. First, the Spearman correlation coefficient for the age of onset was 0.85. Second, participants who reported age of gout onset in both 2003 and 2007 reported their age of onset as being older in 2007 than in the 2003 questionnaire (mean age difference 0.84 years, median 0 years, SD 7.3). Third, the 95% CI between the 2 measures of self-reported age of gout onset in the Bland-Altman plot ranged from −12 years to 12 years (Figure 3). Additionally, the scatter of the difference in reported gout ages did not appear to change as the average age at diagnosis increased (Figure 3).
Sensitivity of self-reported, physician-diagnosed gout in ARIC
Figure 2 displays the number of ARIC participants who were included in the sensitivity analyses. Overall, there were 273 ARIC participants with either a hospital discharge diagnosis of gout or a prescription for a gout medication during one of the study visits (Table 2). At visit 4, specifically, there were 196 returning participants (72%) who reported their gout status and constituted the sensitivity study population. We found the sensitivity of a self-report of physician-diagnosed gout in relation to a discharge diagnosis of gout or prescription of a gout medication to be 84%.
Table 2.
No. Self-reported Gout Cases (%) | Gold Standard Definition of Gout | ||
---|---|---|---|
Discharge Diagnosis of Gout or Prescription for a Gout Medication, n = 196* | Discharge Diagnosis of Gout, n = 64 | Prescription for a Gout Medication, n = 165 | |
Overall | 165 (84.2) | 59 (92.2) | 136 (82.4) |
Women | 30/37 (81.1) | 7/9 (77.8) | 25/32 (78.1) |
Men | 135/159 (84.9) | 52/55 (94.6) | 111/133 (83.5) |
African American | 36/45 (80.0) | 16/17 (94.1) | 24/33 (72.7) |
White | 128/150 (85.3) | 43/47 (91.5) | 111/131 (84.7) |
Education, < 12 yrs | 32/44 (72.7) | 24/35 (68.6) | 10/12 (83.3) |
Education, ≥ 12 yrs | 133/152 (87.5) | 112/130 (86.2) | 49/52 (94.25) |
Hyperuricemic | 127/144 (88.2) | 51/54 (94.4) | 99/114 (86.8) |
Not hyperuricemic | 38/52 (73.1) | 8/10 (80.0) | 37/51 (72.6) |
There were 33 participants with both a prescription and hospitalization for gout.
In the stratified analyses, the sensitivity was similar for men and women, as well as for African American and white participants (Table 2). However, the sensitivity differed by education level and hyperuricemia category. The sensitivity of a self-report of physician-diagnosed gout was higher in participants with a high school education or higher (88% vs 73%). Additionally, the sensitivity of a self-report of physician-diagnosed gout was higher among participants who were hyperuricemic at either visit 1 or visit 2 than among those who were not (88% vs 73%). With regard to the sensitivity analyses, which involved 2 alternative definitions for gout, we noted that prior to visit 4, 65 participants were hospitalized with a diagnosis of gout. All but one attended visit 4 and self-reported their gout status at visit 4 (98% returning participants). In this group, the sensitivity of a self-report of physician-diagnosed gout, using a hospital discharge diagnosis of gout as the standard, was 92%. In addition, we noted there were 241 participants with at least one prescription for a gout medication at any of the first 3 visits; 165 (68% returning participants) attended visit 4 and reported their gout status. The sensitivity of a self-report of physician-diagnosed gout, using prescription of gout medication as the standard, was 82%. Results were similar by sex and race (Table 2). Of note, sensitivity was higher in participants with hyperuricemia and among those who completed high school or a higher level of education. The sensitivity analysis suggests that the self-report of physician-diagnosed gout may depend on the gold standard definition of gout (Table 2). The sensitivity was slightly higher when the standard was hospital discharge diagnosis of gout compared to a prescription for a gout medication (92% vs 82%).
DISCUSSION
Our study suggests that a self-report of physician-diagnosed gout has both good reliability and sensitivity to ascertain gout status in epidemiologic studies. Moreover, the age of gout onset was reliably reported. Using a Bland-Altman plot, we found that participants’ recall of when their gout first occurred had similar precision across the full range of ages at gout onset. The sensitivity of self-reported physician-diagnosed gout was similar in men and women and in African American and white participants. However, sensitivity was higher among participants with a higher education and among those with hyperuricemia.
Our study is the first to include an evaluation of both the reliability and the sensitivity of a self-report of physician-diagnosed gout over an extended period of followup of up to 7 years. In previous studies, only the short-term reliability (< 2 years) of self-reported gout was evaluated. Picavet and Hazes found that 64% of a Dutch population (n = 2338) aged 25 years and older who reported gout at base-line similarly reported gout 6 months later5. The κ value for reliability was 0.64. In the Potsdam component of the European Prospective Study into Cancer and Nutrition (EPIC), the agreement between self-reported gout obtained in an interview and on a questionnaire about 2 years later was 45% (κ = 0.61)6. Notably, the measures of short-term reliability in these 2 studies were less than the longterm reliability we observed in CLUE II. One possible explanation for the differences in reliability is that CLUE II included shorter questionnaires, which allowed participants more time to consider their response than in the longer Dutch population or EPIC study questionnaires5,6.
A self-report of gout has been confirmed in 2 previous studies by querying only those participants with self-reported gout regarding fulfillment of the ACR gout criteria2,3,4. Those 2 studies found a high confirmation rate of self-reported gout against the ACR criteria. However, the performance of the ACR criteria across the entire study population was not ascertained. Further, neither study used hospitalization and prescription drug data. In contrast, we were able to assess our gold standard gout definition for every ARIC participant, and thereafter, to calculate the sensitivity of the self-report of physician-diagnosed gout across all study participants. Additionally, we assessed whether the sensitivity differed by important demographic and clinical characteristics that are associated with gout.
The ability of a medical record review and physician interview to confirm self-reported gout was previously evaluated in only one study9. Twenty-two out of 50 patients with gout (44%) who resided in Sudbury, Massachusetts, USA, in 1964 were confirmed as having gout9. These measures of sensitivity for other self-reported chronic diseases have been reported10,11. For example, the validity of self-reported stroke had a low sensitivity (32%)10. Similar to our findings, high sensitivity (above 80%) for self-reported hip fracture, Parkinson’s disease, diabetes, cancer, stroke, and myocardial infarction were found in a study of community-dwelling disabled women aged 65 years and older11.
There are limitations to our study. ARIC participants with less severe gout, those neither hospitalized nor treated with gout-specific medication, would not satisfy our definition for gout. Our sensitivity results may thus be particularly generalizable to chronic, severe gout. Participants who truly have gout — at the mild end of the disease spectrum —would dilute the group defined as not having gout, precluding an accurate assessment of specificity. Specificity could not be assessed because of a lack of a clear standard for defining those not having gout. Similarly, the sensitivity results may not be generalizable to milder gout cases. Additionally, we defined gout medications as colchicine, probenecid, and allopurinol, which are most commonly used to treat gout in a random sample of US communities. We note that colchicine is also indicated for disorders other than gout (e.g., Behçet’s disease, familial Mediterranean fever). However, in US population–based cohorts such as ARIC and CLUE, these non-gout indications have a low prevalence relative to the prevalence of gout. Nonsteroidal antiinflammatory drugs are also used to treat gout, but are not good markers for gout status. In addition, a limitation of our use of Cohen’s κ statistic is that it assumes incident cases are minimal over the 3-year period of serial ascertainment. However, previous literature suggests that the incidence rate of gout is 1.73 per 1000 person-years12. Finally, this study was not intended to assess the sensitivity of a clinical diagnosis of gout, but rather the self-report of gout for use in an epidemiologic study.
A major strength of our study was the use of 2 large cohorts: CLUE II and ARIC. In CLUE II, there were 428 individuals who reported physician-diagnosed gout over 2 or more questionnaires, separated by as many as 7 years, allowing estimation of reliability over a relatively long period (up to 7 years). Additionally, comparison of the agreement between ages of gout onset across 2 survey periods could be performed using CLUE II data. Moreover, in ARIC, there were 706 participants who reported physician-diagnosed gout, with systematic collection of discharge records for all participants hospitalized within the catchment areas of study sites. While not the clinical standard for gout in terms of synovial fluid analysis for urate crystals or satisfaction of ACR criteria, physician diagnosis of gout on a discharge summary, we believe, nevertheless provides strong evidence of a gout diagnosis. Further, we note that other investigators have used large administrative databases, in both the United States and the United Kingdom, to similarly ascertain gout medications as a proxy for a physician-diagnosis of gout13,14, but we are the first to use medication data to validate the self-report of gout.
While the clinical standard diagnosis of gout is not available in large cohorts, the ARIC study is the first to contain data on use of both gout-specific prescription medications and hospital discharge diagnoses for gout. We also note that the measures of sensitivity were not highly sensitive to varying the gold standard definition of gout.
Our study provides quantitative estimates of reliability and sensitivity of a self-report of physician-diagnosed gout, supporting its use in a future epidemiologic cohort and in population-based studies.
Acknowledgments
The ARIC study was carried out as a collaborative study supported by National Heart, Lung, and Blood Institute contracts N01-HC-55015, N01-HC-55016, N01-HC-55018, N01-HC-55019, N01-HC-55020, N01-HC-55021, and N01-HC-55022. CLUE II was supported by the National Institute of Aging (Grant no. U01 AG18033) and the National Cancer Institute (Grant no. R01 CA105069). The work described in this article was supported in part by grants 5T32HL007024 and 1KL2RR025006-01 from the National Institutes of Health. M. McAdams was supported by a T32 training grant from the National Heart, Lung, and Blood Institute (5T32HL007024). Dr. Maynard was supported by a KL2 Research Grant from the National Center for Research Resources (1KL2RR025006-01). Dr. Gelber was supported by the Donald B. and Dorothy Stabler Foundation.
We thank the staff and participants of the CLUE II and ARIC studies for their important contributions.
References
- 1.Lawrence RC, Felson DT, Helmick CG, Arnold LM, Choi H, Deyo RA, et al. Estimates of the prevalence of arthritis and other rheumatic conditions in the United States. Part II. Arthritis Rheum. 2008;58:26–35. doi: 10.1002/art.23176. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 2.Wallace SL, Robinson H, Masi AT, Decker JL, McCarty DJ, Yu TF. Preliminary criteria for the classification of the acute arthritis of primary gout. Arthritis Rheum. 1977;20:895–900. doi: 10.1002/art.1780200320. [DOI] [PubMed] [Google Scholar]
- 3.Gelber AC, Klag MJ, Mead LA, Thomas J, Thomas DJ, Pearson TA, et al. Gout and risk for subsequent coronary heart disease. The Meharry-Hopkins Study. Arch Intern Med. 1997;157:1436–40. [PubMed] [Google Scholar]
- 4.Choi HK, Atkinson K, Karlson EW, Willett W, Curhan G. Purine-rich foods, dairy and protein intake, and the risk of gout in men. N Engl J Med. 2004;350:1093–103. doi: 10.1056/NEJMoa035700. [DOI] [PubMed] [Google Scholar]
- 5.Picavet HS, Hazes JM. Prevalence of self reported musculoskeletal diseases is high. Ann Rheum Dis. 2003;62:644–50. doi: 10.1136/ard.62.7.644. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 6.Bergmann MM, Jacobs EJ, Hoffmann K, Boeing H. Agreement of self-reported medical history: comparison of an in-person interview with a self-administered questionnaire. Eur J Epidemiol. 2004;19:411–6. doi: 10.1023/b:ejep.0000027350.85974.47. [DOI] [PubMed] [Google Scholar]
- 7.Harrold LR, Saag KG, Yood RA, Mikuls TR, Andrade SE, Fouayzi H, et al. Validity of gout diagnoses in administrative data. Arthritis Rheum. 2007;57:103–8. doi: 10.1002/art.22474. [DOI] [PubMed] [Google Scholar]
- 8.Bland JM, Altman DG. Statistical methods for assessing agreement between two methods of clinical measurement. Lancet. 1986;1:307–10. [PubMed] [Google Scholar]
- 9.O’Sullivan JB. Gout in a New England town. A prevalence study in Sudbury, Massachusetts. Ann Rheum Dis. 1972;31:166–9. doi: 10.1136/ard.31.3.166. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 10.Reitz C, Schupf N, Luchsinger JA, Brickman AM, Manly JJ, Andrews H, et al. Validity of self-reported stroke in elderly African Americans, Caribbean Hispanics, and Whites. Arch Neurol. 2009;66:834–40. doi: 10.1001/archneurol.2009.83. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 11.Simpson CF, Boyd CM, Carlson MC, Griswold ME, Guralnik JM, Fried LP. Agreement between self-report of disease diagnoses and medical record validation in disabled older women: factors that modify agreement. J Am Geriatr Soc. 2004;52:123–7. doi: 10.1111/j.1532-5415.2004.52021.x. [DOI] [PubMed] [Google Scholar]
- 12.Roubenoff R, Klag MJ, Mead LA, Liang KY, Seidler AJ, Hochberg MC. Incidence and risk factors for gout in white men. JAMA. 1991;266:3004–7. [PubMed] [Google Scholar]
- 13.Gurwitz JH, Kalish SC, Bohn RL, Glynn RJ, Monane M, Mogun H, et al. Thiazide diuretics and the initiation of anti-gout therapy. J Clin Epidemiol. 1997;50:953–9. doi: 10.1016/s0895-4356(97)00101-7. [DOI] [PubMed] [Google Scholar]
- 14.Mikuls TR, Farrar JT, Bilker WB, Fernandes S, Schumacher HR, Jr, Saag KG. Gout epidemiology: results from the UK General Practice Research Database, 1990–1999. Ann Rheum Dis. 2005;64:267–72. doi: 10.1136/ard.2004.024091. [DOI] [PMC free article] [PubMed] [Google Scholar]