Association of Environmental and Genetic Factors and Gene-Environment Interactions with Risk of Developing Rheumatoid Arthritis

Abstract

Background

We developed RA risk models based on validated environmental factors (E), genetic risk scores (GRS), and gene-environment interactions (GEI) to identify factors that can improve accuracy and reclassification.

Methods

Models including E, GRS, GEI were developed among 317 Caucasian seropositive RA cases and 551 controls from Nurses’ Health Studies (NHS) and validated in 987 Caucasian ACPA positive cases and 958 controls from the Swedish Epidemiologic Investigation of RA (EIRA), stratified by gender. Primary analyses included age, smoking, alcohol, parity, weighted GRS using 31 non-HLA alleles, 8 HLA-DRB1 alleles and HLA X smoking interaction. Expanded models included reproductive, geographic, and occupational factors, and additional GEI terms. Hierarchical models were compared for discriminative accuracy using AUC and reclassification using Integrated Discrimination Improvement (IDI) and continuous Net Reclassification Index.

Results

Mean (SD) age of RA diagnosis was 57 in NHS and 50 in EIRA. Primary models produced an AUC of 0.716 in NHS, 0.728 in EIRA women and 0.756 in EIRA men. Expanded models produced improvements in discrimination with AUCs of 0.738 in NHS, 0.728 in EIRA women and 0.769 in EIRA men. Models including G or G + GEI improved reclassification over E models; the full E+G+GEI model provided the optimal predictive ability by IDI analyses.

Conclusions

We have developed comprehensive RA risk models incorporating epidemiologic and genetic factors and gene-environment interactions that have improved discriminative accuracy for RA. Further work developing and assessing highly specific prediction models in prospective cohorts is still needed to inform primary RA prevention trials.

Rheumatoid arthritis (RA), an autoimmune disease that causes inflammatory and disabling arthritis, is thought to develop in individuals with inherited genetic risk factors after exposure to environmental factors, including cigarette smoking, residential history, air pollution, occupational exposures, alcohol, female reproductive factors, and low socioeconomic status^1-31. The identification of risk alleles for RA through genome-wide association studies and meta-analyses, along with the discovery of gene-environment interactions, could potentially allow the prediction of RA risk among individuals without symptoms^{2, 32-42}. The Framingham Risk Score⁴³ was developed with the specific goal of clinical risk prediction aiding clinicians in making both recommendations about risk factor modification and decisions about preventive treatment. This successful paradigm of individualized risk factor assessment and stratification has led to a reduction of cardiovascular morbidity and mortality worldwide^{44, 45}. Efforts are now underway to develop similar predictive models for the early identification of individuals at high risk of developing RA among asymptomatic populations who could be enrolled in primary prevention trials ⁴⁶. The first step in this process is to determine the optimal variables to include in such models.

The goal of this study was to develop a risk model based on epidemiologic factors that can be collected easily at a clinical visit, and to study the benefit of adding genetic and gene-environment interaction terms to the model. We used novel statistical methods to choose the optimal combination of predictors among epidemiological factors, genetic susceptibility alleles, and gene-environment interactions for training and validation in an independent dataset. Our hypothesis is that models including G and GEI terms will have the optimal predictive accuracy.

PATIENTS AND METHODS

Study Population

NHS

We conducted a nested case-control study of RA susceptibility among the Nurses’ Health Study (NHS) and the Nurses’ Health Study II (NHSII) prospective cohorts. Among 121,700 female nurses aged 30 to 55 years in NHS, 32,826 (27%) participants provided blood samples and another 33,040 (27%) provided buccal cell samples. Of 116,609 female nurses aged 25 to 42 years in NHSII, 29,611 (25%) provided blood samples. The two cohorts were combined in this manuscript and are referred to as ‘NHS’. Incident RA cases in NHS were confirmed using a two-stage screening method with a connective tissue disease screening questionnaire (CSQ) for RA symptoms⁴⁷ and confirmed by chart review by two board-certified rheumatologists. Rheumatoid factor (RF) was determined by chart review and anti-citrullinated antibodies (ACPA) status was determined by chart review and/or direct assay for RA cases with banked plasma samples from prior to diagnosis⁴⁸. For each confirmed RA case, a healthy control was chosen, matched on cohort (NHS/NHSII), year of birth, menopausal status and postmenopausal hormone use. There were 585 women with validated RA who provided blood samples, 21 (4%) cases were excluded for non-self-reported Caucasian and an additional 22 (4%) were excluded due to missing HLA information. For anyone missing other SNPs, we assigned them a value equal to the expected value (2*risk allele frequency defined in cases or controls separately). Finally, since prior genetic association studies focused on seropositive RA³⁹, analyses were limited to seropositive RA cases (n=317) compared to healthy controls (n=551).

EIRA

The Epidemiologic Investigation of Rheumatoid Arthritis (EIRA) is a population based case-control study which enrolled newly diagnosed cases of RA in Sweden aged 18-70 years between May 1996 and December 2009. Controls were randomly selected, matched to cases on age, sex and geographic location^{7, 40}. A total of 987 anti-citrullinated protein antibodies positive (ACPA+) cases, (702 women and 285 men) and 958 controls (715 women and 243 men) with information on the 31 non-HLA loci and the HLA alleles were used for analyses. In EIRA, a total of 1218 ACPA positive RA cases and 1129 controls recruited from May 1996 to the end of 2009 were selected for genome-wide genotyping. After sample quality control (sample genotype call rate>0.95, ethnicity outliers removed), 988 citrullinated protein antibodies positive (ACPA+) cases (81%) and 958 (85%) controls remained. One male case with 3 HLA alleles was further removed from the analyses. A final dataset with 987 cases (702 women and 285 men) and 958 controls (715 women and 243 men) with information on the 31 non-HLA loci and the HLA alleles was used for analyses.

All aspects of these studies were approved by Partners’ HealthCare or the Karolinska Institutet Institutional Review Boards.

Epidemiologic Factors

We selected epidemiologic factors that had been demonstrated in the literature by other groups and replicated in our datasets to be significantly associated with RA susceptibility including age, smoking, alcohol, education and parity (in women only), and could be easily ascertained in clinical practice^{3, 4, 8, 9, 16, 18, 19, 24, 30, 31, 49}. In the primary analysis, we only included factors that were available in both cohorts and not discovered in either NHS or EIRA. For a secondary analysis we also explored expanded models that also considered risk factors first published by our groups including additional reproductive factors, geographic region, and occupational exposures (in men only)^{10, 11, 14, 15, 24}, as well as gene-environment interactions (GEI) for GST and HMOX1 genes⁴².

NHS

Epidemiologic factors for the primary models in NHS included year of birth, smoking (pack-years), alcohol consumption (cumulative average of daily intake), husband’s educational attainment (as a marker of socioeconomic status) and parity (for females). For expanded NHS models we included region of residence at age 30, age at menarche, menstrual regularity, breastfeeding, oral contraceptive use, menopausal status and postmenopausal hormone use^{10, 24}. All epidemiologic variables were updated through the biennial questionnaire before RA diagnosis (or index date for controls).

EIRA

Epidemiologic factors for the primary models in EIRA included year of birth, smoking exposure before disease onset (pack-years), alcohol consumption, education level (as a marker of socioeconomic status) and parity (for females). For expanded models in EIRA males, we also included occupational exposure to silica, mineral oil and solvents, and HLA-SE X exposure interaction terms^{14, 15}. All data were collected from subjects at the time of incident RA and pertained to exposures prior to RA onset.

Genetic Factors

Genetic Risk Score (GRS)

Thirty-nine validated risk alleles for RA were combined to form a continuous GRS, a weighted combination of 8 HLA-DRB1 ‘shared epitope’ (HLA-SE) alleles and 31 non-HLA risk alleles (GRS39), weighted by the natural log of the published OR⁵⁰ (Supplemental Table 1). To assess the independent contribution of the HLA-SE alleles, we created a GRS limited to only the non-HLA-SE alleles (GRS31). Genotyping and quality control procedures for both NHS and EIRA are described in detail elsewhere^{2, 37, 41, 42}.

Other genetic factors

GSTT-1 homozygous deletion (GSTT1 null) and HMOX-1 variants, GSTT1 X smoking, HMOX1 X smoking interaction terms were included in the expanded models as both were found to significantly interact with smoking and RA risk in the NHS, with replication of the GSTT1-smoking interaction in EIRA⁴².

Statistical Analysis

Logistic regression models were used to estimate the predicted odds of seropositive or ACPA+ RA. GEI was modeled on a multiplicative scale using a product term (GxE). Analyses were performed in accordance with the 25 recommendations for evaluations of risk prediction models (GRIPS Statement)⁵¹.

Model building

The lists of all variables and interactions considered for inclusion are summarized in Table 1 in two categories described as 1)’primary’ variables available in both NHS and EIRA and 2) ‘expanded’ variables available in either NHS or EIRA cohorts. NHS was used as the primary dataset to determine the optimal combination of variables. EIRA was not used as the primary dataset since many of the genetic risk alleles and the HLA*smoking interaction were discovered in this study and could have led to overfitting of the models. Variables were selected based on contribution to the overall model prediction based on the IDI (described below). The final list of variables included in the NHS model was then assessed in the EIRA validation datasets.

Table 1.

Factors included in Primary model in NHS and Expanded models in NHS and EIRA

Factors	Primary Model	Expanded NHS Model	Expanded EIRA Model
Epidemiological and Environmental	year of birth	year of birth	year of birth
	smoking	smoking	smoking
	alcohol	alcohol	alcohol
	education	education	education
	parity (F1)	parity/breastfeeding(F1)	parity (F1)
		menses <12 yrs old(F1)	silica (M2)
		menstrual
		irregularity(F1)	mineral oil (M3)
		menopause(F1)	solvents (M)
		PMH use(F1)
		region (US) (F1)	region (Sweden)
	HLA-SE (0, 1, 2)	HLA-SE (0, 1, 2) (F1)	HLA-SE (0, 1, 2)
Genetic	GRS31	GRS31	GRS31
		GSTT1	GSTT1
		HMOX1	HMOX1
Gene-Environment Interactions	HLA-SE*smoking	HLA-SE*smoking	HLA-SE*smoking
		GSTT1*smoking	GSTT1*smoking
		HMOX1*smoking	HMOX1*smoking
			silica*smoking (M2)
			mineral oil*smoking (M3)
			solvent*smoking (M)

¹F = in Females only, M = in Males only

²Subjects with occupational exposure to rock drilling or stone crushing, or stone dust were classified as silica exposed

³Subjects with occupational exposure to cutting oil, motor oil, form oil, hydraulic oil or asphalt were classified as exposed to mineral oil

As a secondary analysis, we studied expanded risk models that considered all variables including those available only in one cohort or discovered in either NHS or EIRA. For EIRA males, we excluded parity but added silica, mineral oil and solvents to the expanded models. These models were further categorized according to variable groups: epidemiological and environmental factors (E), genetic factors (G) and gene-environment interactions (GEI). In order to assess the benefit of adding G variables and GEI terms to risk models we compared models with just E variables, to those with E+G and finally models with E+G+GEI. These models were developed in a stepwise fashion in each dataset (NHS, EIRA women, EIRA men). The optimal combination of variables was chosen based on significant contribution to the model using the IDI explained below.

Model assessment and comparisons

Models were assessed on variance explained, goodness-of-fit and discrimination ability. The Nagelkerke R² was used as a measure of variance explained by the model⁵². Goodness-of-fit was measured using the Hosmer-Lemeshow χ², (χ²_HL), where a non-significant χ²_HL indicates a good model fit⁵³. Finally, Receiver Operating Characteristic (ROC) curve and the Area Under the ROC Curve (AUC) were used to assess how well each model discriminates between RA cases and controls. Confidence intervals (95%) were obtained via bootstrapping. The models were compared within each dataset using the Integrated Discrimination Improvement (IDI), both to decide on inclusion in the primary NHS model (then tested in EIRA) and also between models in the expanded analysis. Finally, the continuous Net Reclassification Improvement (cNRI) was used to compare the primary and validation models to the optimal model from the expanded analyses, within each dataset.

Developed by Pencina and D’Agostino^{54, 55}, the IDI is a measure of overall improvement in sensitivity and specificity between 2 models (eg. E model compared with E+G model). It is calculated using the predicted probability of the outcome (seropositive RA or ACPA positive RA) in 2 models as follows:

$$\mathit{IDI}=\left({\overline{\widehat{p}}}_{\text{new},\text{events}}-{\overline{\widehat{p}}}_{\text{new},\text{nonevents}}\right)-\left({\overline{\widehat{p}}}_{\text{old},\text{events}}-{\overline{\widehat{p}}}_{\text{old},\text{nonevents}}\right)$$

One limitation is that IDI is calculated using predicted probabilities, which are biased in a case-control design since the probability of RA in the sample is not an estimate of probability of RA in the population. However, for our analyses we used the IDI to compare models within the same cohort, not to generalize across cohorts or to the general population.

To further address this issue, we also use the continuous Net Reclassification Improvement (cNRI) developed by Pencina and D’Agostino⁵⁵. Whereas the original NRI required categories of risk and quantified the overall upward and downward movement between categories⁵⁴, the new cNRI does not require categories. The cNRI⁵⁵ quantifies any movement in predicated probability from the models. This can also be thought of as the amount of correct reclassification among event and non-events without respect to the magnitude of the change in predicted risk.

The cNRI is calculated as

$$\begin{array}{cc}\hfill & \mathit{cNRI}\left(\text{events}\right)=\left(\frac{\#\text{events}({\widehat{p}}_{\mathit{new}}>{\widehat{p}}_{\mathit{old}})}{\#\text{events}}-\frac{\#\text{events}({\widehat{p}}_{\mathit{new}}<{\widehat{p}}_{\mathit{old}})}{\#\text{events}}\right)\hfill \\ \hfill & \mathit{cNRI}\left(\text{nonevents}\right)=\left(\frac{\#\text{nonevents}({\widehat{p}}_{\mathit{new}}<{\widehat{p}}_{\mathit{old}})}{\#\text{nonevents}}-\frac{\#\text{nonevents}({\widehat{p}}_{\mathit{new}}>{\widehat{p}}_{\mathit{old}})}{\#\text{nonevents}}\right)\hfill \\ \hfill & \mathit{cNRI}=\mathit{cNRI}\left(\text{event}\right)+\mathit{cNRI}\left(\text{nonevent}\right)\hfill \end{array}$$

Depending upon the future application of a prediction model, an improvement in reclassification of cases may be more important than that of controls, or vice versa. The cNRI can thus be interpreted for cases (cNRI(events)) and for controls (cNRI(non-events)) separately.

RESULTS

Primary models

In NHS, 317 seropositive RA cases had a mean age at diagnosis of 56 (±10) years and 195 (63%) were current or former smokers (Table 2). In EIRA, 987 ACPA-positive RA cases had a mean age at diagnosis of 51 (±12), 644 (73%) were current or former smokers and 702 (71%) were female. All variables available were included in the final primary model since the addition of each variable improved the predicative ability of the model in NHS as measured by the IDI. This combination of variables showed an AUC of 0.716 (0.681-0.755) in NHS, 0.722 (0.692-0.746) in EIRA women and 0.756 (0.725 – 0.808) in EIRA men (Figure 1, Table 3).

Table 2.

Characteristics of participants in the NHS and EIRA

	NHS		EIRA
	Seropositive RA cases (n=317)	Controls (n=551)	ACPA+ RA cases (n=987)	Controls (n=958)
Age, mean (SD)1	55.1 (±8.1)	55.5 (±7.9)	51.2 (±12.0)	52.5 (±11.7)
Female, n (%)	317 (100%)	551 (100%)	702 (71%)	715 (75%)
Current or past smoker, n (%)	195 (63%)	309 (56%)	644 (73%)	518 (59%)
Pack-years among smokers, mean (SD)	25.0 (±18.0)	22.7 (±20.9)	19.2 (±14.8)	16.5 (±14.6)
GRS39 (with HLA-SE), mean (SD)	5.14 (±0.86)	4.70 (±0.79)	5.39 (±0.95)	4.72 (±0.81)
GRS31 (without HLA-SE), mean (SD)	4.27 (±0.60)	4.10 (±0.86)	4.51 (±0.59)	4.26 (±0.58)
RA features
Mean age at symptom onset, mean (SD)	55.5 (±10.4)	-	50.3 (±12.1)
Mean age at diagnosis, mean (SD)	56.1 (±9.8)	-	51.1 (±12.1)
RF positive, n (%)	297 (94%)	-	835 (88%)
ACPA2-positive, n (%)	112 (55%)	-	987 (100%)
Seropositive3, n (%)	317 (100%)	-	987 (100%)

¹Age at blood draw for blood samples (n=328 cases, n=334 controls)

²Anti-citrullinated protein antibodies (ACPA) assayed in subset of NHS cases (n=202) with available blood samples collected at different points with respect to RA onset, up to 14 years prior to onset or up to 12 years after diagnosis

³RF and/or ACPA

Figure 1.

ROC curves for predicting seropositive RA in NHS (a) and ACPA positive RA in EIRA women (b) and EIRA men (c). Variables included in the primary model (NHS) and Validation models (EIRA) are described in Table 1. ROC: Receiver Operating Characteristic, AUC: Area Under the Curve, NHS: Nurses’ Health Study, EIRA: Epidemiologic Investigation of Rheumatoid Arthritis, ACPA: anti-citrullinated antibody, E: Epidemiologic model, E + G: Epidemiologic + Genetic model, E + G + GEI: Epidemiologic + Genetic + Gene Environment Interaction model.

Table 3.

Results of primary model development in NHS and validation in EIRA females and EIRA males

Statistical Parameters	Primary Dataset NHS Women	Validation Dataset EIRA Women	Validation Dataset EIRA Men
Nagelkerke R2	0.185 (0.129 – 0.246)	0.185 (0.147 – 0.241)	0.275 (0.189 – 0.360)
AUC	0.716 (0.681-0.755)	0.716 (0.693 – 0.749)	0.756 (0.725 – 0.808)
χ 2 hl	5.48 (0.71)	8.71 (p=0.367)	4.40 (p=0.810)

¹Primary model includes year of birth, smoking (pkyrs), alcohol, education, parity (Females only), GRS31, HLA-DR and HLA*smoking

Secondary models

Secondary analyses were done specific to the expanded list of variables available in each dataset. All models that included G performed significantly better than the E models, based on the Nagelkerke R², AUC, IDI and cNRI (Table 4). The expanded model in NHS women that included additional reproductive variables, and additional genetic and GEI terms produced the highest AUC (in NHS) of 0.738 (95%CI: 0.721 – 0.790) (Figure 1a). An optimal expanded model in EIRA women produced an AUC of 0.728 (95%CI: 0.705 – 0.758) and included the GRS and HLA-SE, but not GEI with smoking as these did not increase prediction (eg. addition of the HLA-SE*smoking interaction term did not show improvement with the IDI) (Figure 1b). The optimal expanded model in EIRA men that included occupational exposures produced the highest AUC of 0.769 (95%CI: 0.747 – 0.830), and included the GRS, HLA*smoking interaction, but not other GEI (Figure 1c).

Table 4.

Comparisons of the expanded models including epidemiologic and environmental factors (E), plus genetic factors (E+G), plus gene environment interaction terms (E+G+GEI) models for NHS and EIRA women, and EIRA men.

	NHS Expanded Models 1
Statistical Parameters	E	E+G 2	E+G+GEI3
Nagelkerke R2	0.094 (0.076 – 0.172)	0.186 (0.162 – 0.279)	0.209 (0.188 – 0.323)
AUC/c-statistic	0.655 (0.635 – 0.709)	0.718 (0.702 – 0.774)	0.738 (0.723 – 0.795)
IDI (p-value) compared to
E model	ref	0.069 (p=6.0 × 10−14)	0.088 (p=7.4 × 10−17)
E+G model	−0.069 (p=6.0 × 10−14)	ref	0.019 (p=0.0001)
E+G+GEI model	−0.088 (p=7.4 × 10−17)	−0.019 (p=0.0001)	ref
	EIRA Women Expanded Models 1
Statistical Parameters	E	E+G 2	E+G+ GEI
Nagelkerke R2	0.069 (0.055 – 0.117)	0.199 (0.169-0.265)	0.200 (0.171 – 0.267)
AUC/c-statistic	0.632 (0.614 – 0.671)	0.724 (0.705 – 0.758)	0.724 (0.706 – 0.762)
IDI (p-value) compared to
E model	ref	−0.101 (1.0 × 10−35)	0.102 (8.4 × 10−36)
E+G model	−0.101 (1.0 × 10−35)	ref	0.0002 (0.499)
E+G+GEI model	−0.102 (8.4 × 10−36)	--0.0002 (0.499)	ref
	EIRA Men Expanded Models 1
Statistical Parameters	E	E+G 2	E+G+GEI 3
Nagelkerke R2	0.125 (0.098 – 0.237)	0.273 (0.231 – 0.401)	0.282 (0.240 – 0.414)
AUC/c-statistic	0.685 (0.657 – 0.752)	0.767 (0.744 – 0.828)	0.769 (0.747 – 0.830)
IDI (p-value) compared to
E model	ref	0.116 (p=1.1 × 10−14)	0.123 (p=2.2 × 10−15)
E+G model	−0.116 (p=1.1 × 10−14)	ref	0.006 (p=0.12)
E+G+GEI model	−0.123 (p=2.2 × 10−15)	−0.006 (p=0.12)	ref

¹Expanded E model includes year of birth, smoking (pkyrs), alcohol, education, region of US (NHS) or Sweden (EIRA), parity (Females only), breastfeeding years (NHS only), age of menses (<12 vs ≥12, NHS only), menstual irregularity (NHS only), silica and mineral oil exposure (EIRA men only);

²E+G includes E model + GRS31 and HLA-DR (0,1,2);

³E+G+GEI includes E+G model + HLA-DR*smoking, [HMOX1, HMOX1*smoking, GSTT1, GSTT1*smoking (NHS Only)]

Model comparisons across groups of risk factors

In NHS, the primary model with age, smoking, alcohol, education, parity, GRS 31, HLA-SE and HLA*smoking explained 19% of the variance, whereas in the NHS expanded models the variance explained from just the epidemiologic factors (E model) was 9%. As expected, however, as more variables were added to the expanded model the variance explained increased, first to 19% with inclusion of genetic factors, and finally to 21% with the addition of GxE interaction (E+G+GEI mode). In EIRA women, the primary model explained 19% of the variance in women and 28% of the variance in men. In the expanded analysis among EIRA women, the variance explained from just the epidemiologic factors (E model) was 7%, and as more variables were added, the variance explained increased to 21% with inclusion of genetic factors and was 20% with inclusion of GEI factors. In EIRA men, the primary model explained 27.5% of the variance. In the expanded model for EIRA men, the variance explained by the epidemiologic factors (E model) was 12.5%, and as more variables were added, the variance explained increased to 27.3% with inclusion of genetic factors and to 28.2% with the inclusion of GEI variables.

In NHS expanded models, the addition of G, showed a significant improvement in prediction over the E model as measured with the IDIp=6×10⁻¹⁴); addition of GEI further improved the IDI and (p=0.0001), respectively. For EIRA women, addition of G in the expanded models showed significant improvement in prediction over the E model as measured by the IDI (p < 6.7 × 10⁻³⁷), however, no improvement in IDI was seen with the addition of GEI terms (p>0.05). In EIRA men. expanded models, the addition of G and G+GEI each showed a significant improvement in prediction over the E model (p=1.1 × 10⁻¹⁴ and p=2.2 × 10⁻¹⁵, respectively). However, there was no significant improvement in IDI adding GEI to the E+G model.

The stratified cNRI results presented in Table 5 illustrate the change in sensitivity (reclassification of cases) and change in specificity (reclassification of controls) of the primary model compared to the expanded models. In NHS overall, the primary model performed more poorly than the E+G+GEI, with a total cNRI of −0.26 (p=0.0004), indicating lower specificity. However, from the stratified results we can see that the primary model performed similarly in reclassifying cases (same sensitivity), (cNRI = −0.03, p=0.64), but worse in reclassifying controls (lower specificity) (cNRI = −0.23, p=1.3 × 10⁻⁷). This is interpreted as an excess of 23% of controls reclassified with a higher predicted probability of RA in the primary model vs. the E+G+GEI model. In EIRA women, the primary model performed similarly to the E+G model (p=0.08). However, in the stratified analysis the primary model correctly reclassified controls with lower predicted probabilities (higher specificity) (cNRI = 0.33, p=5.9 × 10⁻¹⁸), and also reclassified cases with lower probabilities of developing RA (lower sensitivity) (cNRI = −0.42, p=5.0 × 10^{− 28}). In EIRA men, the primary model performed worse than the E+G+GEI model according to the cNRI of −0.27 (p=0.004) (lower specificity). However, again in the stratified analysis, the primary model performed similarly to the E+G+GEI model in reclassifying controls (same specificity), (cNRI = 0.08, p=0.24), but did more poorly in reclassifying cases (lower sensitivity) (cNRI = −0.36, p=2.8 × 10⁻⁸).

Table 5.

Stratified analysis of reclassification of cases and controls comparing the primary or validation model to the optimal expanded model¹

Cohort	cNRI for Cases	cNRI for Controls	Total cNRI
NHS	−0.027 (p=0.64)	−0.233 (p=1.3 × 10−7)	−0.260 (p=0.0004)
EIRA (Women)	−0.424 (p<1.0 × 10−16)	0.328 (p<1.0 × 10−16)	−0.096 (p=0.08)
EIRA (Men)	−0.355 (p=2.8 × 10−8)	0.081 (p=0.24)	−0.274 (p=0.004)

¹Expanded model for NHS is the E+G+GEI and for EIRA men and women are the E+G.

DISCUSSION

Leveraging an extensive body of epidemiologic and genetic research on the risk of developing RA among asymptomatic cohorts along with modern statistical techniques, we developed and validated comprehensive risk models for RA. We demonstrate that inclusion of information on genetic variants and gene environment interactions models, significantly improves the predictive power of epidemiologic models. Using the AUC to assess discriminative ability, the optimal primary model among a US female cohort included gene-environment interaction terms for HLA-DRB1, the strongest genetic risk factor for RA and smoking, the strongest environment risk factor for RA, with an AUC of 0.716. The optimal primary model among a Swedish cohort was seen for women with an AUC of 0.716 and for men with an AUC of 0.756. Expanded models demonstrated improved AUCs to 0.738 among US women, 0.724 among Swedish women and 0.769 among Swedish men. The variance explained by the expanded epidemiologic, genetic, and gene-environment interaction models ranged from 21% for women to 28% for men suggested that there are more risk factors yet to be discovered.

We demonstrate that optimal models for prediction of RA should include epidemiologic, genetic and, in some cases, their interaction terms. In other diseases and complex traits, for example type II diabetes^56-61 and cardiovascular disease^{62, 63} variants have at most provided at only a modest increase in predictive ability of clinical models. Willems and colleagues reviewed 20 studies assessing risk prediction for type II diabetes and found that adding genetic factors (up to 40 polymorphisms) did not add significantly to the discriminative ability of any of the clinical models⁶⁴. They showed that where the epidemiologic models are strong (AUCs ranging from 0.68 – 0.92), adding genetic variants with weak effects (even as a cumulative genetic risk score) does not add much to prediction. In a simulation study to explore the potential improvement in discrimination with models that include G−G and G−E interactions, Aschard, et. al demonstrated that inclusion of interaction effects in models for three diseases (breast cancer, type 2 diabetes, and rheumatoid arthritis) was unlikely to dramatically improve the discrimination ability of these models⁶⁵. This study of RA risk is one of the few examples where the inclusion of G and GEI factors in a model resulted in a significant increase in discrimination and reclassification.

Using a primary model consisting of 5 clinical variables that could be collected at a routine clinic visit could be advantageous when screening large numbers of individuals for RA risk. These results are timely as selection of high risk individuals in enrollment in primary prevention trials is an exciting in RA research ^{66, 67}. Thus, we strived to construct a model that performed well statistically, but also included clinical variables that could be collected on a larger scale. The variables for our model were chosen such that all the epidemiological factors could be easily attained by survey (eg. year of birth, smoking, alcohol, education and parity). We studied whether the predictive ability of the model improved with an expanded set of predictors that could be assessed with longer surveys along with genetic risk alleles. We showed that although there were slightly lower AUC statistics in the primary models, the predictive accuracies were significantly lower than the expanded models, as demonstrated by the cNRI results stratified into case and control subgroups. This demonstrates the importance of considering statistical evaluation beyond analysis of AUC statistics ^{54, 55, 68-70}.

If these results were applied to selecting high risk individuals for a prevention trial, the expanded set of variables should be collected by survey, and genetic risk factors should be assessed. Alternatively family history may be a good proxy for genetic risk alleles (although family history information on all subjects was not available in our study). Among men in EIRA, the primary model classified cases as having a lower predicted probability than the expanded model, or lower sensitivity, but had similar specificity to the full model. Among EIRA women, the primary model resulted in a significant decrease in sensitivity, with 42% of cases being classified with lower predicted probabilities; however, there was also an increase in specificity, with 33% of controls being classified with lower predicted probabilities. This would result in fewer potential cases qualifying to be randomized in a trial among men and women, but also reduced false positive fraction (FPF) among women, lowering the chance of unnecessary treatment. However, the performance of the primary model compared to the expanded model in the NHS dataset suggests lower specificity, with 22% of the controls being incorrectly reclassified with a higher predicted probability and no change in sensitivity. This would lead in an increase in FPF, which would result in more women being enrolled and treated unnecessarily. When considering a prognostic model for enrollment in a prevention trial targeted to high-risk groups, maintaining high specificity (and thus low false positive rate) has a higher priority than increasing sensitivity in the setting of treatment with a potentially toxic medication.

One limitation of this study is that models were developed in primarily Caucasian populations. There may be epidemiologic, genetic, or gene-environment interaction factors that might lead to differences in risk models in non-Caucasian populations. Hence, the predictive models developed in this dataset need to be validated in other populations. Another limitation of this study is some factors in the models were originally discovered in NHS or EIRA. These include region of residence and GSTT1 by smoking interaction (NHS), and HLA by smoking, silica and solvents (EIRA)^{10, 14, 15, 42, 71}. Thus, the results of the expanded models could be inflated due to over-fitting. We also recognize that assessing the models on the dataset in which they were developed leads to optimistic measures of variance explained and discrimination. The primary limitation of the study design is that in a matched case control study, we cannot estimate weights for each factor that could be used in other studies, or used in clinical prediction. However, the wealth of data allowed us to gain insights into the optimal collection of variables that should be included in prospective studies for development of prediction rules. Further, although the NHS cohorts involved > 230,000 women, blood was collected on < 25% of women, thus limiting the sample size for analyses that include genetic factors. Finally, blood samples were collected after RA diagnosis in most subjects in NHS and all subjects in EIRA, thus we do not have biomarker data among pre-clinical RA collected prior to the onset of RA symptoms, such as such as autoantibodies or cytokines that have been shown to be strongly associated with risk of development of RA ^{48, 72-76}.

The strength of this study is the use of statistical metrics to parse the effect of the addition of each factor to a model. Over the last decade, limitations of using change in AUC as a primary outcome when comparing both diagnostic and prognostic models has been widely discussed^{54, 55, 69, 77}. Cook ⁷⁷ and Pepe, et al ⁶⁹ separately showed that any new factor would require an exceptionally large OR to show an impact on the AUC. Both Pencina^{54, 55} and Cook⁷⁷ pointed out that for assessing the utility of a prognostic model we are interested in reclassification to a more appropriate risk category (higher for cases and lower for controls), rather than discrimination. By using the IDI, which measures reclassification, we can perform model comparison because the same scale is used in all situations. Even with our relatively modest sample size in NHS, we had >90% power to find an IDI as small as 0.02 and >80% to find a cNRI as small as 0.15. We were able to show the benefit of adding G and GEI terms to an E model in the expanded models.

These results illustrate the challenges of creating a simple risk prediction model for primary prevention trials of RA. Thus, further work on development of highly specific prediction models using prospective cohorts to assess weights is still needed for primary prevention trials. Our data suggests that addition of cumulative RA genetic variants as a genetic risk score and, in some cases, gene-environment interactions to an RA prediction model with epidemiologic factors significantly improves the predictive ability of the model for this complex human autoimmune disease. However, the inclusion of highly specific biomarkers such as ACPA in risk models is likely to improve risk stratification of asymptomatic individuals. Further, we show that identifying risk factors separately in men and women is important, particularly if occupational exposures differ. Ultimately, this collection of variables should be used to estimate weights in other cohorts, then the models validated and their performance assessed before using them for risk stratification.

Supplementary Material

01

Supplemental Table 1. Loci, risk alleles, weights, genotype frequencies and associations with seropositive RA in NHS and ACPA+ RA EIRA for 39 alleles, 8 alleles in HLA-SE and 31 alleles in GRS31

Significant findings.

• Environmental factors, genetic factors and gene-environmental interactions are associated with the risk of developing rheumatoid arthritis in asymptomatic populations.

• RA risk models that include genetic factors and gene-environmental interaction terms are more accurate in modeling risk than those with environmental factors alone.