Abstract
Objectives
To develop and validate a clinical prediction rule to identify inflammatory arthritis (IA) using routinely collected triage data, without ACPA.
Methods
Prospective observational data from 184 patients referred to a tertiary care rheumatology clinic for joint pain over a 9-month period and meeting inclusion criteria were used to derive a clinical risk prediction rule for the diagnosis of IA versus non-IA, utilizing penalized and stepwise logistic regression modelling, including age, sex, CRP and RF. Internal validation was performed using 5-fold cross-validation. A population within the UK Biobank with a diagnosis of incident IA or non-IA, within 180 days of baseline assessment (N = 2828), was used for broad external domain validation. Model performance was assessed by the c-statistic (cvAUC) and the Hosmer–Lemeshow test.
Results
In the derivation cohort, CRP had the strongest association with diagnosis of IA in both models, other important predictors being sex and CRP twice the upper limit of normal in the stepwise model (cvAUC = 0.72, 95% CI 0.58, 0.85) and additionally RF and age in the penalized regression model (cvAUC = 0.77, 95% CI 0.59, 0.95). When applied in the UK Biobank, AUC values decreased (0.54–0.55); however, limitations in accurately defining the timing of incident arthritis indicate the need for geographical validation in a cohort that better fits the target population and setting.
Conclusions
This simple prediction rule, utilizing demographic and laboratory data, performed well in discriminating IA from non-IA in the derivation population. Additional validation studies in similar clinical settings are required to further refine the model and determine generalizability.
Keywords: rheumatology, triage, arthritis, clinical decision rule
Key messages.
A prediction rule using basic demographic and laboratory data had good discriminative ability for IA.
Elevated CRP was the most important predictor of subsequent diagnosis of IA.
This study highlights challenges in accurately identifying timing of incident inflammatory arthritis in population-based studies.
Introduction
Joint pain is a significant cause of healthcare resource utilization and one of the most common reasons for presentation to a primary care provider and referral to rheumatology [1–3]. Increasing worldwide prevalence of OA and RA has led to rising referrals for rheumatology assessment [4, 5]. Deficits in the rheumatology workforce in the USA and Canada, which are projected to grow over the next decade, make timely assessment of all referrals for joint pain impossible, in many regions [6, 7]. Thus, as the global burden of arthritis grows, so too does the rheumatology workforce deficit, highlighting the urgency to implement robust triage systems to ensure accurate prioritization of rheumatology referrals for inflammatory conditions.
Inflammatory arthritis (IA), including RA and PsA, requires prompt diagnosis and initiation of DMARDs to ensure optimal patient outcomes [8]. The first 3 months following symptom onset has been deemed a therapeutic window of opportunity, during which timely intervention improves rates of long-term disease remission and prevents joint damage [9]. The benefits of early treatment in RA are far-reaching beyond preservation of joint function, at both the individual and societal level, including reduction in co-morbidities attributable to chronic inflammation and lost work productivity [10, 11]. Given the importance of early diagnosis and the limited capacity for rheumatology assessment in many places, accurately differentiating IA from non-inflammatory joint pain (non-IA) at referral triage is of paramount importance to ensure appropriate utilization of finite resources and timely access to care for those with IA.
There is an urgent need to develop novel tools to assist rheumatology triage. Several patient-reported tools, containing clinical symptoms and demographic information, have been developed to identify IA with a focus more recently on fully digitalized symptom checkers [12–16]. Despite this, the EULAR has acknowledged that an appropriately validated tool is lacking [17]. Furthermore, the implementation and widespread uptake of such tools are limited in most routine practice settings. The feasibility of a tool utilizing purely demographic and laboratory data has not been studied.
This research aimed to explore whether basic demographic and laboratory data, excluding history and physical examination components, could be used to develop a clinical risk prediction rule for use at the point of rheumatology triage to accurately predict which patients referred for joint pain have a higher risk of having IA and thus should be assessed urgently.
Methods
Derivation cohort
Population
Prospective observational data collected for a separate study were utilized to derive and internally validate the clinical prediction rule [18]. The derivation study population included all new referrals received at one tertiary care clinic in Halifax, Nova Scotia, Canada, between December 2019 and September 2020 for assessment of undifferentiated peripheral joint pain, stiffness or swelling in patients 18 years of age or older. Referrals were randomized in a 1:1 fashion to paper-based triage either by a rheumatologist or physiotherapist. Referrals with previously diagnosed IA and those for assessment of predominantly axial symptoms were excluded. The dataset contained pertinent clinical and laboratory data obtained from referrals and subsequent clinic visits on 184 patients, 108 with non-IA and 76 with IA.
Predictors
Potential predictors were identified a priori based on prior studies and considering those that are readily available to primary care providers when assessing and referring patients [12, 16, 19]. These included age (continuous variable in years), sex assigned at birth (binary variable male or female), RF positivity (binary variable yes or no, utilizing 30 IU/ml as the cut-off), CRP value (continuous variable in mg/L) and CRP more than twice the upper limit of normal (binary variable yes or no, utilizing 16 mg/L as the cut-off). We excluded ACPA from the algorithm, given a lack of routine testing in primary care at our site.
Outcome
The primary outcome was the diagnosis of IA and was defined as any subtype of autoimmune IA, including RA, PsA, undifferentiated IA, palindromic rheumatism, reactive arthritis and enteropathic arthritis. In all cases of IA, the clinical diagnosis was made by a rheumatologist at the time of clinic visit. Non-IA was defined as those cases without a confirmed inflammatory aetiology for their symptoms, with the most common diagnoses being OA and fibromyalgia. Seventy-five of the 108 non-IA cases were confirmed by a rheumatologist, while 33 were declined based on referral information received and not assessed in clinic. Sensitivity analysis was performed excluding these 33 referrals.
Validation cohort
Population
Broad domain external validation was performed using the UK Biobank, a large prospective population-based cohort study, with over half a million participants aged 40–69 and living within 25 miles of one of 22 assessment centres, recruited between 2006 and 2010 from England, Wales and Scotland [20, 21]. Within the UK Biobank, we excluded those who reported OA, RA or PsA at study recruitment, as these individuals represented prevalent cases. Incident IA cases and non-IA cases made up our population within the UK Biobank and were identified based on the first occurrence of a relevant International Classification of Diseases, 10th Revision (ICD-10) code within 6 months following the baseline study assessment, where relevant laboratory tests, including RF and CRP, were obtained. We restricted incident cases to greater than 0 but less than 180 days after baseline bloodwork, to ensure the relevant laboratory tests (CRP, RF) predated the diagnosis of IA and were within an appropriate time frame to make their association with joint disease more likely and thus replicable of the derivation cohort.
Predictors
Values for age, sex, CRP and RF were obtained from the UK Biobank baseline visit [21]. We created a binary variable for CRP twice the upper limit of normal, using the cut-off of 16 mg/L as we did for the derivation cohort. Similarly, we created a binary variable for RF positivity using a cut-off of 30 IU/ml. Given the high amount of RF missingness in the UK Biobank, we considered RF reportability (data field 30826). Those values of RF which were ‘not reportable at assay (too low)’ were coded as RF negative, and those which were ‘not reportable at assay (too high)’ were coded as RF positive based on the manufacturer’s analytical range of 10–120 IU/ml used in the UK Biobank [22].
Outcome
Ascertainment of incident cases of IA was done using the first occurrence of health outcomes field and the ICD-10 codes for RA and PsA, including M05, M06 and M07 (Fig. 1). The first occurrence field includes linked health data obtained from several sources, including primary care, hospital inpatient data, death register and self-report. We excluded those classified based on self-report to improve validity of the diagnosis and increase similarity between outcome definition in the derivation and validation cohorts. Non-IA cases were defined using ICD codes for polyosteoarthritis (M15), other/unspecified OA (M19), first CMC joint OA (M18) and non-specific soft tissue disorder (M79). We excluded OA of the hip and knee to create a population more reflective of the derivation cohort.
Figure 1.
Consort diagram showing flow of participants in the UK Biobank and ICI-10 codes included
Statistical analysis
All continuous variables were standardized using z-score standardization. Univariable logistic regression analysis was conducted for each predictor in the model to explore statistical significance and direction of effect. Age was modelled as a continuous variable, quadratic variable, and categorical variable. Akaike information criterion and Bayesian information criterion of the three models for age were compared to determine the model of best fit. All predictors identified a priori were then included in the multivariable logistic regression modelling, and the clinical prediction rule was derived using both stepwise regression and elastic net regression to perform variable selection and coefficient shrinkage in the case of elastic net regression. The rule was then internally validated using 5-fold cross-validation. For elastic net regression, the lambda with the lowest mean squared error was identified via 5-fold cross-validation and used in the final penalized regression model. Performance of the clinical prediction rule was evaluated, including discrimination (ROC curve) and calibration (goodness of fit/Hosmer–Lemeshow test) using cross-validation and in the UK Biobank.
Further sensitivity analysis to account for missing data in the derivation dataset was completed by imputing all RF missing values as negative (zero) and imputing the missing CRP values with the median value of 6.89 mg/L. These values were chosen to minimize the introduction of bias away from the null hypothesis. Descriptive statistics were reported for this subset, and the clinical prediction rule was developed and internally validated on this sample as per the complete case analysis. Imputation methods were used for a sensitivity analysis only, as we did not feel the missing at random assumption would hold.
Summary statistics were reported as mean and S.D. for continuous variables and number and proportions for categorical variables. Student’s t-test was used to compare continuous variables, and Fisher’s exact test for categorical variables. Statistical significance was defined as a two-sided P-value less than 0.05. Analysis was completed using STATA version 17.0 [23].
We used the Transparent Reporting of a multivariable prediction model for Individual Prognosis or Diagnosis (TRIPOD) checklist to facilitate transparency in reporting (Supplementary Table S1, available at Rheumatology Online) [24].
Ethics
Full IRB review was waived for the collection of the derivation dataset as the project was deemed a quality improvement project; however, a privacy impact assessment was completed at the time of the original study [18]. Derivation data utilized for this study were coded, and full IRB review was waived by the IRB, as this was deemed to be an extension of the original study. For the UK Biobank data, volunteers gave informed consent, and the study was conducted under generic approval from the NHS National Research Ethics Service (approval letter dated 29 June 2021, Ref 21/NW/0157) and under UK Biobank project approval 129024 (principal investigator: J.H.R.).
Results
In the derivation dataset, those with IA were slightly older (57 versus 51) as compared with those with non-IA. Females made up 78% of the non-IA group and 60% of the IA group (Table 1). Mean CRP was higher in the IA group (25.04 mg/L, S.D. = 28.97 versus 7.47 mg/L, S.D. = 8.96), with individuals being three times more likely to have a CRP greater than twice the upper limit of normal and twice as likely to have a positive RF than those with non-IA (Table 1). In the UK Biobank, there were statistically significant differences in all baseline clinical predictors as compared with the derivation cohort, except age in the IA groups, which was similar in both cohorts (Table 1). While positive RF and elevated CRP were at least twice as likely in IA compared with non-IA in the UK Biobank, the percentage of those with IA in the UK Biobank with a positive RF was much less (12 versus 33%) and the average CRP at diagnosis much lower (3.98 mg/L, S.D. = 5.43 versus 25.04 mg/L, S.D. = 28.97) as compared with the derivation cohort (Table 1).
Table 1.
Distribution of model predictors based on arthritis type in the derivation and external validation cohorts.
| Derivation cohort |
External validation cohort/UK Biobank |
|||||
|---|---|---|---|---|---|---|
| Predictors | Non-IA n = 108 | IA n = 76 | Non-IA n = 2573 | P a | IA n = 255 | P a |
| Age (years): mean (S.D.) | 51.39 (14.46) | 56.73 (16.35) | 58.65 (7.66) | <0.001 | 58.2 (7.87) | 0.28 |
| Standardized age: mean (S.D.) | −0.143 (0.94) | 0.203 (1.05) | 0.30 (0.94) | <0.001 | 0.25 (0.97) | 0.72 |
| Sex | ||||||
| Male (%) | 24 (22) | 30 (40) | 1027 (40) | <0.001 | 255 (61) | <0.001 |
| Female (%) | 84 (78) | 46 (60) | 1546 (60) | 100 (39) | ||
| CRP > 2× ULN | ||||||
| No (%) | 86 (80) | 36 (47) | 2529 (98) | <0.001 | 241 (95) | <0.001 |
| Yes (%) | 14 (13) | 37 (49) | 44 (2) | 13 (5) | ||
| Missing (%) | 8 (7) | 3 (4) | ||||
| CRP (mg/L) value | ||||||
| Mean (S.D.) | 7.47 (8.96) | 25.04 (28.97) | 2.88 (4.28) | <0.001 | 3.98 (5.43) | <0.001 |
| Missing (%) | 13 (12) | 7 (9) | ||||
| Standardized CRP value: mean (S.D.) | −0.34 (0.41) | 0.47 (1.33) | 0.09 (1.02) | <0.001 | 0.35 (1.29) | <0.001 |
| RF-positive | ||||||
| No (%) | 76 (70) | 45 (59) | 2514 (98) | <0.001 | 224 (88) | <0.001 |
| Yes (%) | 16 (15) | 25 (33) | 59 (2) | 31 (12) | ||
| Missing (%) | 16 (15) | 6 (8) | ||||
ULN = upper limit of normal.
Comparing derivation cohort to external validation cohort.
Baseline clinical predictors were relatively unchanged in the sensitivity analyses that excluded the patients without a rheumatologist-confirmed diagnosis and following simple imputation of data for missing RF and CRP (Table 2). Following imputation of RF, the difference between groups (non-IA versus IA) in terms of RF negativity was increased (85 versus 67%) as compared with the complete case analysis (70 versus 59%). To further explore our assumption that lab values were not missing at random, we assessed model predictors and outcome based on RF status and found that those with RF missingness appeared more similar to those with RF negativity with respect to the outcome and CRP (Supplementary Table S2, available at Rheumatology Online).
Table 2.
Distribution of model predictors based on arthritis type in the derivation cohort, comparing complete case count and two sensitivity analyses: (1) excluding the 33 individuals without rheumatology assessment and (2) simple imputation of missing data.
| Complete case count |
Excluding 33 non-IA non-confirmed cases |
Simple imputation |
||||
|---|---|---|---|---|---|---|
| Predictors | Non-IA n = 108 | IA n = 76 | Non-IA n = 75 | IA n = 76 | Non-IA n = 108 | IA n = 76 |
| Age (years): mean (S.D.) | 51.39 (14.46) | 56.73 (16.35) | 51.73 (14.56) | 56.73 (16.35) | 51.39 (14.46) | 56.73 (16.35) |
| Standardized age: mean (S.D.) | −0.143 (0.94) | 0.203 (1.05) | −0.16 (0.93) | 0.16 (1.04) | −0.143 (0.94) | 0.203 (1.05) |
| Sex | ||||||
| Male (%) | 24 (22) | 30 (40) | 20 (26.67) | 30 (39.47) | 24 (22) | 30 (40) |
| Female (%) | 84 (78) | 46 (60) | 55 (73.33) | 46 (60.53) | 84 (78) | 46 (60) |
| CRP 2× ULN | ||||||
| No (%) | 86 (80) | 36 (47) | 55 (73.33) | 36 (47.37) | 94 (87) | 39 (51) |
| Yes (%) | 14 (13) | 37 (49) | 14 (18.67) | 37 (48.68) | 14 (13) | 37 (49) |
| Missing (%) | 8 (7) | 3 (4) | 6 (8) | 3 (3.95) | ||
| CRP value | ||||||
| Mean (S.D.) | 7.47 (8.96) | 25.04 (28.97) | 8.95 (10.05) | 25.04 (28.97) | 7.40 (8.40) | 23.37 (28.09) |
| Missing (%) | 13 (12) | 7 (9) | 8 (10.67) | 7 (9.2) | ||
| Standardized CRP value: mean (S.D.) | −0.34 (0.41) | 0.47 (1.33) | −0.35 (0.43) | 0.34 (1.25) | −0.32 (0.41) | 0.45 (1.36) |
| RF-positive | ||||||
| No (%) | 76 (70) | 45 (59) | 49 (65.33) | 45 (59.21) | 92 (85) | 51 (67) |
| Yes (%) | 16 (15) | 25 (33) | 15 (20) | 25 (32.89) | 16 (15) | 25 (33) |
| Missing (%) | 16 (15) | 6 (8) | 11 (14.67) | 6 (7.89) | ||
Non-IA = non-inflammatory joint pain; IA = inflammatory arthritis; ULN = upper limit of normal.
The elastic net logistic regression model retained five predictors, while the stepwise logistic regression model retained only three predictors (Table 3). CRP value was the strongest predictor of IA in both models, with an OR of 2.23 (95% CI 1.25, 13.97) in the elastic net and 4.35 (95% CI 1.31, 14.48) in the stepwise model. In the complete case analysis, RF positivity was not retained in the stepwise regression model, but in the elastic net model, it was retained as a negative predictor of subsequent diagnosis of IA, although the value of the coefficient was very low with a non-significant CI. In simple imputation analysis, where RF was imputed as negative, RF was also retained in the stepwise regression model, now becoming a positive predictor of diagnosis of IA, suggesting the unexpected direction of effect of RF in the elastic net model may be due to overfitting of the model in our data and the small sample size. Comparing the AUC between models, the elastic net model had a higher AUC in both the complete case count and sensitivity analysis, ranging from 0.77 to 0.73 (Fig. 2, Table 4). In external validation, the stepwise model had a higher AUC, which may indicate overfitting of the elastic net model in the derivation dataset, suggesting the stepwise model may perform better when applied to other datasets. Results from the Hosmer–Lemeshow goodness-of-fit test suggest the models were well calibrated in the derivation and validation cohorts (Table 4). Various cutpoints, and their associated sensitivity, specificity, positive/negative likelihood ratios and positive/negative predictive values, are listed in Supplementary Table S3a and b, available at Rheumatology Online.
Table 3.
Predictors selected and corresponding coefficients for the elastic net regression model and stepwise logistic regression model for the complete case count and sensitivity analysis in the derivation dataset.
| Complete case count |
33 non-IA cases without confirmation removed |
Simple imputation |
||||
|---|---|---|---|---|---|---|
| Elastic net coefficients 95% CI | Stepwise coefficients 95% CI | Elastic net coefficients 95% CI | Stepwise coefficients 95% CI | Elastic net coefficients 95% CI | Stepwise coefficients 95% CI | |
| Intercept term | −0.0394 (−0.58, 1.18) | 0.3313 (−0.54, 1.19) | 0.2799 (−0.29, 1.63) | 0.1962 (−0.20, 0.59) | −0.3240 (−0.94, 0.67) | −0.5341 (−1.13, 0.06) |
| CRP (mg/L) value | 0.8032 (0.22, 2.63) | 1.47 (0.27, 2.67) | 0.7183 (0.12, 2.64) | 1.3439 (0.64, 2.04) | 0.7541 (0.14, 0.67) | 1.2502 (0.19, 2.31) |
| CRP > 2× ULN | 0.6453 (−1.23, 1.59) | 0.1510 (−1.24, 1.55) | 0.3838 (−1.49, 1.32) | – | 0.6596 (−1.01, 1.63) | 0.3011 (−1.01, 1.61) |
| Age (years) | 0.1369 (−0.20, 0.56) | – | 0.0969 (−0.25, 0.53) | – | 0.1629 (−0.14, 0.55) | – |
| Sex | −0.5607 (−1.49, 0.11) | −0.7934 (−1.57, −0.02) | −0.4223 (−1.38, 0.27) | – | −0.4508 (−1.28, 0.20) | – |
| RF | −0.0039 (−0.02, 0.01) | – | −0.0036 (−0.02, 0.01) | – | 0.6293 (−0.06, 1.53) | 0.8675 (0.08, 1.65) |
Non-IA = non-inflammatory joint pain; ULN = upper limit of normal.
Figure 2.
Cross-validated AUC for the elastic net and stepwise regression models. (A) Elastic net model derivation dataset. (B) Stepwise model derivation dataset. (C) Externally validated elastic net model. (D) Externally validated stepwise model
Table 4.
Model performance measures, including discrimination, calibration and number of predictors included for the complete case, sensitivity analysis and external validation cohorts.
| Model type |
Model performance |
||
|---|---|---|---|
| Complete case count analysis | Discrimination cv-AUC (S.E.)a | Calibration: cv-HL statistic (P-value)b | Number of predictors included |
| Elastic net | 0.77 (0.09) | 6.28 (0.281) | 5 |
| Stepwise | 0.72 (0.07) | 5.63 (0.23) | 3 |
| Exclusion of non-confirmed cases |
|
|
|
| Elastic net | 0.73 (0.04) | 2.72 (0.44) | 5 |
| Stepwise | 0.72 (0.04) | 0.66 (0.88) | 3 |
| Simple imputation analysis |
|
|
|
| Elastic net | 0.75 (0.09) | 7.9 (0.12) | 5 |
| Stepwise | 0.69 | 27.61 (0.16) | 3 |
|
AUC | Calibration in the large | |
| Elastic net | 0.54 (0.02) | 8.00 (0.43) | – |
| Stepwise | 0.55 (0.02) | 7.60 (0.47) | – |
AUC = Area under the curve.
HL = Hosmer-Lemeshow.
Discussion
We explored the feasibility of using readily available demographic and laboratory data to develop a clinical risk prediction rule to identify IA in the rheumatology triage setting. Despite the absence of patient-reported symptoms or physical exam findings, these models had acceptable discriminative ability. Elevated CRP was the most important predictor of subsequent diagnosis of IA, highlighting the importance of incorporating inflammatory markers into a triage algorithm.
Triage is vitally important to optimal resource utilization in rheumatology practice, particularly in areas with limited resources. The accuracy of triage is dependent on information received in the referral letter, which has been consistently shown to lack pertinent clinical data [19, 25]. Furthermore, primary care practitioners often lack confidence and proficiency in detecting synovitis, making reliance on physical examination findings in referral letters challenging [26]. Many self-administered patient questionnaires have been developed to partially overcome these challenges and incorporate important symptomology, including the early inflammatory arthritis (EIA) detection tool, the Rotterdam Early Arthritis Cohort Tool (REACH) and the Clinical Arthritis Rule (CARE) with area under the receiver operating curves ranging from 0.72 for REACH (externally validated) to 0.90 for the EIA (internally validated only) [12–14]. The discriminative ability of a triage tool incorporating a fully digitalized symptom checker, RhePort, was improved from an AUC of 0.53 to 0.74 using machine learning models that incorporated laboratory parameters [16]. Consistent with our study, elevated inflammatory markers were one of the most important predictors of subsequent diagnosis of an inflammatory rheumatic disease [16]. A recent cross-sectional study aimed to identify positive and negative predictors of IA identified in referral letters and found both inflammatory markers and RF were significant positive predictors of IA in univariable analysis; however, a high proportion of missing data meant laboratory data were not included in the multivariable analysis [19]. The importance of laboratory investigations in the diagnosis of IA has been further highlighted in a prospective examiner-blinded study, whereby deprivation of laboratory data significantly impacted accurate diagnosis, even after a rheumatologist completed a history and physical examination [27]. Other groups have identified factors associated with progression to RA based on anti-CCP status and developed prediction scores for those with musculoskeletal (MSK) symptoms, a positive ACPA but no synovitis, to identify those less likely to need secondary care [28, 29]. In contrast, our clinical prediction rule does not include ACPA as it is not always readily available, particularly in resource-limited settings.
In many centres, rapid access early IA clinics have evolved to help centres meet targets of early assessment in those with a high probability of IA. These clinics often include patients with high titre antibodies such as RF and ACPA, where there is clearly a high probability of IA. Triage of referrals with intermediate probability of IA is often more challenging. EULAR has recently highlighted the potential benefit of pre-visit telehealth to assist with prioritization of referrals, but many centres lack adequate resources to implement this on a large scale [30]. We propose this rule, which could be used to assist with triage of referrals that fall in this intermediate probability group, to determine when collecting additional clinical information would be most beneficial for accurate triage (Supplementary Fig. S1, available at Rheumatology Online). Determination of optimal cutpoints and development of a simple score calculator, for ease of use, will be the focus of future work after further external validation studies are performed.
In broad domain external validation using the UK Biobank, the AUC values dropped, which is explicable for several reasons. Some prior studies assessing incident RA in the UK Biobank have utilized solely hospital admissions data to define incident cases [31, 32]. There are clear drawbacks to this approach, as most patients with IA are diagnosed and managed in an outpatient setting, meaning there could be a significant delay between diagnosis and admission to hospital. In an attempt to overcome this, we excluded all those with arthritis at baseline and utilized the first occurrence field to establish dates for incident diagnosis. The first occurrences field incorporates data from several different sources, including hospital inpatient, primary care, self-report and death register. Despite this, for ICD codes included in this study, over 85% of this field was still sourced from hospital admissions data for IA. This reliance on inpatient data to determine first occurrence has been recognized as potentially problematic by the UK Biobank, and it has been acknowledged that some conditions will be ascertained later if coded through hospital admissions data after they were diagnosed through primary care [33]. Thus, the reliability of incident diagnosis date remains unclear. An important difference between the UK Biobank and our derivation dataset is timing of laboratory data. In the UK Biobank, laboratory data, including CRP and RF for most participants was available only at a single time point (the baseline study assessment) and was performed for research purposes and not for the diagnostic evaluation of joint pain. Our population was restricted to those with a relevant first occurrence code within 6 months of this baseline bloodwork, given the time-sensitive nature of CRP in relation to onset of symptoms and diagnosis of IA. Despite this, there was very little difference between CRP levels in the IA and non-IA groups, thus given the importance of CRP in our model, it did not perform well in this cohort, and we conclude that this tool may not be generalizable to a wider population of patients with arthritis outside the rheumatology triage setting. It should also be noted that the derivation and validation cohorts came from differing countries where differences in access to primary care, genetic predispositions, lifestyle factors and environmental exposures may have contributed to model performance and generalizability. Furthermore, the effects of a healthy volunteer selection bias in the UK Biobank need to be considered, as it has been shown that UK Biobank participants are less likely to be obese, to smoke and drink alcohol than the general population, which could also impact IA risk [34].
Strengths of our study include the development of the clinical prediction rule using robust methodology, including both stepwise and penalized logistic regression techniques and external domain validation using a large prospective cohort. In addition, our derivation dataset consisted of prospectively collected data with very little missing data on candidate predictors included in the rule. Limitations of our study include the relatively small sample size and the fact that 33 of those with non-IA were not assessed in clinic. Although sensitivity analysis excluding these individuals did not reveal significant differences in the performance of the clinical prediction rule, misclassification was possible. In addition, modelling RF as a continuous variable may have improved model performance; however, a high amount of missing data in the derivation dataset prevented us from exploring this further. It is also possible that the derivation cohort may be enriched with cases of IA, as given limited resources, we typically do not accept referrals purely for management of non-IA. As such, geographical validation will be important to determine generalizability to other rheumatology clinics. Due to cohort availability and time constraints, only broad external domain validation in a different clinical setting was performed in this study. As such, while internal validation is one method to identify and mitigate overfitting, further evaluation with temporal validation (same centre, different time point), and geographic validation at a different tertiary care rheumatology clinic will be valuable to further explore this.
Our findings suggest a simple rule utilizing demographic and laboratory data available at the point of rheumatology triage can assist with differentiation between IA and non-IA cases. The discriminative ability of such a tool may be improved by combining it with clinical data in appropriate patients, and this tool can be used to define such populations who may benefit from gathering further clinical information.
Rheumatologists are facing increased referrals for joint pain and novel triage tools are needed to meet this demand. This clinical prediction rule is a simple tool that can be utilized to assist triage pathways and ensure patients are assessed based on need.
Supplementary Material
Contributor Information
Janet H Roberts, Division of Rheumatology, Department of Medicine, QEII Health Sciences Centre and Dalhousie University, Halifax, NS, Canada; Arthritis Research Canada, Richmond, BC, Canada.
Olga Demler, Harvard Chan School of Public Health, Harvard University, Boston, MA, USA.
Alexandra Legge, Division of Rheumatology, Department of Medicine, QEII Health Sciences Centre and Dalhousie University, Halifax, NS, Canada; Arthritis Research Canada, Richmond, BC, Canada.
Claire E H Barber, Arthritis Research Canada, Richmond, BC, Canada; Department of Medicine, Cumming School of Medicine, University of Calgary, Calgary, AB, Canada; Department of Community Health Services, Cumming School of Medicine, University of Calgary, Calgary, AB, Canada.
Supplementary data
Supplementary data are available at Rheumatology Online.
Data availability
Data are available upon request.
Funding
This research was funded by Translating Research into Care (TRIC) grant from the Queen Elizabeth II Health Sciences Centre Research Foundation. C.E.H.B. is supported by funding from the Arthritis Society Canada Stars Career Development Award, funded by the Canadian Institutes of Health Research–Institute of Musculoskeletal Health and Arthritis Society STAR-19-0611/CIHR SI2-169745.
Disclosure statement: The authors have declared no conflicts of interest.
References
- 1. Finley CR, Chan DS, Garrison S et al. What are the most common conditions in primary care? Systematic review. Can Fam Physician 2018;64:832–40. [PMC free article] [PubMed] [Google Scholar]
- 2. Haas R, Gorelik A, Busija L et al. Prevalence and characteristics of musculoskeletal complaints in primary care: an analysis from the population level and analysis reporting (POLAR) database. BMC Prim Care 2023;24:40. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 3. Graydon SL, Thompson AE. Triage of referrals to an outpatient rheumatology clinic: analysis of referral information and triage. J Rheumatol 2008;35:1378–83. [PubMed] [Google Scholar]
- 4. Long H, Liu Q, Yin H et al. Prevalence trends of site-specific osteoarthritis from 1990 to 2019: findings from the Global Burden of Disease Study 2019. Arthritis Rheumatol 2022;74:1172–83. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 5. Shi G, Liao X, Lin Z et al. Estimation of the global prevalence, incidence, years lived with disability of rheumatoid arthritis in 2019 and forecasted incidence in 2040: results from the Global Burden of Disease Study 2019. Clin Rheumatol 2023;42:2297–309. [DOI] [PubMed] [Google Scholar]
- 6. Bolster MB, Bass AR, Hausmann JS et al. 2015 American College of Rheumatology Workforce Study: the role of graduate medical education in adult rheumatology. Arthritis Rheumatol 2018;70:817–25. [DOI] [PubMed] [Google Scholar]
- 7. Kulhawy-Wibe SC, Widdifield J, Lee JJY et al. Results from the 2020 Canadian Rheumatology Association’s Workforce and Wellness Survey. J Rheumatol 2022;49:635–43. [DOI] [PubMed] [Google Scholar]
- 8. Fraenkel L, Bathon JM, England BR et al. 2021 American College of Rheumatology Guideline for the treatment of rheumatoid arthritis. Arthritis Care Res 2021;73:924–39. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 9. van Nies JA, Krabben A, Schoones JW et al. What is the evidence for the presence of a therapeutic window of opportunity in rheumatoid arthritis? A systematic literature review. Ann Rheum Dis 2014;73:861–70. [DOI] [PubMed] [Google Scholar]
- 10. Plein S, Erhayiem B, Fent G et al. Cardiovascular effects of biological versus conventional synthetic disease-modifying antirheumatic drug therapy in treatment-naïve, early rheumatoid arthritis. Ann Rheum Dis 2020;79:1414–22. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 11. Rat AC, Boissier MC. Rheumatoid arthritis: direct and indirect costs. Joint Bone Spine 2004;71:518–24. [DOI] [PubMed] [Google Scholar]
- 12. van Delft ETAM, Barreto DL, van der Helm-van Mil AHM et al. Diagnostic performance and clinical utility of referral rules to identify primary care patients at risk of an inflammatory rheumatic disease. Arthritis Care Res 2022;74:2100–7. [DOI] [PubMed] [Google Scholar]
- 13. Bell MJ, Tavares R, Guillemin F et al. Development of a self-administered early inflammatory arthritis detection tool. BMC Musculoskelet Disord 2010;11:50. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 14. Tavares R, Wells GA, Bykerk VP et al. Validation of a self-administered inflammatory arthritis detection tool for rheumatology triage. J Rheumatol 2013;40:417–24. [DOI] [PubMed] [Google Scholar]
- 15. Knitza J, Mohn J, Bergmann C et al. Accuracy, patient-perceived usability, and acceptance of two symptom checkers (Ada and Rheport) in rheumatology: interim results from a randomized controlled crossover trial. Arthritis Res Ther 2021;23:112. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 16. Knitza J, Janousek L, Kluge F et al. Machine learning-based improvement of an online rheumatology referral and triage system. Front Med 2022;9:954056. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 17. Combe B, Landewe R, Daien CI et al. 2016 update of the EULAR recommendations for the management of early arthritis. Ann Rheum Dis 2017;76:948–59. [DOI] [PubMed] [Google Scholar]
- 18. Roberts JH, Gunn C, Mackinnon JE et al. Feasibility of physiotherapist-led rheumatology triage: a randomized study. J Rheumatol 2024;51:715–20. [DOI] [PubMed] [Google Scholar]
- 19. Thoms BL, Bonnell LN, Tompkins B, Nevares A, Lau C. Predictors of inflammatory arthritis among new rheumatology referrals: a cross-sectional study. Rheumatol Adv Pract 2023;7:rkad067. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 20. Sudlow C, Gallacher J, Allen N et al. UK Biobank: an open access resource for identifying the causes of a wide range of complex diseases of middle and old age. PLoS Med 2015;12:e1001779. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 21. Allen N, Sudlow C, Downey P et al. UK Biobank: current status and what it means for epidemiology. Health Policy Technol 2012;1:123–6. [Google Scholar]
- 22. UKbiobank.ac.uk [Internet]. Cited November 1, 2024. Available from: https://biobank.ndph.ox.ac.uk/showcase/showcase/docs/serum_biochemistry.pdf
- 23. StataCorp. Stata Statistical Software: Release 17. College Station, TX: StataCorp LLC, 2021. [Google Scholar]
- 24. Collins GS, Reitsma JB, Altman DG, Moons KG. Transparent reporting of a multivariable prediction model for individual prognosis or diagnosis (TRIPOD): the TRIPOD statement. BMJ. 2015;350:g7594. [DOI] [PubMed] [Google Scholar]
- 25. Jack C, Hazel E, Bernatsky S. Something’s missing here: a look at the quality of rheumatology referral letters. Rheumatol Int 2012;32:1083–5. [DOI] [PubMed] [Google Scholar]
- 26. Meyfroidt S, Stevens J, De Lepeleire J et al. A general practice perspective on early rheumatoid arthritis management: a qualitative study from Flanders. Eur J Gen Pract 2015;21:231–7. [DOI] [PubMed] [Google Scholar]
- 27. Ehrenstein B, Pongratz G, Fleck M, Hartung W. The ability of rheumatologists blinded to prior workup to diagnose rheumatoid arthritis only by clinical assessment: a cross-sectional study. Rheumatology 2018;57:1592–601. [DOI] [PubMed] [Google Scholar]
- 28. Garcia-Montoya L, Nam JL, Duquenne L et al. Prioritising referrals of individuals at-risk of RA: guidance based on results of a 10-year national primary care observational study. Arthritis Res Ther 2022;24:26. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 29. Duquenne L, Hensor EM, Wilson M et al. Predicting inflammatory arthritis in at-risk persons: development of scores for risk stratification. Ann Intern Med 2023;176:1027–36. [DOI] [PubMed] [Google Scholar]
- 30. de Thurah A, Bosch P, Marques A et al. 2022 EULAR points to consider for remote care in rheumatic and musculoskeletal diseases. Ann Rheum Dis 2022;81:1065–71. [DOI] [PubMed] [Google Scholar]
- 31. Zhang J, Fang X-Y, Wu J et al. Association of combined exposure to ambient air pollutants, genetic risk, and incident rheumatoid arthritis: a prospective cohort study in the UK Biobank. Environ Health Perspect 2023;131:37008. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 32. Chen L, Wu B, Mo L et al. Associations between biological ageing and the risk of, genetic susceptibility to, and life expectancy associated with rheumatoid arthritis: a secondary analysis of two observational studies. Lancet Healthy Longev 2024;5:e45–e55. [DOI] [PubMed] [Google Scholar]
- 33. UKbiobank.ac.uk. Cited November 1, 2024. Available from: https://biobank.ndph.ox.ac.uk/ukb/ukb/docs/first_occurrences_outcomes.pdf
- 34. Fry A, Littlejohns TJ, Sudlow C et al. Comparison of sociodemographic and health-related characteristics of UK Biobank participants with those of the general population. Am J Epidemiol 2017;186:1026–34. [DOI] [PMC free article] [PubMed] [Google Scholar]
Associated Data
This section collects any data citations, data availability statements, or supplementary materials included in this article.
Supplementary Materials
Data Availability Statement
Data are available upon request.