The reliability of a novel magnetic resonance imaging-based tool for the evaluation of forefoot bursae in patients with rheumatoid arthritis: the FFB score

Abstract

Objective. The aim of this study was to determine the reliability of an MRI-based score that evaluates forefoot bursae (FFBs) in patients with RA.

Methods. Items for inclusion, grading criteria and MRI sequences were determined iteratively. The score was evaluated in 30 patients with established RA. Reader agreement was evaluated using the percentage of exact/close agreement, Bland–Altman plots, kappa and intraclass correlation coefficient analyses.

Results. The FFB score assesses nine forefoot regions and contains four items: presence, shape, enhancement and magnetic resonance characteristics. The FFB score showed moderate to good intra- and interreader agreement (κ range = 0.5–0.9 and 0.47–0.87, respectively).

Conclusion. The FFB score is adequately reliable in the evaluation of bursa-like lesions of the forefoot in patients with RA.

Introduction

A high prevalence of bursa-like lesions [forefoot bursae (FFBs)] in the forefoot of patients with RA has been reported using US [1]. FFBs are associated with increased RA disease activity [1, 2] and appear to be a prognostic indicator of foot-related disability [3]. Investigations of FFBs have not yet characterized the underlying pathophysiology and it is possible that a range of bursa-like lesions have been reported. Accurately characterizing FFBs would inform future studies of targeted intervention.

MRI allows multiplanar visualization of the forefoot with differentiation and characterization of soft tissue structures [4, 5]. MRI has greater sensitivity and specificity for RA disease activity within peripheral joints than US or clinical examination, which can both be highly operator dependent [6], or clinical examination [7–9]. MRI is potentially an observer-independent method of characterizing FFBs in patients with RA. The aim of this study was to determine the reliability of a novel MRI-based score for the evaluation of FFBs in patients with RA.

Patients and methods

Study design

An iterative process of score design was completed by a team of rheumatologists, radiologists and podiatrists in a cross-sectional cohort of patients with RA. Ethical approval was obtained from the Southwest Hampshire Ethics Committee. All participants gave written informed consent.

Study population

Participants with an ACR diagnosis of RA, aged 18–80 years, were recruited consecutively from a rheumatology clinic. Participants were excluded if they had received forefoot corticosteroid injection within 12 weeks, had concomitant musculoskeletal disease or were unable to provide consent.

Protocol for score development

Score development adhered as closely as possible to the OMERACT (2009) recommendations for MRI-based quantification of RA. Items included were FFB anatomical location, shape, enhancement and magnetic resonance signal characteristics. The categorization of FFBs based upon descriptive characteristics rather than aetiology or clinical importance was considered the most objective approach and is consistent with principles of radiological investigation.

Protocol for image acquisition

A 1.5T whole-body scanner was used for MRI acquisitions and a four-channel flex extremity radio frequency surface coil was used to image the forefoot (Siemens, Erlangen, Germany). Two-dimensional and three-dimensional sequences of between 29 and 96 slices with 3–0.6 mm slice thickness were completed after orientation with a T1 localizer image. The MRI sequence protocol is provided in section A of the supplementary material (available at Rheumatology Online). Pulse sequence type and timing were selected to visualize (i) the anatomical structure [coronal T1 spin echo (SE)], (ii) the high contrast between fluid and soft tissue [coronal short tau inversion recovery (STIR)] and (iii) synovial inflammation (coronal and sagittal T1-weighted fat suppressed sequences after intravenous contrast administration). The 3D SPACE sequence allowed the three-dimensional reconstruction of identified lesions and orientation with adjacent features. Coronal scans were oriented approximately perpendicular to the metatarsal parabola and sagittal scans were oriented approximately perpendicular to the coronal slice profile and with the shaft of the third metatarsal. The field of view (FoV) in the read direction was determined as the base of the first metatarsal to the distal aspect of the hallux. The FoV in the phase-encode direction was defined as extending from the medial to the lateral foot borders. The TE/TR ratios were adjusted in an iterative process by the radiologist until appropriate image clarity or contrast was achieved.

Protocol for image reading

Images were viewed using syngo fastView software (Siemens). All images were read by two radiologists who were blinded to each other’s scores and clinical data. L.K. re-read an additional set of 10 image sets, at an interval of 4–6 weeks, for the purposes of intrarater agreement analysis.

Identified FFBs were categorized to one of nine sites (see section B of the supplementary material, available at Rheumatology Online). In the event of an FFB extending across anatomical boundaries, the site that contained the majority of the FFB was recorded. Fluid and soft tissue lesions were differentiated using a STIR sequence. Fluid collections were defined as homogeneous hyperintense structures, with fluid-equivalent signal on the STIR sequence and homogenous intermediate signal on non-contrast T1-weighted images. Soft tissue lesions were defined as non-fluid equivalent/intermediate signal on STIR relative to skeletal muscle.

Statistical analysis

Analysis was completed using Stata version 11.0 (StataCorp, College Station, TX, USA) or SPSS version 18.0 (SPSS, Chicago, IL, USA). The study sample size (n = 30) was pragmatically determined based upon estimates of presence/absence identified in previous work [10]. An additional cohort of 10 patients was recruited to complete preliminary intrarater agreement analysis. Participant demographic and clinical characteristics are reported as mean (s.d.) and range.

A radiologist combined mean score for each item was calculated. Agreement was evaluated using estimations of percentage exact agreement, percentage close agreement (within ±2 scores) for all items and kappa agreement for determination of presence or absence. Weighted kappas were used to determine interrater agreement for fluid and soft tissue lesion shape and enhancement scores where both readers identified a lesion as being present.

Results

Study population

The score was evaluated in 30 RA patients (n = 23 female, n = 7 male) with a mean age of 61.7 years (s.d. 4.1), disease duration of 15.3 years (s.d. 10.3) and a 28-joint DAS with CRP (DAS28-CRP) of 3.4 (s.d. 4.5). A total of 300 joints and 540 possible FFB sites per reader were reviewed.

The FFB score

The FFB score items, definitions and grading criteria as well as the FFB score image atlas and user guide can be found in section C of the supplementary material (available at Rheumatology Online).

FFB score reader reliability

A range of lesion characteristics was observed. The FFB score was demonstrated to have moderate to substantial intrareader agreement in the additional set of 10 scans that were re-read (Table 1). Weighted kappa analysis demonstrated that, where both readers identified a lesion as being present, there was moderate to substantial agreement in shape and enhancement grading. Intraclass correlation coefficient (ICC) analysis demonstrated low to moderate agreement, with greater agreement regarding intermetatarsal lesions than plantar lesions. As illustrated in the Bland–Altman plots (Fig. 1), the s.e. mean difference between reader scores was 0.69 (95% CI −8.9 to 6.72). The majority of scores were bounded between the upper and lower limits of 2 s.d. from the mean.

Table 1.

FFB score intra- and interreader agreement

PEA: percentage exact agreement; PCA: percentage close agreement; ICC: intraclass correlation coefficient; SDD: smallest detectable difference.

Fig. 1.

Bland–Altman plots of interrater agreement (both feet combined)

FFB: forefoot bursa.

Discussion

This study is the first to propose a systematic method for the semi-quantitative characterization of bursa-like lesions of the forefoot in patients with RA. In this preliminary analysis, the FFB score was demonstrated to have moderate to substantial reliability.

The evaluation of FFBs, completed as part of the FFB score development, identified differences in the tissue characteristics of observed lesions. Previous authors have suggested that such differences are related to FFB aetiology [2, 11], although characterization by pathological or aetiological means has arguably contributed to confusion within the literature. It is proposed that the FFB score can be utilized to characterize a range of forefoot bursa-like lesions without bias towards their potential aetiology or clinical importance. It should be noted, however, that despite all identified lesions meeting our study definition of bursa (fluid-filled cavity), a range of bursa-like lesions were observed and the clinical significance of this remains unclear. Further evaluation of the clinical importance of MRI-detected FFBs in patients with RA is warranted.

This study has strengths and potential limitations. Bland–Altman plots, kappa values and ICC analyses were used to determine the agreement between readers for the presence of lesions and the characteristics thereof. These analyses consistently demonstrated moderate to substantial agreement between readers across all items. However, these methods do not account for instances where the same lesion may be observed by each reader but scored as occurring in neighbouring locations. The studied population is a cross-sectional consecutive sample of well phenotyped patients with established RA. Thus the generalizability of the study findings to those patients with early or high disease activity should be explored. Similarly, the studied population was recruited from a single site and therefore external validation of the FFB score is required. In addition, the measures of MRI-determined localized disease activity used in this study have not been validated, although they are reproducible [12]. Development of a tool for the evaluation of RA disease activity in the forefoot would significantly enhance work in this area.

Rheumatology key messages.

• The FFB score is adequately reliable for the characterization of bursa-like lesions of the forefoot in patients with RA.

• Further validation of the FFB score for use in patients with RA is required.

• Validation of MRI-determined RA disease activity within the forefoot joints would be beneficial.

Supplementary data

Supplementary data are available at Rheumatology Online.