Evolution of semiquantitative whole joint assessment of knee OA: MOAKS (MRI Osteoarthritis Knee Score)

Abstract

Objective

In an effort to evolve semi-quantitative scoring methods based upon limitations identified in existing tools, integrating expert readers’ experience with all available scoring tools and the published data comparing the different scoring systems, we iteratively developed the MRI Osteoarthritis Knee Score (MOAKS). The purpose of this report is to describe the instrument and its reliability.

Methods

The MOAKS instrument refines the scoring of BMLs (providing regional delineation and scoring across regions), cartilage (sub-regional assessment), and refines the elements of meniscal morphology (adding meniscal hypertrophy, partial maceration and progressive partial maceration) scoring. After a training and calibration session two expert readers read MRIs of 20 knees separately. In addition, one reader re-read the same 20 MRIs 4 weeks later presented in random order to assess intra-rater reliability. The analyses presented here are for both intra- and inter-rater reliability (calculated using the linear weighted kappa and overall percent agreement).

Results

With the exception of inter-rater reliability for tibial cartilage area (kappa=0.36) and tibial osteophytes (kappa=0.49); and intra-rater reliability for tibial BML number of lesions (kappa=0.54), Hoffa-synovitis (kappa=0.42) all measures of reliability using kappa statistics were very good (0.61-0.8) or reached near perfect agreement (0.81-1.0). Only intra-rater reliability for Hoffa-synovitis, and inter-rater reliability for tibial and patellar osteophytes showed overall percent agreement < 75%.

Conclusion

MOAKS scoring shows very good to excellent reliability for the large majority of features assessed. Further iterative development and research will include assessment of its validation and responsiveness.

Introduction

Magnetic resonance imaging (MRI) semi-quantitative scoring of knee osteoarthritis (OA) has proven to be a valuable method for performing multi-feature joint assessment [1,2]. Such approaches score, in a semi-quantitative manner, a variety of features that are currently believed to be relevant to the functional integrity of the knee and/or are potentially involved in the pathophysiology of OA. These articular features include articular cartilage morphology, subchondral bone marrow lesions (BMLs) and cysts, osteophytes, the menisci, the anterior and posterior cruciate ligaments, the collateral ligaments, synovitis, joint effusion, bone attrition, intra-articular loose bodies, and periarticular cysts / bursitis.

Several methods for semi-quantitative assessment of knee OA have been developed [3-5]. The first of the published instruments, the Whole-Organ Magnetic Resonance Imaging Score (WORMS), was developed cognizant of the potential of MRI to provide a measure of all potentially relevant knee OA structures [4]. After identifying some limitations with WORMS [6,7], work was undertaken to develop the Boston Leeds Osteoarthritis Knee Score (BLOKS) [5]. Both of these tools are in widespread use in various observational studies and clinical trials.

Both WORMS and BLOKS have been widely disseminated and used, although the number of direct comparisons of the two instruments to date was quite limited [5]. Recently the validity and reliability of WORMS and BLOKS was compared [8-10]. This comparison has been helpful in identifying the relative merits and weaknesses of these instruments with regards to certain features assumed to be most relevant to the natural history of the disease including cartilage, meniscus and bone marrow lesions. Both instruments had certain limitations and if one were to use the extant literature on published semi-quantitative scoring instruments the discerning investigator would need to choose from a complex array of measures from these two different instruments. For example, WORMS meniscal scoring method mixes multiple different constructs and for BLOKS, application of the BML scoring system was cumbersome and complex and parts of it seemed redundant.

Both of these tools have also undergone unpublished iterations that have made it difficult for the naïve reader to determine the differences between the original instruments description and that which has been used in published research. To facilitate appraisal it is important that iterative developments be made public.

A number of large epidemiologic studies and clinical trials are maturing to the point where large scale scoring is commencing, and it is important that iterative evolution of existing instruments occur with past research findings in mind. In an effort to evolve semi-quantitative scoring methods based upon limitations identified in existing tools, integrating expert readers’ experience with all available scoring tools and the published data comparing the different scoring systems, we iteratively developed the MRI Osteoarthritis Knee Score (MOAKS). The purpose of this report is to describe the instrument and its reliability.

Methods

Upon recognition of the limitations in existing scoring methods iterations were made to elements of the original instruments. The experts (DJH, AG, GHL, AJG, PGC, FR, DG) involved met to consider the limitations of the existing scoring for each pathological feature: Bone marrow lesions (BMLs) and meniscal abnormalities were the most important areas for revision. The focus of the current exercise was therefore to refine the scoring of BMLs, to include sub-regional assessment, to omit some areas of redundancy in cartilage and BML scoring, and to refine the elements of meniscal morphology. After consensus was reached on the new definitions, a intra- and inter-rater reliability exercise was undertaken.

Description of MOAKS

The MOAKS instrument was developed and tested on images obtained on a 3.0 T MRI system with a dedicated peripheral knee coil. Other systems with different field strengths will need to be evaluated.

Delineation of subregional divisions

Osteoarthritis can affect one or multiple compartments in the knee. In MOAKS the knee is divided into 14 articular subregions for scoring articular cartilage and BMLs and in addition the subspinous region is added for BML scoring:

1. The patella is divided into 2 subregions, the medial and lateral patella on the axial view (see Figure 1). The patellar crista (also called apex) is allocated to the medial subregion.

2. The femur is divided into 6 subregions- medial and lateral trochlea, medial and lateral central femur, and the medial and lateral posterior femur. Method of dividing the subregions (see Figure 2):

   1. Trochlea is defined as the femoral articular surface of the PF joint For the division between the trochlea and central regions, on the sagittal image a line is drawn tangentially to the anterior aspect of the proximal tibia (at the margin of the tibial plateau) until it intersects the femoral surface. The division between the central femur and the posterior femur is a line constructed vertically from the posterior aspect of the tibia. (The rationale for choosing these divisions, as opposed to choosing the meniscal boundaries, was the concern that meniscal subluxation and degeneration would introduce variability into this delineation if used).

   2. Anterior and posterior tibial margins are defined irrespective of the presence of osteophytes

   3. The femoral notch is defined as being part of the medial femur.

   4. The superior border of the femur is the physeal line. The posterior border of the trochlea region is the anterior 50% of this region (as measured along the length of the epiphyseal line) (see figure 2)

3. The tibia is divided into 3 medial (anterior, central and posterior) and 3 lateral (anterior, central and posterior) subregions covered by articular cartilage, and the subspinous subregion (SS-region delineated by the tibial spines (Figure 3)).

4. For the anterior, central and posterior divisions the tibia is divided into equal thirds excluding the presence of osteophytes.

Figure 1.

Subregional division of the patella in the axial plane. Axial T2w image shows the medial (M) and lateral (L) portions of the patella as divided in MOAKS. Note that the patellar apex is part of the medial subregion (arrow).

Figure 2.

Anatomical delineation of femur into trochlea (T), central (C) and posterior (P) regions on sagittal projection. Sagittal projection depicts delineation of the tibia into anterior, central and posterior subregions, which is divided into equal thirds.

Figure 3.

Coronal IW image shows anatomical delineation of the tibia into medial, subspinous (SS) and lateral subregions. The femur is divided into the medial and lateral femoral condyle. The intercondylar notch is considered to be part of the medial femur.

Special considerations for scoring in any region

If a lesion spans more than one subregion, the lesion needs to be scored in all subregions, especially as the MOAKS system is modified into a volume-oriented approach and not lesional. Also, if a feature occurs within the subspinous region, but extends also into one or more of the medial or lateral tibial subregions, the lesion will be scored in the subspinous region but also in the additional tibial subregions separately.

Description of scoring for individual features

Each feature is scored separately. We have chosen a number of commonly recognized features based on their likely relevance to pain and structural damage or progression of OA. The scoring for BMLs and articular cartilage described is by subregions as outlined above. Scoring of osteophytes will be performed at defined locations or sites.

For each joint morphologic feature, we have listed the number (if there are multiple abnormalities), its location and grade. For BML and cartilage which are scored by region the score is assigned for the region. We have also described preferred pulse sequences to delineate each feature.

• 1Bone Marrow Lesions (BMLs) and cysts- include areas of presumed bone marrow lesion (BML) (areas of ill-delineated signal within the trabecular bone that are hypointense on T1-weighted images and hyperintense on T2-weighted fat-suppressed images) and associated subarticular bone marrow cysts (defined as well-delineated lesions of fluid equivalent signal directly adjacent to the subchondral plate).

  BMLs will be scored based on the standardized regions outlined previously (p 4-5). Multiple BMLs can occur within each region. Each region will generate a single grade for size inclusive of all BMLs into one score, the number of BMLs per subregion will be counted and % of lesions that is BML as distinct from cyst will be coded (see Table 1).

  1. Each subregion will be graded for BML (including ill-defined lesion and cysts) size in regard to the total volume of the subregion occupied by BML(s) (see Figure 4). Consequently, a BML in a smaller subregion will be smaller when compared to a BML that is assigned the same grade but is present in a larger subregion, or two BMLs of the same absolute size might be assigned different grades if present in subregions with differing volume.

  2. Percentage of the volume of each BML that is BML (as distinct from cyst) is graded as; grade 0= none, grade 1 <33%, grade 2= 33-66% and grade 3 >66%.

  3. If a cyst is present without an associated bone marrow lesion, then cysts will be scored as a 0 for size % of lesion that is BML. The scoring for size should be identical to that for bone marrow lesions (Table 1). In contrast, if a BML does not include any portion of a cyst, the % of the lesion that is BML v. cyst score would be 3.

  4. Do not score marrow signal within osteophytes, however if the lesion extends beyond the osteophyte then it should be scored.

• Pulse sequences:

  Suggested pulse sequences to evaluate BMLs are turbo spin echo T2-/intermediate- or proton-density-weighted fat-saturated or Short TI Inversion Recovery (STIR) sequences in the axial, coronal, and sagittal planes [11]. T1-weighted fat suppressed images after intravenous administration of a gadolinium-based contrast agent may be used alternatively [12]. 3D turbo spin echo (TSE) sequences such as Cube (GE), XETA (eXtended Echo Train Acquisition) or SPACE (Sampling Perfection with Application optimized Contrasts using different flip angle Evolution, Siemens) might possibly be of use for BML assessment in the future [13-15].

  Gradient echo-type sequences, even with robust fat suppression or water excitation, are notoriously insensitive to bone marrow abnormalities due to trabecular magnetic susceptibility of T2* effects, which may result in underestimation of the size of non-cystic BMLs [16,17]. Recent studies have demonstrated that these sequences are also less sensitive in the detection of non-cystic BMLs when using fluid-sensitive FSE sequences as the reference standard [16,18-21]. We do not recommend the use of these sequences for scoring BMLs.

• 2Articular Cartilage

  Although many joint structures are affected in OA, articular cartilage is one of the main tissues involved in the disease process. The rationale for the cartilage score was to provide separate scores for the size (i.e. area affected) and depth of cartilage damage in each of the subregions.

  Each articular cartilage region (except the subspinous region) is graded for size of any cartilage loss (including partial and full thickness loss) as a % of surface area as related to the size of each individual region surface (see figure 5) and % of loss in this subregion that is full-thickness loss (table 2 and figure 5).

  Description of morphology at individual sites is not to be used to create cumulative scores of the knee as histopathology is unclear and this scale is not necessarily linear.

• Pulse Sequences:

  Optimal pulse sequences to evaluate cartilage are still undergoing extensive review. We refer to current state of the art reviews of imaging of articular cartilage [22]. For assessment of early OA commonly used gradient-echo sequences such as spoiled gradient recalled acquisition(SPGR), Fast Low Angle Shot Water Excitation (FLASH), dual echo steady state (DESS) or similar are not suitable as these depict focal defects in an inferior manner when compared to standard T2-/IW (Intermediate Weighted)- or PD (Proton Density)-w fat suppressed turbo-spin echo sequences [23-26].

• 3Osteophytes

  Osteophytes are osteo-cartilaginous protrusions growing at the margins of osteoarthritic joints from a process that involves endochondral ossification. Previous radiographic studies have highlighted this feature as a hallmark of disease [27].

  For MOAKS, osteophytes are scored in each of the 12 locations or sites outlined below (Table 3, Figure 6) and graded according to size where: Grade 0= none; Grade 1=small; Grade 2= medium; Grade 3= large.

  Osteophytes along the trochlea, central and posterior margins of the femoral condyles and tibial plateaus, and along the medial, lateral, superior and inferior margins of the patella (see Figure 6). Posterior femoral osteophytes are assessed peripherally and centrally. The larger osteophyte for either, peripheral or central, location will be scored (Fig 6).

  1. Size of osteophyte should reflect protuberance (how far the osteophyte extends from the joint) rather than total volume of osteophyte (see Figure 7).

  2. Score the largest osteophyte within a given location

• Pulse sequences:

  Optimal pulse sequences to evaluate osteophytes are standard non-fat-suppressed short TE-weighted (preferably T1 over proton density) or gradient- echo-type such as SPGR, FLASH, DESS etc. images in the axial, coronal, and sagittal planes. They may also be assessed on fat suppressed water-sensitive sequences [17].

• 4Hoffa’s Synovitis and Synovitis - Effusion

  Synovitis and effusion are frequently present in OA and in some studies this feature correlated with pain and other clinical outcomes although the literature is conflicting [28-30]. Quantitative MRI markers of synovitis include thickness (or volume) of synovial tissue and the rate of synovial enhancement following intravenous injection of contrast material [31].

  As commonly employed contrast-enhanced sequences are not available in large OA studies due to cost concerns and possible side reactions, a surrogate of signal changes in Hoffa’s fat pad has been applied that has been shown on biopsy to represent mild chronic synovitis [32]. This abnormality is best described as diffuse hyperintense signal on T2/PD/IW-w ft suppressed sequences within the fat pad. It has to be noted that in addition to synovitis these signal changes could also be attributed to other etiologies such as post-arthroscopic changes or Hoffa’s disease [33]. Despite this non-specificity these signal changes are referred to as “Hoffa-synovitis” in MOAKS.

  1. Hoffa-synovitis score (on sagittal image) (one single score for assessment of degree of hyperintensity in Hoffa’s fat pad) based on the region highlighted in Figure 8. Score is based on size: 0= normal; 1= mild, 2= moderate, 3= severe.

• Pulse sequences:

  Suggested pulse sequences to evaluate regions for Hoffa-synovitis are T2/IW- or PD-weighted fat-saturated images in the mid-slices of the sagittal plane [28,29].

  Effusions occur frequently in OA. Recent studies suggest that large synovial effusions may be associated with pain and stiffness in patients with OA [28]. It is important to note that “effusion” (fluid equivalent signal within the joint cavity) on T2/IW/PD w images includes synovitis and effusion [34]. Thus, this imaging measure should preferably referred to as “effusion-synovitis”.

  Scores (Table 4) should be obtained from axial views (Figure 9). Paraarticular cysts and ganglia should not be included in this score as they are considered separately. Pulse Sequences: True amount of joint effusion may be assessed as intraarticular hypointensity on T1w images after i.v. contrast administration. T2/IW/PDw images show intraarticular hyperintensity that represents a composite of effusion and synovitis [34].

• 5Meniscus

  The meniscus has many functions in the knee, including load bearing, shock absorption, stability enhancement and lubrication [35]. Degenerative meniscal lesions such as horizontal cleavages, oblique or complex tears are associated with older age [36]. By the time radiographic disease develops, the overwhelming majority of persons have meniscal lesions [36,37]. The studies that have explored the relationship between the meniscus and risk of disease progression in OA provide a clear indication of the increased risk inherent with damage to this vital tissue [38,39]. Changes in position (also termed subluxation or extrusion) and meniscal morphologic change manifest as tears or loss of substance have both been shown to predispose to cartilage loss. We chose a scoring system that delineated both of these items and provided detail on what the abnormality was and where in the meniscus this occurred.

  1. Extrusion: Four areas where extrusion is scored:

     1. Medial Meniscus: Medial extrusion relative to medial tibial margin (coronal image)

     2. Medial Meniscus: Anterior extrusion (sagittal image) – where extrusion is maximum

     3. Lateral Meniscus: Lateral extrusion relative to lateral tibial margin (coronal image)

     4. Lateral Meniscus: Anterior extrusion (sagittal image) where extrusion is maximum

        Grading for extrusion: Grade 0:<2mm; Grade 1: 2 to 2.9mm, Grade 2: 3-4.9mm; Grade 3: >5mm.

        Note: we wanted to capture anterior extrusion to see if this is independently predictive of cartilage loss/pain/etc compared with medial or lateral extrusion. For each measurement the reference will be the edge of the tibial plateau (excluding any osteophytes).

  2. Morphology: (scored on medial and lateral meniscus for the anterior, body and posterior horn). The anterior and posterior horn regions are scored using the sagittal sequences and the body is scored using the coronal sequences. Morphologic features scored:

     1. Signal (not extending through meniscal surface i.e. not a tear): Y/N

        1. Signal is defined as above as compared with “tears” which are defined as high signal extending to an articular surface on at least 2 slices

     2. Vertical tear (includes radial and longitudinal tears) – must extend to both the femoral and tibial surfaces: Y/N

     3. Horizontal and radial tear: – must extend from the periphery of the meniscus to either a femoral or tibial surface. Y/N

     4. Complex tear: as defined by high signal that extends to both the tibial and femoral surfaces and ≥ 3 points on those surfaces). Y/N

     5. Root tear (posterior horn): - Y/N

     6. Partial maceration: as defined by loss of morphological substance of the meniscus and with or without associated increased signal in the remaining meniscal tissue. Y/N

     7. Progressive partial maceration: Progressive partial maceration as compared to the previous visit. Y/N

     8. Complete maceration: No meniscal substance is visible. Y/N

     9. Meniscal cyst: Y/N

     10. Meniscal hypertrophy is defined as definite increase in meniscal volume in given subregion when compared to normal : Y/N

• Pulse sequences:

  Optimal sequences to evaluate menisci are T1, T2w or proton density fat-saturated images in both coronal and sagittal planes [40,41]. As for BMLs, newly developed 3D FSE sequences might alternatively be used for meniscal assessment [42].

• 6Ligaments/Tendon

  Anterior cruciate ligament tears will be recorded as either absent or present. Partial tears are infrequent and reportedly prone to poor reliability on interpretation [43]. Thus, partial tears are scored as “normal” in MOAKS. Only definite complete tears will be scored.

  1. Anterior Cruciate Ligament (ACL): Score: normal (0) / complete tear (1)

     1. Associated with BML/cyst at site of insertion or origin?: Y/N

     2. ACL Repair: Y/N

  2. Posterior Cruciate Ligament (PCL): Score: normal (0) / complete tear (1)

     1. Associated with BML/cyst at site of insertion or origin?: Y/N

  3. Patellar tendon: Score: 0: no signal abnormality/ 1: signal abnormality present

• Pulse Sequences:

  Optimal sequences to evaluate the aforementioned ligaments are coronal, axial and sagittal, PD or intermediate-weighted, fat Turbo Spin Echo (TSE) images, while 3D sequences also appear promising [17,44,45]. Additional paracoronal T2-w sequences might be helpful to differentiate partial from full thickness tears [46].

• 7Periarticular features

  1. Pes anserine bursitis – absent / present

     This is a bursa that lies anterior and inferior to the medial tibial plateau and is a potential source of pain around the knee. If there is increased signal in this bursa, bursitis is scored as being present.

  2. Iliotibial band (ITB) signal – absent / present

     The iliotibial band is a strong, dense, broad layer of fascia that is part of the fascia lata. The iliotibial band encases the tensor fasciae lata which helps to steady the trunk on the thigh. ¾ of the gluteus maximus inserts into the iliotibial tract and the distal end inserts at the lateral tibial plateau. High signal between the iliotibial band and the femoral cortex may represent irritation of the iliotibial band and is common in medial OA. However, this is also a non-specific finding and often asymptomatic [47]. If there is high signal in this structure, ITB signal should be scored as being present.

  3. Popliteal cyst-absent/ present

     Popliteal cysts are not true cysts and represent fluid in the semimembranosus–medial gastrocnemius bursa. The communication between the joint space and gastrocnemius–semimembranosus bursa allows intraarticular joint fluid to communicate with this bursa [48]. If there is any fluid in this region, this feature should be scored as being present.

     Pulse sequence: Optimal sequences for evaluating popliteal cysts are Proton density or T2-weighted axial images.

  4. Infrapatellar bursa signal – absent / present

     This is a bursa located inferior to Hoffa’s fat pad adjacent to the patellar tendon and is a potential source of pain around the knee. If there is high signal in this bursa, this feature should be scored as being present.

  5. Prepatellar bursa signal – absent / present

     This is a bursa that lies anterior to the patella and is a potential source of pain around the knee. High signal anterior to the patella tendon is a very common non-specific finding of often no clinical relevance. True bursitis is characterized by well demarcated fluid equivalent signal within this structure; only then should this feature be scored as being present.

  6. Ganglion cyst

     1. Associated with the tibio-fibular joint: present/absent

     2. Associated with PCL and ACL: present/absent

     3. Other: present/absent

• Loose bodies: Scale: absent/present

Table 1.

Scoring system for bone marrow lesions

Size of BML (including volume of any associated cysts) by volume	No. of BMLs counted	% of lesion that is bone marrow lesion (v. cyst)
0: none		0: none
1: < 33% of subregional volume		1: < 33%
2: 33 – 66% of subregional volume		2: 33 – 66%
3: > 66% of subregional volume		3: > 66%

Figure 4.

BML grading. Grade 0= none, grade 1 <33% of subregional volume, grade 2= 33-66% of subregional volume and grade 3 >66% of subregional volume. A. Coronal T2 w image shows small grade 1 BML in central subregion of medial tibia. B. A grade 2 BML is depicted in central subregion of medial femur. C. Grade 3 BMLs in the central subregions of the medial femur (arrows) and central medial tibia (arrowheads). D. Coronal image shows BML consisting of non-cystic/ill-defined portion (arrowheads) and cystic (arrow) part.

Figure 5.

Grade for size of any cartilage loss as a % of surface area as related to the size of each individual region. Grade 0= none, grade 1 < 10% of region of cartilage surface area, grade 2= 10-75% of region of cartilage surface area and grade 3 >75% of region of cartilage surface area. (Drawing courtesy of Daichi Hayashi, MD)

Table 2.

Delineation of grading for Cartilage

Size of any cartilage loss (including partial and full thickness loss) as a % of surface area as related to the size of each individual region	% full thickness cartilage loss of the region
0: none	0: none
1: < 10% of region of cartilage surface area	1: < 10% of region of cartilage surface area
2: 10 – 75% of region of cartilage surface area	2: 10 – 75% of region of cartilage surface area
3: > 75% of region of cartilage surface area	3: > 75% of region of cartilage surface area

Table 3.

Sites for osteophyte scoring

Osteophyte Location		Slice Orientation
Anterior femur (trochlea)	Medial	Sagittal/ Axial
	Lateral
Posterior Femur	Medial	Sagittal/ Axial
	Lateral
Central Femur	Medial	Coronal
	Lateral
Patella	Superior	Sagittal
	Inferior
	Medial	Axial
	Lateral
Tibia	Medial	Coronal
	Lateral

Figure 6.

Locations for osteophytes scoring. A. Coronal plane. Osteophytes are scored at the marginal locations of the medial and lateral femur and tibia, respectivel (arrowheads). B. Sagittal plane. Osteophytes are scored at the superior and inferior patellar pole (arrowheads). C. Axial plane. Osteophytes are scored at yhe medial and lateral patella poles (arrowheads), at the anterior medial and lateral femur (black arrows) and the posterior femur medial and lateral (white arrows). Note that there are two locations medially and laterally for osteophytes scoring at the posterior femur, the central and peripheral location. Only the larger osteophyte for either the central or peripheral location will be scored.

Figure 7.

Scoring of osteophytes. Grade 0= none, grade 1=small, grade 2= medium and grade 3= large. A. Grade 1 osteophyte medial femur. B. Grade 2 osteophyte lateral femur. C. Grade 3 osteophyte lateral femur. Size of osteophyte should reflect protuberance (how far the osteophyte extends from the joint) rather than total volume of osteophyte

Figure 8.

Hoffa-synovitis. Sagittal T2w image shows grade 2 hyperintense signal changes in Hoffa’s fat pad consistent with Hoffa-synovitis.

Table 4.

Delineation of grading for Effusion-synovitis

Size of effusion-synovitis
0: physiologic amount
1: small – fluid continuous in the retropatellar space
2: medium – with slight convexity of the suprapatellar bursa
3: large – evidence of capsular distention

Figure 9.

Effusion-synovitis. Hyperintensity within the articular cavity represents a composite of effusion and synovial thickening that cannot be distinguished from each other in the absence of contrast. Grade 0= none, grade 1=small, grade 2= medium and grade 3= large.

A. Grade 0 effusion-synovitis. Normal intraarticular hyperintensity is depicted. B. Grade 1 effusion-synovitis. C. Grade 2 effusion synovitis. D. Grade 3 effusion-synovitis.

Table 5 provides a summary of the changes in the new score (MOAKS) from older instruments including BLOKS [5] and WORMS [4].

Table 5.

Features that are scored in MOAKS in comparison to the original BLOKS instrument and WORMS Score. We used BLOKS as the starting point, and the alterations in MOAKS reflect deviation from BLOKS

MRI feature	Original BLOKS score	Original WORMS score	Alteration in MOAKS
BML size	Score of 0-3 applied for BML volume in 9 different articular subregions. Each BML within a subregion receives an individual size score.	Summed BML size/volume for subregion from 0 to 3 in regard to percentage of subregional bone volume	Modify thresholds from 10-85% to 33-66%. Instead of scoring EACH BML, the entire subregion receives one size score based on the threshold listed above. Grade for regions-use same sub-regions as proposed for cartilage – with the addition of the subspinous region. Count no. of lesions.
BML % area	Score of 0-3 applied for % surface area adjacent to subchondral plate	-	Omitted from new scoring system.
% of lesion BML rather than cyst	Score of 0-3 for % of lesion that is bone marrow lesion as distinct from cyst	Summed cyst size/volume for subregion from 0 to 3 in regard to percentage of subregional bone volume	No change
Cartilage 1	Score of 0-3 for size of loss and % of loss in region that is full thickness	Subregional approach:Scores from 0 to 6 depending on depth and extent of cartilage loss. Intrachondral cartilage signal additionally scored as present/absent	Further subdivision of the medial and lateral tibial region into anterior, central and posterior and subdivision of the weight-bearing femur into central and posterior
Cartilage 2	Extent of any cartilage loss at specified points	-	Omitted from new scoring system.
Osteophyte	Score of 0-3 applied for osteophyte size in12 locations	Score of 0-7 applied for osteophyte size at 16 sites.	No change
Synovitis	Score of 0-3 applied for synovial volume	Combined effusion/synovitis score.	Score of 0-3 applied to signal changes in Hoffa’s fat pad. Feature renamed as “Hoffa-synovitis”
Effusion	Score of 0-3 applied for size of effusion	Score of 0-3 applied for size of effusion	Feature renamed to “Effusion-synovitis”Scoring parameters are unchanged.
Meniscal extrusion	Score of 0-3 applied for amount of extrusion in 4 locations	Not scored	No change
Meniscal signal	Scored as present or absent in 6 regions.	Not scored	No change
Meniscus tear	Type of tear or degenerative process scored as present or absent in 6 regions.	Anterior horn, body, posterior horn scored separately in medial/lateral meniscus from 0 to 4: minor radial or parrot beak tear non-displaced tear or prior surgical repair displaced tear or partial resection complete maceration/destruction or complete resection	Add meniscal hypertrophy, partial maceration and progressive partial maceration
Ligaments	Presence/ absence of tear	Presence/ absence of tear	No change
Periarticular features	Presence/ absence	Presence/ absence	No change

Study Images for Reliability Exercise

The images for the reliability exercise were chosen at random from MRI scans undertaken within the National Institutes of Health (NIH) Osteoarthritis Initiative (OAI, http://www.oai.ucsf.edu). The OAI is a multi-center, longitudinal, prospective observational study of knee OA. The overall aim of the OAI is to develop a public domain research resource to facilitate the scientific evaluation of biomarkers for osteoarthritis as potential surrogate endpoints for disease onset and progression. The OAI database provides an unparalleled state-of-the-art database showing both the natural progression of the disease and information on imaging and biochemical biomarkers and outcome measures.

From 329 participants in the OAI Progression subcohort within OAI Group C (those enrolled through 4/30/2005) with MRI performed on both knees at baseline, all knees with K&L grade 2 or 3 were identified (n=339). A 2×2 factorial design was formed incorporating the factors of side (left vs. right knee) and KL grade (2 or 3). For subjects with both knees having K&L grade 2 or 3, only one knee was randomly selected for potential inclusion. The final 2×2 factorial random sample of 20 knees for the reliability study included 5 randomly selected knees in each of the four cells. A computerized uniform random number between 0 and 1 was assigned to each knee, sorted, then the 5 knees in each cell with the lowest assigned numbers comprised the randomly selected knees in each cell (e.g. n=5 of 90 knees in cell “left knee with K&L grade=3” were randomly selected, etc.).

MRI Acquisition

MRI of both knees was performed on 3 T systems (Siemens Trio, Erlangen, Germany). MRIs were acquired with a USAI quadrature transmit/ receive knee coil. The coronal intermediate-weighted (IW) 2D turbo spin-echo, a sagittal 3D dual echo steady state (DESS) sequence, coronal and axial multiplanar reformations of the 3D DESS and a sagittal IW fat-suppressed (fs) fast spin-echo (FSE) sequence were used for scoringMOAKS. Details of the full OAI pulse sequence protocol and the sequence parameters have been published in detail recently [49]. The sagittal T2 mapping and coronal 3D T1-weighted Fast Low Angle Shot Water Excitation (FLASH WE) sequences were not used for semi-quantitative assessment.

Assessment of Reliability

After an initial training and calibration session on 10 cases that were not included in the reliability exercise, two expert readers with 10 and 8 years experience in semi-quantitative MRI assessment of knee OA respectively, (AG, FR) independently read MRIs of the 20 knees from the OAI. In addition, one reader (FR) re-read the same 20 MRIs 4 weeks later presented in random order to assess intra-rater reliability.

The analyses presented here are for both intra- and inter-rater reliability (calculated using the linear weighted kappa (95%CI)) and the percent agreement of the scoring exercise.

Results

The reliability of the instrument was assessed on 20 subjects with a mean age of 57.8 years (SD 9.8) and a mean BMI of 31.4 kg/m2 (SD 5.0). Fifty-five percent (n=11) were female. Of the 20 knees that were assessed their Kellgren and Lawrence Grades were K&L=2 in 10 knees, K&L=3 in 10 knees. The reliability for the features described above is reported in Table 6.

Table 6.

The reliability for reading of MOAKS features (weighted kappa and percent agreement)

MOAKS Feature	Region	Intra-rater		Inter-rater
		Weighted kappa (95%CI)	Percent agreement	Weighted kappa (95%CI)	Percent agreement
Cartilage Area	Femoral	0.92(0.77-1.00)	0.95	0.62(0.27-0.98)	0.80
	Tibial	0.73(0.47-1.00)	0.80	0.36(0.05-0.67)	0.70
	Patella	0.71(0.43-0.99)	0.75	0.71(0.42-0.99)	0.75
Cartilage Depth	Femoral	0.95(0.85-1.00)	0.95	0.79(0.60-0.98)	0.80
	Tibial	0.83(0.69-0.96)	0.80	0.73(0.52-0.94)	0.75
	Patella	0.92(0.82-1.00)	0.90	0.85(0.72-0.98)	0.80
BML Size	Femoral	0.85(0.69-1.00)	0.85	0.82(0.69-0.95)	0.80
	Tibial	0.76(0.55-0.98)	0.75	0.96(0.86-1.00)	0.95
	Patella	1.00(1.00-1.00)	1.00	0.87(0.74-1.00)	0.85
	Subspinous	0.79(0.60-0.99)	0.80	0.76(0.57-0.94)	0.75
BML, %cyst	Femoral	0.57(0.18-0.95)	0.85	0.74(0.40-1.00)	0.90
	Tibial	0.50(0.23-0.97)	0.80	0.74(0.46-1.00)	0.85
	Patella	0.88(0.73-1.00)	0.90	1.00(1.00-1.00)	1.00
	Subspinous	0.70(0.36-1.00)	0.85	0.65(0.36-0.93)	0.75
BML, No. of lesions	Femoral	0.84(0.68-1.00)	0.85	0.78(0.64-0.92)	0.75
	Tibial	0.54(0.30-0.77)	0.65	0.82(0.62-1.00)	0.85
	Patella	0.90(0.77-1.00)	0.90	0.93(0.82-1.00)	0.90
	Subspinous	0.80(0.63-0.97)	0.80	0.72(0.51-0.93)	0.75
Meniscus Morphology	Medial	1.00(1.00-1.00)	1.00	0.97(0.89-1.00)	0.95
	Lateral	0.91(0.77-1.00)	0.90	0.95(0.86-1.00)	0.95
Meniscal extrusion	Medial	0.82(0.67-0.98)	0.80	0.66(0.46-0.86)	0.60
	Lateral	0.89(0.75-1.00)	0.90	0.79(0.67-0.91)	0.80
Osteophytes	Femoral	0.64(0.36-0.92)	0.75	0.80(0.58-1.00)	0.95
	Tibial	0.70(0.50-0.91)	0.75	0.49(0.24-0.74)	0.55
	Patella	0.84(0.67-1.00)	0.85	0.64(0.39-0.89)	0.65
Hoffa-Synovitis		0.42(0.14-0.70)	0.55	0.70(0.47-0.93)	0.75
Effusion-Synovitis		0.90(0.78-1.00)	0.90	0.72(0.52-0.92)	0.70

With the exception of inter-rater reliability for tibial cartilage surface area (kappa=0.36), tibial osteophytes (kappa=0.49) and intercondylar synovitis (kappa=0.49); and intra-rater reliability for tibial BML number of lesions (kappa=0.54), infrapatellar synovitis (kappa=0.42) and intercondylar synovitis (kappa=0.57) all measures of reliability were very good (0.61-0.8) or reached near-perfect agreement (0.81-1.0) according to the criteria developed by Landis and Koch [50]. The low prevalence of certain features in certain sub-regions may have adversely affected the kappa results hence the percent agreement was also calculated. The majority of features were scored with overall percent agreement above 75% for both, the intra- and inter-reader exercise. Only intra-rater reliability for Hoffa-synovitis (55%), and inter-rater reliability for tibial (55%) and patellar (65%) osteophytes showed overall percent agreement < 75%.

Discussion

The structural determinants of mechanical dysfunction and pain in arthritis are presently not well understood, but probably involve a multitude of interactive pathways characterized by changes in structure and function of the whole joint [51]. The current practice of monitoring only a few of these features (usually radiographically-assessed joint-space narrowing and osteophytes) provides only a restricted view of the disease process and lessens the utility of such assessments. Semi-quantitative MRI scoring will continue to provide important insights into the etiopathogenesis of disease as well as structure-function relationships. As new insights continue to develop the field, this scoring instrument, used to assess structural change in knee OA will similarly need to evolve.

This report provides data on the reliability of structural features scored using MOAKS: a semi-quantitative scoring instrument for MRI assessment of knee OA that builds on the prior experiences of BLOKS and other semi-quantitative scoring tools. The MOAKS instrument refines the scoring of BMLs (providing regional delineation and scoring across regions), cartilage (sub-regional assessment), and refines the elements of meniscal morphology (adding meniscal hypertrophy, partial maceration and progressive partial maceration) and subluxation scoring. In addition we decided to omit some areas of redundancy including the second cartilage scoring method that was part of the original BLOKS instrument and assessed cartilage at defined locations in a specific image and also omitted BML adjacency to subchondral plate scoring.

The timing of this work is relevant as large scale scoring exercises are about to commence in a number of studies including the Osteoarthritis Initiative. Importantly, the measurement properties (including construct and predictive validity and the responsiveness) of these modifications will need to be assessed to ensure their credibility.

The reliability of most of the features that were scored in this exercise was substantial or almost perfect agreement according to Landis and Koch criteria [50]. Some of the features however had only moderate agreement when using kappa statistics including intra-rater reliability for femoral and tibial cystic portions of BMLs, count of tibial BMLs, and Hoffa synovitis, and inter-rater reliability for tibial cartilage area and tibial osteophytes. For some of these features low frequencies of non-zero scores are accountable for these kappa values. Overall percent agreement was good to perfect for almost all features.

Some of the features listed are exploratory in nature and warrant further investigation, including measures of meniscal extrusion. The magnitude of the meniscal extrusion is the same that we have previously used in BLOKS and that has been used in prior analyses [39,52,53]. The extent of analysis in the research literature describing the importance of anterior or posterior extrusion is limited. In our previous analyses anterior extrusion has been associated with an increased risk of cartilage loss [39].

In conclusion, we have refined an existing scoring instrument and assessed its reliability. MOAKS demonstrates reasonable reliability. Further iterative development and research will include assessment of its validation and responsiveness for use in both clinical trials and epidemiological studies. This will allow determination of the significance of specific features to determine if there are subsets that can be consolidated or eliminated to simplify the instrument.