Research Diagnostic Criteria for Temporomandibular Disorders (RDC/TMD): Development of Image Analysis Criteria and Examiner Reliability for Image Analysis

Abstract

Introduction

As a part of a multi-site RDC/TMD Validation Project, comprehensive TMJ diagnostic criteria were developed for image analysis using panoramic radiography, magnetic resonance imaging (MRI), and computed tomography (CT).

Methods

Inter-examiner reliability was estimated using the kappa (k) statistic, and agreement between rater pairs was characterized by overall, positive, and negative percent agreement. CT was the reference standard for assessing validity of other imaging modalities for detecting osteoarthritis (OA).

Results

For the radiological diagnosis of OA, reliability of the three examiners was poor for panoramic radiography (k = 0.16), fair for MRI (k = 0.46), and close to the threshold for excellent for CT (k = 0.71). Using MRI, reliability was excellent for diagnosing disc displacements (DD) with reduction (k = 0.78) and for DD without reduction (k = 0.94), and was good for effusion (k = 0.64). Overall percent agreement for pair-wise ratings was ≥ 82% for all conditions. Positive percent agreement for diagnosing OA was 19% for panoramic radiography, 59% for MRI, and 84% for CT. Using MRI, positive percent agreement for diagnoses of any DD was 95% and for effusion was 81%. Negative percent agreement was ≥ 88% for all conditions. Compared to CT, panoramic radiography and MRI had poor to marginal sensitivity, respectively, but excellent specificity, in detecting OA.

Conclusion

Comprehensive image analysis criteria for RDC/TMD Validation Project were developed, which can reliably be employed for assessing OA using CT, and for disc position and effusion using MRI.

INTRODUCTION

Given the complex nature of classifying temporomandibular disorders (TMD), several systems have been developed and widely used.^1,2 Research Diagnostic Criteria for Temporomandibular Disorders (RDC/TMD) is a widely used diagnostic system for TMD.² Proposed in 1992, this system has two assessment components. Axis I, a clinical and radiographic assessment, is designed to differentiate myofascial pain, disc displacement, and arthralgia, arthritis, and arthrosis. Axis II evaluates psychological status and pain-related disability.

Axis I of the RDC/TMD briefly describes the image analysis criteria for temporomandibular joint (TMJ) disc displacement using arthrography and magnetic resonance imaging (MRI) and osteoarthritis based on tomography.² Although panoramic radiology was not included as an imaging option in the original RDC/TMD, it has been recommended as a screening tool for TMJ pathology.^3-5 With the increasing use of computed tomography (CT) and MRI, it was necessary to develop comprehensive criteria for image analysis using these techniques as a part of the RDC/TMD.

The multi-site RDC/TMD Validation Project was initiated, in part, to assess the reliability and validity of the current RDC/TMD Axis I clinical disorders and to revise them, if indicated. To enhance the operational specification of the RDC/TMD, we developed a set of criteria for acquiring and analyzing panoramic, MRI, and CT images to evaluate the TMJ. Radiologists and TMD clinicians, some of whom were part of the original RDC/TMD proposal, developed these image analysis criteria. This diagnostic criteria was developed from a review of the literature,^2,6-8 recommendations by the members of External Advisory Panel appointed by National Institute of Dental and Craniofacial Research (NIDCR) for the project, and suggestions solicited from members of the TMD and radiology community. This methodology for developing the image analysis criteria suggests that these have content validity.

For the RDC/TMD Validation Project’s outcomes to be meaningful, its criteria should have acceptable inter-examiner reliability. The clinical assessment for RDC/TMD has shown high reliability for both intra-examiner or inter-examiner agreement.^9,10 Due to lack of imaging specifications and detailed criteria in the original RDC/TMD, the reliability of the radiologists in interpreting TMJ images using these criteria has not been previously assessed.

The purpose of this paper is to report the image analysis criteria for RDC/TMD Validation Project. This study also assessed the inter-examiner reliability of the radiologists in the RDC/TMD Validation Project to interpret panoramic, MR, and CT images of the TMJ for osteoarthritis (OA), and MR images for disc displacement or effusion. Finally, we assessed the criterion validity of panoramic radiography and MRI to assess osseous tissue changes using CT as the reference standard.

MATERIAL AND METHODS

Participants and Research Locations

Study participants were consecutively recruited from August 2003 to September 2006 at the University of Minnesota, Minneapolis, MN, University of Washington, Seattle, WA, and University at Buffalo, Buffalo, NY. The study was approved by the institutional review boards of the three universities. Informed consent was obtained for all participants. HIPAA guidelines were followed.

Imaging Modalities

The RDC/TMD assessment protocol specified the use of three imaging modalities: panoramic radiography, MRI, and CT. At the University of Minnesota, a Sirona Orthophos digital panoramic machine, Siemens Sensation 16 MDCT, and for MRI, Siemens Vision 1.5T and Siemens Avanto 1.5T equipment were used. At the University of Washington, Siemens Orthophos panoramic machine, GE LightSpeed VCT machine and GE Signa 1.5 T MRI scanner were used. At the University at Buffalo, Siemens Orthophos 3 panoramic machine, Toshiba Aquilion CT, and Siemens Symphony 1.5T system were used.

Image Acquisition

Panoramic radiography

Panoramic radiographs were obtained without any modification of the protocols used in the respective clinics. Diagnostic quality digital or film-based panoramic images were of acceptable density and contrast as determined by the radiologist. The radiographs were acquired with proper subject positioning as recommended by the manufacturer of the panoramic equipment. The radiographs showed the maxilla and mandible, including both of the condyles as well as the dentition.

Magnetic resonance imaging

Participant preparation

To prepare for closed-mouth MR examination, the TMD clinician instructed the participants to put their back teeth together in the position where these fit the best. The clinician then verified this position visually. The same written instructions were given to the radiology technologist, to read to the participant prior to acquiring the MRI in the closed-mouth position. To prepare for the open-mouth position MR examination, the TMD clinician instructed the participants to open as wide as they could tolerate. The maximum open position was determined clinically. The TMD clinician then placed a mouth opening device (Burnett BiDirectional TMJ Device, Medrad Inc, Pittsburgh, PA) between the participant’s teeth and opened it to the maximum that the participant could tolerate. The amount of opening was recorded by the clinician and this information was given to the radiology technologist so that they could place the Burnett BiDirectional TMJ Device to the desired opening. The technologist had permission to reduce the amount of mouth opening on the Burnett device during the procedure if the participant could not tolerate it.

Magnetic resonance image acquisition

The closed mouth MR images were acquired in proton density (PD) and T2 algorithm by using a dedicated TMJ surface coil. For open-mouth MR images, only PD images were acquired, as effusion was not evaluated in open-mouth views. A minimum of six slices of each joint were obtained in sagittal and axially corrected coronal views. The PD images had TR = 2000.0, TE = 17.0, while the T2 images had TR = 2000.0, TE = 102.0. The axially corrected coronal views were obtained in closed-mouth views only, where the sections were made through the long axis of the condyles.

Computed tomography

Participant preparation

Similar to preparing the participants for MRI examination, the TMD clinician instructed the participants to put their back teeth together in the position where these fit the best. The TMD clinician verified this position visually. The same written instructions were given to the radiology technologist to read to the participant prior to acquiring the CT scan.

Computed tomography image acquisition

All images were obtained from a multi-detector CT and reconstructed in a hard-tissue algorithm. A minimum of 12 sections of each condyle (1-mm thickness slices) were generated in sagittal and axially corrected coronal views. On axially corrected coronal views, the sections were created through the long axis of the condyles. No soft tissue analysis was made in the CT images. Because range of translation of the condyle was evaluated in MRI and to reduce radiation exposure, open-mouth views were not obtained with CT.

Image Interpretation Criteria

The radiologists at the UM and UW were diplomates of the American Board of Oral and Maxillofacial Radiology and the radiologist at UB was a diplomate of the American Board of Radiology and Neuroradiology; radiologists had between 12 and 23 years of experience interpreting TMJ images. In the Validation Project, the diagnostic criteria used by the radiologists to identify panoramic, MRI and CT-disclosed osteoarthritis, and MRI-disclosed disc displacements are shown in Tables 1-4. The images were interpreted in the following sequence: panoramic radiographs, MR images (both osseous- and non-osseous component assessments), and the CT images. All the available slices from CT and MRI were evaluated. For the reliability studies, a different protocol was used and is described in the appropriate section. The radiologists in both situations were blind to the clinical history or clinical diagnosis of the participants.

Table 1.

Hard tissue assessment using Panoramic, MRI and CT.

Scoring option	Scoring Criteria
Condylar head: (Score Yes/No for each of the eleven criteria)	Condylar Hypoplasia: Condylar morphology is normal but the size is small from all dimensions. This is associated with either an increase in the joint space in a normal articular fossa, or a small articular fossa. Condylar Hyperplasia: Condylar morphology is normal but the size is large in all dimensions. This will be associated with either lack of joint space in a normal articular fossa, or an enlarged articular fossa to accommodate the large condyle. Articular Surface Flattening: A loss of the rounded contour of the surface. Subcortical Sclerosis: Any increased thickness of the cortical plate in the load bearing areas relative to the adjacent non-load bearing areas. With MRI, this is identified as low signal intensity in bone marrow on proton density and T2 study. Subcortical Cyst: A cavity below the articular surface that deviates from normal marrow pattern. Surface Erosion: Loss of continuity of articular cortex. Osteophyte: Marginal hypertrophy with sclerotic borders and exophytic angular formation of osseous tissue arising from the surface. Generalized Sclerosis: No clear trabecular orientation with no delineation between the cortical layer and the trabecular bone that extends throughout the condylar head. Loose Joint Body: A well-defined calcified structure(s) that is not continuous with the disc or osseous structures of the joint. With MRI, this is identified as low and/or high signal on proton and T2 study. Deviation in Form: Condylar deviation in form is defined as a departure from normal shape, such as concavity in the outline of the cortical plate, and not attributable to flattening, erosive changes, osteophytes, hyper or hypoplasia. Boney Ankylosis: Continuous osseous structure between the condyle and temporal bone associated with no discernable joint space and no translation of the condyle in the open mouth views.
Fossa/eminence (Score Yes/No for each of the three criteria).	Articular Surface Flattening: A loss of the rounded contour of the surface. Subcortical Sclerosis: Any increased thickness of the cortical plate in the load bearing areas relative to the adjacent non-load bearing areas. With MRI, this is identified as low signal intensity in bone marrow on proton and T2 study. Surface Erosion: Loss of continuity of cortical margin.
Condylar position: (Select one or more of the four options)	Concentric position with normal joint space Concentric position with decreased joint space Anterior position Posterior position
Condylar translation: Sagittal open mouth MRI. (Select one of the three options)	Apex of the condyle translates to less than the apex of the articular eminence Apex of the condyle translates to the apex of the articular eminence Apex of the condyle translates beyond the apex of the articular eminence
Panoramic radiographs only: (Score Yes/No)	Odontogenic pathology(ies) Non-odontogenic pathology(ies)
MRI only: (Score Yes/No).	Condylar Edema: Any high signal intensity within the bone marrow of the condyle present on T2 study

Table 4.

Disc Diagnosis for TMJ Using MRI. (Scoring options are A, B, C, D, or E as in the table below)

Normal: Disc location is normal on closed and open images. Disc Displacement with Reduction: Disc location is displaced on closed-mouth images but normal in open-mouth images. Disc Displacement without Reduction: Disc location is displaced on closed-mouth and open-mouth images. Indeterminate: Disc location is not clearly normal or displaced in the closed-mouth position. Disc Not Visible: Neither signal intensity nor outlines make it possible to define a structure as the disc in the closed-mouth and open-mouth views. If the images are of adequate quality in visualizing other structures in the TMJ, then this finding is interpreted to indicate a deterioration of the disc, which is associated with advanced disc pathology.

Overview

Scoring criteria

For CT and MRI, multiple slices of a joint were evaluated, and the “worst case” scenario was scored when there were different findings in the different slices. For example, in the case of disc displacements, if in the closed-mouth position the disc was clearly anteriorly displaced only in one section but “normal” in all other sections, the diagnosis was anteriorly displaced disc. If in the open-mouth position the disc reduced in all but one of the sections, it was diagnosed as a non-reducing disc.

Two exceptions to the above rule were allowed: 1) in case of disc deformity, “best case” scenario was recorded, i.e., if the disc was deformed in the closed-mouth position but not in the open-mouth position, then it was not considered deformed; and 2) for condylar concentricity, radiologists could check multiple answers when there were different impressions in different sections. For example, if the condyle was anterior in one section and posterior in another, the radiologist checked both applicable responses.

The scoring options are mostly Yes/No. To overcome the limitations of categorizing by Yes/No choice, the scoring form provided options to indicate conditions that did not fall within the diagnostic scheme for osteoarthritis. In addition, for each image type, the radiologist had an area on the scoring form to comment on the findings or on the quality of the images.

Osseous component

The temporomandibular joint osseous component features were assessed using panoramic radiography, MRI, and CT. Evaluation of the joints was recorded on a scoring form. Each scoring factor had a Yes/No option. Definitions of these factors are provided in Table 1. For panoramic radiography, in addition to the observation of TMJs, odontogenic and non-odontogenic findings were recorded.

Osseous component analysis criteria

For the condylar head, features to note were gross hypoplasia or hyperplasia, flattening of the articular surface, subcortical sclerosis or cyst, surface erosion, osteophytes, generalized sclerosis, loose joint bodies, and deviation in form. For the fossa, the criteria included flattening of the articular eminence, subcortical sclerosis, and surface erosion. For the joint, condylar position and ankylosis was also noted. On completion of these observations, a diagnosis was made, categorizing the joint as normal, indeterminate, or affected with osteoarthritis (OA). A diagnosis of ankylosis was also allowed. Condylar edema was noted on MRI.

Table 2 describes the diagnostic conclusions. Examples of osseous changes are displayed in Figures 1, 2 and 3.

Table 2.

Osseous Diagnoses for the TMJ from Panoramic Radiographs, CT, and MRI. (Scoring options are A, B, or C as in the table below)

No Osteoarthritis Normal relative size of the condylar head; and No subcortical sclerosis or articular surface flattening; and No deformation due to subcortical cyst, surface erosion, osteophyte, generalizedsclerosis, or loose calcified bodies. Indeterminate for Osteoarthritis Normal relative size of the condylar head; and Subcortical sclerosis with/without articular surface flattening; or Articular surface flattening with/without subcortical sclerosis; and No deformation due to subcortical cyst, surface erosion, osteophyte, generalized sclerosis, or loose calcified bodies. Osteoarthritis Deformation due to subcortical cyst, surface erosion, osteophyte, generalized sclerosis or loose calcified bodies.

Figure 1.

Sagittal CT views of condyles representing examples of nonosteoarthritic or indeterminate osseous changes observed. A-B. Rounded condylar head, and well-defined cortical margin. C. Rounded condylar head, and well-defined noncortical margin. D-E. Indeterminate for OA: slight flattening of anterior slope and well-defined cortical margin. F. Indeterminate for OA: flattening of anterior slope and a pointed anterior tip that is not sclerosed, well-defined cortical margin, fossa is shallow. G. Well-defined cortical margin has a notch on the superior part, a deviation in form, fossa is shallow. H. Narrowed appearance of the condylar head near medial part, close position of the cortical plates gives the impression of sclerosis, a nonosteoarthritic condyle,

Figure 2.

Sagittal CT views of condyles representing examples of osseous changes observed, and corresponding osteoarthritis (OA) diagnoses. A. Indeterminate for OA: subcortical sclerosis without any flattening, without erosion. B. OA: subcortical sclerosis, osteophytic growth on the anterior part of the condyle. C. OA: subcortical sclerosis, flattened posterior slope of the eminence, osteophytic growth on the anterior part of the condyle, limited joint space superiorly. D. OA: flattened superior margin, osteophytic growth at the anterior, fossa is shallow. E. OA: flattened posterior slope of the eminence, condylar margin is eroded and lacks corticated border, osteophytic growth. F. OA: flattened superior margin, decreased condylar height, margin is eroded and lacks corticated border, osteophytic growth, outline of the fossa is irregular. G. OA: a bony cavity below the articular surface margin (i.e., subcortical cyst), osteophytic growth, posterior slope of the eminence is sclerosed. H. OA: generalized sclerosis, surface erosion, osteophytic growth, sclerosed fossa.

Figure 3.

Axially corrected coronal CT views of condyles representing examples of osseous changes observed, and corresponding osteoarthritis (OA) diagnoses. A-B. Nonosteoarthritic condyles, rounded condylar head, and well-defined cortical margin. C. Nonosteoarthritic condyle, flattened superior margin, and well-defined cortical margin. D. Nonosteoarthritic condyle, flattened lateral slope, and well-defined cortical margin. E. Indeterminate for OA: rounded condylar head and subcortical sclerosis. F. Indeterminate for OA: subcortical sclerosis. G. OA: subcortical sclerosis, surface erosion. H-I. OA: surface erosion. J. OA: generalized sclerosis, and subcortical cysts. K. Nonosteoarthritic condyle, well-defined corticated margin, bifid appearance, deviation in form. L. Nonosteoarthritic condyle, subcortical sclerosis in nonarticulating surface, bifid appearance, deviation in form.

Position and translation of the condyle

Position of the condyle in relation to the articular fossa was evaluated on sagittal PD MR images and on sagittal CT images. Translation of the condyle was evaluated on open-mouth sagittal PD MR images.

Non-osseous component

Assessment of the non-osseous component was limited to MRI. Range of motion was assessed only with MRI sections in the open-mouth position. Evaluation of the joints was recorded on a scoring form. Each scoring factor had a Yes/No option. Definitions of these factors are provided in Table 3.

Table 3.

Non-osseous Component Assessment Criteria Using MRI.

Scoring option	Scoring Criteria
Disc Position: Closed-mouth sagittal and axially corrected coronal views (Score Yes/No for each criteria)	Normal Disc Position In the sagittal plane, relative to the superior aspect of the condyle, the border between the low signal of the disc and the high signal of the retrodiscal tissue is located between the 11:30 and 12:30 clock positions; and In the sagittal plane, the intermediate zone is located between the anterior-superior aspect of the condyle and the posterior-inferior aspect of the articular eminence; and In the oblique coronal plane, the disc is centered between the condyle and eminence in the medial, central, and lateral parts.
	Indeterminate In the sagittal plane, relative to the superior aspect of the condyle, the low signal of the disc and the high signal of the retrodiscal tissue is located anterior to the 11:30 position, but the condyle contacts the intermediate zone located between the anterior-superior aspect of the condyle and the posterior-inferior aspect of the articular eminence; or In the sagittal plane, relative to the superior aspect of the condyle, the low signal of the disc and the high signal of the retrodiscal tissue is located between the 11:30 and 12:30 clock positions, but the intermediate zone of the disc is located anterior to the condyle; and In the axially corrected coronal plane, the disc is positioned between the condyle and eminence in the medial, central, and lateral parts.
	Disc Displacement In the sagittal plane, relative to the superior aspect of the condyle, the low signal of the disc and the high signal of the retrodiscal tissue is located anterior to the 11:30 clock position; and In the sagittal plane, the intermediate zone of the disc is located anterior to the condyle; or In the axially corrected coronal plane, the disc is not centered between the condyle and eminence in either the medial or the lateral parts.
	Disc Not Visible: Neither signal intensity nor outlines make it possible to define a structure as the disc.
Disc Position: Open-mouth sagittal views (Score Yes/No for each criteria)	Normal Disc Position: The intermediate zone is located between the condyle and the articular eminence.
	Persistent Disc Displacement: The intermediate zone is located anterior to the condylar head.
	Disc Not Visible: Neither signal intensity nor outlines make it possible to define a structure as the disc.
Disc Shape: Closed-mouth sagittal views (Score Yes/No for each criteria)	Normal: The disc in the sagittal plane is biconcave.
	Deformed: All shapes other than biconcave in the sagittal plane.
	Disc Not Visible: Neither signal intensity nor outlines make it possible to define a shape of the disc.
Effusion: Open-or closed-mouth sagittal views (Score Yes/No for each criteria)	None: No bright signal in either joint space in the T2 weighted images.
	Slight Effusion: A bright signal in either joint space that conforms to the contours of the disc, fossa/articular eminence, and/or condyle.
	Frank Effusion: A bright signal in either joint space that extends beyond the osseous contours of the fossa/articular eminence and/or condyle and has a convex configuration in the anterior or posterior recesses.
Loose calcified Bodies closed-mouth sagittal views (Score Yes/No)	Single or multiple discrete low signal intensity objects are present in the joint spaces, and are not attached to the condyle, fossa or eminence in any plane.

Non-osseous component analysis criteria

Using PD and T2 weighted MR images, the following non-osseous features were observed: position of the intermediate zone and posterior band of the disc in relation to the condylar head in the closed- and open-mouth sagittal views, disc shape, disc rotation (position in the medio-lateral direction), effusion, and presence of loose calcified bodies in the soft tissues. Subsequent to such observation, the diagnoses for the soft tissues were: normal, anterior disc displacement with reduction, anterior disc displacement without reduction, disc not visible, or indeterminate. Table 4 describes the diagnostic conclusions for non-osseous components. Examples of various changes associated with the non-osseous components of the joint are displayed in Figures 4-7.

Figure 4.

Sagittal proton density MR views representing examples of changes in the soft tissue components of the TMJ observed, and corresponding disc diagnoses. A-B. Normal disc location: normal biconcave disc shape, posterior band is at 11:30-12:30 position, intermediate zone is in contact with the condylar head. C. Normal disc location: thinning of the disc, posterior band is at 11:30-12:30 position, intermediate zone is in contact with the condylar head. D-E. Indeterminate for disc location: normal biconcave disc shape, posterior band is at 11:30-12:30 position, intermediate zone is not in contact with the condylar head. F. Anteriorly displaced disc: normal biconcave disc shape, posterior band is at <11:30 position, intermediate zone is not in contact with the condylar head. G. Anteriorly displaced disc: intermediate zone is not visible. H Anteriorly displaced disc: thickened disc.

Figure 7.

Axially corrected coronal MR views in the closed-mouth position representing examples of changes in the soft tissue components of the TMJ observed. A-C. Centrally located disc. Image C shows bifid condyle. D-F. Disc is laterally displaced. G-H. Medial displacement of the disc.

Calibration of the Radiologists

Two board-certified oral and maxillofacial radiologists and two board-certified medical radiologists representing the three research locations participated in the calibration and reliability studies. The first calibration and reliability study was conducted at one of the research locations (UB). This exercise spanned over two days. The training and calibration was done on the first day by projecting and discussing slides of panoramic radiographs, CT and MRI showing all characteristics of normal and indeterminate osseous conditions and OA. In addition, slides of MRI were used for demonstrating all characteristics of soft tissues including disc position, shape and joint effusion.

Initial Reliability Study Methods

On the second day of the exercise, the reliability of the radiologists was evaluated. Each radiologist viewed panoramic radiographs; representative axially corrected coronal and sagittal slices from CT; and open- and closed-mouth sagittal views of PD-MRI, and T2-MRI. For the initial reliability study, the images were collected from prior studies or teaching files from the three research locations. For subsequent annual reliability studies, the images used were from the participants in the current project. For the initial reliability testing, 59 joints from 30 panoramic radiographs were evaluated. For OA, representative slices of 70 joints were evaluated on CT and 70 joints on MRI. For disc position, representative MRI slices of open-and closed mouth views of 68 joints were evaluated. The selected images represented the full scope of possible diagnoses. The radiologists independently interpreted the images on computer monitors housed in separate rooms. No time limit was imposed to view the images and no clinical data was provided. The results were scored on scannable forms.

Annual Reliability Studies

The RDC/TMD Validation Project was completed in four years. After the calibration and initial reliability test, three subsequent annual reliability studies were performed at individual research locations using images obtained on the participants of the project. For each study, separate sets of images were compiled by one of the radiologists into a computer disc for distribution to all the radiologists. The images composed of 20 panoramic radiographs (40 joints), 25 sets of CT and MRI (each set containing a representative sagittal and axially corrected slice in closed-mouth position) and 25 sets of MRI (each set containing a representative sagittal slice in closed- and open-mouth position). Panoramic, CT and MR images included examples of normal and OA, were included in random order. The MR images included examples of normal disc position, indeterminate, disc displacement with reduction, and disc displacement without reduction. These four types of disc conditions were included in random order. For effusions, the MR images included examples of frank, slight or no effusion. All the radiologists in the three research sites reviewed the same set of images and were blinded to the diagnosis of the other radiologists. The images were scored according to the criteria developed for RDC/TMD Validation Project

Data Analysis for Reliability Studies

For the reliability studies, the RDC/TMD requires dichotomous radiological ratings to be used in the clinical TMD diagnosis algorithm. Therefore, OA ratings were categorized as present (frank ratings) versus absent (normal or indeterminate), disc position was categorized as displaced versus nondisplaced where not visible, indeterminate, or other ratings were excluded, and effusion was categorized as present (frank rating) versus absent (normal or indeterminate). Reliability was estimated using the kappa (k) statistic. According to Fleiss et al., k values < 0.40 are considered poor reliability, values between 0.40 and 0.75 are considered fair to good reliability, and values > 0.75 are considered excellent reliability.¹¹ To account for the dependence of left and right images from one individual, 95% confidence intervals (CIs) for kappas were calculated using the bootstrap method with 5000 replications. ¹²

In addition to the reliability coefficients, agreement for pairs of raters was calculated. For example, in the case of four raters, six pair-wise comparisons exist. Overall percent agreement is calculated as the sum of the two numbers in the diagonal of a 2 by 2 pair-wise agreement table divided by the total number of ratings. It represents the percentage of ratings where raters agree. The positive percent agreement is defined as the percentage of positive readings that both readers agree on in pair-wise comparisons divided by all of the positive readings for both readers. Negative agreement is defined as the percentage of negative readings that both readers agree on in pair-wise comparisons divided by all of the negative readings for both readers.

RESULTS

In the RDC/TMD Validation Project, a total of 1247 participants were screened, and 734 participants were enrolled. After excluding 10 drop-outs or incomplete assessments, 724 participants (1448 joints) were assessed with panoramic, CT and MRI.

For osseous tissue diagnosis of OA based on panoramic radiographs, the inter-examiner reliability of the radiologists was poor (k = 0.16) (Table 5).¹¹ The reliability of the radiologists on diagnosing hard tissue status was fair (k = 0.47) when using PD MR images. Reliability was good when diagnosis of hard tissue status was conducted using CT images (k = 0.71), almost reaching the threshold for excellent reliability (k> 0.75).

Table 5.

Sample Characteristics and Reliability Coefficients for Radiological Diagnoses.

Diagnosis	No. of joints	Prevalence of diagnosis [%]*	Kappa	95% CI
Osteoarthritis – Panoramic radiography	179	9	0.16	0.04–0.27
Osteoarthritis – MRI	145	20	0.47	0.33–0.58
Osteoarthritis – CT	145	41	0.71	0.63–0.79
Any disc displacement	143	64	0.84	0.76–0.91
Disc displacement with reduction	143	30	0.78	0.68–0.86
Disc displacement without reduction	143	33	0.94	0.89–0.98

^*Prevalence of diagnosis is from the images used for reliability studies.

For analysis of non-osseous components using MRI, the reliability was excellent (k = 0.84) for disc displacement. Reliability for disc displacement with reduction (k = 0.78) was lower than for disc displacement without reduction (k = 0.94) although both had excellent reliability (k>0.75).

Agreement in diagnosing hard and soft tissue conditions between pairs of raters was always high; overall percent agreement was no lower than 82% for OA or disc displacement (Table 6). However, although percent negative agreement was always high (no lower than 88%) for both osseous and non-osseous conditions, percent positive agreement varied substantially among diagnoses. For diagnosing OA using panoramic radiographic images, 19% agreement was observed. The percent positive agreement increased to 59% for diagnosing OA using MRI. The diagnosis of OA reached 84% positive agreement only when CT images were assessed. Percent positive agreement for diagnosing disc displacement equaled or exceeded the percent positive agreement for diagnosing OA using CT. For disc displacement without reduction, the percent positive agreement (96%) almost equaled the percent negative agreement (98%), which indicated that raters agreed on the presence and on the absence of the condition to a similar degree.

Table 6.

Overall, Positive, and Negative Percent Agreement for Radiological Diagnoses.

Diagnosis	% agreement	% positive agreement	% negative agreement
Osteoarthritis – Panoramic radiography	88	19	93
Osteoarthritis – MRI	82	59	89
Osteoarthritis – CT	86	84	88
Any disc displacement	93	95	91
Disc displacement with reduction	91	84	94
Disc displacement without reduction	97	96	98

% positive agreement = the % of images radiologists agreed on the presence of the condition.

% negative agreement = the % of images radiologists agreed on the absence of the condition.

The reliability of the radiologists on diagnosing effusion based on T2-weighted MR images was good (k = 0.64; 95% CI: 0.39 to 0.88), and effusion was present in 53% of the observations. Overall percent agreement for effusion was 81%, and positive and negative percent agreements were similar (82% and 80%, respectively).

For assessing the criterion validity of the criteria, we analyzed the images of all the participants in the project. Using the CT diagnosis as the reference standard, the sensitivity and specificity for OA diagnoses based on panoramic radiographs and MRI were determined (Table 7). For MRI and CT, 1,448 joints were compared. On panoramic radiographs, 13 joints were nondiagnostic, therefore, 1,435 joints were evaluated against CT. The sensitivity of panoramic radiography in detecting osteoarthritis was low, while the specificity was high. The sensitivity of proton density MR images was marginal, while the specificity was high. When OA was detected on CT, 26% of the panoramic radiographs and 59% of MRI displayed positive finding of OA. When OA was not detected on CT, 99% of panoramic radiographs and 98% MRI were also negative for OA.

Table 7.

Diagnostic Accuracy of panoramic radiography and MRI for OA.

	OA of TMJ
	Sensitivity	Specificity
Panoramic radiography vs CT	26.2	99.3
95% confidence interval	21.0-31.6	98.6-99.7
MRI vs CT	59.4	98.0
95% confidence interval	53.7-64.9	97.0-98.8

For this analysis, the diagnoses of normal and indeterminate were combined as no OA, which was then compared with frank OA.

DISCUSSION

The image analysis criteria reported here have content validity as these were developed from review of the literature,^2,6-8 recommendations by the members of External Advisory Panel appointed by NIDCR for the project, and suggestions from members of the TMD and radiology community. The study demonstrated that using these criteria, the reliability of the radiologists for assessment of osseous diagnosis with CT was good, disc diagnosis with MRI was excellent, and effusion diagnosis with MRI was good. Using CT as the reference standard for diagnosing OA, panoramic radiography and MRI had poor to marginal sensitivity, respectively, but excellent specificity.

The RDC/TMD is currently used by at least 45 research groups and has been translated into 18 languages.^9,13,14 The clinical component of the RDC/TMD has also been tested in various ethnic communities.^15,16 Although the scope of the RDC/TMD is robust, its application in image interpretation was not equally useful due to lack of well-defined diagnostic criteria. With cross-sectional imaging modalities widely available, reliable criteria for image analysis are essential for both research endeavors as well as for use by TMD clinicians.

The image analysis criteria reported in this manuscript were developed to fill the void in the RDC/TMD as well as for clinical use. The image analysis criteria were established before initiating the RDC/TMD Validation Project.¹⁷ On all 724 participants, we acquired panoramic, MRI, and CT images. The interpretations of the images were done strictly according to these criteria. In this large group of research participants, the criteria were suitable and comprehensive for recording all findings related to OA, disc position as well as other joint-related conditions. After the initial reliability study, the annual reliability studies used images obtained from participants of current project. Therefore, the results of this imaging reliability study contribute to the demonstrated diagnostic reliability and accuracy of the RDC/TMD Validation Project.

Nomenclature

This study raised issues related to diagnostic nomenclature.¹ The original RDC/TMD classifies joint disorders as arthralgia, osteoarthritis, and osteoarthrosis. While arthralgia and osteoarthritis have pain as a major discerning component, osteoarthrosis is a degenerative change without any pain-related signs or symptoms. In medical literature, such differentiation is usually not noted and the terms osteoarthrosis and osteoarthritis are often used interchangeably, with osteoarthritis the more prevalent and common term.^18,19 Another term used for this condition is degenerative joint disease.^20,21 Stegenga has reviewed terminologies, such as TMD, osteoarthrosis, and osteoarthritis, and has advocated the use of osteoarthritis as the preferred term.²² For the present radiographic criteria, we have adopted the use of the term osteoarthritis (OA) although degenerative joint disease might be the best term to use for interpretation of radiographs and images when no clinical information is available.

Classification schemes for other joints

The radiographic classification of OA for any joint is challenging. Major work on OA classification has focused on knee joints. The pioneer work on classifying OA was proposed by Kellgren and Lawrence using plain-film radiography.²³ In this classification, the presence of both osteophytes and subcortical sclerosis are used to grade osteoarthritis (grades ranging 0-4). In another classification system by the American College of Rheumatology/Knee Arthroscopy Osteoarthritis Scale, osteophytes are the radiographic marker for OA.²⁴ This classification system uses criteria based on clinical findings as well as radiographic or arthroscopic observation to arrive at the diagnosis. Yet another classification system of knee osteoarthritis was proposed by Ahlback, based on radiographs of patients in an upright standing position.²⁵ This system evaluates the range of attrition from minimal narrowing of the articular space to the maximum attrition of more than 15 mm. Ahlback’s classification does not use osteophytes as an indicator of osteoarthritis. The Brandt Radiographic classification is based on that of Kellgren-Lawrence, and considers joint space narrowing, osteophytes, subcortical sclerosis, and subcortical cysts.²⁶ Using MRI as the diagnostic tool, a new classification system named Boston-Leeds Osteoarthritis Knee Score (BLOKS) has been developed.²⁷ This classification uses bone marrow lesions, osteophytes, effusion, and meniscal lesions as criteria for OA. Erosion femoral cortex is also a feature of knee OA.²⁸ For analyzing OA of the hand, interphalangeal joint erosion is a common feature.²⁹

The radiographic criteria for knee or hand arthritis cannot be directly translated to TMJ disorders. We primarily modeled our criteria with the criteria of the Kellgren-Lawrence knee OA analysis system, and incorporated published radiographic criteria for TMJ analysis.^30-32 This produced a comprehensive list of radiographic features for osseous and non-osseous tissue analysis on panoramic, CT, and MR images as well as criteria for OA.

Classification scheme for osseous components

Joint space

Joint-space narrowing is an important radiologic diagnostic feature for knee joints, which bear weight when radiographs are acquired in a standing position. Unlike knees that depend on gravity to “standardize” the load on the joint, it would be difficult to determine the “standard” loading force to use when assessing the TMJ. For the TMJ, wide anterior joint space has been correlated to anterior displacement of the disc, and presence of osteophyte.^33-35 In addition, narrowed joint space may be a feature of TMJ OA.³⁶ Because the joint space can vary during pressure, such as mastication, as well as in the open-mouth position,³⁷ the current criteria determine the joint space only when the mouth is closed in a comfortable position. Because deviation of the joint space can occur in normal joints, and with OA (due to osteophytes) or without OA (due to disc displacement only), we have not included joint space variation as a reliable indicator of OA.

Sclerosis

In the knee joints, localized subcortical sclerosis is not a reliable indicator of OA.³⁸ In patients with TMJ pain, subcortical sclerosis is present in one-third of the patients, and changes slightly with progression of disease.³⁹ For diagnosing OA, we considered subcortical sclerosis of the condylar surface or the fossa as an indeterminate, that is, as an indication of variation on normal, especially as it relates to aging, or remodeling. However, in our criteria, generalized sclerosis of the subcortical bone was considered a sign of OA, as it is associated with cartilage degradation.⁴⁰ When generalized scleroses were considered as a sign of OA, such lesions were adjacent to the articular surfaces.

Flattening of the margins

Flattening of the articulating surface of the condyle, fossa and eminence, without the evidence of osteophyte formation, is not a reliable indicator of OA.³⁸ Kurita et al. have indicated that flattened eminence is related to OA of TMJ; however, they did not indicate if such flattening was associated with erosion of the cortical margin.⁴¹ When the cortical margin was intact, but the condyle or eminence showed flattened appearance, we graded them as indeterminate. In addition, flattening and localized subcortical sclerosis were viewed as a sign of remodeling, and graded as indeterminate for OA. Remodeling is a function of age as well as duration and degree of disc displacement. Whether it will progress to frank osteoarthritis is not currently predictable.

Erosion of the cortical plate and subcortical cyst formation

As surface erosion is one of the features of OA in hand or knee joints,^28,42 and for TMJ.^43-47 The current criteria uses erosion as an important feature of OA. In advanced erosion, several joints also displayed subcortical cyst formation, which we considered another feature of OA.⁴³ Subcortical cysts is a misnomer since it is not a true cyst but rather an area of osseous degeneration.

Osteophytes

Most knee joint classification use osteophyte as a commonly used radiographic feature for OA. Osteophytes indicate cartilage degradation,³⁸ and are associated with pain in the joint.⁴⁸ An osteophyte, even if small, is an indicator of progression of OA.⁴⁹ Several atlases were developed to orient the clinicians in identifying osteophytes in the knee, hip, or hand.^50-52 In the RDC/TMD, all three imaging modalities were used to determine the presence of osteophytes. Figure 2 provides examples of osteophytes as viewed on sagittal CT views. On panoramic radiography, we can see only limited areas of the anterior surface of the condyles.⁴⁷ Most osteophytes are located on the anterior surface,⁵³ and are not adequately displayed on panoramic radiographs unless large or located on the antero-lateral aspect of the joint. This limitation partially explains the low sensitivity of panoramic radiographs in detecting OA. The moderate sensitivity of the MRI is likely due to the difficulty in detecting small osteophytes or small erosive changes in the cortical plates.

Calcified loose bodies in the soft tissues

We treated presence of loose calcified bodies as a sign of OA when other features of OA were also present. Without other signs, presence of loose calcified bodies was not as a sign of OA, as these can also represent synovial chondromatosis which are calcified cartilage embedded in synovial tissues,⁵⁴ or chondrocalcinosis. Although loose calcified bodies are more common in large joints, these produce similar clinical symptoms in TMJ (e.g., pain, limitation of movement, crepitation, and inflammation).⁵⁵

In summary, the imaging analysis criteria use erosion, subcortical cyst, osteophyte, and generalized sclerosis to diagnose TMJ OA. Flattening and/or sclerosis, unless the latter is generalized, are considered indeterminate signs for diagnosing OA. As such, the criteria do not over-diagnose frank OA. We believe that the review of the radiological literature of the TMJ and other joints justifies the items in our criteria for diagnosing OA.

Classification scheme for non- osseous components

The non-osseous criteria included shape and position of the disc in both open and closed mouth and the presence of joint effusion. The non-osseous components were evaluated in proton density and T2 weighted MR images only.

Disk shape and position

We adopted the classification by Orsini et al. for identifying the location of the posterior band of the disc and location of the intermediate zone both in closed and open mouth.⁸ Tasaki et al. classified the disc position into 9 categories, with an additional category of indeterminate.⁶ Although our criteria considered and evaluated all of these 10 possible positions, the results were classified into only five types (Table 4). In general, increasing the diagnostic options often results in reduced reliability. Therefore the proposed RDC/TMD imaging criteria for disc position are simple, have excellent reliability and still include all possible positions/shapes of the disc in relation to the osseous components as used in the current literature.

Effusion

The relationship of fluid effusion to pain and OA is not yet clear.^56-60 It is now understood that effusion occurs with disc displacement,^61,62 and is a sign that may appear before osteoarthritic changes occur.⁶³ Therefore, we have not included effusion as a criterion for identifying OA. However, in a knee-joint study, moderate or frank effusion and osteophytes reliably correlated with symptoms of OA.⁴⁸ In our grading, effusion was rated as absent, slight, or frank in its presence. Both the upper and lower joint spaces were evaluated to arrive at a diagnosis of effusion.

Translation of the condyles

In our study, we used open-mouth MRI to view translation of the condyles. The same images were also used to identify disc position or shape.⁶⁴ As CT does not reveal disc shape or position, and to minimize radiation exposure, we did not acquire open-mouth CTs since condylar translation is adequately assessed with open mouth MRI. From MRI, we graded the translation of the condyle in relation to the apex of articular eminence in one of three ways: the condyle translated (1) less than the apex, (2) to the apex, or (3) beyond the apex. Takatsuka et al. have classified the range of motion as positive, limited, or negative,⁶⁵ which appears to be subjective for research or clinical use. These authors also report that the presence of OA does not always limit the translation.

Consistent terminology and proper diagnosis are essential in clinical TMD practice.¹ To eliminate or minimize the bias of the radiologists, the research diagnostic criteria described in this report can reliably be used to interpret panoramic, CT and MR images for osteoarthritis and MR images for disc displacement. The criteria specifically avoid charting the range or extent of disorders. Although range designations such as mild, moderate, and severe are useful for clinical description, these are often subjective, difficult to standardize and reduce reliability. Likewise, we have avoided terminologies such as acute or chronic. To overcome the limitations of categorizing, the scoring form we used provided options to indicate conditions that do not fall within the diagnostic scheme for OA. In addition, for each image type, an area on the scoring form was available for the radiologist to comment on the images or findings.

In the current study, our results show that comprehensive image analysis criteria developed for the RDC/TMD Validation Project can effectively be employed in assessing TMD, as the study radiologists achieved good reliability in detecting OA using CT, excellent reliability in identifying disc position, and fair to good reliability in detecting fluid effusion.

Reliability for interpreting radiographs and images

In the current study, the scoring options for panoramic radiography were normal, indeterminate, or frank OA. The reliability of the radiologists was poor in interpreting panoramic radiographs for OA. When the scoring of normal and indeterminate were grouped together and compared against frank OA, the agreement improved to a moderate level. A position paper by the American Academy of Oral and Maxillofacial Radiology (AAOMR) indicated that panoramic radiographs may be useful to detect gross TMD pathoses only.⁷ Earlier studies have also indicated that inter-examiner reliability of detecting TMJ pathosis from panoramic radiographs was low to moderate.^66-68 In addition to the inability of the panoramic radiographs to reveal osteophytes, erosion of the articular margin of the condyle is not properly revealed due to superimposition by other bony structures. The inherent limitation of panoramic radiographs in demonstrating the contours of the condyle and the articular fossa is the likely reason for the poor agreement in interpreting such images.

When MR images are used for diagnosing OA, the reliability of the radiologists was fair (k= 0.47). In knee-joint MR studies, radiologists have been found to have similar moderate reliability in diagnosing OA (for osteophytes, k = 0.65).²⁷ For the current reliability study, the radiologists examined only one representative image from open- and closed-mouth sagittal views. It is possible that the reliability would change if all the sections through the condylar head were made available to the radiologists. In another study, high agreement (94%) was achieved in detecting osseous changes on MRI when two examiners were calibrated for seven months using several hundred images.⁶⁹ A recent study for TMD reported that higher reliability in clinical diagnosis can be achieved by recalibrating the examiners during the course of the study.⁷⁰ In the present project, we recalibrated the radiologists on an annual basis.

Our results indicate that the reliability of the radiologists was excellent (k = 0.84) when they diagnosed the disc position using PD MR images. Previous studies reported moderate to good inter-examiner agreement on disc position.^71-75 One study indicated that the inter-examiner reliability can be improved to the level of ‘good’ by selecting of high quality MR images, and by calibrating the examiners.⁷⁵

A systematic review of the literature on the efficacy of MRI on diagnosing TMJ disorders showed that the results from the literature are inconsistent and the evidence is insufficient.⁷⁶ The authors of this review indicate that the poor performance of the examiners is partly due to different diagnostic procedures employed by the investigators and lack of defined diagnostic criteria. Another study has shown that when the examiners were not calibrated, the kappa values for inter-examiner agreement on TMJ disc position or configuration were poor.⁷⁷ These issues were addressed in our study, as the radiologists were calibrated and used clearly defined image analysis criteria. In detecting effusion, a study reported poor reliability (k = 0.36) for noncalibrated examiners;⁷⁷ however, the calibrated examiners in our study had good reliability (k =0.64). Our data suggest that the image analysis criteria developed for the RDC/TMD Validation Project and calibrated examiners led to more reliable interpretation than other reports.

The superiority of CT over panoramic radiography or MRI in displaying the features of TMJ OA has been well documented and has wide acceptance.^47,78,79 Using multi-detector CT images in our study, the reliability of the radiologists was good (k = 0.71) and the agreement of the radiologists in diagnosing OA was high (86%). A systematic review indicated that cone beam CT (CBCT) maybe superior to multi-detector CT, although the reviewers suggested that further studies were needed to determine the usefulness of CBCT over than multi-detector CT.⁷⁹ When the current project was proposed and conducted, the availability of CBCT was limited and was not accessible to the project radiologists. Previously, several investigators had used tomography and arthrotomography as a method for cross-sectional imaging to assess osseous changes in the TMJ. These studies indicated that the inter-examiner reliability using this technique was fair (k-0.56)⁸⁰ to excellent(k=0.40-0.80).^81,82

Criterion Validity

In our validation assessment study, we considered the CT findings as the reference standard for diagnosing OA using images.^78,83 The sensitivity of panoramic radiography in detecting OA was low (Table 7). Our results conform to the recommendation of the position paper from AAOMR that panoramic radiography is only useful in diagnosing advanced OA.⁷ In identifying OA, the sensitivity of MR images was marginal, when compared to CT findings. We used 1.5 Tesla (T) magnets in acquiring our MR images. Stehling et al. have reported that visualization of the disc is similar at 1.5 and 3.0 T, while the anatomic details were substantially better at 3.0 T.⁸⁴ TMJ surface coils were used in all three research sites for achieving high-quality images. Although the sensitivity of panoramic radiography and MR ranged from low to marginal, the specificity of these techniques in detecting OA was high when compared to reference standard CT (Table 7).

The results indicate that about 75% of CT-diagnosed OA is not detected using panoramic radiography, and about 40% with MRI. Therefore, this suggests that clinical or research studies of OA should use CT when possible despite the increased radiation. As in this study, radiation exposure can be decreased with limiting CT study to the closed mouth position. Another option may be to use CBCT, which can provide diagnostic information equal to MDCT.⁷⁹ CBCT has been shown as a better diagnostic option for TMJ erosion compared to panoramic radiography or linear tomography.⁴⁷ The image resolution of different brands of CBCT differs; therefore it remains to be determined if images generated from different brands of CBCT are indeed more diagnostic than those generated from MDCT. Finally, future research should attempt to improve the resolution and definition of MR images so it can be used instead of CT for assessing osseous structures.

There were several limitations to this study. The initial calibration and reliability study was done at one site, with all four radiologists in the same location. Subsequent recalibration and reliability studies were done separately, which reduced the discussions between the radiologists. This may have led to a lower reliability than if the radiologists were together for the annual studies. The three geographical locations were chosen to assemble an experienced team of investigators and to increase the number of participants in the study. However, the distance created logistical problems. Conference calls are a substitute for physical presence of the radiologists, but a better option is to conduct recalibration in one site. In addition, for logistic purposes a single representative slice of CT or MRI from a joint was selected for the reliability examination in the study. Evaluation of multiple slices would have reflected clinical situations for total assessment of the joint. The four-year length of the study added an unexpected variable. The hospital at the UM site replaced its MRI unit during the course of this study; therefore, the images obtained at the beginning of the study were different than the images obtained later. The three study sites used different brands of panoramic, CT, and MR units. Although the images were completely diagnostic, there were differences in the image quality of each unit, which may have affected reliability outcomes.

In conclusion, panoramic radiography had poor reliability and low sensitivity, compared to CT, for detecting TMJ related osseous changes. These findings suggest that this imaging modality has limited utility for assessing the TMJ. MRI has fair reliability and marginal sensitivity in diagnosing osseous changes when compared to CT. Therefore MRI is not an ideal imaging technique for detecting osseous changes, thus CT remains the image of choice for assessing osseous tissues. Relative to soft tissue assessment, MRI has excellent reliability for assessing disc position and good reliability for detecting effusions. Overall, the criteria proposed in this study for image analysis covered all possible osseous and non-osseous conditions of TMJ in a large group of participants in a multi-site RDC/TMD Validation Project.¹⁷ The image analysis criteria presented in this paper are reliable for diagnosing osseous and non-osseous components of TMJ using CT and MRI, respectively. We recommend that they be used in both clinical and research settings.

Figure 5.

Sagittal proton density MR views representing examples of changes in the soft tissue components of the TMJ observed, and corresponding disc diagnoses. Images on the upper panel are in closed-mouth position. Images in the lower panel are in open mouth position. A. Anteriorly displaced disc: thickened disc. B. Anteriorly displaced disc: deformed disc shape, posterior band is thickened, generalized sclerosis of the condylar head. C-D. Anteriorly displaced disc: deformed disc shape. E. Reduction of the disc position (while the mouth is open): normal disc shape, intermediate zone is in contact with the condylar head. F. Reduction of the disc position (while the mouth is open): posterior band is thickened, intermediate zone is in contact with the condylar head. G. Non-reduction of the disc position (while the mouth is open): intermediate zone is not detectable. H. Non-reduction of the disc position (while the mouth is open): intermediate zone is detectable.

Figure 6.

Sagittal T2 MR views representing examples of changes in the soft tissue components of the TMJ observed, and presence of effusion. A-B. Anteriorly displaced disc, and effusion in the superior joint space. C-D. Anteriorly displaced disc, and effusion in both the joint spaces.