Changes in inter observer variation of Schatzker and AO/OTA classification of tibial plateau fractures on addition of CT scan

Abstract

Objective: Schatzker and AO/OTA classification are commonly used to classify tibial plateau fractures, but they are defined using plain radiographs and may not capture fully the complexity of these fractures. CT scan offers better visualization of occult fractures, joint depression and overall fracture morphology, but its impact on these classifications in term of interobserver variation is unclear. There is paucity of literature on this aspect, hence this study. Methods: A total of 38 cases of tibial plateau fractures were classified by five different observers (four senior residents and one consultant) on the basis of Schatzker and AO/OTA classification. Initially, the observers classified the cases using plain radiographs and then reclassified the cases after supplementing the radiographs with CT scan images. The interobserver reliability was calculated using kappa coefficient. Results: The interobserver agreement for Schatzker classification was found to be moderate on plain radiographs (mean κ_X-ray = 0.593) and substantial after addition of CT scan images (mean κ_{(X-ray + CT scan)} = 0.630). The interobserver agreement for AO/OTA classification was found to be fair on plain radiographs as well as after addition of CT scan images (mean κ_X-ray = 0.313 and mean κ_{(X-ray + CT scan)} = 0.320). Conclusion: After providing advanced imaging both the classification systems showed improvement in the interobserver reliability. However, the change was found to be non-significant. This highlights the weakness of the plain radiograph based classification systems and indicates adoption of classifications based on advanced imaging.

Introduction

A classification system for tibial plateau fractures is necessary to identify the fracture pattern, predict the prognosis and aid in planning and evaluating surgical outcomes. According to the available literature there are as many as 38 classification systems described and the number is increasing [1]. However, only a few of them are used in routine clinical practice. There is no universally accepted classification system for tibial plateau fractures. A good classification system should have less intraobserver and interobserver variability and should have high reproducibility and validity [1].

Schatzker Classification and AO/OTA (Arbeitsgemeinschaft für Osteosynthesefragen/Orthopaedic Trauma Association) classification systems are the most commonly employed classification systems for tibial plateau fractures [2,3]. Both these classification systems were originally described for evaluation of fracture pattern on plain radiographs [4,5].

The Schatzker classification was developed in 1979. The advantage of this classification is that it is simple and easy to use in clinical setting. As it is developed using plain X-rays thus it is accessible in low resource settings [5,7]. However this system does not describe comminution, displacement or posterior column involvement well [7].

AO/OTA system offers a more detailed and hierarchical approach, considering fracture morphology, location, and extent. It includes both articular and metaphyseal components and better accommodates complex fractures. The disadvantages are that this classification system is complex and difficult to apply in a busy clinical setting and it requires lot of training and familiarity for consistent application [4,7].

The advanced imaging techniques, particularly CT (Computerised Tomography) scans are increasingly being utilized for management of tibial plateau fractures [6,7]. Various studies have demonstrated the superiority of CT scans in the formulation of surgical treatment plan for these fractures [6,8-10]. CT scans are better at visualizing occult fracture lines, depression and overall fracture morphology [3,9,11,12].

Fracture pattern classification has been a persistent challenge due to the inherent difficulties associated with the complex nature of tibial plateau fractures for which several groups have discussed an additional trial application of CT scans to improve judgement [13].

The Interobserver variation of Schatzker and AO/OTA classification systems are well studied on plain radiographs [3,6,13-17]. The inter observer reliability for Schatzker classification on plain radiographs ranges from fair to substantial with majority of the studies reporting it to be moderate [2,3,6,8,13-19]. The inter observer agreement for AO/OTA classification on plain radiographs ranges from fair to moderate with majority of the studies reporting it to be moderate [2,8,13-16,19].

There is a lack of consensus in the limited literature available on the interobserver variation of these classification systems on addition of CT scan. Hence, the current study was envisaged to assess the interobserver variation of Schatzker and AO-OTA classification and evaluate the impact of addition of CT scan in their interobserver variation.

Materials and methods

This prospective observational study was carried out in the department of Orthopedics and Radiology in a tertiary care center of a developing nation. Institutional Ethical Clearance was obtained.

Patient selection

A total of thirty eight patients of either gender between the age group of 21-65 years presenting to a tertiary care centre with tibial plateau fractures of less than 3 weeks duration were enrolled for this study after careful screening for the mentioned eligibility criteria (Table 1). The informed consent of each patient was obtained. The demographic data of each patient was recorded in a pre-designed case record form.

Table 1.

Eligibility criteria for case selection

Inclusion Criteria	Exclusion Criteria
Consenting adults more than 18 years of age of any gender.	Patients with pathological fractures.
Patients with closed proximal tibia fractures less than 3 weeks old.	Patients with previous surgery around Proximal Tibia.
	Patients with incomplete radiological workup.
	Contraindications to X rays or CT scans. Example pregnancy.

Radiology

Standard radiological workup including the AP (antero-posterior)/lateral views of knee with leg on plain radiography. Exposure settings with kVp (kilovoltage peak) 60-70 and mAs (miliampere-second) 7-10 was used. Non-contrast Computerised Tomography scan of the involved side was done with a 64 slice 32 channel CT scanner (Seimens Somatom Definition AS) with a slice thickness of 1 mm (millimetre) on with 3D (three dimensional) reconstruction at 1 mm interval was done for each patient (Figure 1).

Figure 1.

This image is showing AP and lateral plain radiographs of a knee joint showing tibial plateau fracture on right side.

Observing the fracture patterns

In order to minimise observer bias five Orthopaedic surgeons with similar level of experience were chosen to observe and classify the fracture patterns according to the Schatzker and AO/OTA classification. Each observer of this study was provided with printouts of appropriate descriptive details of Schatzker and AO/OTA classification. They were allowed to refer to the classifications anytime during the evaluation without any time constraints. To minimise bias the observers were blinded for the demographic data of the patients, radiology reports and observations by fellow observers of the study.

At first, the observers classified the tibial plateau fractures according to Schatzker and AO/OTA classification using the AP and lateral views on plain radiographs alone (Figure 1). Then the observers were made to reclassify the fractures after supplementing the radiographs with CT scan images of the same patient with axial, sagittal and coronal cuts & 3D reconstructed image (Figures 2, 3). All responses were recorded in a Microsoft Excel Sheet.

Figure 2.

The image is showing plain CT scan with Coronal, Axial and Saggital cuts of the knee joint showing the tibial plateau fracture.

Figure 3.

The image is showing 3D reconstruction of a knee joint showing a tibial plateau fracture.

Statistical analysis

The data obtained was converted into a computer based spreadsheet and analysed. The continuous variables were analysed for mean and standard deviation. The categorical variables were analysed for proportions. The interobserver variation was calculated using the kappa coefficient. The values were interpreted according to Landis and Koch’s criteria (Table 2) [20]. A p-value of < 0.05 was considered as statistically significant.

Table 2.

Kappa values as described by Landis and Koch [20]

0-0.2	Poor agreement
0.21-0.4	Fair agreement
0.41-0.6	Moderate agreement
0.61-0.8	Substantial agreement
0.81-1	Almost perfect agreement

Results

Agreement on observations

The mean kappa value for interobserver variation of the Schatzker classification using plain radiographs was 0.593, indicating moderate agreement according to Landis and Koch’s interpretation of kappa. With the addition of CT scan images, the kappa value increased to 0.630, reflecting substantial agreement. The Increase is a kappa value of 0.037 in observer agreement, but the change was not statistically significant, with a p-value of 0.285 (Table 3).

Table 3.

Comparison of inter-observer variation of Schatzker classification on plain radiographs alone and in combination with CT scan images

	kappa value for Schatzker classification on plain radiographs alone	p-value	kappa value for Schatzker classification on plain radiographs + CT scan	p-value
O 1 & 2	0.664	.0001	0.786	.0001
O 1 & 3	0.629	.0001	0.653	.0001
O 1 & 4	0.638	.0001	0.683	.0001
O 1 & 5	0.610	.0001	0.625	.0001
O 2 & 3	0.492	.0001	0.618	.0001
O 2 & 4	0.539	.0001	0.610	.0001
O 2 & 5	0.542	.0001	0.661	.0001
O 3 & 4	0.703	.0001	0.655	.0001
O 3 & 5	0.537	.0001	0.460	.0001
O 4 & 5	0.580	.0001	0.557	.0001
Mean kappa	0.593		0.630
Change in mean kappa	0.037
p-value for change in kappa	0.285

O 1 to 5 = Observers.

Mean kappa value

The mean kappa value for interobserver variation of the AO/OTA classification using plain radiographs was 0.313, with the addition of CT scans, the kappa value increased to 0.320, reflecting fair agreement in both scenarios. The increase in kappa value of 0.007 was not statistically significant, with a p-value of 0.890 (Table 4).

Table 4.

Comparison of inter-observer variation of AO/OTA classfication on plain radiographs alone and in combination with CT scan images

	kappa value for AO/OTA classification on plain radiographs	p-value	kappa value for AO/OTA classification on plain radiographs + CT scan	p-value
Observer 1 & 2	0.460	.0001	0.559	.0001
Observer 1 & 3	0.294	.0001	0.305	.0001
Observer 1 & 4	0.355	.0001	0.449	.0001
Observer 1 & 5	0.378	.0001	0.422	.0001
Observer 2 & 3	0.262	.0001	0.188	.0001
Observer 2 & 4	0.289	.0001	0.389	.0001
Observer 2 & 5	0.324	.0001	0.287	.0001
Observer 3 & 4	0.251	.0001	0.270	.0001
Observer 3 & 5	0.182	.0001	0.123	.016
Observer 4 & 5	0.337	.0001	0.208	.0001
Mean kappa	0.313		0.320
Change in mean kappa	0.007
p-value for change in kappa	0.890

Discussion

In the current study the mean kappa value for Schatzker classification on plain radiographs was found to be 0.593 [20]. The mean kappa value for Schatzker classification on combined plain radiographs with CT scan images was found to be 0.630 [20]. We observed a net change of 0.037 in mean kappa value after the addition of CT scan images. This change was small compared to the categorical increment of 0.2 in the Landis and Koch’s interpretation of kappa [20]. Even though the kappa values obtained in this study changed the level of agreement from moderate on plain radiographs alone to substantial after the addition of CT scan images for Schatzker classification, the change was not found to be statistically significant (p-value > 0.05).

The mean kappa value of inter-observer variation of AO/OTA classification on plain radiographs was found to be 0.313 which falls in the category of fair agreement. The mean kappa value of inter-observer variation of AO/OTA classification on combined plain radiographs and CT scan images was found to be 0.320 also falling in the category of fair agreement [20]. There was a net change of 0.007 in the mean kappa value which was also not found to be statistically significant (p-value > 0.05).

In contrast to our study, Brunner et al [8] (2010) evaluated the impact of CT scan on intra and inter observer reliability of Schatzker, AO and Hohl and Moore classification systems. In this study four independent observers analysed 45 consecutive intra-articular tibial plateau fractures. The interobserver reliability of the Schatzker classification on plain radiographs improved from moderate with a mean kappa of 0.418 to good with a mean kappa of 0.755 on supplementing the data with 2D CT scans. The interobserver reliability of the AO/OTA classification also improved from moderate with a kappa of 0.429 on plain radiographs to good with a mean kappa of 0.728 on 2D CT scans. In this study, there was no comment on the statistical significance of the difference of the mean kappa values. In comparison to our study this study had a bigger sample size however with lesser observers [8].

A study by Doornberg et al (2011) evaluated the impact of 3D reconstructed CT images (slice thickness < 2 mm) over plain radiographs + 2D CT images. Six different observers analysed 45 complex tibial plateau fractures. The results were analysed on two different occasions 2 weeks apart. In the first round plain radiographs and 2D CT scan images were used. In the second round 3D reconstructed images were added. The observers found six different characteristics of complex tibial plateau fractures and also classified the fractures on the basis of Schatzker, AO/OTA and Hohl and Moore classification systems. The kappa value for the interobserver reliability of the Schatzker classification slightly improved from 0.545 on 2D CT scan and plain radiographs to 0.596 with addition of 3D CT scan. Similarly, kappa value for the interobserver reliability of the AO/OTA classification improved from 0.536 on 2D CT scan and plain radiographs to 0.545 with addition of 3D CT scan. However, the results were not found to be statistically significant. This suggested limited role of 3D CT after 2D CT for classification of tibial plateau fractures [9].

Mellema et al studied the reliability of the Schatzker and Luo classification among a large number of observers on two dimensional CT images (2D CT) and also studied the effect of addition of 3 dimensional CT images (slice thickness < 1.25 mm). The study recruited 81 observers and they were randomised for the analysis of 15 complex tibial plateau fractures to either 2D-CT or 2D- and 3D-CT. The study used the Siegel and Castellan multirater kappa measure. The interobserver agreement of the Schatzker classification on 2D-CT scan was fair with a mean kappa of 0.37. On supplementing 3D data the level of agreement between different observers declined with a mean kappa of 0.29. This study did not include AO/OTA classification [21].

In a recent 2022 retrospective cohort study conducted by Yao et al 90 2D and 3D CT scans were subjected to analysis by six different observers. The observers classified the fractures according to 4 different classification systems; Schatzker, AO/OTA, updated three column concept and ten segment classification. The inter observer reliability of the Schatzker classification on 2D CT-scan was found to be substantial with a mean kappa of 0.64 with a slight improvement on 3D CT-scan 0.66. The interobserver reliability of the AO/OTA classification on 2D CT-scans was found to be moderate with a mean kappa of 0.54 there was an improvement on 3D CT-scans with a mean kappa of 0.59 [22].

Ahmad et al studied 35 cases of tibial plateau fractures were assessed by 5 different observers who classified them according to Schatzker and Four Quadrant classification. The study found the level of agreement between the observers for Schatzker classification on plain radiographs alone to be Substantial with a mean kappa of 0.61 which remained in the substantial category with a slightly improved mean kappa of 0.64 when CT scan images were added. This study did not comment on the statistical significance of the difference in the kappa values after addition of CT scan images [17].

There were only a few studies which evaluated the impact of addition of CT scans on the interobserver variation of Schatzker and AO/OTA classification systems [24]. There was no consensus amongst the various studies as the results were variable. Studies by Brunner et al [8], Gicquel et al [2], Taskesan et al [19], Ahmad et al [17] showed a positive impact of CT scan on interobserver variation. Study by Masouros et al [16] showed a neutral impact of CT scan on interobserver variation. A study by Chan et al [6] demonstrated a negative impact of CT scan on interobserver variation. Mellema et al [21] showed a negative impact of 3D CT scans over 2D CT scans on interobserver variation of Schatzker classification whereas Hu et al [10] showed a positive impact of 3D CT scan in its study. These differences could have been due to the different methodologies used in different studies and varying levels of experience of the observers. A direct comparison of kappa values for Schatzker and AO/OTA classification according to the various studies employing plain radiographs and CT scans as imaging modalities is tabulated (Table 5).

Table 5.

Comparison of interobserver agreement of schatzker and AO/OTA classification across various studies

Studies	No. of Subjects	No. of Observers	Inter-observer Agreement and Kappa for Schatzker classification on X-rays	Inter-Observer agreement and kappa for Schatzker classification on CT scan	Inter-Observer agreement and kappa for AO/OTA classification on X-rays	Inter-Observer agreement and kappa for AO/OTA classification on CT scan
Chan et al [6] (1997)	21	6	Substantial (0.62)	Substantial (0.61)
Walton et al [13] (2003)	30	3	Fair (0.38)		Moderate (0.41)
Charalambous et al [14] (2007)	50	6	Moderate (0.41)		Moderate (0.43)
Maripuri et al [15] (2008)	50	4	Moderate (0.47)		Fair (0.36)
Hu et al [10] (2009)	21	4		2D CT + X-ray - Substantial (0.74)		2D CT + X-ray - Substantial (0.71)
				3D CT - Almost perfect (0.85)		3D CT - Almost perfect (0.83)
Brunner et al [8] (2010)	45	4	Moderate (0.418)	Significant (0.755)	Moderate (0.429)	Significant (0.729)
Te Stroet et al [18] (2011)	15	8	Moderate (0.47)	Moderate (0.46)
Doornberg et al [9] (2011)	45	6		2D CT + X-ray - Moderate (0.545)		2D CT + X-ray - Moderate (0.536)
				3D CT - Moderate (0.596)		3D CT - Moderate (0.545)
Gicquel et al [2] (2013)	50	6	Moderate (0.404)	Moderate (0.476)	Fair (0.357)	Moderate (0.479)
Mellema et al [21] (2016)	15	81		2D CT - Fair (0.37)
				3D CT - Fair (0.29)
Taskesan et al [19] (2017)	60	4	Moderate (0.51)	Significant (0.61)	Moderate (0.43)	Moderate (0.54)
Anwar et al [23] (2019)	44	4		X-ray + CT scan - Substantial (0.723)
Gupta et al [3] (2022)	53	5	Moderate (0.41)
Yao et al [22] (2022)	90	6		2D CT - Substantial (0.64)		2D - CT Moderate (0.54)
				3D-CT Substantial (0.66)		3D CT Moderate (0.59)
Masourous et al [16] (2022)	25	12	Fair (0.361)	Fair (0.364)	Fair (0.204)	Fair (0.231)
Ahmad et al [17]	35	5	Substantial (0.61)	Substantial (0.64)
Current study	38	5	Moderate (0.593)	Substantial (0.630)	Fair (0.313)	Fair (0.320)

The strength of this study lies in its prospective study design. However, there is a limitation of relatively small sample size of 38 patients.

Conclusion

In the current study, the interobserver agreement on Schatzker classification is fair on plain radiographs and moderate after addition of 3D-CT. The interobserver agreement of AO/OTA classification is moderate on both plain radiographs and after addition of 3D-CT. This study fails to demonstrate significant superiority of CT scans over plain radiographs for the classification of tibial plateau fractures using either the Schatzker or AO/OTA classification systems. We attribute the inability to achieve better interobserver reliability with advanced imaging to the inherent weaknesses of the classification systems, which were originally designed for use with plain radiographs only. CT scans remain useful for surgical planning and fragment-specific fixation, which is beyond the scope of the current study.

Disclosure of conflict of interest

None.