Comparison of PACS and Hard-copy 51-inch Radiographs for Measuring Leg Length and Deformity

Abstract

Background

The picture archiving and communication system (PACS) eventually will replace the use of standard hard-copy radiographs. It is unknown whether measurements of limb length discrepancy (LLD) and deformity on PACS compare in accuracy and reproducibility with those from hard-copy radiographs.

Questions/purposes

We compared the reproducibility and reliability of LLD and deformity measurements for each of these two media.

Methods

We retrospectively reviewed 51-inch standing lower extremity images obtained for LLD or deformity analysis from 40 patients to compare the measurements and their reliability on hard-copy film with those performed on soft-copy PACS. Two observers independently performed measurements twice using each system at 1-week intervals to minimize interobserver or intraobserver bias. Intraclass correlation coefficients (ICCs) were determined to test intraobserver and interrater reliability of Rater 1 and Rater 2.

Results

Interrater reliability of measurements made on hard copy ranged from 0.69 to 0.99 and PACS-derived measurements ranged from 0.66 to 0.98. Intraobserver reliability for Rater 1 for measurements made on hard copy ranged from 0.853 to 0.999 and PACS-derived measurements ranged from 0.80 to 0.996. Intraobserver reliability for Rater 2 for measurements made on hard copy ranged from 0.931 to 0.999 and PACS-derived measurements ranged from 0.962 to 0.999.

Conclusions

Each system yielded comparable reliability for measurements, therefore, transition to PACS to perform measurements in patients with LLD or deformity can be made with confidence.

Level of Evidence

Level III, diagnostic study. See the Guidelines for Authors for a complete description of levels of evidence.

Introduction

Conventional radiographs are being transitioned in many office and hospital practices to digital imaging. Hard-copy conventional radiographs are reliable for diagnosis and operative planning; however, digital imaging acquisition, computed radiography (CR), and digital radiography (DR) along with soft-copy PACS are becoming predominant methods through which radiographic images are being analyzed for diagnosis and measurements.

CR uses a plate instead of a cassette and film similar to the card used in some digital cameras and equipment. The plate, as with the card, must be processed to be read. DR transmits the image directly via cable or wireless to the image display monitor. In our practice, we routinely use the 51-inch standing radiographs of lower extremities for measuring LLD and deformity [5] for the initial evaluation and followup [9].

As limb lengthening practices transition to digital acquisition and PACS, differences between manual measurements performed on hard-copy radiographs and digital measurements performed on soft-copy images must be validated. Several studies have compared the reliability, accuracy, and diagnostic performance of digital imaging versus hard-copy radiographs for fracture interpretation [1], interpretation of sonograms [2], hand radiographs [4], chest, brain, and abdominal CT scans [6, 13], MRI for meniscal tears [7], chest radiographs [13], and comparison of standing full-length radiographs versus scanograms for patients with LLD [9]. These studies found no differences for either system. However, none of these studies assessed the reliability of measurements for LLD and limb deformities using CR and DR-acquired images and soft-copy PACS.

The purpose of this investigation was to establish the intraobserver and interobserver reliability of digitally acquired images displayed on PACS as soft-copy images as compared to digitally acquired images, printed as hard-copy radiographs in the same patient cohort.

Patients and Methods

We retrospectively reviewed the images of the 40 most recent patients in our clinic with either a LLD or lower limb deformity. All patients were adults ranging from 30 to 60 years old. Measurements were collected on a routine clinical basis during preoperative and postoperative examinations. This is a retrospective study approved by the hospital Institutional Review Board.

All images were acquired digitally, stitched, 51-inch AP bilateral standing hip-to-ankle images. We placed a calibration ball on all images in the midline of the AP dimension. Blocks were used to level the pelvis for patients with LLD. Block size is selected based on the clinical examination, taking into consideration spinal curvatures and iliac crest level. Radiographs confirm our findings during clinical examination. If a block size selected does not level the pelvis, the total leg length difference is calculated by adding the height of the block used and the remaining leg length difference to level the pelvis radiographically. The x-ray beam was centered at the level of the knees at a 10-foot distance. We printed the hard-copy images using a fixed adjustment for magnification to be equivalent to that of traditional analog film. Two observers (SK, Rater 1, and DB, Rater 2) independently measured each patient’s images on the hard-copy film and the soft copy using PACS (Philips Easy Vision Healthcare, Bothell, WA, USA) monitor display. The PCR Eleva S Plus was used for CR, and the Digital Diagnose (DIDI) Version 1.4 with a resolution of 1600 × 1200 was used for DR.

Each observer measured hard-copy and soft-copy images for all 40 patients. A pencil, ruler, and goniometer were used for manual hard-copy image measurement and performed by displaying the long x-ray film on a long light box with landmarks marked directly on the radiograph. Using a circle center guide, the center of the femoral head was located. The center of the knee joint and the center of the ankle relative to the talus was determined (Fig. 1). The Ortho Toolbox (SECTRA NA, Shelton, CT, USA) and autocalibration were used for soft-copy image length and angular measurements on the PACS monitor. Autocalibration is a computer software calculation that standardizes the image to an object of known dimension. When using the Ortho Toolbox, magnification is calibrated to a known standard. The 25-mm calibration marker was consistently placed during each image acquisition. The measurement scale on the soft-copy images is based on the calibration tool. The images were zoomed in (magnified) to optimize positioning consistency of the measuring tool points of reference. To avoid intraobserver bias, two observers independently performed measurements twice using each system at two separate times (with at least a 1-week interval). Measurements were repeated four times by each observer. For example, at Week 1, hard-copy radiographs were measured by Rater 1. At Week 2, digital radiographs were measured by the same rater. These measurements then were repeated in Weeks 3 and 4 by Rater 1. A similar protocol was followed by Rater 2.

Fig. 1.

A 51-inch radiograph shows the measurements for the lateral distal femoral angle (LDFA) and medial proximal tibial angle (MPTA) used for patients with LLD or deformity.

Measurements consisted of bilateral total limb length (hip to ankle), femur length, tibia length, lateral distal femoral angle (LDFA), and medial proximal tibial angle (MPTA) [5]. The femur length difference (Δ femur), tibial length difference (Δ tibia), and LLD also were calculated, and the most precise and least precise measurement in both systems were identified.

Joint orientation angles were measured in reference to the mechanical axis of the femur and the tibia. The joint line was drawn along the articular surface of the distal femoral condyles. The LDFA is the angle formed laterally between the femoral mechanical axis (center of the hip to center of the knee) and the knee line. The MPTA (medial proximal tibial angle) is the angle formed medially between the tibial axis (from the center of the knee to the center of the talus) and the knee line (Fig. 2).

Fig. 2.

The radiograph shows the lateral distal femoral angle (LDFA) and medial proximal tibial angle (MPTA) measurements for patients with genu varum deformity.

The length of the femur and the tibia was measured along the mechanical axis lines. The LLD was the difference between the mechanical axis length of the right and left lower extremities. The Δ femur and Δ tibia were the differences between the mechanical axis lines of the femur and tibia, respectively, between the right and left sides (Fig. 3). The measurements shown in Figs. 1 and 2 are standard measurements validated by Paley [5].

Fig. 3A–B.

The radiographs show measurements made on 51-inch long films. (A) This radiograph shows full leg lengths. Leg lengths are measured from the top of the femoral head to the center of the talus and ankle. (B) This radiograph shows segmental leg lengths. Femur length is measured from the top of the femoral head to the center of the knee and tibia length is measured from the center of the knee to the center of the talus and ankle.

To assess the global agreement and consistency of measurements made for each reader (differences between readings at various times for each reader), intraclass correlation coefficients (ICC) were calculated. To evaluate the global agreement of measurements between readers (Rater 1 versus Rater 2), interobserver reliability was calculated. All p values were two-sided with significance set at an alpha equal to 0.05. Guidelines for interpreting reliability and agreement coefficients vary. To assess the precision of the reported reliability estimates, 95% confidence intervals (95% CI) were calculated. All analyses were performed using SPSS Version 14.0 (SPSS Inc, Chicago, IL; USA).

Results

Intraobserver reliability between Time 1 and Time 2 for hard-copy and soft-copy measurements ranged from 0.80 to 0.996. The most and least precise measurements on soft copy are shown (Table 1). Reported correlation values for measurements made on hard copy ranged from 0.853 to 0.999. The most and least precise measurements on hard copy are shown (Table 2). Intraobserver reliability also was performed for Rater 1 to assess the reliability of measurements between hard-copy and soft-copy systems. ICC values for this ranged from 0.858 (right LFDA) to 0.980 (delta femur) (Table 3). For Rater 2, correlation values for soft-copy measurements ranged from 0.962 to 0.999. The most and the least precise measurements on soft copy done by Rater 2 are shown (Table 1). Reported correlation values for measurements made on hard copy ranged from 0.931 to 0.999. The most and the least precise measurements on hard copy done by Rater 2 are shown (Table 2). Intraobserver reliability also was performed for Rater 2 to assess the reliability of measurements between hard-copy and soft-copy systems. ICC values for this ranged from 0.546 (delta femur) to 0.999 (left femur) (Table 3). The delta femur variable had a correlation coefficient less than 0.90. All other variables had coefficient values of at least 0.96 and greater. These correlation coefficients indicated that both systems were equally reliable for Raters 1 and 2.

Table 1.

Intraclass correlations between PACS measurements at Times 1 and 2 by Raters 1 and 2

Radiographic measurement	Number*	Rater 1			Rater 2
		Correlation	Lower 95% confidence limit	Upper 95% confidence limit	Correlation	Lower 95% confidence limit	Upper 95% confidence limit
Right LDFA	40	0.796	0.604	0.989	0.967	0.886	1.000
Left LDFA	40	0.958	0.867	1.000	0.964	0.880	1.000
Right MPTA	40	0.873	0.717	1.000	0.962	0.875	1.000
Left MPTA	40	0.951	0.853	1.000	0.980	0.916	1.000
LLD	39	0.996	0.966	1.000	0.997	0.973	1.000
Right femur	40	0.975	0.904	1.000	0.988	0.940	1.000
Left femur	40	0.996	0.970	1.000	0.999	0.989	1.000
Right tibia	39	0.951	0.851	1.000	0.999	0.989	1.000
Left tibia	40	0.976	0.907	1.000	0.998	0.979	1.000
Delta tibia	39	0.996	0.967	1.000	0.982	0.922	1.000
Delta femur	40	0.842	0.670	1.000	0.969	0.890	1.000

LDFA = lateral distal femoral angle; MPTA = medial proximal tibial angle; LLD = limb length discrepancy; * the number of pairs analyzed for that particular analysis (radiographic measurement).

Table 2.

Intraclass correlations between hard-copy radiographic measurements at Times 1 and 2 by Raters 1 and 2

Radiographic measurement	Number*	Rater 1			Rater 2
		Correlation	Lower 95% confidence limit	Upper 95% confidence limit	Correlation	Lower 95% confidence limit	Upper 95% confidence limit
Right LDFA	40	0.884	0.736	1.000	0.957	0.864	1.000
Left LDFA	40	0.968	0.888	1.000	0.941	0.832	1.000
Right MPTA	40	0.853	0.687	1.000	0.950	0.851	1.000
Left MPTA	40	0.963	0.877	1.000	0.953	0.857	1.000
LLD	39	0.973	0.899	1.000	0.993	0.956	1.000
Right femur	40	0.999	0.986	1.000	0.976	0.906	1.000
Left femur	40	0.999	0.984	1.000	0.999	0.988	1.000
Right tibia	39	0.999	0.987	1.000	0.999	0.989	1.000
Left tibia	40	0.951	0.853	1.000	0.970	0.893	1.000
Delta tibia	39	0.990	0.946	1.000	0.989	0.942	1.000
Delta femur	40	0.982	0.952	1.000	0.931	0.815	1.000

LDFA = lateral distal femoral angle; MPTA = medial proximal tibial angle; LLD = limb length discrepancy; * the number of pairs analyzed for that particular analysis (radiographic measurement).

Table 3.

Intraclass correlations between hard-copy radiographs and PACS measurements for Raters 1 and 2

Radiographic measurement	Number*	Rater 1			Rater 2
		Correlation	Lower 95% confidence limit	Upper 95% confidence limit	Correlation	Lower 95% confidence limit	Upper 95% confidence limit
Right LDFA	80	0.858	0.779	0.909	0.980	0.969	0.987
Left LDFA	80	0.953	0.927	0.970	0.968	0.951	0.980
Right MPTA	80	0.906	0.853	0.939	0.969	0.952	0.980
Left MPTA	80	0.950	0.922	0.968	0.970	0.953	0.980
LLD	78	0.987	0.979	0.992	0.993	0.990	0.996
Right femur	80	0.941	0.909	0.962	0.951	0.924	0.968
Left femur	80	0.948	0.919	0.967	0.999	0.998	0.999
Right tibia	78	0.946	0.915	0.965	0.998	0.997	0.999
Left tibia	80	0.932	0.894	0.956	0.981	0.971	0.988
Delta tibia	78	0.969	0.952	0.980	0.986	0.977	0.991
Delta femur	80	0.980	0.969	0.987	0.546	0.381	0.668

LDFA = lateral distal femoral angle; MPTA = medial proximal tibial angle; LLD = limb length discrepancy; * the number of pairs analyzed for that particular analysis (radiographic measurement).

Interrater reliability for hard-copy measurements ranged from 0.697 to 0.998 and soft-copy system measurement reliability ranged from 0.665 to 0.984 (Table 4). A high interrater reliability coefficient value indicated high associations between measurements taken by both raters. The lowest correlation value testing the interrater reliability of x-ray film was right MPTA (0.697). The highest were for right femur length (0.987) and right tibia length (0.998). The lowest correlation values testing the interrater reliability of PACS were right MPTA (0.665) and right LDFA (0.789). The highest was for LLD (0.984). Measurements performed using either soft copy or hard copy yielded high interrater reliability values with narrow confidence intervals, indicating each system was reliable and precise.

Table 4.

Interclass correlations between Raters 1 and for PACS and hard-copy radiographic measurements

Radiographic measurement	Number*	PACS			Hard-copy radiographs
		Correlation	Lower 95% confidence limit	Upper 95% confidence limit	Correlation	Lower 95% confidence limit	Upper 95% confidence limit
Right LDFA	80	0.789	0.652	0.925	0.918	0.829	1.000
Left LDFA	80	0.930	0.848	1.000	0.945	0.872	1.000
Right MPTA	80	0.665	0.499	0.831	0.697	0.537	0.856
Left MPTA	80	0.931	0.850	1.000	0.943	0.870	1.000
LLD	78	0.984	0.945	1.000	0.963	0.902	1.000
Right femur	80	0.801	0.668	0.934	0.987	0.951	1.000
Left femur	80	0.886	0.783	0.989	0.978	0.931	1.000
Right tibia	78	0.896	0.796	0.996	0.998	0.986	1.000
Left tibia	80	0.893	0.794	0.993	0.932	0.852	1.000
Delta tibia	78	0.915	0.824	1.000	0.941	0.865	1.000
Delta femur	80	0.925	0.804	0.677	0.959	0.885	1.000

LDFA = lateral distal femoral angle; MPTA = medial proximal tibial angle; LLD = limb length discrepancy; * the number of pairs analyzed for that particular analysis (radiographic measurement).

Discussion

For limb lengthening and deformities, reliable measurements are particularly crucial because lower extremity alignments are measured routinely to formulate the course of treatment. Therefore, the reliability and confidence in the measurements, using PACS for 51-inch standing AP images, are key. The literature to date lacks a substantial number of comparison investigations regarding the reliability of using measurements through PACS for documenting LLDs and deformities, specifically on 51-inch stitched images. The purpose of this study was to determine the reliability of soft-copy measurements to evaluate limb length and angular deformity of the lower extremities. Intraobserver and interobserver correlation coefficient values of soft-copy digital measurements made directly on the PACS monitor were compared with manual (ruler/goniometer) measurements made on printed hard-copy film.

This study has certain limitations. First, was our use of a square measuring tool pointer, as opposed to a smaller more precise pointer, for measuring images with PACS. However, this limitation was overcome by magnifying the field to identify the measurement landmarks and to help increase reliability and reproducibility. Second was performing the measurements on digitally acquired hard-copy radiographs, which has 20% magnification inherent in analog hard-copy film. We accounted for this magnification when our digitally acquired images were printed for hard-copy film because all images at our institution are printed at 100% of analog magnification. Third, the 51-inch digitally acquired image required stitching, which is performed automatically by the software with manual adjustment when required. In our institution, each stitched film is inspected manually for crosshair congruity and restitched if not accurate. Fourth, the images were acquired with a 2.5-cm calibration ball in place on which the Ortho Tool box in PACS automatically calibrates the image. The accuracy of the calibration tool is in the consistency of ball placement at the middle AP dimension. The highest possible consistency was achieved by limiting the technologists performing the examination.

Several studies have reported the reliability and accuracy of using conventional and digital radiographs. Sailer et al. [10] compared angle measurements and assessed axial alignment of the lower limb using mechanical axis measurements on conventional and digital total-leg radiographs of 24 patients. They found no difference in angle measurements on conventional and digital radiographs. Rozzanigo et al. [8] studied alignment and articular orientation of lower limbs (femoral length, tibial length, distal medial femoral angle, proximal medial tibial angle, and tibiofemoral angle) by manual versus computer-aided measurements on digital radiograms. They found no differences between the measurements taken with the manual and computer-aided methods (p < 0.05). Sabharwal et al. [9] determined the intraobserver and interrater reliability of measuring LLD with standing full-length radiographs of the lower limb and scanograms. ICC and mean absolute difference (in millimeters) were calculated to assess intraobserver and interrater reliability. They concluded that standing full-length radiographs are as equally reliable as a scanogram for measuring LLD. Other studies report no differences in the reliability, accuracy, and diagnostic performance of digital imaging versus hard-copy radiographs [1, 2, 4, 6, 7, 9, 13] . By comparing all the extremity lengths and angular variations, and on valuating the difference in data, the digitally acquired images in PACS permit a smooth transition in clinical reliability. The dual comparisons using the intraobserver and interobserver reliability support the data in eliminating any bias.

Digitally acquired images distributed on a PACS system are rapidly gaining acceptance as state-of-the-art radiology [2, 12]. Sanfridsson [11] warned that protocols must be established to appropriately measure distances and angles on these digital images. An investigation of the use of soft-copy digital imaging concluded PACS had minimal influence in diagnostic errors [7], and as long as high-quality monitors are used at the workstations, interpretation of PACS-derived radiographs is substantially similar to that for hard-copy radiographic analysis [3, 7]. Digital measurement on PACS allows enhancements of images, for example, brightness, contrast, and magnification can be adjusted to better see important landmarks that are used for measurement. Benefits of transitioning to PACS include improved archiving, transmission, and the ability to access images simultaneously at multiple sites [11]. An added advantage is the reduction in printing of films (in multiple), storing film files, and eventually destroying printed copies. Use of PACS is reliable in a clinical setting [1, 8, 9, 11].

Our observations suggest that regarding the 51-inch standing bilateral leg radiograph, soft-copy deformity and length measurements are as reliable as manual hard-copy measurements. The slight differences observed between intraobserver and interobserver correlation coefficients can be attributed to random error.