Evaluation of reproducibility of robotic knee testing device (GNRB) on 60 healthy knees

Abstract

Introduction

There is a main concern about the usefulness and the accuracy of the different laximeters. The GnrB device is becoming more popular, but according to the literature it's not clear whether it is fully reliable.

Objective

GNRB laxity measurements are not reproducible for measurement of anterior translation of the tibia.

Material and methods

We analyzed the reproducibility of GNRB measurements in healthy subjects by the Test-Retest method in 2 sessions.

Results

30 pairs of healthy knees were assessed. Test-Retest agreement was poor for both absolute values and for differentials at 134 and 200 N with an intra-class correlation ranging from 0.210 to 0.486. There was a significant differance in anterior tibial translation, in side-to-side difference, according to the patellar pressure.

Conclusion

The reproducibility is found to be poor under optimum conditions of comparability. The patellar pressure influences strongly thelaxity value.

1. Introduction

Accurate assessment of knee laxity is critical for many steps in the management of anterior cruciate ligament (ACL) injury. Classic physical exam maneuvers, while essential, depend on subjective factors such as clinician experience, muscle relaxation, and inherent knee variability.¹

Therefore, objective assessment of joint laxity, or laximetry, is often desired to supplement physical exam findings.¹²

Major recent developments include the introduction of a new robotic arthrometer, the GNRB, designed to improve the objectivity of arthrometry for measuring anterior knee laxity.² The GNRB (Genourob, Laval, France) was first described in 2009 and has been a major focus of arthrometry literature in the past few years. This laximeter offers some technological advances in attempt to quantify the anterior laxity and improve the accuracy of the measurements recording.⁴,12 It reproduces the lachman test in an automating manner, by testing the laxity at 20 ° of knee flexion, controlling the patellar stabilization pressure, recording of translation in the absence of hamstring muscles contractions and by a motorized push on the calf.²

Preliminary results from the developers of the GNRB demonstrate good reproducibility of the measurements. In addition, they reported diagnostic validity measures and found that using 1.5 mm as a side to side difference threshold value is used for diagnosis of partial ACL tears with an 87% specificity at 134 N.² This value is controversial because this system has undergone other evaluations and appear to be poorly reproducible and unreliable. Vauhnik et al.³ reported a low precision of the device with measurement errors up to 3.8 mm.

Furthermore, an ideal patellar stabilization pressure and a threshold difference of this pressure between two measurements is not well described in literature.⁷ Bouguennec et al.⁷ studied the relation between patellar pressure and anterior tibial translation and didn’t find a significant correlation between these two parameters. However, their method and results were not clearly presented. It seems to be of great interest to control the effect of the variation of this patellar pressure on the results.

In clinical settings, knowing the intra-reliability of the instrument is crucial in order to be accurately interpreted and limitations fully understood. It was hypothesized that the GNRB is not reproducible by 0.8 mm. This would mean that the diagnosis of partial rupture of the ACL at 1.5 mm is not valid. An error of 0.8 mm for each knee could therefore lead to a false diagnosis of partial lesion. The primary objective was to assess the reproducibility of the GNRB knee arthrometer on healthy knees. The second objective was to assess the relation between the anterior tibial translation and patellar stabilization pressure.

2. Material and methods

2.1. Type of study

We conducted a prospective continuous study between January and February 2017. The reproducibility of GNRB measurements was analyzed by the Test-Retest method on 30 subjects with 30 pairs of healthy knees. The study protocol had been previously approved by our ethics committee.

2.2. Subject enrollment

One experienced examiner (D.M) performed knee laxity measurements on 30 volunteer medical students and doctors, aged between 23 and 48 years, with an intact ACL, pain free knees, and without history of present or previous knee injuries or surgery, using the GNRB knee arthrometer.

2.3. Method

All subjects were tested by the same examiner on 2 occasions in the same way and at the same time of a day approximately 2 weeks apart. Their age, sex, body weight and size were recorded prior to knee testing. Subjects were positioned and GNRB was applied according to manufacturer’s instructions. The patient was lying on standard examination table in the supine position with the arms placed along the body. The leg to be tested first was determined with randomization. The lower limb was placed in a rigid adjustable leg support, with the knee placed in neutral rotation so that the patella is facing anteriorly. The knee was fixed so that the inferior pole of the patella is covered by the lower border of the patellar support.

A displacement transducer (0.1 mm precision) positioned on the tibial tubercle recorded the relative displacement of the anterior tibial tubercle with respect to the femur.

The same conditions were applied for the second session, that occured two weeks following the first one, with a symmetrical pressure on the patella (<10 N difference) controlled by a pressure sensor and identical length positioning of each limb controlled by a scale on the leg support.

Each session consisted of 2 consecutive tests with patellar stabilization pressure between 75 and 90 N and >90 N respectively. A force of 134 and 200 N was applied on the calf for each test. A dry run was performed first for each patient to ensure that the patient was relaxed. Absolute values of anterior translation for each knee and side to side difference were recorded. Data were collected on a distant PC. A laxity file was built up for each patient including measurements conditions (patellar stabilization pressure, leg length, thrust forces). The device was in good working order, and its accuracy of the calibration was rechecked independently between sessions.

2.4. Statistics analysis

Statistical analysis was performed on STATA software 11.2 (STATA Corp., College Station, TX). We described patients’ characteristics using number and frequency for qualitative data, and mean, standard deviation (SD) and range (minimum-maximum) for quantitative data. Anterior tibial translation was described as quantitative data. The reproducibility between the test-retest values ​​of the absolute displacement as well as the test-retest values ​​of the side to side difference displacement (left-right leg) was studied using Intra class correlation (ICC) together with 95% Confidence Interval. The relation between patellar stabilization pressure and knee anterior laxity was assessed using Paired T test (in test measures). All reported p-values were two-sided and the significance threshold was <0.05.

3. Results

30 subjects with 30 pairs of healthy knees, 26 (87%) males and 4 (13%) females, were tested. Their mean age was 28.6 years (SD = 5.2) and mean body mass index 24.1(SD = 2.2). The mean (and standard deviation) of the absolute and differential values of the anterior tibial translation in test and retest are presented in Table 1, Table 2.

Table 1.

Absolute values of knee anterior translation.

Patellar pressure	Test force	Leg	Test			Retest
			Mean(SD)	Min	Max	Mean(SD)	Min	Max
P 75 - 90	134	Right	3.4 (1.2)	2	8	3.3 (0.9)	2.2	6
		Left	3.1 (1.0)	1.6	6.4	3.3 (1.2)	1.8	7.1
		both	3.2 (1.1)	1.6	8	3.3 (1.0)	1.8	7.1
	200	Right	5.1 (1.5)	2.9	10.2	5.0 (1.0)	3.4	7.6
		Left	4.8 (1.3)	2.8	8.8	5.0 (1.2)	3.2	8.3
		Both	4.9 (1.4)	2.8	10.2	5.0 (1.1)	3.2	8.3
P > 90	134	Right	3.0 (1.2)	1.5	7.9	3.0 (0.8)	1.9	4.9
		Left	2.7 (1.0)	1.5	5.6	2.9 (1.0)	1.7	5.9
		Both	2.9 (1.1)	1.5	7.9	3.0 (0.9)	1.7	5.9
	200	Right	4.6 (1.5)	2.6	10	4.5 (1.0)	2.8	6.9
		Left	4.1 (1.1)	2.5	7.5	4.3 (1.0)	2.9	7.1
		Both	4.4 (1.3)	2.5	10	4.4 (1.0)	2.8	7.1

SD: standard Deviation.

Table 2.

Differential values between left and right leg of knee anterior laxity.

Patellar pressure	Test force	Test			Retest
		Mean(SD)*	Min	Max	Mean(SD)a	Min	Max
P 75 - 90	134	−0.3 (0.9)	−1.6	1.6	−0.2 (1.1)	−3	3
	200	−0.3 (1.1)	−2.4	1.9	−0.3 (1.3)	−2.8	2.4
P > 90	134	−0.3 (0.9)	−2.3	1.5	−0.3 (1.0)	−2.1	2.2
	200	−0.5 (1.2)	−2.9	1.6	−0.3 (1.2)	−2.6	2.3

SD: standard Deviation.

^a(Left–Right).

All intra-class correlation coefficients are less than 50%. The ICC ranged from 0.414 to 0.486 for absolute values and 0.210–0.356 for differential values, indicating low intra-reliability of the machine. Slight more reliability was observed with 200 N force on the calf with ICC 0.486 and 0.356 than with 134 N force with ICC 0.414 and 0.210 for absolute and side-to-side differential values respectively. The results of ICC and 95% limits of agreement are presented in Table 3.

Table 3.

Intra-class correlation (ICC) results.

Values	Patellar pressure	Test force	ICC	95% Confidence Interval
Absolute values	P 75–90	134	0.414	0.180–0.603
		200	0.486	0.267–0.657
	P > 90	134	0.460	0.236–0.638
		200	0.481	0.261–0.654
Differential values (Left–Right)	P 75–90	134	0.210	−0.157–0.526
		200	0.356	0.001–0.631
	P > 90	134	0.332	−0.026–0.614
		200	0.348	−0.008–0.626

There was a significant difference in anterior tibial translation, in absolute value, according to the patellar pressure (75–90 versus >90) both for a thrust of 134N and 200N (p ​​< 0.05).

There was a significant difference in anterior tibial translation, in side to side difference value, according to the patellar pressure (75–90 versus >90) for a 200 N thrust (p = 0.02 < 0.05) but not for a thrust of 134N (p = 0.54 > 0.05). The results are showed in Table 4.

Table 4.

Paired T test for correlation between patellar stabilization pressure and knee anterior laxity (Test measures).

Values	Test force	Patellar force	Obs	Mean	SD	95% Confidence Interval	P – value
Absolute values	134	P 75–90	60	3.22	1.142862	2.933101–3.523566	P < 0.0001
		P > 90	60	2.86	1.106784	2.580754–3.152579
		Difference	60	0.36	0.2828976	0.2885865–0.4347469
	200	P 75–90	60	4.91	1.378983	4.553771–5.266229	P < 0.0001
		P > 90	60	4.36	1.327629	4.023704–4.70963
		Difference	60	0.54	0.34216	0.454944–0.6317226
Differential values (Left–Right)	134	P 75–95	30	−0.29	0.8774899	−0.6243267 to 0.3009935	P = 0.5440
		P > 90	30	−0.34	0.9438001	−0.6924208 to 0.124207
		Difference	30	0.04	0.3865706	−0.1010144 to 0.1876812
	200	P 75–90	30	−0.30	1.084669	−0.7116885 to 0.983552	P = 0.0245
		P > 90	30	−0.46	1.166368	−0.8955289 to −0.244711
		Difference	30	0.15	0.3540245	0.021384 to 0.2855283

SD: standard Deviation.

4. Discussion

Our hypothesis is confirmed: this device is not fully reliable and the 1.5 mm threshold should not be validated for an ACL partial tear. The ICCs ranged from 0.210 to 0.486 indicating low intra-reliability of the machine. Moreover, the laxity varies according to the patellar pressure. This is a major technical point that must be raised and lead to determine a standard value for this pressure.

The general desire to produce objective and repeatable data of lachman test in clinical practice led to a variety of devices to measure anterior knee laxity.¹² The most commonly used are the Telos, a stress imaging laximeter, the KT-1000 and the Roliometer.⁴ These devices had varying levels of sophistication and cost and have been subject to numerous studies and trials.⁵,14, 15, 16, 17, 18 Their utility and reliability have been extensively studied.⁵ New devices continue to arise as technological advancements continue to make the devices more sophisticated. Major recent developments include the introduction of a robotic arthrometer, the GNRB device, designed to improve the objectivity of arthrometry.² This laximeter has received increased attention owing to its automated testing protocol and promising reliability measures.⁴

Because of the recent availability of the GNRB arthrometer, only few clinical studies in the literature have been conducted to assess its inter and intra-reliability, and to compare it to other devices.⁸,19,20

Robert et al. ² have shown a good inter-reliability of the GNRB, wherever the examiner’s experience stands and whatever the evaluated side condition could be. Similarly, Colette et al. ⁶ reported that the GNRB gives reproducible measurements and not examiner dependent, but they did not reported their analysis method, reporting only mean values and standard variations of the testing. Both authors have found that GNRB reliability to be significantly superior to KT 1000 device, which they attributed to the higher precision translation probe and the automated force application. Bouguennec et al. ⁷ also reported a better reproducibility of the GNRB over the Telos. However, they choose 89 N as a force applied on the calf and studied healthy knees in longitudinal postoperative follow-up of the opposite knee with 6 months duration between the test and retest, which limits their conclusion.

On the other hand, significant discrepancy exists in the literature. Other authors doubt on the superior reliability of the instrument over other arthrometers and on its ability to overcome subjective sources of variation in clinical practice. GNRB was found to underestimate anterior laxity as measured intraoperatively using the navigation system. A systematic measurement error of −3.7 mm occurred for preoperative measurements versus intraoperatively using a non-image-based navigation system.¹³ This systematic bias may be relevant to treatment decision-making. In addition, Vauhnik et al. ³,9 were the first to describe the results of inter and intra reliability of the GNRB arthrometer using ICC values. They reported low ICC values for intra-reliability (ICC: 0.338–0.786) and inter-reliability (ICC: 0.220–0.424). He showed that an anterior knee laxity less than 2–3 mm from one test session to the next in a single knee is likely due to measurement error and not due to changes in actual knee laxity.

The results in our trial showed substantial variations in measurements using GNRB arthrometer. There was intra-operator variation whether in recording absolute values of displacement in single knee or differences of displacement between pairs of knees. The test re-test concordance is bad for both absolute values with an ICC ranged from 0.414 to 0.481 and differential values with ICC ranged from 0.210 to 0.356. Furthermore, slight more reliability was observed with 200 N force on the calf than 134 N.

This emphasizes on the need to exceed 200 N to improve the reproducibility of any instrumented ACL laxity measurement as demonstrated by Markolf et al.¹⁰

For a robotic device such as the GNRB, changes in laxity noted from one test session to another in terms of directions, amounts and speeds of force application is minimal at most because all these parameters are somehow automatized. Instead, there may be several reasons for the measurement errors found in this device. As described by Vauhnik et al.,⁹ the sensor of the GNRB arthometer is flat and since it is positioned directly on the skin of the tibial tuberosity, this makes this sensor very susceptible to errors related to soft tissue movement and consequently to the tibial rotation which is occurring during the test, since the tibial tuberosity is not flat. Another possible factor for large variance might lie in the position of the device relative to the knee from session to session and to the degree of patient’s relaxation. The GNRB arthrometer is a robotic device and it is therefore very sensitive to changes in positioning. This highlights the importance of examiner performance in standardization of patient positioning and testing protocol in order to decrease measurement error and ensure better reproducibility even with robotics laxity testing devices like the GNRB. In addition, we suspect other reasons that may be responsible for changes in its performance. It seems difficult to position the knee in an identical way in rotation between two sessions despite the control of the rotation of the foot available in the GNRB. Changes in knee position lead to changes in tibial rotation and therefore in the amount of anterior tibial displacement measured.¹¹ This problem is obvious in obese patients for whom the thickness of soft tissues at the level of displacement sensor could underestimate anterior tibial translation.

The patellar shell cap fixates the patella against the femoral trochlea, and thus, fixates the femur upon anterior tibial translation, with an individualized pressure. It has to be positioned accurately, and consists of 3 straps (Fig. 1), each one has the ability to alter this pressure. We found, in our experience with this machine, a problem to determine a fix patellar pressure. It is quite difficult to keep exactly the same value between measurements. This is maybe caused by the changes in relaxation of the patient and leg’s muscles firing patterns instantly. Adding to this, these straps can be tightened in a different manner for the same patellar pressure. We think that this arbitrary fixation of these straps can play a role in variation of tibial translation. Under the same pressure, a strap tightened too proximal put rigid strain on the thigh with posterior translation of the femur, and thus, can give falsely a large tibial anterior translation. In contrary, a strap tightened distally will place the femur in more flexible position and thus translation of the tibia will be smaller.

Fig. 1.

GNRB device.

Whatever the subjective causes of variation, the GNRB was in our hand not capable of overcoming them and providing a reliable reproducible measurement of laxity of the ACL. Because the system is non-invasive, it is still susceptible to errors related to soft-tissue or skin artifact, and it is also open to errors related to patient muscle guarding.¹²

From their observations, Robert et al. have sought to establish measurements which identify pathological laxity. They tested 21 complete ACL tears and 24 partial ACL tears with the GNRB and suggested a difference of more than 1.5 mm between the measured laxities in two knees at 134 N as a threshold for partial tear of the ACL in one of them with a sensitivity of 80% and specificity of 87% and 3 mm threshold for complete rupture with sensitivity of 70% and specificity of 99%. This cut off is criticable because there is high variability of the laxity measurements. Indeed if there is a variability higher or equal to 0.8 mm in each knee, this could conduct to a misdiagnosis of partial lesion (with difference between the 2 knees higher than 1.5 mm [0.8 mm × 2]). For example in our study, in the 60 test-retests (with patellar stabilization pressure between 75 and 90 N and a force of 200 N), 50% had a difference between test-retest > = 0.8 mm. Moreover, the cut off of 3 mm for difference between the 2 knees for the diagnosis of complete lesion is also questionable. Indeed in our study, 25% of these test-retests had a difference between test-retest > = 1.5 mm which could lead to a misdiagnosis of complete lesion (with difference between the 2 knees higher or equal to 3 mm [1.5 mm × 2]). Thus, the finding of 1.5 mm and 3 mm as the threshold for diagnosis of ACL partial and complete rupture respectively are to be judged. On this criterion, the GNRB is liable to indicate false positive and false negative results. Lefevre⁸ found 2.5 mm differential threshold for partial tears with sensibility of 84% and specificity of 81%, which seems to be more pertinent. Beldame et al. ¹⁹ also didn’t find the same diagnostic value proposed by the designer for partial tears, while their series was four times larger in effective. For 1.5 mm differential, a low sensibility of 62% and specificity of 75,9% was described.

The ideal patellar stabilization pressure and a threshold difference of this pressure between two measurements is not well described in literature.⁷ The choice of minimum values ​​and threshold differences between 2 tests has therefore been made arbitrarily. Bouguennec et al. studied the relation between patellar pressure and anterior tibial translation and didn’t find a significant correlation between these two parameters. However, their method and results were not clearly presented.⁷ Our study is the first in literature to describe a significant correlation between the patellar pressure and the anterior tibial translation. We found a significant decrease in anterior tibial translation with an excessive patellar pressure. This finding may be explained by a possible involuntary reflex or defensive contraction of the quadriceps or the hamstring with high patellar pressure, thus decreasing significantly the anterior translation as has been demonstrated. Yet, particular attention is needed regarding the patellar stabilization pressure during the test. The patellar pressure strongly influences the results and must be reproduced identically in order to compare the measurements. The determination of a sufficient theoretical value of maximum difference between two measurements is however necessary.

5. Conclusion

In this study, the reproducibility is found to be poor under optimum conditions of comparability. We did not find the proposed critical values for diagnosis of a partial LCA tear to be reliable in diagnosis. A discussion of side-to-side differential displacement values for the diagnosis of partial tear with 1.5 mm should be reconsidered. Moreover, the patellar stabilization pressure strongly influences the results.

Conflict of interest

All authors declare that they have no conflict of interest.