An automatic 2D–3D image matching method for reproducing spatial knee joint positions using single or dual fluoroscopic images

Abstract

Fluoroscopic image technique, using either a single image or dual images, has been widely applied to measure in vivo human knee joint kinematics. However, few studies have compared the advantages of using single and dual fluoroscopic images. Furthermore, due to the size limitation of the image intensifiers, it is possible that only a portion of the knee joint could be captured by the fluoroscopy during dynamic knee joint motion. In this paper, we presented a systematic evaluation of an automatic 2D–3D image matching method in reproducing spatial knee joint positions using either single or dual fluoroscopic image techniques. The data indicated that for the femur and tibia, their spatial positions could be determined with an accuracy and precision less than 0.2 mm in translation and less than 0.4° in orientation when dual fluoroscopic images were used. Using single fluoroscopic images, the method could produce satisfactory accuracy in joint positions in the imaging plane (average up to 0.5 mm in translation and 1.3° in rotation), but large variations along the out-plane direction (in average up to 4.0 mm in translation and 2.28 in rotation). The precision of using single fluoroscopic images to determine the actual knee positions was worse than its accuracy obtained. The data also indicated that when using dual fluoroscopic image technique, if the knee joint outlines in one image were incomplete by 80%, the algorithm could still reproduce the joint positions with high precisions.

Introduction

Fluoroscopic imaging technique has been increasingly used to measure in vivo human knee joint motion (Banks and Hodge 1996; Zuffi et al. 1999; Mahfouz et al. 2003; Li et al. 2004). This technique reproduces the joint spatial positions through a 2D–3D image matching procedure that combines the information of the 3D knee model and the captured fluoroscopic images of the knee. Although application of a single fluoroscopic imaging technique has been mostly used to measure total knee replacement kinematics (Banks and Hodge 1996), the dual fluoroscopic imaging technique has been extensively used to examine the native knee joint motion (Li et al. 2008). The accuracy or precision of using these imaging modalities for studying knee joint kinematics has been reported in various sophistications in the literature (Banks and Hodge 1996; Hoff et al. 1998; You et al. 2001; Kaptein et al. 2003; Bingham and Li 2006; Li et al. 2008). However, few studies have compared the differences in application of single and dual fluoroscopic imaging techniques using a consistent fluoroscopic set-up and a consistent 2D–3D image matching method (Hanson et al. 2006).

Recently, fluoroscopic imaging techniques have been used to measure the dynamic range of motion of the knee such as gait or dynamic flexion of the knee (Li et al. 2008). Due to the limited size of the imaging intensifier, the fluoroscope may not capture the entire knee joint image when the joint moves in a large range of motion. The incomplete information of the joint captured on the fluoroscopic images may affect the reproduced joint position in space. However, no study has reported on the effect of incomplete knee joint information captured on the images on the accuracy of the reproduced knee joint positions.

In this paper, we presented a systematic evaluation of an automatic 2D–3D image matching method in the determination of spatial knee joint positions using either single or dual fluoroscopic imaging techniques. The algorithm of the automatic matching procedure was introduced first. The accuracy and precision of the method in determination of knee joint positions in space were then evaluated through a series of designed tests. Finally, the effect of the incomplete knee joint information on the captured images on the reproduced knee joint positions was investigated.

Materials and methods

Combined MRI and fluoroscopic image technique

Fluoroscopic system set-up

The dual fluoroscopic image system (DFIS) consists of two fluoroscopes (BV Pulsera®, Philips, Bothell, WA, USA) positioned to formulate a common imaging zone (Figure 1(a)). A subject can freely move his/her knee within the DFIS, whereas the two fluoroscopes capture the images of the knee simultaneously (Bingham and Li 2006; Hanson et al. 2006; Li et al. 2008). The in vivo poses of the knee are recorded as a series of paired fluoroscopic images. These images are corrected for distortion using the Gronenschild method and automatically segmented using Canny edge detection method (Bingham and Li 2006).

Figure 1.

(a) A dual-orthogonal fluoroscopic system for capturing in vivo knee joint kinematics; (b) a virtual dual-orthogonal fluoroscopic system constructed to reproduce the in vivo knee joint kinematics.

A virtual DFIS is then created in a solid modelling program (Rhinoceros®, Robert McNeel & Associates, Seattle, WA, USA) to replicate the geometry of the actual DFIS (Bingham and Li 2006; Figure 1(b)). The two fluoroscopic images (labelled as F₁ and F₂ in Figure 1(b)) are positioned to represent the two virtual image intensifiers. Two virtual cameras are created to represent the two actual X-ray sources. A global coordinate system is created in the virtual DFIS, where the X–Y plane coincides with the plane of a fluoroscopic image intensifier (F₁ in this study; Figure 1(b)).

In application of a single fluoroscopic image technique, only one fluoroscope is needed (for example, F₁ in Figure 1). Practically, this is much easier to establish than the DFIS as no synchronisation between the fluoroscopes is needed, and a single fluoroscope is accessible in many medical institutions.

3D knee model reconstruction

Each subject will be MR scanned to acquire 3D MR images of the target joint. The 3D MR images are used to construct 3D anatomic surface models of the tibia and femur using an established protocol in Rhino software (Li et al. 2008). The surfaces are tessellated, and the vertices of the mesh are used to create a 3D point cloud of the model. A local coordinate system is created for each model (Figure 1(b)) that is related to the global coordinate system through a position vector and a rotation matrix (Bingham and Li 2006). The 3D model can be manipulated in the virtual environment in 6DOF, and the two virtual cameras can project the model onto the two virtual image intensifiers. The in vivo knee position is reproduced when the projections of the 3D model match the outlines of the knee captured on the fluoroscopic images during actual activities.

Automatic 2D–3D image matching process

Transformation of model points

In the global coordinate system, a point in the 3D model is noted as P_i, which can be transformed with the model by six unknown independent variables from an initial pose to a new position, noted as P̃_i. The transformation matrix M is composed by a 3D translation vector T = {x, y, z}^T and a rotation matrix defined in a X–Y–Z Euler sequence using three angles (α, β, γ).

$$\{\begin{array}{c}{\tilde{P}}_i=M\cdot P_i=\left[\begin{array}{cc}\hfill R_{3\times 3}\hfill & \hfill T\hfill \\ \hfill 0\hfill & \hfill 1\hfill \end{array}\right]\cdot P_i,\hfill \\ R_{3\times 3}=\left[\begin{array}{ccc}\hfill \text{cos}\beta \text{cos}\gamma \hfill & \hfill \text{sin}\alpha \text{sin}\beta \text{cos}\gamma -\text{cos}\alpha \text{sin}\gamma \hfill & \hfill \text{cos}\alpha \text{sin}\beta \text{cos}\gamma +\text{sin}\alpha \text{sin}\gamma \hfill \\ \hfill \text{cos}\beta \text{sin}\gamma \hfill & \hfill \text{cos}\alpha \text{cos}\gamma +\text{sin}\alpha \text{sin}\beta \text{sin}\gamma \hfill & \hfill \text{cos}\alpha \text{sin}\beta \text{sin}\gamma -\text{sin}\alpha \text{cos}\gamma \hfill \\ \hfill -\text{sin}\beta \hfill & \hfill \text{sin}\alpha \text{cos}\beta \hfill & \hfill \text{cos}\alpha \text{cos}\beta \hfill \end{array}\right]\hfill \end{array}\phantom{\}}.$$ (1)

Projection of 3D model

Once the 3D model is transformed to a new pose, the ith point P̃_i of the model is projected onto the image intensifier of the kth fluoroscope and is noted as P̃_k,i (Bingham and Li 2006),

$${\tilde{P}}_{k,i}=S_k+\frac{l_k}{({\tilde{P}}_i-S_k)\cdot n_k}({\tilde{P}}_i-S_k),$$ (2)

where the scalar l_k is the distance between the X-ray source and the image intensifier of the kth fluoroscope; the vector S_k locates the kth X-ray source and n_k the unit vector normal to the kth intensifier plane.

A transformation matrix M_k is used to transform the projection P̃_k,i to a 2D local coordinate system, which is defined by three points containing an origin, the X-axis and the Y-axis in the kth image intensifier:

$${\tilde{P}}_{k,i}^L=M_k\cdot {\tilde{P}}_{k,i}.$$ (3)

The automatically segmented outline of the fluoroscopic image in the kth image intensifier plane is manually reviewed and saved as a list of 2D spatial points (Bingham and Li 2006). These points of the outline are used to create splines and transformed by M_k to the local coordinate system of the kth image intensifier, noted as g_k(t), t is the parametric variable, for which the range corresponds to the recorded endpoints of the splines.

Thus, both the projection points of the 3D model and the points on the outline of the actual joint image are transformed to the local coordinate system. These points can be used to minimise the average distance between the projection of the 3D model and the outline of the actual joint on the fluoroscopic images (Bingham and Li 2006).

Objective function for optimisation

In this paper, only the points on the outline of the projected 3D model are compared with the segmented fluoroscopic outline of the joint. Using a left-looking, contour-following algorithm, the grids that outline the projected points ${\tilde{P}}_{k,i}^L$ in the kth image intensifier were obtained (Bingham and Li 2006; Figure 2).

Figure 2.

(a) Outlining procedure, (b) compartmentalise projected points, (c) determine boundary grids with left-looking outlining technique, (d) and (e) select point in each grid that is closest to the outer edge and (f) completion of algorithm with selected outline points.

The segmented fluoroscopic image outline g_k(t) is the target that the projected points of the 3D model need to match. For the DFIS, this is realised by minimising the distances between the projected points and the fluoroscopic outline points represented by the two matching groups: ${\tilde{P}}_{1,i}^L$ and ${\tilde{P}}_{2,i}^L$ (Figure 3), i.e.

$$\begin{array}{cc}\hfill I& =\sum _{k=1}^2\frac{1}{N_k}\sum _{t=1}^{N_k}{d}_{t,i}^2=\sum _{k=1}^2\frac{1}{N_k}\sum _{t=1}^{N_k}\left\{\underset{i}{\text{min}}\mid {\left[g_k\left(t\right)-{\tilde{P}}_{k,i}^L\right]}^2\phantom{\mid }\right\}\hfill \\ \hfill & =\sum _{k=1}^2\frac{1}{N_k}\sum _{t=1}^{N_k}\left\{\underset{i}{\text{min}}\mid {\left[g_k\left(t\right)-M_k\left(S_k+\frac{l_k}{({\tilde{P}}_i-S_k)\cdot n_k}({\tilde{P}}_i-S_k)\right)\right]}^2\phantom{\mid }\right\}.\hfill \end{array}$$ (4)

Figure 3.

Representation of calculating the minimum distance between fluoroscopic outline points and model projected outline points.

As the objective function I approaches zero, the 3D model closely approaches the pose of the actual knee joint in space. Therefore, to accurately replicate the actual knee joint pose, the objective function is minimised by adjusting position and orientation of the local coordinate system of the femur or tibia {x, y, z, α, β, γ} in global system (Figure 1(b)):

$$J=\underset{\{x,y,z,\alpha ,\beta ,\gamma \}}{\text{min}}I.$$ (5)

In this paper, if a single fluoroscopic image is used, the information related to the fluoroscope F₂ is removed from the above optimisation procedure. Minimisation of this function is accomplished by using the Broyden, Fletcher, Goldfarb and Shanno quasi-Newton method that is implemented in the MatLab2010a® software. Convergence is controlled by terminating the minimisation routine when the differential change in variables meets the required tolerance or the number of objective function calls exceeds a specified number during the optimisation procedure.

Determination of initial 3D model position

To reproduce the in vivo knee position using the above automatic matching process, an initial position (or initial guess) of each model is needed. This can be done by manually manipulating the model in the software to make an initial guess (Bingham and Li 2006). As in any optimisation process, an initial guess may influence the convergent rate of the optimisation. We assume that the error of manual initial guessing is within ± 10 mm in translation and ± 10° in rotation from the actual pose of the knee joint. For each in vivo position of the knee, we generate 50 initial guessing positions for the knee using a Monte Carlo method implemented in the MatLab. The knee position is determined as the average of the multiple optimisation processes.

Evaluation of the accuracy and precision of the matching procedure

The algorithm presented above has been programmed in a personal computer (Intel® Core™ i7 processor: 2.67 GHz, 9 GB RAM). A validation was performed to demonstrate the accuracy and repeatability of the algorithm in reproducing the knee joint position in space. The validation was carried out using idealised images, artificial images and actual images captured from a living human knee.

Idealised testing

An idealised environment was created to evaluate the accuracy and repeatability of the automatic matching algorithm in determination of the spatial position of the knee under controlled conditions. To do this, the 3D femur and tibia models of one human subject were positioned in the system to create a knee bending position, and this position was used as a gold standard knee position. The models were then projected onto the virtual image intensifiers, and the projections were used to create a pseudo fluoroscopic outline of the joint (Figure 4). The ideal joint outline was used to reproduce the known joint pose using the automatic matching procedure. The error of the automatic matching process was determined by comparing the optimised model pose with the gold standard position. The knee pose was reproduced using both the dual fluoroscopic images and the single fluoroscopic image (F₁ fluoroscope).

Figure 4.

To project the cloud points model onto the virtual intensifiers to create ideal outlines on the two virtual image intensifiers.

Artificial testing

To determine the effect of the fluoroscopic image segmentation on accuracy and repeatability of the automatic matching algorithm, we created artificial fluoroscopic images of the knee by projecting the meshed 3D model of the knee onto the virtual image intensifiers (Figure 5). The artificial images were segmented using B-splines, the same procedure when segmenting a real fluoroscopic image (Bingham and Li 2006). The knee model position was used as a gold standard to examine the reproduced knee pose.

Figure 5.

(a) To project the meshed model onto the virtual intensifiers to create artificial images and (b) to find the outlines from the artificial images.

Actual testing

The actual test employed the fluoroscopic images of the left knee of one living subject taken by the DFIS during a stair climbing activity. The images were acquired under the Internal Review Board approval and with informed consent signed by the subject. A low flexion position and a high flexion position of the knee were selected to test the automatic matching algorithm (Figure 6). As the actual positions and orientations of the subject's femur and tibia were unknown a priori, this test was designed to evaluate the repeatability (or precision) of the algorithm in determination of the in vivo positions and orientations of the knee. The results of this actual testing were used as the golden standard in examination of the effect of the incomplete knee joint images on prediction of the knee joint positions in space.

Figure 6.

(a) Actual images of full extension and (b) maximum flexion in stair poses.

Effect of incomplete joint imaging

As the fluoroscopes may capture only part of the knee joint if the knee moves in a large range in space, we examined the effect of incomplete knee joint images on the reproduced knee joint positions. In the first case, the image of the knee was partially blocked in one fluoroscopic image. In the second case, the images of the knee were partially blocked in both fluoroscopic images. The images were blocked from either the medial/lateral direction or the superior/inferior direction of the knee.

Results

Idealised testing

Using dual fluoroscopic images, this test yields a maximal error of 0.01 ± 0.01 mm in translation and 0.02 ± 0.09° in orientation for the femur (Table 1); a maximal error of 0.02 ± 0.12 mm in translation and 0.12 ± 0.36° in orientation for the tibia (Table 1). Using single fluoroscopic images, this test yields a maximal in-plane position error of 0.27 ± 0.70 mm and an orientation error of 0.17 ± 0.81°, but an out-plane position error of 2.79 ± 7.73 mm and an orientation error of 0.95 ± 2.50° for the femur (Table 1); a maximal in-plane position error of 0.16 ± 0.35 mm and an orientation error of 0.03 ± 0.608, but an out-plane position error of 3.31 ± 8.02 mm and an orientation error of 0.92 ± 2.58° for the tibia (Table 1).

Table 1.

Accuracy (average error values) and repeatability (standard deviations of the automatic matching procedure) in idealised, artificial and actual environments.

		Translation (mm)			Rotation (deg)
Test	Plane	Δ x	Δ y	Δ z	Δ α	Δ β	Δ γ
Femur: error in femoral pose parameters Avg ± Std
Ideal	Dual	0.00 ± 0.02	0.00 ± 0.02	0.01 ± 0.01	0.02 ± 0.06	0.02 ± 0.09	0.01 ± 0.06
	Single	0.13 ± 0.45	0.27 ± 0.70	2.79 ± 7.73	0.40 ± 1.48	0.95 ± 2.50	0.17 ± 0.81
Artificial	Dual	0.21 ± 0.02	0.03 ± 0.02	0.14± 0.03	0.34 ± 0.11	0.40 ± 0.15	0.01 ± 0.08
	Single	0.47 ± 0.45	0.25 ± 0.42	4.00 ± 7.00	0.12 ± 2.15	2.19 ± 2.36	0.19 ± 1.11
Actual full flexion	Dual	0.00 ± 0.05	0.00 ± 0.05	0.00 ± 0.06	0.00 ± 0.34	0.00 ± 0.24	0.00 ± 0.25
	Single	0.00 ± 0.28	0.00 ± 0.78	0.00 ± 8.23	0.00 ± 1.56	0.00 ± 2.54	0.00 ± 0.69
Actual full extension	Dual	0.00 ± 0.02	0.00 ± 0.05	0.00 ± 0.03	0.00 ± 0.12	0.00 ± 0.14	0.00 ± 0.11
	Single	0.00 ± 0.32	0.00 ± 0.26	0.00 ± 4.38	0.00 ± 1.94	0.00 ± 1.48	0.00 ± 0.94
Tibia: error in tibial pose parameters Avg ± Std
Ideal	Dual	0.01 ± 0.06	0.00 ± 0.04	0.02 ± 0.12	0.12 ± 0.36	0.06 ± 0.37	0.08 ± 0.29
	Single	0.16 ± 0.35	0.11 ± 0.24	3.31 ± 8.02	0.26 ± 1.10	0.92 ± 2.58	0.03 ± 0.60
Artificial	Dual	0.12 ± 0.05	0.09 ± 0.04	0.14± 0.10	0.21 ± 0.24	0.26 ± 0.41	0.41 ± 0.19
	Single	0.03 ± 0.60	0.23 ± 0.46	3.67 ± 7.81	2.01 ± 4.13	1.65 ± 2.62	1.32 ± 2.07
Actual full flexion	Dual	0.00 ± 0.08	0.00 ± 0.05	0.00 ± 0.13	0.00 ± 0.37	0.00 ± 0.36	0.00 ± 0.28
	Single	0.00 ± 0.29	0.00 ± 0.45	0.00 ± 7.13	0.00 ± 2.99	0.00 ± 1.36	0.00 ± 1.77
Actual full extension	Dual	0.00 ± 0.09	0.00 ± 0.06	0.00 ± 0.14	0.00 ± 0.35	0.00 ± 0.39	0.00 ± 0.27
	Single	0.00 ± 0.26	0.00 ± 0.23	0.00 ± 4.80	0.00 ± 1.62	0.00 ± 1.05	0.00 ± 0.93

Note: Each case was evaluated using 50 initial positions.

Artificial testing

Using dual fluoroscopic images, this test yields a maximal error of 0.21 ± 0.02 mm in translation and 0.40 ± 0.15° in orientation for the femur (Table 1); a maximal error of 0.14 ± 0.10 mm in translation and 0.41 ± 0.19° in orientation for the tibia (Table 1). Using single fluoroscopic images, this test yields a maximal in-plane position error of 0.47 ± 0.45 mm and an orientation error of 0.19 ± 1.11°, but an out-plane position error of 4.00 ± 7.00 mm and an orientation error of 2.19 ± 2.36° for the femur (Table 1); a maximal in-plane position error of 0.23 ± 0.46 mm and an orientation error of 1.32 ± 2.07°, but an out-plane position error of 3.67 ± 7.81 mm and an orientation error of 2.01 ± 4.13° for the tibia (Table 1).

Actual testing

Using dual fluoroscopic images, this test yields a maximal precision of ±0.06 mm in translation and ±0.34° in orientation for the femur (Table 1); a maximal precision of ±0.14 mm in translation and ±0.39° in orientation for the tibia (Table 1). Using single fluoroscopic images, this test yields a maximal in-plane position precision of ±0.78 mm and an orientation precision of ±0.94°, but an out-plane position precision of ±8.23 mm and an orientation precision of ±2.54° for the femur (Table 1); a maximal in-plane position precision of ±0.45 mm and an orientation precision of ±1.77°, but an out-plane position precision of ±7.13 mm and an orientation precision of ±2.99° for the tibia (Table 1).

Effect of incomplete joint imaging

Using the dual image technique, if one image was partially blocked by 20% in the medial/lateral direction, the automatic matching process could determine the femur position with an accuracy of 0.09 ± 0.06 mm in translation and 0.46 ± 0.15° in orientation and the tibia position with an accuracy of 0.21 ± 0.3 mm in translation and 0.6 ± 0.68° in orientation. When the image was blocked by 80%, the femur position could be determined with an accuracy of 0.25 ± 0.22 mm in translation and 0.7 ± 0.57° in orientation and the tibia position with an accuracy of 0.42 ± 0.06 mm in translation and 0.87 ± 0.47° in orientation (Table 2). If one image was partially blocked by 20% in the superior/inferior direction, the automatic matching process could determine the femur position with an accuracy of 0.06 ± 0.06 mm in translation and 0.69 ± 0.73° in orientation and the tibia position with an accuracy of 0.03 ± 0.13 mm in translation and 0.51 ± 0.44° in orientation. When the image was blocked by 80%, the femur position could be determined with an accuracy of 0.57 ± 0.33 mm in translation and 1.3 ± 2.73° in orientation and the tibia position with an accuracy of 0.09 ± 0.49 mm in translation and 1.71 ± 1.14° in orientation (Table 3).

Table 2.

Accuracy (average error values) and repeatability (standard deviations of the automatic matching procedure) when partly blocking one fluoroscopic image or both fluoroscopic images by 20–80% from the medial/lateral direction.

Error: Avg ± Std			Translation (mm)			Rotation (deg)
Femur		Block	Δ x	Δ y	Δ z	Δ α	Δ β	Δ γ	F 1	F 2
20%	Dual	F1	0.03 ± 0.03	0.09 ± 0.06	0.04 ± 0.08	0.12 ± 0.23	0.46 ± 0.15	0.05 ± 0.13
		F1 & F2	0.17 ± 0.10	0.09 ± 0.05	0.16 ± 0.09	0.22 ± 0.25	0.04 ± 0.31	0.21 ± 0.22
	Single	F1	0.08 ± 0.45	0.13 ± 0.52	0.51 ± 5.26	0.52 ± 3.55	0.48 ± 3.33	0.40 ± 1.26
40%	Dual	F1	0.03 ± 0.04	0.08 ± 0.05	0.02 ± 0.08	0.05 ± 0.30	0.43 ± 0.23	0.22 ± 0.17
		F1 & F2	0.20 ± 0.07	0.09 ± 0.04	0.16 ± 0.09	0.45 ± 0.31	0.16 ± 0.30	0.01 ± 0.20
	Single	F1	0.15 ± 0.46	0.22 ± 0.67	0.10 ± 5.01	0.21 ± 3.65	0.49 ± 3.86	0.08 ± 1.71
60%	Dual	F1	0.04 ± 0.03	0.04 ± 0.03	0.19 ± 0.14	0.22 ± 0.19	0.64 ± 0.31	0.11 ± 0.12
		F1 & F2	0.36 ± 0.16	0.17 ± 0.65	0.28 ± 0.91	0.50 ± 1.48	1.23 ± 2.81	0.53 ± 1.76
	Single	F1	0.38 ± 0.45	0.11 ± 0.73	0.33 ± 4.73	1.19 ± 3.80	1.15 ± 4.59	0.80 ± 2.29
80%	Dual	F1	0.03 ± 0.05	0.03 ± 0.07	0.25 ± 0.22	0.31 ± 0.39	0.70 ± 0.57	0.04 ± 0.24
		F1 & F2	0.39 ± 0.18	0.25 ± 0.95	0.40 ± 0.91	0.72 ± 1.39	1.65 ± 3.01	0.74 ± 2.74
	Single	F1	0.52 ± 0.47	0.08 ± 1.33	0.26 ± 5.24	1.84 ± 4.25	2.24 ± 4.64	0.75 ± 3.69

Tibia		Block	Δ x	Δ y	Δ z	Δ α	Δ β	Δ γ
20%	Dual	F1	0.03 ± 0.11	0.00 ± 0.09	0.21 ± 0.30	0.60 ± 0.68	0.46 ± 0.72	0.43 ± 0.45
		F1 & F2	0.59 ± 0.25	0.39 ± 0.09	0.11 ± 0.46	0.09 ± 0.90	1.37 ± 1.51	0.72 ± 0.65
	Single	F1	0.39 ± 0.35	0.45 ± 0.35	0.01 ± 4.77	1.41 ± 1.92	0.75 ± 2.30	0.62 ± 1.22
40%	Dual	F1	0.06 ± 0.09	0.14 ± 0.09	0.31 ± 0.28	0.52 ± 0.65	0.83 ± 0.53	0.64 ± 0.42
		F1 & F2	1.17 ± 0.55	0.47 ± 0.14	0.40 ± 0.77	0.39 ± 1.29	3.57 ± 2.93	0.48 ± 0.99
	Single	F1	0.77 ± 0.78	0.47 ± 0.84	0.49 ± 5.38	1.73 ± 3.29	2.76 ± 4.68	1.15 ± 2.05
60%	Dual	F1	0.04 ± 0.09	0.34 ± 0.10	0.34 ± 0.28	0.49 ± 0.75	0.60 ± 0.49	0.38 ± 0.49
		F1 & F2	1.01 ± 0.85	0.03 ± 0.40	0.50 ± 0.92	0.66 ± 1.55	3.55 ± 4.09	0.42 ± 1.15
	Single	F1	0.81 ± 0.79	0.70 ± 1.20	0.43 ± 5.40	1.93 ± 3.48	2.90 ± 4.56	1.48 ± 2.33
80%	Dual	F1	0.03 ± 0.08	0.42 ± 0.06	0.22 ± 0.25	0.29 ± 0.60	0.87 ± 0.47	0.29 ± 0.37
		F1 & F2	0.88 ± 0.74	0.21 ± 0.33	0.64 ± 0.91	1.07 ± 1.57	3.10 ± 3.82	0.78 ± 1.14
	Single	F1	0.91 ± 0.82	0.21 ± 0.93	0.44 ± 5.35	1.97 ± 3.20	2.87 ± 4.65	1.34 ± 2.04

Table 3.

Accuracy (average error values) and repeatability (standard deviations of the automatic matching procedure) when partly blocking one fluoroscopic image or both fluoroscopic images by 20–80% from the superior/inferior direction.

Error: Avg ± Std			Translation (mm)			Rotation (deg)
Femur		Block	Δ x	Δ y	Δ z	Δ α	Δ β	Δ γ	F 1	F 2
20%	Dual	F1	0.06 ± 0.06	0.03 ± 0.06	0.06 ± 0.05	0.69 ± 0.73	0.25 ± 0.36	0.23 ± 0.37
		F1 & F2	0.01 ± 0.07	0.07 ± 0.06	0.06 ± 0.05	0.76 ± 0.67	0.14 ± 0.38	0.16 ± 0.25
	Single	F1	0.06 ± 0.32	0.00 ± 0.23	1.21 ± 4.54	0.80 ± 2.31	1.33 ± 1.70	0.19 ± 0.90
40%	Dual	F1	0.06 ± 0.14	0.04 ± 0.10	0.10 ± 0.13	0.39 ± 1.93	0.51 ± 0.72	0.06 ± 0.82
		F1 & F2	0.22 ± 0.16	0.18 ± 0.17	0.06 ± 0.13	0.89 ± 2.98	0.21 ± 0.74	0.34 ± 0.83
	Single	F1	0.06 ± 0.27	0.02 ± 0.26	0.86 ± 5.05	1.34 ± 3.30	1.57 ± 1.56	0.42 ± 0.87
60%	Dual	F1	0.03 ± 0.14	0.02 ± 0.09	0.23 ± 0.18	0.88 ± 2.32	0.65 ± 0.76	0.48 ± 1.00
		F1 & F2	0.27 ± 0.27	0.08 ± 0.20	0.31 ± 0.31	1.42 ± 4.11	0.11 ± 0.85	0.65 ± 0.72
	Single	F1	0.04 ± 0.33	0.04 ± 0.32	0.85 ± 4.80	2.02 ± 4.30	1.01 ± 2.52	0.41 ± 0.69
80%	Dual	F1	0.09 ± 0.20	0.07 ± 0.10	0.57 ± 0.33	1.30 ± 2.73	1.15 ± 0.97	0.53 ± 1.10
		F1 & F2	0.62 ± 0.82	0.17 ± 0.16	0.46 ± 0.58	2.90 ± 4.46	0.75 ± 1.77	1.38 ± 0.96
	Single	F1	0.54 ± 1.52	0.08 ± 0.33	0.82 ± 5.10	2.45 ± 4.58	0.86 ± 4.93	0.67 ± 0.92

Tibia		Block	Δ x	Δ y	Δ z	Δ α	Δ β	Δ γ
20%	Dual	F1	0.01 ± 0.09	0.01 ± 0.06	0.03 ± 0.13	0.51 ± 0.44	0.15 ± 0.35	0.39 ± 0.32
		F1 & F2	0.06 ± 0.08	0.03 ± 0.07	0.04 ± 0.15	0.57 ± 0.50	0.32 ± 0.48	0.31 ± 0.33
	Single	F1	0.11 ± 0.28	0.43 ± 0.29	1.65 ± 4.84	0.42 ± 1.86	0.69 ± 1.42	0.24 ± 1.01
40%	Dual	F1	0.10 ± 0.13	0.03 ± 0.05	0.01 ± 0.18	1.37 ± 0.85	0.55 ± 0.43	0.86 ± 0.46
		F1 & F2	0.17 ± 0.13	0.01 ± 0.06	0.12 ± 0.22	1.98 ± 0.93	0.63 ± 0.77	0.83 ± 0.61
	Single	F1	0.04 ± 0.35	0.43 ± 0.44	0.97 ± 4.90	0.35 ± 3.57	1.53 ± 1.90	0.36 ± 1.37
60%	Dual	F1	0.11 ± 0.11	0.03 ± 0.06	0.15 ± 0.21	1.93 ± 1.00	0.99 ± 0.62	1.26 ± 0.66
		F1 & F2	0.13 ± 0.20	0.21 ± 0.08	0.33 ± 0.26	2.66 ± 1.17	0.71 ± 1.15	1.42 ± 0.68
	Single	F1	0.27 ± 0.54	0.15 ± 0.57	0.02 ± 5.10	0.35 ± 3.49	0.01 ± 3.15	0.59 ± 1.12
80%	Dual	F1	0.07 ± 0.12	0.01 ± 0.09	0.09 ± 0.49	1.71 ± 1.14	1.19 ± 0.74	1.18 ± 0.73
		F1 & F2	0.75 ± 0.32	0.28 ± 0.12	0.38 ± 0.59	0.88 ± 1.54	1.32 ± 2.54	0.65 ± 1.37
	Single	F1	0.97 ± 0.65	0.04 ± 0.70	0.55 ± 5.05	1.12 ± 3.73	3.29 ± 5.06	0.06 ± 1.29

However, using the dual image technique, if both the images were blocked by 20% in the medial/lateral direction, the automatic matching process could determine the femur position with an accuracy of 0.17 ± 0.10 mm in translation and 0.22 ± 0.25° in orientation and the tibia position with an accuracy of 0.59 ± 0.25 mm in translation and 1.37 ± 1.51° in orientation. When both the images were blocked by 80%, the femur position could be determined with an accuracy of 0.4 ± 0.91 mm in translation and 1.65 ± 3.01° in orientation and the tibia position with an accuracy of 0.88 ± 0.74 mm in translation and 3.1 ± 3.82° in orientation (Table 2). If both the images were partially blocked by 20% in the superior/inferior direction, the automatic matching process could determine the femur position with an accuracy of 0.07 ± 0.06 mm in translation and 0.76 ± 0.67° in orientation and the tibia position with an accuracy of 0.04 ± 0.15 mm in translation and 0.57 ± 0.5° in orientation. When the image was blocked by 80%, the femur position could be determined with an accuracy of 0.62 ± 0.82 mm in translation and 2.9 ± 4.46° in orientation and the tibia position with an accuracy of 0.75 ± 0.32 mm in translation and 1.32 ± 2.54° in orientation (Table 3).

Using the single fluoroscopic image technique, if the image was blocked by 20% in the medial/lateral direction, the automatic matching process could determine the femur position with an accuracy of 0.51 ± 5.26 mm in out-plane translation and 0.52 ± 3.55° in orientation and the tibia position with an accuracy around 0.01 ± 4.77 mm in out-plane translation and 1.41 ± 1.92° in orientation (Table 2). If the image was blocked by 20% in the superior/inferior direction, the automatic matching process could determine the femur position with an accuracy of 1.21 ± 4.54 mm in out-plane translation and 0.8 ± 2.31° in orientation and the tibia position with an accuracy around 1.65 ± 4.84 mm in out-plane translation and 0.42 ± 1.86° in orientation (Table 3).

Discussion

This study investigated the accuracy and precision of an automatic matching method in determination of spatial positions of the native knee using either single or dual fluoroscopic images. Using the dual fluoroscopic image technique, the data indicated that for the femur and tibia, their spatial positions could be determined with an accuracy and precision in less than 0.21 mm in translation and 0.41° in orientation in all the testing cases. The data also indicated that even if only part of the joint outline is available in fluoroscopic images, the algorithm could still accurately reproduce the joint positions. When single fluoroscopic images were used, the method could produce satisfactory joint positions in the imaging plane, but generating large variation in positions and orientations along the out-plane direction.

Various methods have been proposed to determine the joint positions and orientations using single fluoroscopic images (Banks and Hodge 1996; Hoff et al. 1998; Zuffi et al. 1999; Kanisawa et al. 2003; Mahfouz et al. 2003; Zihlmann et al. 2006; Acker et al. 2010; Scarvell et al. 2010; Tsai et al. 2010). Most of these studies were focused on the measurements of the kinematics of artificial joint implants, such as total knee arthroplasty. Banks et al. reported an accuracy of approximately 0.2 mm in the in-plane translations and approximately 1.3° in orientation using a TKA shape library (Banks and Hodge 1996). In a recent study, Acker et al. reported average errors within 1.3 mm in the in-plane translation and 2.1° in orientation of 3D TKA models using an automatic matching method (Acker et al. 2010). Kanisawa et al. studied the in vivo normal knee kinematics and reported an accuracy of 1.2 mm in in-plane translation and 0.8° in orientation (Kanisawa et al. 2003). While all these methods reported satisfactory results for in-plane motion, the error for out-plane motion has been reported from 0.65 to 4 mm. The data obtained using our method on single plane fluoroscopic images are consistent with the published data in determination of in-plane positions and orientations of the knee, but with larger errors in out-plane positions.

There are few studies that have reported on the validation of using DFISs to measure total knee arthroplasty kinematics where both a manual matching method and an automatic matching method have been examined (Kaptein et al. 2003; Bingham and Li 2006; Hanson et al. 2006). Recently, several studies have examined the accuracy of manual matching methods in reproducing knee (Li et al. 2008), ankle (Wan et al. 2008), spine (Wang et al. 2008) and shoulder (Massimini et al. 2010) positions in space. These data indicated that the accuracy and precision are within 0.4 mm in translation and 1.2° in orientation in the determination of joint positions in space. We found that the automatic matching method of this paper has improved the accuracy using the artificial fluoroscopic images (less than 0.21 mm in translation and 0.41° in orientation) when compared with the above reports. However, when using the actual fluoroscopic images, the automatic matching procedure has higher precision in determining joint position (<0.15 mm) and orientation (<0.4°) than the manual matching method (Massimini et al. 2010). It should be noted that in using the manual matching method, fewer times of matching were used than the 50 times of matching used in the automatic matching procedure due to the laborious nature of the manual matching procedure. This may cause certain differences in the resulted precisions.

The method developed in this paper minimises the distances between the actual bony outlines captured on the fluoroscopic images and the projection outlines of the 3D bony models. This is similar to that used by Bingham and Li (2006). The advantage of this method is the simplicity in practical application. However, this method does not fully utilise the internal structural information of the joint, such as the mass density distribution of the bone in the images. Several other papers have proposed to match the bony density distribution in the captured joint images by creating a bony density image using the 3D joint models (You et al. 2001; Scarvell et al. 2010; Tsai et al. 2010). Such method, however, requires the construction of 3D anatomic solid models of the joint using CT scan images. In general, its accuracy in determination of joint positions and orientations could reach 0.23 mm and 0.42°, respectively. These are close to the accuracy presented in this paper using the automatic matching method.

Considering the effect of initial guessing position on the optimisation procedure, we proposed to randomly choose 50 initial guesses using a Monte Carlo method for each pose of the joint. The results indicated a Gaussian distribution of the optimisation results (Figure 7). This indicated that different initial guesses do not generate a unique solution for the knee position using the automatic matching method. However, the output variation is relatively small when the dual fluoroscopic images were used, as evidenced by the small standard deviations of the results of the 50 trials (Tables 1–3). Therefore, in actual application of this method, we suggest to use 50 initial guess positions for optimisation of each pose and to calculate the means of the 50 convergent results as the position and orientation of the bone.

Figure 7.

The random error of initial guessing used in femoral test is ± 10 (mm) in translation and ± 10 (degree) in rotation. The 50 results obtained from these randomly chosen initial positions are the Gaussian distribution.

The fluoroscopic images of the actual test used in this study were obtained for a person performing stair ascending. The validation indicated that the automatic matching procedure is applicable to investigate the dynamic knee joint positions. However, in actual dynamic knee joint motion, the knee always moves in a large range in space. For example, the knee moves over 40 cm during stance phase of gait on a treadmill (Li et al. 2008). Therefore, the fluoroscopes may only capture part of the knee when the knee moves to the edge of the image intensifiers. There are studies that have proposed to move the fluoroscope with knee during knee motion to capture the entire knee joint motion (Zihlmann et al. 2006). However, it is in general a challenge to construct a moving fluoroscopic system and to synchronise it with the knee motion. We evaluated the sensitivity of the automatic matching procedure to the incomplete information of the knee on fluoroscopic images. Using dual fluoroscopic image technique, the automatic matching method can still reproduce accurate knee joint position in space even when one image was blocked by 80%. It showed much better precision when compared with the results obtained using the single image technique. When both images were partially blocked, the accuracy of the reproduced knee joint position was decreased with the increase in the amount of blocked images.

There are certain limitations in the current validation study. Due to the difficulty in determining a real gold standard position for the knee in space, we used the 3D knee models in computer to represent a gold standard to create ideal fluoroscopic images. This procedure might have the advantage to avoid errors from image segmentation and distortion and to provide a rigorous evaluation of the efficiency of the automatic matching process. We also created artificial fluoroscopic images using the 3D knee models. This procedure may have the advantage to avoid the errors from image distortion and focus on the effect of fluoroscopic image segmentation on the automatic matching algorithm. The actual knee images could only be used to evaluate the precision of the matching process as the exact positions of the living knee in space are unknown. With these tests, we are confident that the automatic matching process is accurate and repeatable in determination of spatial knee positions when combined with the dual fluoroscopic images of the knee. For application of the single fluoroscopic image method, if the in-plane knee poses are the main interests, it could be a viable method for knee kinematics study due to its simplicity in real application.

In summary, this paper presented an automatic matching method that utilised either single or dual fluoroscopic images to analyse the knee joint positions and orientations in space. Its accuracy and precision were evaluated using different tests. Furthermore, the method was used to examine the knee joint positions when only partial knee joint was captured by the fluoroscopic images. The data demonstrated that the automatic matching method, when using dual fluoroscopic images, is highly accurate in the determination of human knee joint spatial positions. Using single fluoroscopic image is appropriate in the determination of in-plane knee position variables, but generates large errors in out-plane variables.