Autonomous Frameless Spine Registration and Robot‐Assisted Surgery

ABSTRACT

Objective

Accurate and efficient patient registration remains a critical challenge in robot‐assisted spine surgery, where current methods often rely on invasive bone‐mounted frames or intraoperative imaging that increases radiation exposure and procedural complexity.

Method

This article presents a novel non‐invasive and contactless spine registration system that integrates a skin‐affixed, radiopaque floating‐point mask with a robot‐mounted monocular camera. The system was evaluated in phantom experiments to quantify registration accuracy and pedicle‐screw‐axis targeting performance. The system autonomously establishes the pose of the pedicle‐screw axis relative to the mask (patient) and surgical robot coordinate systems. A monocular camera mounted on the robot captures live images of the mask in the operating theater, enabling real‐time visual recognition and tracking of the mask's elements. Advanced segmentation and pose estimation algorithms compute the spatial relationship between the vertebrae and the mask, accounting for accurate angular displacement between preoperative imaging and intraoperative patient posturing (CT‐to‐OT shift).

Results

Experimental validation on anatomically curved spine phantoms demonstrates millimeter accuracy in pedicle‐screw path localization. The system facilitates accounting for changes in patient configuration from CT scan posture to OT surgical posture to provide an objective and accurate reference for image‐guided pedicle screw placement. It eliminates intraoperative imaging while relying on preoperative CT and the registration was completed in under 4 min per vertebra.

Conclusion

The proposed system facilitates accounting for changes in patient configuration from CT scanner posture to OT surgical posture to provide an objective and accurate reference for image‐guided screw placement, achieving registration within 4 min per vertebra. The invention facilitates frame‐free patient registration both in manual and robotic spine surgery. The work lays the foundation for clinical translation through future cadaveric and pilot studies. All results reported in this study are limited to controlled laboratory and phantom conditions. Clinical performance and intraoperative feasibility remain to be established in future patient studies.

Workflow of a contactless, frame‐free, robot‐assisted vertebra registration system. Preoperative CT data establish vertebra {V} and mask {M_pre} coordinate systems. Their relationship is computed, and intraoperatively the robot recognizes the skin‐affixed mask, defining {M_intra}. Tracking mask features enables CT‐to‐OT posture change estimation, ensuring accurate pedicle‐screw axis registration.

1. Introduction

Pedicle‐screw placement is one of the most common spine surgeries. In the United States, over 400,000 spine fusion cases are performed annually [1]. Minimally invasive surgery (MIS), using robot‐assisted or image‐based navigation with preoperative images, is increasingly adopted in spine fusion procedures, as it enhances surgical accuracy compared to conventional freehand methods [2, 3].

Robot‐assisted cranial and spine surgery integrates a variety of tools and devices to locate, visualize, and operate within the confined region of interest (ROI). The workflow for real‐time surgical guidance and execution requires two discrete registration processes: (1) image registration of the patient, which involves associating the ROI in preoperative CT images with respect to an external reference, and (2) physical registration of the patient, which aligns the same ROI in the operating theater (OT) relative to the same external reference. In image‐guided spinal surgery (IGSS), a reference frame is rigidly affixed to the patient's bone and serves as the spatial anchor for ROI. This external reference framework enables tracked instruments to be registered relative to the patient, thus facilitating real‐time navigation. Advances in this domain have led to the development of robot‐assisted systems, in which the spatial relation between the robot‐base coordinate system and the reference frame governs the navigation accuracy and guides the placement of surgical tools. In contrast, conventional freehand techniques rely extensively on the surgeon's proprioceptive skills and anatomical knowledge to internally model spatial relationships, in the absence of explicit external references or image guidance systems.

Commercially available IGSS consists of multiple integrated devices, such as a dynamic reference array (DRA), an electro‐optical (EO) tracking camera, and specialized surgical tools [4]. To ensure accurate patient registration, these systems align preoperative computed tomography (CT) data with real‐time intraoperative 2D or 3D X‐ray images, a process known as 3D2D or 3D3D image registration [5, 6]. A dedicated computer workstation executes the navigation software and acts as the primary interface for the surgical team.

Navigation remains precise provided that the spatial relationship between the frame and the patient is not altered. Because a frame is attached to a single vertebra or the pelvis, it may result in navigational errors if there is movement within the spine during surgery. Intervertebral mobility allows the spine to remain flexible, meaning the configuration of the vertebrae shifts based on the patient's overall posture and the positioning of their limbs [7, 8]. In the case of multilevel spinal surgery, several studies have suggested that accuracy is best maintained when screw placement occurs within two vertebral levels above or below the DRA. For extensive instrumentation, repeated surface matching and re‐registration are often recommended to minimize the error, although these steps prolong operative time [9].

The use of EO cameras is to track the DRA and the tool to facilitate 3D navigation [10]. DRAs are inherently bulky, requiring invasive bone exposure for precise anchoring. Additionally, when multiple vertebrae need to be operated on, the DRA has to be repeatedly anchored to different vertebrae and re‐registered, which can lead to additional bone exposure and higher time consumption and be labor‐intensive [11]. Such interventions may cause trauma and a more difficult recovery.

To simplify the workflow, researchers have proposed the use of non‐invasive, radiopaque, and optical skin markers in combination with intraoperative imaging for patient registration [12, 13, 14, 15]. These recent advancements have provided proof‐of‐concept for utilizing such approaches to enhance accuracy and reliability in spinal and neurosurgical navigation. Nevertheless, these methods have not yet rendered the OT entirely radiation‐free, as intraoperative imaging remains necessary for marker visualization and spatial registration.

Ongoing developments have directed research toward markerless registration and navigation. One such method is by tracking skin features using a multiview camera system, which has demonstrated sub‐millimeter accuracy [16]. The effectiveness of this approach depends on the detection of distinct skin features, such as moles and pigment spots. On the other hand, markerless skin registration using US‐CT/MR fusion, where 3D cameras and skin segmentation enable radiation‐free registration. However, it is sensitive to posture variation and camera placement, with reported errors of 7–17 mm [17]. Re‐attachable fiducial markers improve workflow by reproducible positioning across pre‐ and intra‐operative phases. The reported target registration errors (TRE) of up to 4.65 mm in abdominal phantoms [18] limit their applicability in spine surgery. In addition, inter‐vertebral mobility and soft tissue deformation remain major challenges in accurate patient registration for spinal surgery.

This article presents a novel frameless and contactless system for autonomous spine‐registration (patient registration). The method involves registering the patient's vertebrae using an externally skin‐affixed mask and a vision‐enabled robot. This mask serves as a non‐invasive common reference to relate the preoperative image space and the live patient space in the OT. The risk of surgical trauma or physiological pressure is eliminated by the non‐invasive, frameless protocol. All stages of the workflow, including registration and navigation, prior to the actual surgical intervention are performed using a non‐contact method. The setup at the OT is as shown in Figure 1. The mask is skin‐affixed externally and composed of radio‐opaque floating‐point elements arranged in a pattern. The system further integrates a two‐dimensional camera mounted on the end‐effector of the robot to capture intraoperative images of the mask. A dedicated algorithm has been developed to segment and identify each floating‐point element within both the preoperative CT and intraoperative camera images.

FIGURE 1.

Illustration of floating‐point mask [M], affixed on skin of patient and 2D camera [C], mounted on end‐effector [RP] of robot [RB].

The objective of the article is: (1) to develop segmentation and pose‐estimation algorithms, further provide an end‐to‐end surgical workflow for an autonomous, frameless, and contactless spine‐registration system; (2) to quantitatively evaluate the registration and pedicle‐screw targeting accuracy of the proposed robot‐assisted system on anatomically curved spine phantoms, using Target Registration Error (TRE) and Tool Placement Error (TPE) as primary performance metrics; (3) to investigate the utility of the floating‐point mask as a ground‐truth benchmark reference for a registration system that relies solely on preoperative CT, eliminating the need for bone‐mounted instruments and intraoperative radiological imaging in the operating theater.

The remainder of the article is organized so that Section 2 provides an overview of data and the methodology. Section 3 details the results and discussion. The conclusion is drawn in Section 4, giving an account of the benefits of the new approach and the method.

1.1. Symbol Table: Image Registration and Patient Registration

The article follows a consistent convention where position vectors use a left superscript to indicate the reference frame and a right subscript to denote the corresponding landmark, whereas rotation matrices employ a right superscript to specify the reference frame and a right subscript to denote the target frame Tables 1, 2, 3. The symbol $\left(\circ \right)$ represents matrix multiplication. Also, the term “pedicle‐medial axis” and “screw axis” is used interchangeably.

TABLE 1.

Coordinate frames.

Symbol	Definition
{C}	Preoperative CT machine reference frame
{V}	Vertebral coordinate frame (attached to the ith vertebra)
{M}	Skin‐affixed floating‐point mask coordinate frame
{RB}	Robot base coordinate frame
{RP}	Robot platform (end‐effector) coordinate frame
{CAM}	Monocular camera coordinate frame

TABLE 2.

Position vectors and landmarks.

Symbol	Definition
${}^{\mathrm{C}}\mathrm{S}_0\left(\mathrm{i}\right)$	Volumetric centre of the spinal canal (Origin of {V})
${}^{\mathrm{C}}\mathrm{B}_0\left(\mathrm{i}\right)$	Volumetric centre of the vertebral body
${}^{\mathrm{C}}\mathrm{L}_0\left(\mathrm{i}\right)\text{and}{}^{\mathrm{C}}\mathrm{R}_0\left(\mathrm{i}\right)$	Centres of the left and right pedicle isthmus
${}^{\mathrm{C}}\mathrm{M}_0\left(\mathrm{i}\right)$	Origin of the mask frame (surface centroid of top‐left element)
${}^{\mathrm{C}}V_0\left(\mathrm{i}\right)$	Origin of the vertebra frame
P	Pedicle‐medial axis
${}^{\mathrm{C}}{\mathrm{E}}_{\mathrm{j}}\text{and}{}^{\mathrm{OT}}{\mathrm{E}}_{\mathrm{j}}$	Position of mask element relative to CT frame and intraoperative OT frame
ρ	Radial displacement of the floating‐point

TABLE 3.

Transformations and operators.

Symbol	Definition
$\mathrm{V}^{\mathrm{C}}\left(\mathrm{i}\right)$	Rotation matrix of the i‐th vertebra relative to frame {C}
$\mathrm{V}^{\mathrm{M}}\left(\mathrm{i}\right)$	Orientation of the vertebra relative to the mask frame
${\mathrm{T}}_{{\mathrm{M}}^{\mathrm{C}}}^{{\mathrm{M}}^{\mathrm{OT}}}$	Homogeneous transformation mapping pre‐CT to intra‐OT mask pose
D	Deformation matrix (capturing rigid and non‐rigid effects)
λ	Regularization parameter for non‐rigid deformation
(°)	Matrix multiplication operator

2. Method

2.1. Overall Workflow of the Proposed Contactless Vertebra Registration System

The complete workflow of the proposed robot‐assisted contactless vertebra registration system is illustrated in Figure 2. The process begins with affixing external floating‐point masks to the skin above the targeted vertebrae. Thereafter, the acquisition of a preoperative CT scan of the patient is performed in the prone position. Subsequently, the 3‐D coordinates of both the feature points of the vertebra and the elements' floating‐point of the mask are extracted in CT image space, $\left\{\mathit{C}\right\}$ (Table 1). The mask serves as a reference, bridging the preoperative medical image space and the intraoperative patient space. During surgery, a robot‐guided monocular camera captures real‐time images of the same floating‐point mask affixed to the patient's posterior lumbar region. The distinct features (i.e., the presence of a floating‐point) of each element of the mask enable automatic pose detection, allowing their 3D coordinates to be reconstructed in the intraoperative patient space. The proposed algorithm then aligns the live mask system with the preoperative CT mask, thus computing the transformation between the intraoperative patient space and the preoperative image spaces. This transformation is then propagated to register the vertebra with the robot‐base without direct contact, enabling robot‐assisted spine surgery. The current practice in spinal image guidance systems utilizes preoperative CT scans and intraoperative 2D/3D imaging. These systems typically consist of multiple components, including a dynamic reference array (DRA), an electro‐optical camera array, and customized surgical instruments. DRAs are bulky, which requires significant bone exposure for accurate anchoring to the spine and high discomfort to the patient. Additionally, when multiple vertebrae need to be operated on, the DRA must be anchored to different vertebrae and repeatedly registered, which can lead to additional bone exposure and higher time consumption and be labour‐intensive. A computer workstation executes the system software and serves as the primary interface for the surgical team.

FIGURE 2.

An overview of the entire procedure with the proposed setup.

2.2. System Description and Surgical Setting

The main components of the setup consist of a spine phantom, a 5‐megapixel Dino‐Lite monocular camera, a six‐degree‐of‐freedom (6‐DOF) parallel robot, and a GPU‐enabled workstation. This article does not contain studies with animals or humans. For this type of study, formal consent is not required.

For the purpose of validation of the registration method and system, a standard phantom mimicking the human lumbar spine was fixated in a prone position inside a transparent acrylic enclosure, with lumbar lordosis centred at the L3 vertebra. This posture is assumed to be an operative posture in OT. Through‐holes were drilled in the pedicle regions of the L1 and L3 vertebrae, representing pedicle‐screw holes, prior to the positioning. The mask was manually centred and affixed on the superior surface of the phantom (representing human skin) above the targeted vertebral levels, ensuring that its elements conformed to the local anatomical curvature, as shown in Figure 3a. The surface was contoured to replicate the natural curvature of the human spine, as shown in Figure 3c.

FIGURE 3.

(a) Floating‐point mask affixed over the underlying L1 and L3 vertebrae of the spine, (b) the curvature of the superior surface of the phantom in the camera, and (c) a CT image, where the superior surface illustrates the curve (highlighted in red color).

A mask, composed of a cluster of seven elements (marker), is configured in an “I‐shape” within a 30.8 × 30.8 mm arrangement. Severijns et al. support placement of markers in a cluster because the relative arrangement of the cluster can help compensate for small landmarking mistakes [19]. While three non‐collinear elements are theoretically sufficient to define the mask frame $\left\{\mathit{M}\right\}$, seven (L1 and L3) and eight (L2 and L4) were arranged in a specific geometric pattern to account for skin motion and local deformations in OT, without increasing the footprint. This redundancy allows the system to validate registration by constructing multiple coordinate combinations, ensuring consistent and accurate localization of the pedicle‐screw axis. In addition, the marker must be placed closest to the spinous process of the vertebra. Morl et al. placed skin markers directly above the spinous processes, which are the most accurate for representing underlying vertebral positions and motions. Specifically, they report that the markers closest to spinous processes have the highest correlation with vertebral positions (r > 0.96) and very low skin/soft tissue shift compared to other placements [20]. Preoperative CT images were acquired with a pixel resolution of 0.46 mm× 0.46 mm and a slice thickness of 0.67 mm. Subsequently, the phantom was repositioned in the same prone position as during preoperative imaging and was firmly secured to the operating table to minimize intraoperative misalignment. Figure 3 illustrates the phantom, masks, and a preoperative CT image. {In these experiments, each mask is linked to the corresponding underlying vertebra and then used for intraoperative transformation computation using the segmented mask elements in the CT and real‐time images by the camera.} Each mask is mapped with its underlying vertebra in the preoperative CT. This coordinate map is then used to facilitate intraoperative registration, transforming real‐time camera images of the masks into the anatomical frame of the patient's vertebrae in the OT.

The masks, affixed to the phantom's surface above the target vertebra (Figures 1 and 3), incorporate elements (markers) known as bull's eye levels. Each element comprises a transparent, hermetically sealed disc‐shaped enclosure, measuring 6 mm in height and 10 mm in diameter, filled with low‐viscosity alcohol. It encloses a small air bubble that functions as a floating‐point of diameter $\approx$2.5 m. Alcohol was selected due to its low viscosity, minimal temperature sensitivity within the operating room range, and rapid settling characteristics under gravity. The enclosure is rigid and fully sealed, preventing evaporation or leakage and ensuring stable bubble dynamics throughout the procedure.

The selection of the element is crucial. Unlike conventional radio‐opaque markers that indicate only the position [21], the proposed markers convey both position and orientation information. The floating‐point is a gravity‐sensitive element whose position shifts dynamically in response to the local orientation, placed in the space of gravity and time. One‐time calibration maps the floating‐point position to the element's orientation, referenced to the gravity axis. Thus, a salient feature of the proposed marker is its high sensitivity to orientation, allowing accurate monitoring of positional and orientational variations over time.

Apart from bull's‐eye levels, other orientation‐measuring instruments include inertial‐based sensors, such as Inertial Measurement Units (IMUs), consisting of multiaxis accelerometers and gyroscopes, and sometimes with the addition of magnetometers [22]. These are referred to as non‐image‐based techniques, which are typically characterized by ease of use and reduced costs. They also provide high accuracy, although they may not be as accurate as image‐based techniques in some cases. They require initial calibration and positioning, after which real‐time sensor fusion algorithms interpret movement and orientation, thus maintaining registration without imaging [22]. However, frequent recalibration is often necessary to combat drift and maintain accuracy. Furthermore, real‐time monitoring necessitates a preliminary surgical procedure to affix the sensor to the ROI.

The six‐degree‐of‐freedom (6‐DOF) parallel robot is a 6‐6 UPS (Universal‐Prismatic‐Spherical) parallel kinematic mechanism. This configuration consists of six legs connected to two semi‐regular hexagonal planar structures, referred to as the base and the platform. Each leg is connected at one end to the base via a universal joint and at the opposite end to the platform using a spherical joint. Coordinate systems are at the centres of robot‐base $\left\{\mathit{RB}\right\}$ and the robot‐platform $\left\{\mathit{RP}\right\}$ [23]. To facilitate intraoperative tracking, the robot is equipped with a high‐resolution 5~MP camera mounted directly on the platform (Figure 1). This compact, 60 g camera has a small field of view (16 mm× 12 mm), 25× magnification, and 2 K resolution. The system's portable and compact design allows for easy integration into standard operating rooms. Traditional navigation systems often rely on distantly fixed overhead or wall‐mounted cameras that require a continuous, unobstructed global view. They are highly susceptible to tracking failures if surgical staff or equipment block the line‐of‐sight. By utilizing an “eye‐in‐hand” configuration, mounting the camera on the end‐effector, this system effectively eliminates line‐of‐sight constraints. Additionally, the camera can be orientated at multiple angles, providing comprehensive and diverse visualization of the surgical field within the confined spaces of the lumbar and thoracic spine.

For future clinical application, sterilization of the system components will follow established protocols: the floating‐point mask will be supplied as a single‐use pre‐sterilized disposable device using low‐temperature sterilization (ethylene oxide or hydrogen peroxide gas plasma), suitable for the hermetically sealed alcohol‐filled elements. The robot‐mounted monocular camera, operating in a non‐contact configuration, will be enclosed in a sterile drape consistent with standard practice for optical components in surgical robotic systems.

Lumbar spine surgeries are performed predominantly in the prone position, although specific case variations may occur depending on the surgical objective. Special body supports are often used to secure the patient in the prone position [24]. The prone approach offers several distinct clinical advantages. Smajic et al. justify that the prone position enhances spinal realignment and improves lateral retractor placement [25]. The practice strongly supports the proposed novel mask containing floating‐point elements in a pattern. A floating‐point element of size 10 mm is chosen, as it optimally fits into a pattern for the purpose of accurate real‐time patients. The elements allow an orientation range of ±5°, which is considered sufficient for spinal surgeries. Relvas‐Silva et al. demonstrated a statistically significant increase in global lumbar lordosis of 2.8° when comparing the supine to prone position (p = 0.005), while changes in lower lumbar lordosis did not reach statistical significance [26]. Consequently, the expected intervertebral angular differences between vertebral orientations observed in preoperative CT data and intraoperative camera images are ±1°.

In this study, it is crucial to recognize that patient preparation is deliberately performed in a posture that replicates the intended intraoperative alignment. The data simplicity and accuracy are enhanced by maintaining consistent configuration in the pre‐CT surgical setup, pre‐operative CT scan and intraoperative posture. This is achieved by aligning each of the intraoperative elements' floating‐point with the corresponding preoperative elements' floating‐point. During the patient's preparation for the CT scan, each element of the mask was meticulously placed to maintain the floating‐point approximately at the centre of each element, accommodating an orientation range of ±5°. The occurrence of skin sliding over bone (STA) is minimized by the patient preparation protocol, which recommends relaxed soft tissue positioning without skin stretch during all three‐configuration shown in Figure 4 The figure provides a conceptual schematic of the patient preparation and prone positioning workflow from pre‐CT to the OT. This illustration has been clinically validated by co‐authors to ensure procedural accuracy. It serves as a visual guide only and does not represent or derive from primary experimental data.

FIGURE 4.

The Conceptual illustration of consistent patient positioning from pre‐CT acquisition to OT (the figure is partially generated by AI).

2.3. Registration Module

The autonomous spine‐registration algorithm comprises two key modules: the image‐registration module and the patient‐registration module. The image‐registration module is performed on a patient‐specific CT scan. The output of the module gives position and orientation, that is, the 3D transformation of each vertebra $\left\{\mathit{V}\left(\mathit{i}\right)\right\}\mathit{,}\mathit{i}\mathit{\in }\left\{\mathbf{1}\mathbf{,}\mathbf{2}\mathit{,}\mathbf{3}\mathit{,}\dots \right\}$, with respect to the external floating‐point mask coordinate system $\left\{\mathit{M}\left(\mathit{i}\right)\right\}$ (Table 1). When the patient is shifted to the operating theater (OT), the posture of the patient along with the underlying spine may change, altering the floating‐points' position. Within the OT, the patient‐registration module performs real‐time measurement of floating‐points and establishes them $\left\{\mathit{M}\right\}$ with respect to the robot, followed by the registration of the robot to the intraoperative pose of the vertebra. Moreover, this module establishes a correlation between the transformations obtained in the medical image space and those in the real patient space. The section mentioned below explains in detail how the floating‐points' position and orientation are integrated into the registration system.

The contribution of this work is the development of a system comprising a robot‐mounted monocular camera and a novel floating‐point mask combined with an algorithm for accurate and contactless spine‐registration. This approach has the potential to eliminate the necessity for intraoperative 2D/3D imaging and repeated anchoring of bone‐mounted frames during multilevel spine surgery.

2.3.1. Image Registration Module

In CT image space, a vertebra is segmented and an algorithm autonomously establishes a coordinate frame $\left\{\mathit{V}\right\}$ attached to the vertebra. The frame is established using feature points autonomously identified in medical images, as shown in Figure 5 and previously described in [27].

FIGURE 5.

Location of vertebral frame {V} and pedicle‐medial axis frame {L} with respect to {C} is illustrated over point cloud of L2 vertebra.

The positions vectors $\left[{}^{\mathit{C}}\mathit{S}_{\mathbf{0}}\left(\mathit{i}\right),{}^{\mathit{C}}\mathit{B}_{\mathbf{0}}\left(\mathit{i}\right),{}^{\mathit{C}}\mathit{L}_{\mathbf{0}}\left(\mathit{i}\right)\text{and}{}^{\mathit{C}}\mathit{R}_{\mathbf{0}}\left(\mathit{i}\right)\right]$ Table 2 represents the volume centre of the spinal canal, the vertebral body centre, and the centres of the left and right pedicle isthmus for the ith vertebra, expressed in the CT machine reference frame $\left\{\mathit{C}\right\}$ (indicated by left superscript). The vertebral coordinate system of the ith vertebra is represented as ${\mathit{V}}^{\mathit{C}}\left(\mathit{i}\right)\left({\widehat{\mathit{X}}}_{{\mathit{V}}_{\mathit{i}}},{\widehat{\mathit{Y}}}_{{\mathit{V}}_{\mathit{i}}},{\widehat{\mathit{Z}}}_{{\mathit{V}}_{\mathit{i}}}\right)$ Table 3, is defined with its origin at ${\mathit{V}}_{\mathbf{0}}$, corresponding to the volumetric centre of the spinal canal ${}^{\mathit{C}}\mathit{S}_{\mathbf{0}}\left(\mathit{i}\right)$. The coordinate system ${\mathit{V}}^{\mathit{C}}\left(\mathit{i}\right)$ is established within the medical image space and expressed relative to the CT reference frame $\left\{\mathit{C}\right\}$ (indicated by right superscript). The unit vectors ${\widehat{\mathit{X}}}_{{\mathit{V}}_{\mathit{i}}}\mathit{,}{\widehat{\mathit{Y}}}_{{\mathit{V}}_{\mathit{i}}}\mathit{,}{\widehat{\mathit{Z}}}_{{\mathit{V}}_{\mathit{i}}}$ specify the principal directions of the vertebra, expressed in the CT frame $\left\{\mathit{C}\right\}$. The unit vectors defining, ${\mathit{V}}^{\mathit{C}}\left(\mathit{i}\right)$, are computed as follows:

$$\begin{array}{c}{\widehat{\mathrm{Y}}}_{{\mathrm{V}}_{\mathrm{i}}}=\frac{\left({}^{\mathrm{C}}\mathrm{B}_0\left(\mathrm{i}\right)-{}^{\mathrm{C}}\mathrm{S}_0\left(\mathrm{i}\right)\right)}{\left|{}^{\mathrm{C}}\mathrm{B}_0\left(\mathrm{i}\right)-{}^{\mathrm{C}}\mathrm{S}_0\left(\mathrm{i}\right)\right|};\widehat{{\mathrm{X}}_{{\mathrm{V}}_{\mathrm{i}}}^{\prime }}=\frac{\left({}^{\mathrm{C}}\mathrm{R}_0\left(\mathrm{i}\right)-{}^{\mathrm{C}}\mathrm{L}_0\left(\mathrm{i}\right)\right)}{\left|{}^{\mathrm{C}}\mathrm{R}_0\left(\mathrm{i}\right)-{}^{\mathrm{C}}\mathrm{L}_0\left(\mathrm{i}\right)\right|};\hfill \\ {\widehat{\mathrm{Z}}}_{{\mathrm{V}}_{\mathrm{i}}}=\widehat{{\mathrm{X}}_{{\mathrm{V}}_{\mathrm{i}}}^{\prime }}\times {\widehat{\mathrm{Y}}}_{{\mathrm{V}}_{\mathrm{i}}};{}^{\mathrm{C}}\widehat{{\mathrm{X}}_{{\mathrm{V}}_{\mathrm{i}}}}={\widehat{\mathrm{Y}}}_{{\mathrm{V}}_{\mathrm{i}}}\times {\widehat{\mathrm{Z}}}_{{\mathrm{V}}_{\mathrm{i}}}\hfill \end{array}$$ (1)

Stacking three unit vectors as a column of 3 × 3 matrix in the order, ${\widehat{\mathrm{X}}}_{{\mathrm{V}}_{\mathrm{i}}},{\widehat{\mathrm{Y}}}_{{\mathrm{V}}_{\mathrm{i}}}\text{and}{\widehat{\mathrm{Z}}}_{{\mathrm{V}}_{\mathrm{i}}}$, yields rotation matrix that describes the ith vertebra frame $\left\{\mathit{V}\right\}$. with respect to CT frame $\left\{\mathit{C}\right\}$.

$$\begin{array}{c}{\mathrm{V}}^{\mathrm{C}}\left(\mathrm{i}\right)={\left[{\widehat{\mathrm{X}}}_{{\mathrm{V}}_{\mathrm{i}}} {\widehat{\mathrm{Y}}}_{{\mathrm{V}}_{\mathrm{i}}} {\widehat{\mathrm{Z}}}_{{\mathrm{V}}_{\mathrm{i}}}\right]}_{3\mathrm{x}3}=\left[\begin{array}{ccc}{\mathrm{X}}_{{\mathrm{V}}_{\mathrm{i}}}\circ {\mathrm{X}}_{\mathrm{C}}\hfill & {\mathrm{Y}}_{{\mathrm{V}}_{\mathrm{i}}}\circ {\mathrm{X}}_{\mathrm{C}}& {\mathrm{Z}}_{{\mathrm{V}}_{\mathrm{i}}}\circ {\mathrm{X}}_{\mathrm{C}}\\ {\mathrm{X}}_{{\mathrm{V}}_{\mathrm{i}}}\circ {\mathrm{Y}}_{\mathrm{C}}& {\mathrm{Y}}_{{\mathrm{V}}_{\mathrm{i}}}\circ {\mathrm{Y}}_{\mathrm{C}}& {\mathrm{Z}}_{{\mathrm{V}}_{\mathrm{i}}}\circ {\mathrm{Y}}_{\mathrm{C}}\\ {\mathrm{X}}_{{\mathrm{V}}_{\mathrm{i}}}\circ {\mathrm{Z}}_{\mathrm{C}}& {\mathrm{Y}}_{{\mathrm{V}}_{\mathrm{i}}}\circ {\mathrm{Z}}_{\mathrm{C}}& {\mathrm{Z}}_{{\mathrm{V}}_{\mathrm{i}}}\circ {\mathrm{Z}}_{\mathrm{C}}\end{array}\right]\hfill \end{array}$$ (2)

The image registration algorithm computes the pose of the pedicle‐medial axis $\left(\mathit{P}\right)$ relative to the vertebral coordinate frame $\left\{\mathit{V}\right\}$, detailed as below, enabling precise intraoperative guidance.

$${\left[{\mathrm{P}}^{\mathrm{V}}\left(\mathrm{i}\right)\right]}_{3\mathrm{x}3}={\left[{\mathrm{V}}^{\mathrm{C}}\left(\mathrm{i}\right)\right]}^{\top }\circ {\left[{\mathrm{P}}^{\mathrm{C}}\left(\mathrm{i}\right)\right]}_{3\mathrm{x}3}$$ (3)

$${\left[{}^{\mathrm{V}}\mathrm{P}_0\left(\mathrm{i}\right)\right]}_{3\mathrm{x}1}={\left[{\mathrm{V}}^{\mathrm{C}}\left(\mathrm{i}\right)\right]}^{\top }\circ {\left[{}^{\mathrm{C}}\mathrm{P}_0\left(\mathrm{i}\right)-{}^{\mathrm{C}}\mathrm{V}_0\left(\mathrm{i}\right)\right]}_{3\mathrm{x}1}$$ (4)

where ${\mathrm{P}}^{\mathrm{V}}\left(\mathrm{i}\right)$ describes the rotation matrix of the screw axis and ${}^{\mathrm{V}}\mathrm{P}_0\left(\mathrm{i}\right)$ describes position of the screw axis relative to ith vertebra. The notation ${\mathit{P}}_{\mathbf{0}}$ serves as a general representation for ${\mathit{L}}_{\mathbf{0}}$ and ${\mathit{R}}_{\mathbf{0}}$, corresponding to the left and right pedicles, respectively.

Until now, the article has presented the algorithm for localizing the vertebral frame and pose of the screw axis within the medical image space, independently of the CT reference frame $\left\{\mathit{C}\right\}$. In the following section, the compound transformation enabling the computation of the pedicle‐screw axis pose $\left\{\mathit{P}\right\}$ with respect to the externally skin‐affixed mask frame $\left\{\mathit{M}\right\}$ is described in detail.

Sequentially, segregation of the masks from CT scan was initiated. A marching squares algorithm [28] was applied to segment both the physical elements of the mask and their floating‐point in each slice. Figure 6 illustrates an axial slice with the segmented boundaries of a mask element and its associated floating‐point (highlighted in the enlarged view). The boundaries of these segmented regions were identified and measured in the CT coordinate frame $\left\{\mathit{C}\right\}$ and represented as point clusters. Sequential stacking of these clusters along the Z‐axis of the CT frame $\left\{\mathit{C}\right\}$ produces a 3D point cloud of the mask. Further, each element and its corresponding floating‐point were then segmented and hierarchically labeled using the DBSCAN algorithm [29], a density‐based clustering method, as shown in Figure 7. Subsequently, the algorithmic steps were applied for the consistent identification of individual elements and for the formulation of a coordinate system for the ith mask $\left\{{\mathit{M}}^{\mathit{C}}\right\}$ relative to frame $\left\{\mathit{C}\right\}$.

FIGURE 6.

A CT axial slice containing an element of the mask over the underlying L1 vertebra and the floating‐point of the element in an enlarged view.

FIGURE 7.

3D point cloud of segmented and hierarchically labeled elements (color‐coded) forming a mask.

During the point cloud processing step, each mask and its constituent elements were autonomously identified and hierarchically arranged in ascending order along the X and Z axes (from right to left and from feet to head). Each element was then represented by the centroid of its top surface, referred to as the surface centroid, as illustrated in Figure 8.

FIGURE 8.

Illustration of a surface centroid (highlighted) of the top surface (in magenta) of a segmented element (in green).

Thereafter, the group of surface centroids corresponding to each mask was autonomously clustered, facilitating the construction of a mask‐specific coordinate system $\left\{{\mathit{M}}^{\mathit{C}}\left(\mathit{i}\right)\right\}$. The centroid is sensitive to the position and orientation of the element. Since each vertebra has a different orientation, the mask conforms to the orientation of the vertebra. Accordingly, the mask frame $\left\{{\mathit{M}}^{\mathit{C}}\left(\mathit{i}\right)\right\}$, represents the unique orientation of that vertebra. Henceforth, the analytical formulation of the vertebra coordinate system relative to the mask, $\left\{{\mathit{V}}^{\mathit{M}}\left(\mathit{i}\right)\right\}$, is discussed.

The frame ${\mathit{M}}^{\mathit{C}}\left(\mathit{i}\right)\left({\widehat{\mathit{X}}}_{{\mathit{M}}_{\mathit{i}}},{\widehat{\mathit{Y}}}_{{\mathit{M}}_{\mathit{i}}},{\widehat{\mathit{Z}}}_{{\mathit{M}}_{\mathit{i}}}\right)$ Table 3 is attached to the ith mask with respect to frame $\left\{\mathit{C}\right\}$. The unit vectors ${\widehat{\mathit{X}}}_{{\mathit{M}}_{\mathit{i}}}$, ${\widehat{\mathit{Y}}}_{{\mathit{M}}_{\mathit{i}}}$, ${\widehat{\mathit{Z}}}_{{\mathit{M}}_{\mathit{i}}}$ give the principal directions of the mask frame. The origins, ${}^{\mathrm{C}}\mathrm{M}_0\left(i\right)$, of each mask are autonomously and uniquely defined at the surface centroid of the top‐left element of the mask, referenced to the patient. The relative orientation and position of the ith vertebra frame $\left\{\mathit{V}\right\}$ with respect to the mask frame $\left\{\mathit{M}\right\}$ is given by Equations (5) and (6). Combining these, the homogeneous transformation matrix describing the pose of $\left\{\mathit{V}\right\}$ relative to $\left\{\mathit{M}\right\}$ as computed from the preoperative CT scan is given by Equation (7).

$${\left[V^M\left(\mathrm{i}\right)\right]}_{3\mathrm{x}3}={{\left[M^{\mathrm{C}}\left(\mathrm{i}\right)\right]}^{\top }}_{3x3}\circ {\left[V^{\mathrm{C}}\left(\mathrm{i}\right)\right]}_{3\mathrm{x}3}$$ (5)

$${\left[{}^{M}V_0\left(\mathrm{i}\right)\right]}_{3\mathrm{x}1}=-{{\left[M^{\mathrm{C}}\left(\mathrm{i}\right)\right]}_{3x3}}^{\top }\circ {\left[{}^{\mathrm{C}}M_0\left(\mathrm{i}\right)-{}^{\mathrm{C}}\mathrm{V}_0\left(\mathrm{i}\right)\right]}_{3\mathrm{x}1}$$ (6)

$$T_V^M\left(i\right)={\left[\begin{array}{cc}{{\left[M^{\mathrm{C}}\left(\mathrm{i}\right)\right]}^{\top }}_{3x3}\circ {\left[V^{\mathrm{C}}\left(\mathrm{i}\right)\right]}_{3\mathrm{x}3}& -{{\left[M^{\mathrm{C}}\left(\mathrm{i}\right)\right]}_{3x3}}^{\top }\circ {\left[{}^{\mathrm{C}}M_0\left(\mathrm{i}\right)-{}^{\mathrm{C}}\mathrm{V}_0\left(\mathrm{i}\right)\right]}_{3\mathrm{x}1}\\ 0_{1x3}& 1\end{array}\right]}_{4x4}$$ (7)

2.3.2. Patient Registration Module

Up to this point, the vertebra coordinate system $\left\{V\right\}$ is established with respect to the frame $\left\{C\right\}$ and its pose is estimated relative to the mask frame $\left\{M\right\}$, thus completing the vertebral image registration. The resulting transformation $T_V^M$, computed by the Image Registration Module, is then forwarded to the Patient Registration Module to enable real‐time tracking.

In the operating theater (OT), the camera mounted on the robotic platform functions as the visual sensing unit of the system. The lateral position (X, Y) of the camera with respect to the robot‐platform $\left\{\mathit{RP}\right\}$ is calibrated at the author's lab, and the accuracy is reported in Section 3. The spatial relationship between the camera coordinate frame $\left\{\mathit{CAM}\right\}$, and the robot base coordinate frame $\left\{\mathit{RB}\right\}$ is established through the robot's forward kinematic model. The robot possesses an initial estimate of the patient's posture based on the preoperative setup. At the start of the procedure, the robot is positioned such that the camera's field of view encompasses the top‐left reference element of the mask. The robot sequentially moves to each mask element, ensuring that the camera's field of view captures one element at a time. This motion follows a predetermined sequence based on their selection in the medical image space, guiding the robot to move between element locations in physical space. At each element, the robot pauses to acquire an image and initiate an algorithm that segments the concentric circle and the floating‐point (white glossy spot), as illustrated in Figure 9. The image is taken from a plane orthonormal to the element's surface at a calibrated working depth.

FIGURE 9.

Camera image of an element showing the fixed concentric circle (in red) and the floating point (in blue), captured along the surface normal.

Patient preparation is initiated prior to the preoperative CT acquisition. The patient's posture at this stage is referred to as the pre‐CT surgical setup, which is deliberately configured to match the intended intraoperative posture in the operating theater. In this setup, external floating‐point masks are affixed to the skin overlying the target vertebrae while the patient maintains the desired intraoperative posture. While affixing the mask, a specialized guiding template is used to ensure that the 2 mm air bubbles within all seven elements are initially positioned at their respective centres (see Figure 3a). Aligning the elements' floating‐point to its centre is not mandatory, but doing so will provide a well‐defined reference and maximum orientation range (±5°). This deliberate placement aims to reduce, or potentially eliminate, soft tissue artifact (STA) in the vicinity of the surgical site. The relative motion between skin‐mounted markers and bony anatomy is commonly referred to as a soft tissue artifact (STA).

To facilitate accurate and repeatable posture alignment, the elements' floating‐point initial reference is noted. By utilizing the floating‐point as a precise reference, the clinical team can realign the patient's body into a near‐identical posture during both the CT acquisition and the subsequent intraoperative in OT. This approach is supported by prior work: Mörl and Blickhan minimized apparent STA by placing markers during seated flexion, thereby aligning skin markers with the vertebrae in the flexed posture and achieving negligible relative motion error during static flexion tests [30]. Likewise, Johnson et al. reported reduced STA near the neutral posture when markers were applied with the subject in an upright position [31].

Despite the precautions taken during patient preparation, transferring a patient from the pre‐CT surgical setup to the intraoperative setup may introduce orientation shifts and skin deformation relative to the underlying vertebrae. This phenomenon is often attributed to the hyperelastic behavior of the epidermal and dermal layers under low tensile loading [32]. Given that the floating‐points are gravity‐sensitive points, the setup enables the system to detect and measure rotational displacements from the initial reference with arcminute resolution. This high sensitivity is critical for managing intervertebral orientational variability, which has been recorded between 0.4° and 0.6° in trial‐to‐trial repeatability of intervertebral sagittal rotation across L1‐S1 of clinical populations [33].

To account for these posture‐related deformations, the system employs a mathematical model to compensate for STA relative to the CT acquisition posture, outputting the result as a precise change in vertebral orientation. A representative pattern of floating‐point displacement, observed across various cases, is presented in Figure 10, where the red circles denote the positions in the preoperative CT and the green circles indicate the intraoperative positions. Figure 10 further provides a detailed graphical depiction of the decomposition of these shifts into rigid and non‐rigid components. Figure 10a,b demonstrates a rigid rotation of the vertebra that is characterized by all elements' floating‐points moving in a single, unique direction, and (c‐h) illustrates the positions of floating‐points under the influence of STA, where the skin has deformed about a specific axis.

FIGURE 10.

Decoupling of floating‐point displacements into rigid rotation (a–b) and non‐rigid deformation (c–h), comparing preoperative CT positions (red) to intraoperative positions (green).

The experiments to detect and decompose the non‐rigid STA were conducted separately from the phantom. The experiments were conducted on an elastic substrate with mechanical properties approximating human skin soft tissue (material: silicone, thickness: 0.1 mm). The floating‐point mask was affixed on this substrate in the same element pattern as used in the phantom experiment. Controlled compressive, tensile, and bending forces were applied quasi‐statically along multiple axes to generate deformation patterns covering the range of STA modes described in the literature [34, 35] which include mask size change, shape distortion, and opposing floating‐point displacement directions across element subsets. These experiments validated the system's capacity to detect and classify rigid versus non‐rigid floating‐point displacement patterns, as illustrated in Figure 11. The rigid body rotation experiment (Figure 11a,b) was conducted by tilting the entire mask as a unit, while the non‐rigid STA experiment (Figure 11c–h) was conducted by locally deforming the elastic substrate beneath specific element subsets while keeping the overall mask frame stationary. Also, there might be cases where floating‐points undergo non‐uniform displacements of differing magnitudes and directions, indicating differential skin deformation relative to the underlying vertebra which flags the data for inconsistency. By mathematically isolating these components, the algorithm suppresses the non‐rigid deformation component by setting the regularization parameter λ = 0, ensuring that the underlying rigid bony structures are tracked with sub‐millimeter accuracy despite any superficial skin movement (STA).

FIGURE 11.

Experimental demonstration of decoupling of floating‐point displacements into rigid rotation (a–b) and non‐rigid deformation (c–h), simulated in an OT‐like setup.

The patient registration model is formulated through the transformation ${\mathit{T}}_{{\mathit{M}}^{\mathit{C}}}^{{\mathit{M}}^{\mathit{OT}}}$, which maps the preoperative mask frame $\left\{{\mathit{M}}^{\mathit{C}}\right\}$ (extracted from the CT data) with respect to the intraoperative mask frame $\left\{{\mathit{M}}^{\mathit{OT}}\right\}$ (captured in the OT) for the ith mask. This transformation accounts for posture‐induced changes between the two configurations and represented in homogeneous coordinates as:

$${\mathit{T}}_{{\mathit{M}}^{\mathit{C}}}^{{\mathit{M}}^{\mathbf{OT}}}=\left[\begin{array}{cc}D{\left(\lambda \right)}_{3x3}& t_{3x1}\\ 0_{1x3}& 1_{1x1}\end{array}\right]$$ (8)

where $t\in R^{\mathbb{3}}$ represents the translation component and $D\in R^{\mathbb{3}\times \mathbb{3}}$ is the matrix capturing the combined rigid and non‐rigid effects. To regulate the contribution of non‐rigid deformation, a regularization deformation parameter $\lambda \in \left[0,1\right]$. λ is a design parameter, not estimated from data. It linearly regulates the contribution of non‐rigid deformation in the matrix according to given Equation (9).

$$D\left(\lambda \right)=R\left(I+\lambda \left(U-I\right)\right)$$ (9)

where $R\in \mathit{SO}\left(3\right)$ is a rotation matrix that denotes the rigid‐body rotation and $U\in R^{\mathbb{3}\times \mathbb{3}}$ is a symmetric positive‐definite matrix capturing non‐rigid deformation due to STA. Setting $\lambda =1,$ the full estimated deformation is modeled. In this case, the Matrix $D$ becomes deformation gradient that can be factorized using polar decomposition [36] as follows:

$$D\left(1\right)=R_{3\times 3}\circ U_{3\times 3}$$ (10)

Conversely, when $\lambda =0$, non‐rigid deformation is completely suppressed, such that every point on the skin mapped under purely rigid rotation $R$ and the deformation matrix reduces to a purely rigid motion.

$$D\left(0\right)=R_{3\times 3}$$ (11)

For a given $\lambda$, the optimal transformation (see Equation 8) is obtained by minimizing the residual between preoperative mask elements ${}^{\mathit{C}}{\mathit{E}}_{\mathit{j}}$ and intraoperative mask elements ${}^{\mathit{OT}}{\mathit{E}}_{\mathit{j}}$.

$$\underset{D,t}{\mathit{min}}\sum _j^S{‖D\left(\mathrm{\lambda }\right)\circ {}^C{E}_j+t-{}^{\mathit{OT}}{E}_j‖}^2$$ (12)

where $\mathit{S}$ is numbers of elements. Because the mask comprises seven elements, more than the three non‐collinear points theoretically required for rigid registration, the system incorporates inherent redundancy. This redundancy allows validation via subset consistency analyses, in which the transformation is recomputed from various elements combinations.

Finally, the intraoperative vertebral pose relative to the robot base frame $\left\{\mathit{RB}\right\}$ is then computed via a compound transformation, as shown in Equation (13).

$$T_{V^c}^{\mathit{RB}}=\underset{\text{Mask to Robot}}{\underbrace{T_{M^{\mathrm{OT}}}^{\mathit{RB}}}}\circ \underset{\text{Vertebra to Mask}\left(\mathrm{OT}\right)}{\underbrace{T_{M^C}^{M^{\mathit{OT}}}\circ T_{V^{\mathrm{C}}}^{M^C}}}$$ (13)

The transformations $T_{V^C}^{\mathit{RB}},T_{M^{\mathit{OT}}}^{\mathit{RB}},T_{M^C}^{M^{\mathit{OT}}}\mathit{\text{and}}{T}_{V^{\mathrm{C}}}^{M^C}$ are constructed following the convention where each transformation maps the subscript frame with respect to the superscript reference frame.

In the OT, the robot navigates in a plane perpendicular to the gravity axis. However, the anatomical curvature of the human back necessitates adaptive positioning, as the mask elements do not reside on a single flat plane. To address this, the robot iteratively adjusts its depth along the surface normal of each element using image‐derived feedback (based on sharpness and feature dimensions), ensuring an optimal working depth. This working depth is defined as the separation between the camera frame $\left\{\mathrm{CAM}\right\}$ and the element at optimal sharpness under 25× magnification.

To maintain this precise working depth, the element's orientation is estimated using a pre‐calibrated model (sensitivity constant, $\kappa$ = 27.11 mm/degree) that relates floating‐point radial displacement (ρ) to dual‐axis rotations $\left(\mathrm{\theta },\varphi \right)$. Further, the inverse kinematics of the robot are utilized to compute the platform pose $\left\{\mathit{RP}\right\}$ such that the vision system remains along the normal of the element during measurement, accurately measuring the 3‐D position (X, Y, Z) of each element relative to the robot‐base $\left\{\mathit{RB}\right\}$ thereby allowing us to compute ${\mathit{T}}_{{\mathit{M}}^{\mathit{C}}}^{{\mathit{M}}^{\mathit{OT}}}$.

2.3.3. Summary for Registration Module

This work presents a novel registration approach for robot‐assisted pedicle screw placement. The method introduces a custom floating‐point mask comprising multiple bull's‐eye level elements arranged in a fixed configuration. The masks are attached to the skin overlying adjacent vertebral bodies prior to imaging in intended intraoperative posture.

The geometric features of each mask element, including both the outer level structure and the enclosed floating‐point, are detected in the preoperative CT scan and subsequently in images acquired by a robot‐guided camera. These correspondences are used to register the mask frames from CT coordinate space to the robot coordinate space. In addition, changes in the floating‐point positions enable continuous detection of posture‐related variations in mask orientation and position.

Because the pedicle screw axis is defined relative to the vertebral anatomy in CT space, the established registration allows this trajectory to be expressed directly in the robot coordinate frame. The robot can therefore assist in accurate pedicle screw placement without requiring intraoperative imaging.

The feasibility and accuracy of the proposed approach are demonstrated through experiments on a spine phantom. Figure 12 shows the flowchart for the robot‐assisted autonomous registration.

FIGURE 12.

Robot‐assisted autonomous registration and navigation.

2.4. Performance Metrics

The source of total error has been resolved across multiple layers of the system, ranging from data preparation to measurement and alignment through robotic intervention. The end‐to‐end task accuracy metrics are reported at three major distinct levels: (i) robot and camera positioning accuracy, (ii) data preparation and segmentation error in medical images (e.g., marker localization), and (iii) guidance accuracy during the terminal approach were quantified using Target Registration Error (TRE) and Tool Placement Error (TPE) Table 4. The flowchart in Figure 13 illustrates the source and propagation of these errors, detailing how individual contributions from each layer influence the end task.

TABLE 4.

Performance Metrics.

Symbol	Definition
TRE	Target Registration Error
TPE	Tool Placement Error
RMSE	Root‐Mean‐Square error

FIGURE 13.

Illustration of how micrometer‐scale component precision propagates to millimeter‐scale system‐level precision.

2.4.1. Target Registration Error

Let ${}^{\mathit{RB}}\mathit{L}\in R^{\mathbb{3}}$ denote the landmark point expressed in the robot base coordinate frame $\left\{\mathbf{RB}\right\}$. The actual position of this landmark, obtained from CT‐based segmentation and registration, is denoted by ${}^{\mathit{RB}}\mathit{L}\left(\mathit{act}\right)\in R^{\mathbb{3}}$. The computation for the landmark is given by:

$${}^{\mathit{RB}}L=\underset{\text{Mask to Robot}}{\underbrace{T_{\mathit{CAM}}^{\mathit{RB}}}}\circ {}^{\mathit{CAM}}L$$ (14)

$${}^{\mathit{RB}}L\left(\mathit{act}\right)=\underset{\text{Mask to Robot}}{\underbrace{T_{M^{\mathit{OT}}}^{\mathit{RB}}\circ T_{M^{\mathrm{C}}}^{M^{\mathit{OT}}}}}\circ \underset{\mathrm{CT}\text{to Mask}}{\underbrace{T_C^{M^c}\circ {}^{C}L}}$$ (15)

The Target Registration Error (TRE) is defined as:

$$\mathit{TRE}=‖{}^{\mathit{RB}}L\left(\mathit{act}\right)-{}^{\mathit{RB}}L‖$$ (16)

where $\mid ·\mid$ denotes the Euclidean (straight‐line) distance. If multiple landmark points $L\left(\mathrm{act}\right)$ are to be evaluated (e.g., several vertebral landmarks or pedicle screw targets), the root‐mean‐square (RMS) of TRE is used and defined as:

$$\mathit{TRE}\left(\mathit{RMS}\right)=\sqrt{\frac{1}{S}\sum _{j=1}^S{‖{}^{\mathit{RB}}L\left(\mathit{act},j\right)-{}^{\mathit{RB}}L\left(j\right)‖}^2}$$ (17)

where $\mathit{S}$ denotes the number of targets (i.e., sample size).

2.4.2. Target Placement Error

The tool placement error (TPE) is defined as the vector sum of its translational and orientational components.

$$\mathit{TPE}\left(\mathit{RMS}\right)={\mathit{TPE}}_{\mathit{ang}}+{\mathit{TPE}}_{\mathit{ang}}$$ (18)

Translational error captures lateral deviation of the tool trajectory, while orientational error captures angular misalignment that accumulates with insertion depth.

The TPE was evaluated at anatomically significant targets and is as shown in Figure 14: (i) the desired pedicle entry point (intersection of the pedicle‐screw axis ($\mathit{P}$) with the posterior cortical surface) and (ii) the desired screw tip located along the pedicle axis. The desired (ground‐truth) screw axis is represented by a unit direction vector, $\widehat{P}$ which joins the entry point $P_S$ and pedicle endpoint $P_E$. In the physical operating space, the actual trajectory achieved by the robot is represented by $\widehat{P}\left(\mathit{act}\right)$ passing through the computed points $P_S\left(\mathit{act}\right)$, $P_E\left(\mathit{act}\right)$. The screw axis orientational error is given by the angle $\mathrm{\Delta }{\mathrm{\theta }}_P$ between the actual and desired axes. TPE is ultimately quantified as the perpendicular distance between these desired and actual trajectories. TPE at the pedicle entry point and screw tip is given by Equations (19), (20), respectively.

$${\mathit{TPE}}_{\mathit{\text{entry}}}={‖P_S\left(\mathit{act}\right)-P_S‖}_{\perp }$$ (19)

$${\mathit{TPE}}_{\mathit{end}}={‖P_E\left(\mathit{act}\right)-P_E‖}_{\perp }$$ (20)

FIGURE 14.

Geometric illustration of tool placement error (TPE) and its component.

The orientational component of tool placement error is defined as the angular deviation between the desired and actual screw axes:

$${\mathit{TPE}}_{\mathit{ang}}=\mathrm{\Delta }{\mathrm{\theta }}_P={\cos }^{-1}\left(\widehat{P}\left(\mathit{act}\right)\circ \widehat{P}\right)$$ (21)

3. Results and Discussion

The proposed method was validated using spine phantoms. The experimental setup consisting of a 6‐DOF parallel robot, where a monocular camera and tool‐guiding system were mounted on the robot's end effector; a typical setup is illustrated in Figure 15. The computer screen in the figure displays a magnified image of an element of the mask as seen by the robot‐camera. The monocular camera attached to the robot's end effector avoids line‐of‐sight constraints during registration.

FIGURE 15.

Experimental setup: A robot equipped with a 2D camera, floating‐point masks over an underlying vertebra, and visualization for image‐guided assistance for registration and surgical procedure.

3.1. Robotic System Performance and Calibration

The performance of the system is primarily governed by robot positioning, component‐level calibration, and ROI segmentation accuracies. The performance metrics of the robotic system are evaluated at the National Accreditation Board for Testing and Calibration Laboratories. The robot satisfies all the performance characteristics desired for cranial and spinal surgical procedures. The repeatability values remained within a narrow band of 8–20 μm, with an average of 14 μm. The average absolute error observed was 67 μm, with most values ranging between 50 and 150 μm, peaking briefly at around 160 μm. The output performance metrics confirm that the proposed system maintains the reliable accuracy required for robot‐assisted neurosurgery [37]. The calibration, accuracy, and repeatability assessment of the robot is reported in [38].

3.2. Pose Estimation via Vision‐Level Localization

To account for posture‐induced displacements of the floating‐points between CT imaging and surgical posturing in the OT, a backward mapping transformation is computed for each mask element. This mapping estimates specific rotations $\left({\mathrm{\theta }}_{\mathrm{new}},{\mathrm{\varphi }}_{\mathrm{new}}\right)$ of individual elements based on measured radial displacements $\left({\rho }_{\mathrm{new}}\right)$ of their gravity‐sensitive floating points. This ensures that mask element localization accurately accounts for posture‐induced variations. As illustrated in the calibration result in Figure 16, this relationship between element orientation and floating‐point displacement achieved a root‐mean‐square error (RMSE) of 0.225 mm, providing the high‐precision reference needed to bridge the CT‐to‐OT shift.

FIGURE 16.

Relationship between the radial displacement of the floating‐point and the solid angle tilt of the mask element relative to the gravity axis.

Subsequent error propagation occurs during intraoperative localization of the floating‐point mask. This vision‐level localization uncertainty stems from two primary sources. (i) Lateral localization (X, Y): The uncertainty associated with lateral localization of individual mask elements is less than 50 μm. (ii) Depth measurement (Z): The accuracy of depth measurement depends on the precise estimation of element orientation, which is derived from meticulously calibrated data across multiple sets of experiments.

The adaptive depth adjustment is demonstrated through experiments conducted at three varying heights, as illustrated in Figure 17. In set 1, the checkerboard (dataset) is kept at a baseline distance from the camera frame $\left\{\mathit{CAM}\right\},$ along the axis of view. In set 2, the checkerboard is lowered by 19 mm relative to the base line, and in set 3, it is further lowered by 29.6 mm from the baseline. In response to these displacements, the robot's feedback control system iteratively adjusts its depth with respect to the robot‐base to maintain focal sharpness. The experiments are repeated at multiple levels. For each level, the platform moves until it settles at the point of highest sharpness score, which corresponds to a working depth of 93 mm with an accuracy within 100 μm. The sharpness score is computed as the mean of the absolute values of the Laplacian across the region of interest [39]. The resulting sharpness score‐versus‐depth profiles, which validate the system's ability to maintain a consistent working distance despite non‐planar object displacement, evaluated across three distinct sets, are presented in Figure 18.

FIGURE 17.

Experiments demonstrate adaptive‐depth adjustment by the robot equipped with a monocular camera.

FIGURE 18.

Sharpness‐vs‐depth profile for set 1, set 2, and set 3.

These profiles were generated based on automated image sharpness feedback metrics, providing insight into the optimal focal depth for each dataset. This demonstrates the system's ability to accurately maintain working distance between the camera frame $\left\{\mathit{CAM}\right\}$ and the object, despite significant object displacement. It should be noted that the sharpness scores depend on pattern choice (in this case, it's checkerboard) and the ambience under a fixed magnification of the camera. Therefore, a one‐time calibrated measurement of working depth for the marker type and the ambience is established. Together, these results validate the proposed image‐guided adaptive positioning strategy and further confirm that the inverse kinematics computation effectively accommodates the non‐planar arrangement of the elements during measurement.

3.3. Target Registration Error (TRE) and Target Placement Error (TPE) Analysis

The robotic and vision components operate within the accuracy of the micrometer (μm) range. Despite this, the error in localizing the pedicle‐screw entry point and endpoint relative to the ground truth can be higher due to the cumulative error of the entire workflow. To analyze robot guidance accuracy, the study incorporated a wide range of variations by positioning the phantom at eight different angles (0°–4.5°) about the caudal–cephalic axis. Thus, generating 56 samples for registering each of the target vertebrae (L1 and L3) with respective floating‐point masks. We evaluated these samples to determine the impact on the accuracy between the elements of the corresponding mask positioned in preoperative CT images and the physical space (OT). The performance of the system across all experimental variations is measured and recorded. Figure 19 illustrates the violin curve detailing the registration error of the individual mask elements (markers) placed over the vertebrae.

FIGURE 19.

Position localization error of elements (marker) of floating‐point mask placed over L1 and L3 vertebrae.

The results presented in Table 5, which depict the findings of the phantom registration study, confirm the accuracy of the proposed system. With robot assistance, it demonstrates an accuracy of 0.6078 ± 0.3485 mm. These results are well within the anticipated accuracy range for an image‐guided spine surgery system (IGSS), which is $\le$ 2 mm at a 95% confidence interval [40].

TABLE 5.

Target registration error (TRE) evaluation.

S. No.	N	Vertebra	Target registration error (RMSE)
			Mean ± SD (mm)	95% CI (mm)	Max. error (mm)
1	56	L1	0.5523 ± 0.2008	[0.3713, 0.6688]	0.7614
2	56	L3	0.6633 ± 0.3485	[0.3214, 0.8377]	1.2503

Table 6 shows the tool placement error (TPE) validation results at the tool tip across vertebral levels and pedicles. These results are consistent with the target registration error (TRE) analysis, where sub‐millimeter entry‐point accuracy propagated to millimeter‐scale tool‐tip deviation due to angular error accumulation along the insertion depth. The reported metrics include both positional error (in millimeters) and trajectory error (in degrees), expressed as root‐mean‐square error (RMSE) along with the corresponding 95% confidence intervals and the maximum observed errors. As shown in Figure 14, depth between the tool entry point and the tool tip is measured to be 195 mm, which yielded a mean positional error at the tool tip less than ~1.1 mm, with maximum errors not exceeding ~1.40 mm. Trajectory error was consistently low, with mean angular deviations were less than 0.30° for all the cases.

TABLE 6.

Target placement error at the tool tip (TPE) evaluation.

S. No.	N	Vertebra	Pedicle	Tool placement error at the tool tip (RMSE)
				Positional error (mm)			Trajectory error (°)
				Mean ± SD	95% CI	Max. error	Mean ± SD	95% CI	Max. error
1	8	L1	Left	0.92 ± 0.20	[0.78, 1.06]	1.32	0.27 ± 0.06	[0.23, 0.31]	0.39
2	8	L1	Right	0.83 ± 0.34	[0.59, 1.07]	1.20	0.24 ± 0.10	[0.17, 0.31]	0.35
3	8	L3	Left	1.02 ± 0.36	[0.77, 1.27]	1.25	0.30 ± 0.11	[0.23, 0.37]	0.37
4	8	L3	Right	0.94 ± 0.43	[0.64, 1.24]	1.40	0.28 ± 0.13	[0.20, 0.36]	0.41

3.4. Registration Time Per Vertebra and Clinical Robustness

Variability in patient posture between CT and OT was effectively accounted for, ensuring precise alignment of the vertebrae in both manual and robotic surgical setups. The registration process was completed in under 140 s per vertebra, significantly reducing the time required for preoperative and intraoperative setup. Figure 20 illustrates the validation of the registration of the pedicle‐screw axis in real patient space. A video demonstrating the experimental setup and validation process is attached with the article (Video S1). The floating‐point mask is lightweight, easily affixable and CT‐compatible, offering a non‐invasive alternative to conventional rigid frames.

FIGURE 20.

An enlarged view of the phantom illustrating the tool's entry at the pedicle‐screw axis entry point.

A critical aspect of the system's clinical robustness is its ability to differentiate between global patient movement and localized Soft Tissue Artifacts (STA). In the event where patient translates rigidly on the table (Scenario 1), the inter‐element distances within the mask remain constant. The algorithm identifies this as a rigid‐body translation based on the absolute positions of the mask elements in the camera frame, rather than relying on bubble displacement. While this scenario is largely mitigated by firm patient fixation after alignment, the system remains capable of tracking such shifts. Conversely, Local Skin Translation (Scenario 2) involves skin sliding over the bone, which may occur without a change in the element's tilt relative to gravity. The skin sliding over bone has negligible occurrences because, in the patient preparation phase, it is recommended in the protocol that the patient's body remains relaxed, allowing no skin stretch while fixating the patient relative to the table. If it is present, the algorithm detects non‐rigid shifts and flags the data for inconsistency.

3.5. Comparison With Existing Registration Systems

The proposed system compares favorably with the principal registration paradigms currently in clinical and investigational use. Conventional bone‐mounted DRA‐based systems (e.g., Medtronic StealthStation, Brainlab Spine) achieve TRE in the range of 1–2 mm at a 95% confidence interval, but require invasive anchoring, repeat re‐registration at each vertebral level during multilevel instrumentation, and rely on intraoperative fluoroscopy or O‐arm imaging for spatial confirmation [5, 6, 9, 10, 11]. The present system achieves a mean TRE of 0.61 mm (Table 5) and a maximum tool‐tip positional error of 1.40 mm (Table 6) without any bone‐mounted hardware, without intraoperative imaging, and with registration completed in under 140 s per vertebra, a time comparable to or shorter than single‐level DRA re‐registration workflows.

Skin‐marker‐based approaches with intraoperative 3D–2D registration [13] offer non‐invasive reference but cannot account for the CT‐to‐OT postural shift independently. They retain the radiation burden of intraoperative imaging. Markerless methods using multiview skin‐feature tracking have demonstrated sub‐millimeter accuracy in controlled settings [16], but their performance depends on the presence of distinct cutaneous landmarks and degrades under surgical draping. US–CT/MR fusion registration approaches report errors of 7–17 mm [17], which falls outside the ≤ 2 mm IGSS accuracy criterion. Re‐attachable fiducial markers have achieved TRE of up to 4.65 mm in abdominal phantoms [18], making them unsuitable for the sub‐millimeter demands of pedicle screw guidance. The floating‐point mask system uniquely combines non‐invasiveness, radiation‐free intraoperative operation, orientation‐sensing capability, and autonomous STA decomposition within a single integrated workflow, addressing limitations that individually affect each of the above categories.

4. Strength and Limitations

The proposed system offers several key strengths. First, it is entirely non‐invasive and contactless—eliminating the need for bone‐mounted DRA frames and intraoperative fluoroscopy or O‐arm imaging, thereby reducing surgical trauma, radiation exposure, reducing crowd in OT and workflow complexity. Second, unlike conventional radiopaque skin markers, the floating‐point element encodes both translational and rotational information through a gravity‐referenced air bubble, enabling automated detection of rigid vertebral reorientation and non‐rigid soft tissue artifacts (STA) with arcminute‐scale angular resolution. This capability is absent in conventional radiopaque markers. Third, the “eye‐in‐hand” robot‐mounted camera architecture eliminates line‐of‐sight constraints inherent to conventional overhead electro‐optical tracking systems, and is compatible with standard operating rooms without requiring dedicated infrastructure. Fourth, the registration process was completed in under 140 s per vertebra, with TRE (mean 0.61 mm) and TPE (maximum 1.40 at 195 mm insertion depth) fall within the ≤ 2 mm accuracy criterion established for IGSS and compare competitively with invasive DRA‐based systems, confirming both accuracy and time‐efficiency for real‐time surgical application.

The limitations of the article are that the STA compensation model is not based on the real patient skin data. It is modeled based on a silicone substrate approximating skin, which does not identically replicate the anisotropic, multilayer, and time‐dependent mechanical response of living tissue. The mathematical model relies on a linear regularization parameter (λ) and is not estimated adaptively from patient‐specific data. The scope of the article does not include cadaveric trials, which are an important prerequisite for clinical translation. Future studies should address these limitations through cadaveric experiments, pilot clinical trials under IRB‐approved protocols, and multi‐level spine evaluations.

5. Conclusion

This study validated the proposed system within a controlled laboratory environment using phantoms in a simulated operating theater (OT) setting. The results demonstrate that the integrated device and algorithm significantly enhance the accuracy, efficiency, and safety of pedicle screw placement. By facilitating objective, digitally precise, and autonomous contactless registration, this system represents an advancement over traditional methods that necessitate invasive frames.

The non‐invasive, frameless protocol eliminates the risk of procedure‐induced surgical trauma and physiological pressure. Furthermore, the system reduces reliance on intraoperative imaging by utilizing a novel external mask device acquired during preoperative CT scanning and preserving the same anatomical reference in the OT for vertebral registration. Experimental data derived from robotic testing confirm high accuracy and time‐efficient performance, with registration and procedural times consistently under 140 s per vertebra (Figures 19 and 20; Tables 5 and 6).

To build upon these findings, future research will focus on: (1) human cadaver trials to evaluate anatomical variability and soft‐tissue impacts; (2) pilot clinical studies under IRB‐approved protocols to assess radiation‐free guidance; and (3) comprehensive reliability assessments for seamless surgical integration. Ultimately, this method demonstrates the potential to transform clinical workflows by reducing setup time, minimizing human error, and improving patient outcomes through a system that remains robust against physiological movement and anatomical variations.

Author Contributions

Amit Kumar: conceptualization, methodology, data curation, writing – original draft, writing – review and editing, visualization, validation, formal analysis. T. A. Dwarakanath: conceptualization, validation, supervision, writing – review and editing, methodology. Gaurav Bhutani: validation, supervision. Dwarakanath Srinivas: validation, supervision. Aliasgar Moiyadi: supervision, validation.

Funding

The authors have nothing to report.

Disclosure

The views expressed in this article are those of the authors and do not necessarily represent the official position of their affiliated institutions.

Ethics Statement

The authors have nothing to report.

Consent

The authors have nothing to report.

Conflicts of Interest

The authors declare no conflicts of interest.

Supporting information

Video S1: Demonstration of the autonomous frameless and contactless spine registration and accurate navigation for a surgical procedure. The video illustrates the complete intraoperative workflow: sequential positioning of the camera over each floating‐point mask element affixed to the spine phantom, adaptive depth adjustment of the robot‐mounted monocular camera to maintain focal sharpness along the non‐planar surface, autonomous segmentation and pose estimation of each bull's‐eye level element, and robot‐guided tool alignment to the registered pedicle‐screw axis. Registration is completed in under 140 s per vertebra without bone‐mounted hardware or intraoperative imaging. All experiments were conducted on an anatomically curved lumbar spine phantom under controlled laboratory conditions.

Data Availability Statement

The data that support the findings of this study are available upon reasonable request from the corresponding author. The data are not publicly available due to privacy or ethical restrictions.