Abstract
We present a novel 3D adaptive observer framework for use in the determination of subsurface organic tissue temperatures in electrosurgery. The observer structure leverages pointwise 2D surface temperature readings obtained from a real-time infrared thermographer for both parameter estimation and temperature field observation. We introduce a novel approach to decoupled parameter adaptation and estimation, wherein the parameter estimation can run in real-time, while the observer loop runs on a slower time scale. To achieve this, we introduce a novel parameter estimation method known as attention-based noise-robust averaging, in which surface thermography time series are used to directly estimate the tissue’s diffusivity. Our observer contains a real-time parameter adaptation component based on this diffusivity adaptation law, as well as a Luenberger-type corrector based on the sensed surface temperature. In this work, we also present a novel model structure adapted to the setting of robotic surgery, wherein we model the electrosurgical heat distribution as a compactly supported magnitude- and velocity-controlled heat source involving a new nonlinear input mapping. We demonstrate satisfactory performance of the adaptive observer in simulation, using real-life experimental ex vivo porcine tissue data.
I. Introduction
Over the past decades, the advantages of robot-assisted surgery over conventional laparoscopy have become increasingly apparent [1]. As the robotic surgery performance demands grow, so does the need for a deeper understanding of the physical phenomena governing both the tools and the tissue treated, as well as the control laws for the precise attainment of the surgical objectives.
Electrosurgery relies on the use of high power density radio frequency currents to actively heat organic tissue, allowing it to be denatured, coagulated, desiccated, fulgurated, or incised [2]. One of the key advantages of electrosurgery is its capability of simultaneous cutting and coagulation, providing blood stoppage for complex surgical tasks. Since this technique allows for precise ablation of tissue with very little collateral damage (~100–400 μm), it is commonly used in practice, with over half of the surgical procedures employing it [2]. An illustration of the electrosurgical process along with the observer setup considered in this work is shown in Fig. 1.
Fig. 1:

Illustration of the electrosurgical process as it relates to the adaptive inner-domain temperature observer presented in this work. Given the electrosurgical probe power and position commands, as well as a surface thermography image, we estimate the thermal diffusivity and employ an adaptive observer law to obtain the inner-domain temperature field.
Electrosurgical control can be posed as a boundary control problem characterized by a controllable heat flux, as dictated by the power setting, and cathode position actuation. In previous work [3], [4], the authors have considered a 1D setting in which the tissue thermodynamics are modeled using a Stefan problem formulation, where a homogeneous substance undergoing a moving phase change from virgin to a denatured tissue is considered. In [3] a stationary probe was considered, with a novel approach to non-collocated thermal sensing being introduced. The model in [3] was applied as part of a real-life planar cutting task using a hypermodel-based model predictive control algorithm in [4]. However, to guarantee a tight attainment of the control objective in the case of a moving probe location, which is essential in practice [2], considering the process in 3D becomes a necessity. In this work, we consider a spatially distributed controlled moving heat source, while accounting for chemical reactions by incorporating a bounded disturbance. In this setting, as the first approximation, a homogeneous Fourier heat equation is employed, with a moving heat volumetric source that has compact spatial support. In this work, we develop a 3D observer structure based on surface temperatures with the goal of estimating the subsurface temperature distribution; the latter goal is essential in assessing and preventing subsurface tissue damage. In practice, these inner-domains temperatures cannot be sensed in a noninvasive fashion, thereby significantly limiting any model-based control efforts for safe autonomous electrosurgery. In addition to proving convergence under bounded heat input disturbances in the case of known thermophysical parameters, we also introduce a novel real-time parameter adaptation loop in which only surface temperature measurements are used. This provides the first adaptive surface-temperature-based observer for electrosurgical processes.
In the context of biomechanics and electrosurgery, the first works dealing with temperature distributions in nonhomogeneous tissue models considered the thermal response of layered media when subjected to microwave excitation using microwave radiometry [5], building on earlier work that considered homogeneous tissue models [6]. In all of these works, experimental verification rested on the use of surface temperatures measurements; indeed, obtaining inner-domain temperatures is not possible without invasively altering the heat transfer process, e.g., by inserting thermocouples. Recent work [7] has used multi-angle surface thermography images to estimate the tomographic, or volumetric, temperature field. In [7], an application-specific deep neural network model was trained to deduce the inner-domain temperatures from the surface temperatures after being trained on synthetic data based on an unforced Fourier heat transfer model with constant thermal diffusivity throughout the entire domain. Given this data, such a model is only capable of estimating inner-domain temperatures for experiments that involve a free thermal response and have the same thermophysical parameters, boundary conditions, and geometry. In the context of electrosurgery, it is essential that an external heat input be accounted for, as well as to have an ability to obtain subsurface temperatures for different tissue types and geometries. It is then natural to consider the infinite-dimensional observers based on the surface temperature feedback, as is done in this work.
In constructing model-based observers, it is essential to have a sufficiently accurate model of the thermodynamics. While the thermal properties of the tissue can readily be obtained in a laboratory environment using differential scanning calorimetry (DSC) and thermogravimetric analysis (TGA) [3], these methods are costly and time-consuming, and only provide localized estimates of the tissue properties. More importantly, such methods are necessarily invasive, since they require biopsic samples to be extracted which can often only be used for one form of analysis at a time. Circumventing these issues, which extend to nondestructive testing of materials as well, in the context of subsurface material characterization a technique known as dynamic thermal tomography (DTT) has been introduced by Vavilov et al. [8], and has recently seen use in medical applications [9], [10]. In DTT, thermographic images are captured at different areas on the surface, with the goal of estimating local thermophysical characteristics (thermal conductivity, heat capacity, heat transfer coefficients, etc.) by exciting dynamic thermal responses using embedded collocated heating elements. Using the obtained volumetrically varying parameter estimates, it is possible to formulate a model PDE and obtain the subsurface temperatures as was done in [9]. An early approach to this method for use in biomedical applications is presented in [11], but it relies on Fourier series expansions. A major drawback of DTT is the fact that it is an invasive tomography method, unlike other tomographic imaging techniques such as magnetic resonance imaging. In this work, we introduce a non-invasive real-time parameter adaptation framework based solely on the surface temperature. To enable the latter we propose a novel thermography-based parameter estimation method which we call attention-based noise-robust averaging (ANRA).
The paper has the following structure. The boundary control problem of interest along with the related mathematical model are presented in Sec. II. Sec. III presents a non-collocated surface-temperature-based observer structure for electrosurgical action that is asymptotically convergent in the case of known thermophysical parameters. Sec. IV presents a novel adaptive observer structure, in which the thermophysical parameters are directly estimated from the real-time surface thermography. Sec. V compares the performance of both observer structures through numerical simulation. Conclusions are drawn in Sec. VI. Formal proofs will be presented in a separate publication.
II. Problem Formulation
In this work, we consider the following thermodynamic model on an n-dimensional compact domain , based on a standard model from the literature [12, Eq. 1]:
| (1) |
where a ∈ ℝ+ denotes the thermal diffusivity, and ∇2 is the Laplacian operator. x(t) denotes the temperature field at time t, such that ; by we denote x(t)(η) for η ∈ Ω. The heat source position is denoted by p(t) ∈ Ω, which is used as a parameter in the functional Tp defined as
| (2) |
where χΩ is the indicator function on the set Ω. In our model, denotes a normalized heat source term, which is scaled by the control input . The heat input is perturbed by , which is bounded in the L2(Ω)-norm by known , and is γ-Hölder continuous for γ = 1/2. Finally, is the known controlled heat source velocity, which is such that for all t ∈ [0, ∞). We denote the full state by .
In this work, we assume having access to thermographer data captured on one of the boundaries of Ω, which we denote by . In a three-dimensional setting, this would be the top surface of a slab of material. Furthermore, we consider that temperatures are sampled on a finite Cartesian grid with uniform spacing Δη > 0, denoted by . In this work, we consider , such that .
Let be the output map, defined as . The output is denoted by . We introduce a map C* from ℝM to H1(Ω), which will be used as part of a Luenberger observer structure in the following section:
where δ is the Dirac delta. In the remainder of this work, we make the following assumptions:
-
(A1)
The initial condition has known constant temperature, i.e., for all η ∈ Ω for some known .
-
(A2)
inf u(t) ≥ 0 and sup u(t) < ∞.
-
(A3)
On the boundary of Ω, ∂Ω, a trivial Neumann boundary condition holds, i.e., .
-
(A4)
The disturbance w(t) is bounded in the L2-norm for all t ∈ [0, ∞), for known . In addition, w(t) is γ-Hölder for γ = 1/2, with the Hölder coefficient being upper-bounded by a known constant Cw ≥ 0. Finally, w ∈ C0(𝒲), i.e., w is continuous in time.
-
(A5)
Heat source velocity is continuous and bounded, such that for all t ∈ [0, ∞).
As can be seen in (1), we assume that there are no errors in the probe position. In the remainder of this work, we consider control input pair (u(t), v(t)) to be given for all t ∈ [0, ∞); we also assume that there is no error in the control inputs.
Remark 1. It will be shown in Sec. V that Assumption (A1) does not necessarily need to hold. Assumption (A2) requires boundedness and non-negativity of the control signal, which follows from physical limitations in electrosurgery (i.e., heat can only be supplied by the electrosurgical pencil, and only a finite power can be supplied). Assumption (A3) is standard and can be argued to hold for sufficiently large domains, where the far-field temperature is constant (i.e., in thermodynamic equilibrium).
Assumption (A4) puts constraints on the heat source disturbance. First, it should be noted that the L2-norm bound imposed by requires that the disturbance signal is of bounded energy in space, but not in time. This implies that the disturbance could be persistent across the entire system run, with no ultimate boundedness being imposed. Furthermore, the 1/2-Hölder coefficient bound Cw ≥ 0 imposes a constraint on the sharpness of the disturbance; in particular, this constraint is more general than a Lipschitz bound. While white noise cannot be captured as part of this assumption, wide-band non-decaying signals can be. This L2-norm bound can be found by considering unmodeled dynamics such as tissue perfusion, and accounting for the magnitude of these terms based on [12].
III. Observer Design
We consider the following observer system:
| (3) |
where is the time-varying observer gain, which is diagonal and positive definite for all t ≥ 0, such that at least one of the gi(t) ≔ Gii(t) satisfies
| (4) |
where m(Ω) is the volume of the domain Ω, and
| (5) |
Note that a0 is assumed to be known; the case of unknown thermal diffusivity, is treated in the next section. Here, the disturbance Hölder coefficient bound Cw and the L2-bound ϵw,2 are known based on Assumption (A4). Note that, as mentioned in the previous section, we assume that p ∈ C1(Ω) is known without error. We start by considering asymptotic error convergence for the case where a0 = a.
Observer Error Convergence
Theorem 1. Consider the observer given by (3)–(5). Provided that assumptions (A1)–(A4) hold, the temperature estimation error converges to zero.
Remark 2. The lower bound on the gain given by (4) and (5) can be computed directly given the internal observer state , the observed surface temperature y(t), and the known bounds ϵw,2 and Cw from (A4). Given the assumptions, is continuous, which in turn implies that the lower bound on the gain, (4), is continuous.
Given the evolution of Cγ(t) from (5), it can be seen that the lower gain bound is increasing in time, but is bounded for finite t. In practice, a reasonably large initial choice of gain will result in (4) being satisfied during some time interval [0, T] for T ∈ ℝ+.
Theoretically, the elements gi(t) could be taken arbitrarily large, since no sensor noise is considered as part of this work; this would result in an instantaneous correction of the temperature error at each sensing location. In practice, one would not want to increase the gain too much, since this results in amplification of unwanted sensing noise.
IV. Adaptive Observer Design
We now consider the case where a0 ≠ a, with unknown a. Given that the true diffusivity is unknown, we introduce the following adaptive observer structure:
| (6) |
where G(t) obeys the conditions of Thm. 1, and is defined as
| (7) |
where is the instantaneous thermal diffusivity estimate obtained using the attention-based noise-robust averaging (ANRA), introduced next. The parameter adaptation gain is denoted by L ≥ 0. In this formulation, the parameter estimate in effect acts as an integral correction term to the Laplacian term.
Adaptive Noise-Robust Averaging
To obtain , we note that we may only use the surface temperature readings from the thermographer. To use the latter for our purpose we propose a novel framework further referred to as attention-based noise-robust averaging (ANRA). ANRA relies on noise-robust gradient operators that allow for computing the Laplacian and time derivative while suppressing high-frequency noise, which would ordinarily corrupt any further data processing steps. We denote the thermal diffusivity estimate obtained by ANRA based on discrete-time surface temperature measurements by , where k is the current time sampling index.
Since our parameter estimation architecture depends on direct computation of the spatial and temporal temperature gradients, we wish to suppress the effects of high frequency noise, which is observed both spatially and temporally in thermographic data. For this reason, a noise-robust filter, i.e., one that subdues high frequency noise while leaving low frequency content unattenuated, is desirable [13]. Pavel Holoborodko introduced noise-robust gradient operators in backward differencing form [13], as well as linear filters for spatial gradients [14]; this family of operators is instrumental in our work. We denote noise robust finite difference operators by .
As part of the ANRA framework, we introduce the rate of thermal change (RTC) field indicating the areas of the temperature field exhibiting the most activity. The RTC layer serves to indicate the points likely to be noise-driven, and allowing us to favor the high thermal activity regions. For this reason, we propose to draw the attention of the parameter estimation algorithm to areas of high RTC, using a data-driven attention layer. The RTC field is defined as follows:
| (8) |
where k denotes the sample index in time, denotes the surface temperature matrix captured by the thermographer, and β ≥ 1 is a tuning parameter that increases the RTC value separation between areas of low and high rate of change as it increases; in this work, β = 100. The choice for this definition is as follows. We take the absolute value of the temperature gradient at each location and exponentiate it, so as to assign a larger positive value to a large thermal rate of change, this value scaling nonlinearly in the gradient. We then subtract the smallest element to get the minimum to be zero. Finally, we divide by the sum of all elements, to obtain a matrix whose elements sum to 1. This latter property is instrumental as shown next.
Having obtained the RTC field, we may now introduce a weighted averaging step. The instantaneous thermal diffusivity estimate is obtained as follows:
| (9) |
In this step, we obtain a weighted average that is governed by the magnitude of the rate of temperature change. This is motivated by the observation that the sensor data from the areas that experience little thermal change are often dominated by noise. In light of (1), thermal diffusivity estimates obtained using ANRA are only valid when the power input, u(t), is zero. During surgery, such periods of inactivity frequently occur to assess the quality of an incision that was just made, thereby providing ample opportunity to compute during these times. An overview of the ANRA method is shown in Fig. 2, and the full adaptive observer structure is summarized in Fig. 3.
Fig. 2:

Overall attention-based noise robust averaging (ANRA) model architecture: (I) real-time thermographic imagery is acquired; (II) gradients that appear in the PDE model are computed using noise-robust gradient operators; (III) computed gradients are used to directly estimate parameters; (IV) an attention layer based on the rate of temperature change (RTC) is used to compute weighted (spatially averaged) parameter estimates; (V) based on a confidence threshold on the RTC-field, final parameters are computed.
Fig. 3:

Block diagram of the proposed adaptive observer (6).
It must be noted that when obtaining the diffusivity estimates, temperature gradients in the depthward direction cannot be sensed. This is natural, since thermography cannot determine temperatures inside a material. This does, however, lead to a discrepancy in the thermal diffusivity when compared to a correct estimate based on the full depthward temperature information. We propose a constant scaling factor on to correct for this fact, as outlined in the following lemma.
Lemma 1. Consider an infinitesimal 3D Cartesian volume of a homogeneous material, whose temperature is governed by the heat equation . Let the top surface of this volume be fully insulated (i.e., ). Assume that along each Cartesian axis, the second derivative of temperature along that axis is the same for each direction. Then, if the bottom surface is also modeled to be insulated, while the true system is not insulated at the bottom, the effective thermal diffusivity a′ will be equal to .
Using the result from Lemma 1, it suffices to scale any thermal diffusivity estimate obtained from surface temperature measurements by 4/5.
Conjecture 1. Given the adaptation law introduced in (6), the adaptive observer converges asymptotically, with faster convergence compared to the non-adaptive observer of Thm. 1.
We demonstrate Conjecture 1 in the following section.
V. Numerical Simulation
To demonstrate the performance of the proposed adaptive observer of (6), (7), we have run a simulation of an electrosurgical process. The simulation is performed on a homogeneous domain with size 4 cm × 2 cm × 2 cm with material properties of porcine loin tissue from [4], where mass density ρ = 700 kg/m3, thermal conductivity k = 0.5934 W/kg·K, and isochoric specific heat cp = 4 kJ/kg·K. The heat conduction is governed by the heat equation in (1), and the entire domain starts at constant temperature 300 K, with homogeneous Neumann boundary condition on the top surface and Dirichlet boundary condition on the other faces. Two cuts are made along the longitudinal axis on the top surface at the speed of 1 cm/s. Cut 1 starts from t = 0 s at location (1, 0, 0) and ends at point (1.75, 0, 0), and cut 2 starts from t = 1.25 s at location (2.25, 0, 0) and ends at point (3, 0, 0), at 30 W output.
The heat source q is a normalized 3D Gaussian distribution defined as , where σ = 1 mm is the radius of the electrosurgerical probe needle and q0 is a normalization constant that ensures . The domain is discretized spatially using second-order finite differencing with Δx = 0.05 cm, and temporally using trapezoid integration (Crank–Nicolson) with Δt = 0.02 s and a Gauss–Seidel solver.
First, we run the simulation with the material properties listed above, with the nominal thermal diffusivity being . To introduce an initial parameter mismatch, we take a0 = 2a for the observer system; by Conjecture 1, we expect the parameter mismatch to be corrected for. As mentioned in Sec. IV, the ANRA-based thermal diffusivity is only applicable when the heat source is off, we therefore obtain for t ∈ [0.75, 1.25] s, i.e., the period when the pen is stationary between both cuts. In addition, the heat source disturbance w(t) (continuous in time, and γ-Hölder in space) is injected into the nominal system of (1) so as to capture thermographer noise and unmodeled dynamics. The observer gain G(t) is taken to be a constant identity matrix scaled by a factor of 50. A video1 of the simulated temperature field demonstrates that the adaptation law correctly detects a change in the thermal diffusivity, with the observer consequently correcting the estimated temperature field. In addition to this video, Figs. 7a, 7b show evidence of tracking of the decreasing diffusivity, as indicated by the increased heat spread at t = 1.94 s.
Fig. 7:

3D temperature field in Kelvin at t = 0.54 s and t = 1.94 s using adaptive observer (6) with contour slices at planes y = 0 cm, and z = 0, −1, −2 cm.
Fig. 4 shows the error of the adaptive parameter during the simulation for different values of L. We can see that the parameter estimate converges to the correct value faster as gain L in the adaptive observer increases. In the case of L = 0.5, the error goes to zero within one second. However, in the case of L = 0.1, there is not enough time for the error to reach zero before the source is turned back on and parameter adaptation is suspended; given more time between cuts, this configuration would also have converged.
Fig. 4:

Parameter estimate error for different parameter adaptation gains.
Referring to the nominal system run, we compute two error metrics (∞-norm and L2-norm of the error) over time to demonstrate convergence of the temperature estimate. In Fig. 5 we can see the effect of the adaptive observer on the maximum temperature error. We are able to significantly reduce the error in the second cut even with a small L of 0.1, and the error goes to zero with L = 0.5. From Fig. 6, we can see that the convergence rate is much higher with a larger L; towards the end of the simulation, most of the L2 error is due to discretization errors as evidenced by Fig. 5. In practice, it may not be advantageous to choose exceedingly large values of L, since this may amplify estimation errors due to sensor noise.
Fig. 5:

Maximum temperature error for different parameter adaptation gains.
Fig. 6:

Temperature estimation L2 error for different adaptation gains.
VI. Conclusion
In this work, we have considered the observation of a 3D temperature field arising during an electrosurgical procedure with a moving volumetric heat source, based only on surface temperature feedback. We have presented a nonlinear partial differential equation that governs the process, from which we derived a pointwise noncollocated observer law based on surface temperature. In this work, we have allowed for a smooth and power-bounded heat source disturbance to be present across the entire domain, including an adequate choice of feedback gain. A closed-form lower bound for the feedback gain has been derived based solely on rudimentary assumptions of the process noise, as well as the surface temperature feedback. Using this observer law, asymptotic convergence was shown.
We have also introduced a novel direct parameter estimation technique, which we termed attention-based noise-robust averaging (ANRA) to enable real-time parameter adaptation and shown its use. We have demonstrated convergence of this adaptive observation law through numerical simulation.
Acknowledgments
Research reported in this publication was supported by the National Institute of Biomedical Imaging and Bioengineering of the National Institutes of Health under award number R01EB029766. The content is solely the responsibility of the authors and does not necessarily represent the official views of the National Institutes of Health.
Footnotes
References
- [1].Lanfranco AR, Castellanos AE, Desai JP, and Meyers WC, “Robotic Surgery: A Current Perspective,” Annals of Surgery, vol. 239, no. 1, pp. 14–21, Jan. 2004. [DOI] [PMC free article] [PubMed] [Google Scholar]
- [2].Palanker D, Vankov A, and Jayaraman P, “On mechanisms of interaction in electrosurgery,” New Journal of Physics, vol. 10, no. 12, p. 123022, Dec. 2008. [Google Scholar]
- [3].El-Kebir H and Bentsman J, “PDE-based modeling and non-collocated feedback control of electrosurgical-probe/tissue interaction,” in 2021 American Control Conference. New Orleans, Louisiana, USA: IEEE, 2021, pp. 4045–4050. [DOI] [PMC free article] [PubMed] [Google Scholar]
- [4].El-Kebir H, Lee Y, Berlin R, Benedetti E, Giulianotti PC, Chamorro LP, and Bentsman J, “Online hypermodel-based path planning for feedback control of tissue denaturation in electrosurgical cutting,” in Proc. of the 11th IFAC Symposium on Biological and Medical Systems, Benyo B, Ed., vol. 54, no. 15. Ghent, Belgium: IFAC, 2021. pp. 448–453. [Google Scholar]
- [5].Bardati F and Solimini D, “Radiometric sensing of biological layered media,” Radio Science, vol. 18, no. 6, pp. 1393–1401, 1983. [Google Scholar]
- [6].Erez A and Shitzer A, “Controlled destruction and temperature distributions in biological tissues subjected to monoactive electrocoagulation,” Journal of Biomechanical Engineering, vol. 102, no. 1, pp. 42–49, Feb. 1980. [DOI] [PubMed] [Google Scholar]
- [7].Ledwon D, Sage A, Juszczyk J, Rudzki M, and Badura P, “Tomographic reconstruction from planar thermal imaging using convolutional neural network,” Scientific Reports, vol. 12, no. 1, p. 2347, 2022. [DOI] [PMC free article] [PubMed] [Google Scholar]
- [8].Vavilov VP and Maldague XP, “Dynamic thermal tomography: New promise in the IR thermography of solids,” in Aerospace Sensing, Eklund JK, Ed., Orlando, FL, USA, 1992, pp. 194–206. [Google Scholar]
- [9].Toivanen J, Tarvainen T, Huttunen J, Savolainen T, Orlande H, Kaipio J, and Kolehmainen V, “3D thermal tomography with experimental measurement data,” International Journal of Heat and Mass Transfer, vol. 78, pp. 1126–1134, 2014. [Google Scholar]
- [10].Toivanen J, Tarvainen T, Huttunen J, Savolainen T, Pulkkinen A, Orlande H, Kaipio J, and Kolehmainen V, “Thermal tomography utilizing truncated Fourier series approximation of the heat diffusion equation,” International Journal of Heat and Mass Transfer, vol. 108, pp. 860–867, 2017. [Google Scholar]
- [11].Chen MM, “Computer-aided tomographic thermography,” in Heat Transfer in Medicine and Biology, Shitzer A and Eberhart RC, Eds. Boston, MA, USA: Springer US, 1985, pp. 353–369. [Google Scholar]
- [12].Singh S and Melnik R, “Thermal ablation of biological tissues in disease treatment: A review of computational models and future directions,” Electromagnetic Biology and Medicine, vol. 39, no. 2, pp. 49–88, Apr. 2020. [DOI] [PubMed] [Google Scholar]
- [13].Holoborodko P, “Smooth Noise Robust Differentiators,” http://www.holoborodko.com/pavel/numerical-methods/numerical-derivative/smooth-low-noise-differentiators/, 2008.
- [14].——, “Noise Robust Gradient Operators,” http://www.holoborodko.com/pavel/image-processing/edge-detection/, 2009.
