Task-based evaluation of fluorescent-guided cancer surgery as a means of identifying optimal imaging agent properties in the context of variability in tumor- and healthy-tissue physiology

Abstract

Fluorescent molecular-guided surgery (FGS) is at a tipping point in terms of clinical approval and adoption in a number cancer applications, with ongoing phase 0 and phase 1 clinical trials being carried out in a wide range of cancers using a wide range of agents. The pharmacokinetics of each of these agents and the physiology of these cancers can differ vastly on a patient-to-patient basis, bringing to question: how can one fairly compare different methodologies (defined as the combination of imaging agent, system, and protocol) and how can existing methodologies be further optimized? To this point, little methodology comparison has been carried out, and the majority of FGS optimization has concerned system development—on the level of maximizing signal-to-noise, dynamic detection range, and sensitivity—independently from traditional agent development—in terms of fluorophore brightness, toxicity, solubility, and binding affinity and specificity. Here we propose an inclusion of tumor and healthy tissue physiology (blood flow, vascular permeability, specific and nonspecific binding sites, extracellular matrix, interstitial pressure, etc…) variability into the optimization process and re-establish well-described task-based metrics for methodology optimization and comparing quality of one methodology to another. Two salient conclusions were identified: (1) contrast-to-background variability is a simple metric that correlates with difficult-to-carry-out task-based metrics for comparing methodologies, and (2) paired-agent imaging protocols offer unique advantages over single-imaging-agent studies for mitigating confounding tumor and background physiology variability.

1. INTRODUCTION

Countless different metrics that have been used to evaluate image quality exist across a diverse spectrum of applications: from biomedical, astronomy, to military. In his seminal textbook on the Foundations of Image Science (1), Harrison Barrett proposed that the only metric that should matter is the task-based evaluation: i.e., how well does the proposed methodology perform at its implied task. In molecular-guided surgery, the goal of a methodology is ultimately to prolong patient survival and improve quality of life. Of course, carrying out a 20-year study to assess the qualities of every newly proposed methodology or improvement to an existing methodology is untenable. More simply, the task could be defined as a discrimination of whether a region-of-interest (ROI) is cancerous or not. In principle, one could simplify this to the “ideal-observer” if the truth in an image is known by carrying out a receiver-operating-characteristic (ROC) curve for any specific methodology. Then, any methodology with a higher area-under-the-curve (AUC) of the ROC would generally be expected to perform better at the task of discrimination than a competing methodology with a lower AUC of the ROC. Aside: this precludes other important comparisons, such as cost, potential patient side-effects, and ease of use, which would also need to be considered. Unfortunately, in molecular-guided surgery it can be difficult to know the absolute truth, which might conceivably require tissue from each pixel in an image to be removed and independently assayed to establish whether each was truly negative or positive for cancer.

2. METHODS

2.1. Image quality metrics

This work proposes a much simpler metric for approximating the AUC of the ROC: the contrast-to-variability metric:

$$CVR=\frac{\left|{\mu }_T-{\mu }_B\right|}{\sqrt{{\sigma }_T^2}+{\sigma }_B^2},$$ [1]

where μ_T and μ_B represent the mean signals measured in representative regions of the image that are known to be tumor positive and negative, respectively, and σ_T and σ_B represent the respective standard deviations in these mean signals. A more commonly used metric in molecular-guided surgery has been the “tumor-to-background” ratio, simply:

$$TBR=\frac{{\mu }_T}{{\mu }_B};$$ [2]

however, while this metric has some rudimentary value in comparing methodologies, its inability to account for variation in the signal makes it ill-suited for predicting the result of a task-based evaluation. Figure 1 presents some preliminary simulation results comparing the both CVR and TBR to the AUC of the ROC for a range of μ_T, μ_B, σ_T and σ_B.

Figure 1.

Comparison of tumor-to-background ratio (TBR) and contrast-to-variance ratio (CVR) in example noise-added images, with large and small inclusions.

An example of the relevance of CVR over TBR is presented in Fig. 1 where the left image has a TBR of only 3, but much better noise statistics than the image on the right with a TBR of 5, allowing easier identification of the large and small circular regions of interest. However, evaluation of the CVR in both cases demonstrates a >6 times higher CVR in the left image than the right, matching the clear improvement of the left image over the right in terms of identifying the two inclusions.

2.2. Simulations

Contrast curves of cell-surface receptor targeted imaging agents were created using a six-stage, fifth-order Runge-Kutta ordinary differential equation solver, ode45() in MATLAB (R2019a, Mathworks, Natick, MA), directly applied to the systems of differential equations presented in Fig. 2. Resulting curves were interpolated to 4 h time windows for small-imaging agent like conditions and for 3–6 days for antibody-like imaging agents. Plasma input functions were created from average plasma input functions of ABY-029 (an affibody based fluorescent agent) and fluorescent cetuximab (an antibody based agent) (2, 3). Extravasation and efflux rate constants, K₁ and k₂, were set to match that known for affibodies and antibodies, and equal to 0.1 min⁻¹ and 0.08 min⁻¹, for affibody in EPR tumor for 1–10 kDa sized imaging agents in animal and human tumor imaging experiments (4, 5).

Figure 2.

Mathematical models of measured signal based on biological and imaging agent properties.

3. RESULTS

3.1. Simulations

By identifying a simplistic metric—the CVR—that can act as a surrogate for image quality in molecular-guided surgery, it becomes valuable to develop and validate mathematical models of contrast and signal variation that may be expected clinically, particularly in the context of exogenously administered molecular targeted imaging agents. Such models can all manner of parameters to be evaluated in with respect to their effects on CVR. These parameters can include imaging system characteristics (gain, filtering, dynamic range of the detector, exposure, sensitivity, light source, hyperspectral, multi-modality etc…), imaging agent characteristics (plasma pharmacokinetics, plasma protein binding, size, charge, lipophilicity, binding on and off rates, affinity, nonspecific binding, etc…), tissue physiology characteristics (tumor and healthy tissue blood flow, vascular permeability, interstitial pressure, extracellular matrix, lymphatic drainage, etc…), imaging protocol specifics (timing of image or images, multiple imaging agents, etc…), and image processing specifics (kinetic modeling, machine learning, pattern recognition, etc…). As an initial demonstration of what such a model is capable of, Fig. 3 presents estimation of tumor and background signals (as well as contrast) under four conditions; a slow clearing antibody like agent an “EPR” tumor (one with high enhanced permeability and retention effects) and in a high interstitial pressure tumor (low EPR); comparatively, these two tumor states were evaluated in terms of a fast clearing imaging agent.

Figure 3.

Contrast modeling example results

4. DISCUSSION

In shot-noise limited imaging, TBR works will as an indicator of image quality; however, when the noise statistics change, for instance with paired-agent imaging (6, 7), fluorescence lifetime imaging (8), hyperspectral imaging (9), or Raman imaging (10, 11), TBR can change drastically amongst images that have equal quality, while the CVR remains within 5%. It is recommended that to compare different methodologies/technologies for improving molecular-guided surgery, we all start using CVR instead of TBR to estimate image quality.