Credibility assessment of patient-specific modeling in transcatheter aortic valve implantation—Part 2: Uncertainty quantification and sensitivity analysis

Abstract

Transcatheter aortic valve implantation (TAVI) benefits from patient-specific computational modeling, yet model credibility remains a challenge. The ASME V&V40 standard provides a framework for assessing uncertainty and sensitivity in in silico predictions, ensuring reliability in clinical decision-making. This study evaluates uncertainty quantification (UQ) and sensitivity analysis of a patient-specific TAVI model using the ASME V&V40 standard to enhance model credibility. Four patient-specific TAVI models with 23 and 26 mm SAPIEN 3 Ultra (S3) devices were developed using finite-element simulations for deployment and fluid–structure interaction analysis for hemodynamic analysis. Uncertain parameters included anatomical features, material properties, hemodynamic conditions, and procedural variables. A surrogate model was constructed with Gaussian-process regression, and probabilistic assessment was conducted via quasi-Monte Carlo analysis. Sensitivity analysis identified key parameters influencing model outputs. The surrogate model accurately predicted device diameter (mean relative error <1%), with balloon expansion volume and stent-frame material properties being the most influential. Hemodynamic predictions exhibited greater uncertainty, with effective orifice area and pressure gradient showing deviations beyond the 5% validation threshold. This study establishes a framework for UQ in patient-specific TAVI modeling, demonstrating reliable device deployment predictions. The findings support integrating in silico models into clinical decision-making, benefiting clinicians, manufacturers, and regulatory bodies. This study is complemented by a first part dedicated to the discrete validation of the patient-specific TAVI model against clinical data.

I. INTRODUCTION

In transcatheter aortic valve implantation (TAVI), patient-specific modeling has reached considerable maturity with tangible applications of computer modeling and simulation to predict the medical device position and thus help clinicians in the pre-operative planning of the structural intervention.^1,2 However, the credibility of computational models has only recently gained significant attention from the medical industry, driven by the need to reduce the extensive testing during the development stage and the cost and ethical concerns of preclinical animal modeling. The lag in the adoption of in silico modeling by the medical industry can be attributed to several factors, including the lack of accurate models, standards, and validation processes. A paradigm shift was, however, observed with the publication of the ASME V&V40 standard, which provides a risk-based framework for the credibility assessment of a given computational model when used to inform decisions on the intended medical device.³ Several activities are needed to assess model credibility, including verification, validation, and uncertainty quantification (UQ). The ASME V&V40 standard considers the uncertainty analysis as a pivotal element for credibility assessment.⁴ By assessing uncertainty and sensitivity, a systematic analysis of model robustness can be obtained to identify the conditions where in silico predictions deviate from expected ranges because of perturbations in input parameters.

Examples of UQs have been developed across various domains, including pharmacokinetic models,⁵ physiological closed-loop control systems,⁶ left ventricular assist device simulations,⁷ intracranial aneurysm modeling,⁸ and in-stent restenosis.⁹ In the context of patient-specific modeling, Galappaththige et al.¹⁰ employed an in silico cardiac electrophysiology model to estimate uncertainty arising from population variability, demonstrating the role of UQ in supporting robust model-based decision-making. While global sensitivity and probabilistic analyses based on surrogate modeling are common to these studies, the methodologies for comparison and validation metrics employed to quantify the influence of uncertain parameters and assess model credibility vary significantly across applications.

This study aims to perform UQ and sensitivity analysis of a patient-specific model for TAVI patients in accordance with the ASME V&V40 standard. Following an introduction to the patient-specific TAVI model and the study group, the requirements and application of the ASME V&V40 framework are outlined to account for the definition of the comparator, uncertain parameters, and their evaluation in clinical settings. A surrogate model was developed based on original numerical simulations of structural device deployment and post-TAVI hemodynamics. Later, the surrogate model was adopted to evaluate the model response to uncertain parameters while a quasi-Monte Carlo analysis was carried out to assess the probabilistic distribution of uncertainties between clinical and in silico data within an acceptance criterion. We emphasize that this manuscript constitutes the second part of a two-part study. In Part 1,¹¹ we presented the validation of a patient-specific TAVI model by comparing simulation results with clinical post-TAVI data. To assess the robustness and reliability of the proposed patient-specific TAVI model, the present work (Part 2) builds upon that foundation by performing a comprehensive uncertainty quantification and sensitivity analysis in accordance with the ASME V&V40 framework.

II. RESULTS

Regarding the structural deployment of the S3 device, only a minimal part of the simulations presented numerical failures due to unrealistic combinations of material parameters and TAVI-related procedural variables. This was not, however, observed for the 88 post-TAVI fluid–structure interaction (FSI) analyses for which a fraction of 8.4%–22.7% simulations failed due to the variability in the model inputs resulting from the Latin hypercube sampling (LHS) methodology. Figure 1 illustrates the predictive capability of the surrogate model based on Gaussian process (GP) regression with leave-one-out (LOO) for the quantification of the S3 device expansion across different device heights for two patients with the 23- and 26 mm device sizes. Specifically, the mean relative error (MRE) between surrogate model and actual finite-element values for the device diameter is consistently lower than 1% in all patients. Low LOO root mean square errors (RMSE) and high correlation coefficient (R) were also found for all models as reported in Table I. However, the response of the trained surrogate model highlighted lower effectiveness in capturing the bioprosthesis hemodynamic response estimated with actual FSI outputs (see Fig. 2).

FIG. 1.

Correlation between the surrogate model response and actual structural numerical predictions of the device diameter at (a) inflow, (b) mid, and (c) outflow levels for the Case #4 model with the 26 mm S3 device.

TABLE I.

Fitting response of the surrogate model for each model parameter for both 23-mm and 26-mm S3 devices. Note: LOO RMSE = LOO root mean square error; R = correlation coefficient; MRE = mean relative error.

		23-mm S3 device		26-mm S3 device
		Case #1	Case #2	Case #3	Case #4
Inflow diameter
	LOO RMSE	0.004	0.011	0.001	0.002
	R 2	0.98	0.99	0.99	0.98
	MRE (%)	0.22	0.41	0.10	0.15
Mid diameter
	LOO RMSE	0.001	0.014	0.001	0.001
	R 2	0.97	0.98	0.99	0.91
	MRE (%)	0.58	0.48	0.09	0.43
Outflow diameter
	LOO RMSE	0.006	0.003	0.001	0.001
	R 2	0.96	0.98	0.98	0.99
	MRE (%)	0.26	0.19	0.09	0.08
EOA
	LOO RMSE	12.678	29.404	26.155	46.788
	R 2	0.96	0.94	0.97	0.92
	MRE (%)	2.07	4.37	8.45	11.83
TPG
	LOO RMSE	32.156	34.901	16.583	28.990
	R 2	0.91	0.93	0.86	0.89
	MRE (%)	24.53	15.96	15.70	22.17

FIG. 2.

Correlation between the surrogate model response and actual FSI predictions of the (a) effective orifice area (EOA) and (b) trasmural pressure gradient (TPG) of the 26 mm S3 device for the Case #3 model.

Figures 3 and 4 display the Pareto plots of input parameters on output uncertainties for the device diameter at three cross-section levels and flow parameters for the 23- and 26 mm stent frame, respectively. Blue bars indicate a positive association of the uncertain parameter with the quantity of interest, while red bars display a negative correlation with the model output variable. Pareto plots of the other two patients are shown in the supplementary material (see Figs. 2S and 3S). Global sensitivity analysis revealed that the device adaptation to the stenotic aortic valve is primarily influenced by the balloon expansion and the medical device elastic material modulus. Only for one patient case with the 23 mm device, the implantation depth was found as the key player of the device's biomechanical response, with a sensitivity index of 32.4% at the outflow diameter. The uncertainty in the Young modulus of the S3 device was the most influential parameter with percentage effect ranging from 29.3% to 48%. A less consistent influence of uncertain parameters on the model-related fluid-dynamic response was observed. Sensitivity indexes highlighted that blood pressure, heart rate, and the transaortic flow jet are the most influential parameters of the post-TAVI hemodynamics. For the 23 mm device, as an example, a 16.4 mm Hg deviation of the mean systolic pressure values may result in 27.3% variation in the prediction of the EOA. Similarly, a 0.5 m/s deviation of mean S3 flow velocity may lead to a change of nearly 30% in the TPG estimates.

FIG. 3.

Pareto plot showing sensitivity of model inputs on the output device diameter for the (a)–(c) 23 mm S3 device (Case #1) and (d)–(f) 26 mm S3 device (Case #3) at different cross sections. Note: Ball_Exp = balloon expansion; C1_Calc = Neo-Hookean material parameter of calcification; C1_Aorta = Neo-Hookean material parameter of aortic wall; Depth = implantation depth; S3_E = Young's modulus of S3 device; Thk = aortic wall thickness.

FIG. 4.

Pareto plot showing sensitivity of model inputs on both EOA and TPG for the (a) and (b) 23 mm S3 device (Case #1) and (c) and (d) 26 mm S3 device (Case #3). Note: Ball_Exp = balloon expansion; C1_Calc = Neo-Hookean material parameter of calcification; C1_Aorta = Neo-Hookean material parameter of aortic wall; Depth = implantation depth; HR = heart rate; Pdias = diastolic pressure; Psys = systolic pressure; S3_E = Young's modulus of S3 device; S3_Flow = S3 flow velocity; Thk = aortic wall thickness.

Figures 5 and 6 show the clinical and in silico cumulative distribution function (CDF) curves for all investigated quantities of interest together with the corresponding area metric for two patients with different device sizes. The validation criterion (i.e., area metric ≤5%) was observed in the majority of device diameter predictions. Out of 12 assessments (i.e., three device height and four patients), only three cases had a level of agreement higher than the 5% acceptance threshold, with the highest area metric of 13.0% for the patient with the 23 mm device. Both clinical and in silico CDF curves showed a steep rise, which indicated that the diameter values are tightly clustered around a specific range. Since the CDF comparison visually and quantitatively evaluates how well the computational predictions match the observed clinical measurement, for the mid-level of the 26 mm S3 [Fig. 5(e)], the probability of having a stent frame size ≤25.4 mm as clinically found was 85% for the investigated input model variance. In a different way, the discrepancy between clinical and in silico predictions was remarkably higher for both the EOA and TPG hemodynamic parameters as compared to diameter-related CDFs. Only in one patient with the 23 mm device size, the 5% acceptance threshold was found for the TPG parameter.

FIG. 5.

CDF curves for both clinical and in silico data of the device diameter showing the area metric and its value confined by the two CDF curves (gray area) computed for both 23 and 26 mm S3 devices of Case #1 and Case #3 models (a)–(f).

FIG. 6.

CDF curves for both clinical and in silico data of both EOA and TPG showing the area metric and its value confined by the two CDF curves (gray area) computed for both 23 and 26 mm S3 devices of Case #1 and Case #3 models (a)–(d).

Upon completion of the uncertainty analyses, the ASME V&V 40 framework was applied to evaluate the credibility assessment, where each UQ activity is graded according to increasing levels of investigative rigor based on the model risk assessment (see the supplementary material).

III. DISCUSSION

This study applied the ASME V&V 40 framework to assess uncertainty in patient-specific TAVI modeling, evaluating both model rigor and associated risks. UQ encompassed anatomical and material properties, physiological conditions, and procedural variables. The findings confirm the model's robustness in predicting the structural response of the medical device, supporting its reliability for in silico safety-critical decisions. However, the model predictive accuracy for post-TAVI hemodynamics requires careful attention, as discrepancies of up to 43.68% exceeded acceptance criteria. These differences cannot be attributed only to poor predictive capability but rather to the evaluation criteria and methodologies used to measure the investigated quantities of interest. Nonetheless, this approach advances the standardization and validation of in silico models for medical devices.

Sensitivity analysis indicated that the biomechanical response of the aortic root after device implantation was mainly affected by balloon inflation and the elastic properties of the medical device material. The manual management of fluid expansion by the interventional cardiologist via a syringe is crucial in establishing the final configuration of the device. In clinical practice, deviating from the manufacturer's specifications for the filling of the balloon delivery system is common; yet, quantifying its effect on the final device diameter by post-TAVI surveillance imaging remains challenging.¹² In a prior study, we performed material testing on the Nylon-12 balloon and noted a visco-plastic response under different temperatures and strain rates.¹³ The balloon material demonstrated incremental relaxation over time when maintained at the standard inflated volume, with deformation evaluated via digital image correlation. The second most significant input parameter was the Young's modulus of the medical device, which proved to be the most crucial material property in the patient-specific TAVI model. This finding is significant, as the majority of research has concentrated on enhancing tissue wall models to elucidate ideal material properties using imaging data¹⁴ or investigating material behavior (e.g., isotropy vs anisotropy) via different constitutive relationships.¹⁵ Conversely, Finotello et al.¹⁶ aimed to elucidate the impact of device material characteristics, demonstrating that material uncertainty in self-expandable devices might result in erroneous estimates of tissue wall stress values.

There was significant variation in the sensitivity indices with respect to the fluid-dynamic response. Two factors may elucidate the difference. First, while the patient was still under anesthesia in the hybrid operating room, the EOA and TPG metrics were evaluated right after the structural heart intervention. The observed patient behavior differed from the model because the simulations relied on stochastic limitations of hemodynamic parameters obtained from the physiological norms of the overall patient population. Second, the post-TAVI FSI simulations probably produced a more unpredictable and nonlinear response due to the interplay of ten uncertain parameters. This highlights the importance of tailoring UQ approaches to the intricacies of real-world situations. Notwithstanding these limitations, the suggested patient-specific TAVI model demonstrated sensitivity to variations in flow boundary conditions, which can be deemed credible.

The validation activity, based on CDFs and the area measure, raises numerous key questions about the rigor of the patient-specific TAVI model. The 5% acceptance level for comparing simulated and observed data represents a stringent criterion, especially considering the inherent inter-subject variability in human physiology. As the strict acceptance threshold was not achieved for both EOA and TPG, this raises the question if the acceptance criterion should be adjusted based on the specific quantity of interest and the known limitation/variability of both models and clinical measurement techniques. The suggested validation study implicitly assumes that all model inputs are affected only by measurement errors; nevertheless, certain inputs may represent population-based features instead of individual-specific data. The ASME V&V40 standard does not specifically outline the integration of patient-specific features and the consequent model uncertainty within a population-based validation methodology. Furthermore, measurement errors must also consider intra-observer variability in quantifying the quantities of interest. For instance, although the 5% error threshold for diameter comparison just considers instrumental precision, intra-observer variability in diameter measurement may lead to a wider dispersion of data points around the mean. Scuoppo et al.¹⁷ demonstrated a low interclass correlation coefficient in post-TAVI computed tomography (CT) imaging assessments of the S3 device between expert and non-expert radiologists. The intra-observer variability in evaluating EOA and TPG using echocardiography frequently exceeds that noted for CT-derived diameter values, as indicated by Calleja et al.¹⁸ It is also crucial to note that the ASME V&V40 standard allows for flexibility in the interpretation of validation procedures. The selection of validation measures, such as relative error or area metric, is not explicitly mandated. Relative error with probability is relevant solely when clinical and simulation data are accessible for individual samples. The area metric accurately identifies variance discrepancies between clinical and computational results, even when their mean values are comparable. Nevertheless, when clinical and simulation outcomes diverge, as evidenced by the significant disparities in CDFs for EOA and TPG, the area metric may exhibit diminished sensitivity to variations in differences. Despite these considerations, the area metric values obtained for the structural part of our patient-specific TAVI model exhibit strong concordance between in silico and in vivo data. For the flow-related quantity of interest showing a large deviation from in vivo data, further analyses should be carried out to determine whether the model is credible for the chosen context of use.

This study on UQ has several limitations. First, the findings are specific to the current patient-specific TAVI model implementation. Any modifications, such as incorporating anisotropic material properties or using different imaging comparators, would require reevaluating model credibility. Additionally, population variability estimates used to define error bounds and classify uncertainties as aleatoric or epistemic may lead to different interpretations of uncertain parameter influence. The primary limitation, however, lies in the surrogate model's predictive capabilities. For the device diameter, the surrogate model response was sufficiently prognostic, ensuring that epistemic errors were the predominant component of the uncertainty analysis. This is not the case for the fluid-related surrogate model, for which alternative regression methods and expanded experimental designs may enhance predictive accuracy and improve credibility assessment. The study is restricted by only four patient models, rendering the findings regarding the influence of model inputs on simulation outputs and the extent of uncertainty potentially non-generalizable to the wider patient population.

IV. CONCLUSION

This study applied the ASME V&V40 framework to quantify uncertainty and analyze sensitivity in a patient-specific TAVI model. Using surrogate modeling and quasi-Monte Carlo simulations, the influence of uncertain parameters on S3 device deployment and post-TAVI hemodynamics was assessed. The model reliably predicted device expansion, with balloon inflation volume and stent-frame material properties emerging as key determinants, achieving area metrics within the 5% acceptance threshold. However, limitations in the surrogate model and methodological discrepancies in EOA and TPG measurements likely contributed to the lower agreement between clinical and in silico hemodynamic predictions. Despite these challenges, this work establishes a systematic and rigorous framework for assessing model credibility and supporting the integration of patient-specific simulations in clinical decision-making. The findings hold particular relevance for medical device manufacturers, clinicians, and regulatory bodies seeking robust, simulation-based insights for TAVI planning.

V. METHODS

A. Patient study group

The patient-specific TAVI model encompassed a structural finite-element simulation of the deployment of the SAPIEN 3 Ultra (S3) transcatheter heart valve in the aortic root model, followed by the post-TAVI fluid–structure interaction (FSI) analysis using smooth-particle hydrodynamics technique. This model was built to quantify the device performance in terms of the S3 adaptation to the calcified valve leaflets (i.e., device diameter), pressure gradient, and orifice area during cardiac beating similar to requirements indicated by the standard for cardiac valve prostheses (ISO 5840-1). All details concerning the model development, from segmentation to FE model setup, the numerical verification activities (i.e., element type and size, and solver settings), and in vivo population-based validation in accordance with the ASME V&V40 requirements were reported in the first part of this work.¹¹ Moreover, the accuracy in the patient-specific geometric reproducibility of the TAVI model was deeply investigated to assess the effect in the segmentation process and imaging artifacts on the resulting patient-specific geometries.¹⁷

We here investigated the case of two patients who underwent TAVI with the 23 mm S3 device and two patients who underwent TAVI with the 26 mm device. The restriction to four patients was dictated by the remarkable computational efforts needed for developing the surrogate model for probabilistic analysis. For each patient, a total of 176 structural and fluid–structure interaction (FSI) simulations being 2.81 ± 0.16 days long were carried out on a server system using 64 CPUs (AMD EPYC v75f3) in a parallel job.

B. UQ for ASME V&V40

The ASME V&V40 credibility assessment in computer modeling and simulation employs a systematic approach to guarantee the reliability and accuracy of the simulated physical phenomena. Following the specification of the question of interest, context of usage, and related model risk, a crucial component of the credibility assessment is the UQ and sensitivity analysis as defined in Part 1 of this study.¹¹ The UQ offers probabilistic distributions of model outputs, while the sensitivity analysis determines how uncertainty in the outputs might be imputed to various sources. This analysis facilitates the assessment of the variability intrinsic to the model input and the model's impact on the resultant output. To assess the model's robustness, both uncertainty quantification and sensitivity analysis must be conducted, with evaluations adhering to an adequate level of rigor.

In this study, the model risk was classified as high, indicating the necessity for a stringent credibility assessment. Indeed, the patient-specific TAVI model output can significantly influence the clinical decision-making, as inaccurate predictions could potentially result in severe patient injury. Therefore, a severity level of 5 for model credibility was chosen in the UQ of the patient-specific TAVI model, following the 5-level risk matrix outlined in ASME V&V40. Regarding the rigor of the output comparison of the quantity of interest, a severity level of 5 indicates that comparisons between the comparator (clinical data) and the computational model are based on differences of ≤5%.

Computed tomography (CT) imaging and transesophageal echocardiographic functional assessment were used as comparators of the implanted S3 devices. After ethical approval and signature of the informed consent, a prospective clinical study was performed for each patient to analyze the S3 configuration after TAVI by contrast-enhanced CT imaging. Specifically, the S3 device was assessed at three cross-sectional levels—inflow, mid, and outflow—based on diameter measurements of the deformed device by an experienced radiologist. The minimum and maximum device diameters of the S3 stent frame were measured and then the average value was used for comparison with in silico predictions. Transesophageal echocardiography on the patient under general anesthesia was performed just after the TAVI procedure to quantify the effective orifice area and transvalvular pressure gradient as quantity of interest. The range of characteristics of the test sample (i.e., one male and three females) included patient age (79–94 years), cuff systolic pressure (100–128 mm Hg), cuff diastolic pressure (47–83 mm Hg), and heart rate (62–87 bpm). Uncertainty of measurements incorporated instrument accuracy and repeatability in the device diameter measurements.

C. Uncertain parameters

A comprehensive set of uncertain parameters was evaluated in the patient-specific TAVI model, including imaging and anatomical characteristics, material properties, patient hemodynamic states, and TAVI-related procedural factors, each defined by epistemic or aleatoric uncertainty to reflect both knowledge constraints and intrinsic biological variability. Aleatoric uncertainty results from the system intrinsic randomness or stochasticity, while epistemic uncertainty derives from insufficient knowledge or modeling assumptions on the system. The uncertainties and their corresponding parameter variations are outlined as follows:

• 1.
  Imaging and segmentation: Variability in imaging techniques and resolution (e.g., CT scans) introduces inaccuracies in the segmentation of anatomical structures, which propagate through the model and affect predictions.

  • (a)
    Imaging and patient geometry: Geometric reconstructions can be influenced by semi-automatic thresholding and manual edits. To quantify geometry accuracy, we previously validated reconstructed anatomy against reference reconstructions based on the ISO50 threshold-based methodology. Results showed that calcifications exhibited the highest agreement between user-based and reference segmentations (93.4% ± 3.1%), followed by the aortic wall (85.4% ± 8.4%) and native valve leaflets (75.5% ± 6.3%). Due to the complexities involved in incorporating geometric errors into the in silico model, segmentation-related uncertainties were not explicitly considered in the present study.
  • (b)
    Aortic wall thickness: The tissue thickness of the aortic wall is crucial for accurately simulating the mechanical behavior of the tissue, but CT imaging does not allow precise quantification. For capturing variability in aortic tissue thickness, a nominal value of 2 mm with an epistemic uncertainty range of 1.5–2.5 mm based on engineering judgment of literature data was adopted.¹⁹
• 2.
  Material properties: The biomechanical behavior of the aortic root and the material properties of the S3 device depend on constitutive parameters, which are derived from experimental data with inherent uncertainties or exhibit stochastic variability.

  • (a)
    Neo-Hookean calcification parameter: This parameter describes the stiffness of the calcified aortic valve. The nominal value was set at 67.8 MPa, with an uncertainty range of 40–100 MPa. Due to the lack of experimental characterization, the bounds were determined based on engineering judgment of literature data.¹⁴
  • (b)
    Neo-Hookean aorta parameter: This parameter represents the mechanical properties of the aortic tissue wall and is critical for modeling the elastic recoil of the aortic tissue following device deployment. The nominal value was 0.98 MPa, with an aleatoric uncertainty characterized by a standard deviation of 0.24 MPa. The nominal value was obtained through inverse analysis, estimating the material parameter based on dynamic CT imaging as previously described by our group.²⁰ The aleatoric variability was derived from the standard deviation of the Neo-Hookean parameters estimated through inverse analysis in a study group of 20 TAVI patients.
  • (c)
    S3 stent-frame material parameter: The elastic material properties of the S3 stent frame are essential for determining the radial force exerted on the aortic root, which governs device expansion. The nominal value was 238.5 GPa, with an epistemic uncertainty range of 229.6–247.4 GPa and an additional aleatoric uncertainty component of 30 GPa. These values were derived from literature data and manufacturer guidelines.²¹
• 3.
  Boundary conditions: Physiological variability among patients introduces stochastic variations in boundary conditions, which are often estimated from limited clinical data or generalized assumptions. Although demographic data were collected for each patient, the boundary condition inputs were treated as aleatoric uncertainties due to inherent patient variability. These uncertainties were characterized based on data from a cohort of 100 TAVI patients treated in our hospital institution.

  • (a)
    Diastolic and systolic blood pressure: Blood pressure measurements were obtained using cuff sphygmomanometry at in-hospital admission. For diastolic pressure, the nominal value was set at 63.5 mm Hg, with an aleatoric uncertainty characterized by a standard deviation of 10.64 mm Hg. For systolic pressure, the nominal value was 127.3 mm Hg, with an aleatoric uncertainty standard deviation of 16.4 mm Hg.
  • (b)
    Velocity flow jet: Flow velocity across the S3 valve leaflets was measured using transesophageal echocardiography. The nominal value was 2.4 m/s, with an aleatoric uncertainty standard deviation of 0.5 m/s.
  • (c)
    Heart rate: Heart rate, which influences the simulation time, was recorded during the clinical work-up. For the investigated patient models, the nominal value was 71.2 bpm, with an aleatoric uncertainty standard deviation of 9.7 bpm to account for patient-specific variability.
• 4.
  Procedural variability: Procedural factors, such as valve positioning and balloon expansion, introduce variability that is often beyond the modeler's control. For each patient, procedural variables were measured using post-TAVI CT and echocardiographic imaging and were treated as a combination of epistemic and aleatoric uncertainties.

  • (a)
    Implantation depth of the S3 device: The implantation depth of the S3 device significantly influences both mechanical stability and hemodynamic performance. The nominal implantation depth was set at 6 mm, based on clinical measurements on a large population dataset, with an epistemic uncertainty range of 5–7 mm and an aleatoric uncertainty of 1 mm to account for intra-observer variability during measurement.
  • (b)
    Fluid volume during balloon expansion: The fluid volume used during balloon expansion affects the degree of under- or over-expansion of the device and consequently determines the post-TAVI device diameter. The nominal value was 17 ml, as per manufacturer guidelines, with an epistemic uncertainty range of 16–18 ml and an aleatoric uncertainty of 0.1 ml to account for typical production tolerances of the medical syringe.

Once the model inputs were classified as epistemic and/or aleatoric uncertainties, lower and upper bounds were determined and subsequently used as inputs for the design of experiments (see Table II). In particular, the bounds were simply taken by retaining the largest range of variability after each type of uncertainty was calculated.

TABLE II.

Nominal values, epistemic lower (EUQ LB) and upper (EUQ UB) uncertainty bounds, aleatoric uncertainty (AUQ), and mixed lower (mixed LB) and upper (mixed UB) uncertainty bounds of each model input.

	Nominal	EUQ LB (−2σ)	EUQ UB (+2 σ)	AUQ (σ)	Mixed LB (−2 σ)	Mixed UB (+2 σ)
Aortic thickness (mm)	5.0	1.5	2.5	⋯	1.5	2.5
C1—aorta (MPa)	0.98	⋯	⋯	0.24	0.50	1.46
C1—calcification (MPa)	67.8	40	100	⋯	40	100
S3—Young modulus (GPa)	238.4	229.6	247.4	230	224.8	25.2.2
Implantation depth (mm)	6.0	5.0	7.0	1.0	5.0	7.0
Balloon fluid volume (ml)	17.0	16.0	18.0	0.1	15.8	18.2
Diastolic pressure (mm Hg)	63.5	⋯	⋯	10.6	42.2	84.8
Systolic pressure (mm Hg)	127.3	⋯	⋯	16.4	94.4	160.2
S3 flow velocity (m/s)	2.4	⋯	⋯	0.5	1.4	2.4
Heart rate (bpm)	71.2	⋯	⋯	9.7	51.7	90.7

D. Clinical parameter evaluation and uncertainty

Uncertainty in the clinical data collected after TAVI procedure were considered to obtain a probabilistic distribution of the quantity of interest. These quantities were influenced by uncertainties related to instrument accuracy and intra-operator variability. For each quantity of interest, the level of uncertainty was quantified as follows:

• •Device diameter: Measurements of the stent frame were performed by an experienced radiologist using medical imaging software associated with the 256-row detector CT scanner (Somatom Force, Siemens, Deutschland). In our previous study, voxelization and resolution errors were found to be ≤1% and can therefore be considered negligible when compared to errors arising from model inputs.¹⁷ However, artifacts due to the metallic nature of the device can introduce a measurement error of approximately 5%. This value was used to estimate the standard deviation for CT-based measurements of the device diameter, which was subsequently used for the computation of the cumulative distribution function.
• •Valve area: The EOA was measured by an expert cardiologist using standard ultrasound system (EPIQ 7, Phillips, Netherlands). The primary sources of uncertainty were related to image resolution, voxel distortion, and probe-induced image distortion. Based on manufacturer guideline, a standard deviation of 7 mm² was assigned to EOA measurements of each patient.
• •Pressure gradient: TPG was quantified by the echocardiographic system using the Bernoulli's principles. The TPG was therefore derived by the flow jet velocity ( $v$) across the valve at systole as $\mathit{TPG}=4·v^2$. Based on manufacturer's technical sheet, a standard deviation of 4 mm Hg was considered for each measurement.

E. UQ approach and sensitivity analysis

According to the ASME V&V40, the validation assessment must account for the quantification of uncertainties in the distribution of model outputs arising from the uncertainties of the model inputs. A sensitivity analysis was conducted to assess how uncertainty in the outputs might be attributed to various sources, utilizing the probability distribution functions of the input parameters. The UQ methodology involved the initial creation and later application of a surrogate model to analyze the model's response to uncertain parameters, followed by quasi-Monte Carlo analysis for probabilistic evaluation.

1. Design of experiment

Latin hypercube sampling (LHS) was used to implement the surrogate model and thus design a finite-element simulation plan comprising 88 simulations (11 model inputs × 8 random input variable combinations). While the conventional recommendation is to use at least ten variable combinations to adequately capture the variability inherent in the model inputs, a sensitivity analysis conducted by reducing the number of random combinations from ten to seven demonstrated no appreciable deterioration in the predictive performance of the surrogate model. Although this may yield a pragmatic approach, we successfully diminished the total number of simulations and thus the computational effort without altering the resulting surrogate model development. Moreover, the LHS methodology may yield implausible combinations of input parameters, particularly for flow boundary conditions; these unphysiological configurations must be identified and excluded to avoid numerical failure and ensure model validity. Model input combinations resulting in a pressure drop between systole and diastole lower than 20 mm Hg, as well as those yielding diastolic pressures higher than systolic pressures, were excluded from the design of experiments. No measures were implemented to eliminate unrealistic combinations of material parameters and thickness values. Figure 1S of the supplementary material illustrates examples of model input combinations generated using the LHS methodology.

2. Surrogate modeling

Following each finite-element simulation defined within the design of experiments, the quantities of interest were extracted using Python scripting in Abaqus software. This automated approach accelerated the extrapolation of the large output datasets and minimized potential user error. Various surrogate modeling techniques, including the response surface method, radial basis function interpolation, and Gaussian process (GP) regression, were developed and evaluated using MATLAB (v2024, MathWorks, USA) to fit the simulation output parameters and assess model fitting accuracy. Ultimately, the GP regression approach, combined with leave-one-out (LOO) cross-validation, was selected as the optimal method. In the LOO procedure, one simulation output was excluded in each iteration, and the model was rebuilt using the remaining $N-1$ data points. GP regression approximates the simulation response as a realization of a Gaussian distribution, characterized by a mean function $\mu \left(x\right)$ and a covariance function $k\left(x,x^{\prime }\right)$, also known as the kernel. The output $y\left(x\right)$ at any input point $x$ is assumed to follow a multivariate Gaussian distribution,

$$y\left(x\right)\sim GP\left(\mu \left(x\right),k\left(x,x^{\prime }\right)\right).$$ (1)

This approach enabled efficient approximation of the numerical simulation plan response as a function of input variables while offering relatively low computational cost.

3. Probabilistic analysis and global sensitivity

For probabilistic quasi-Monte Carlo analysis, the Sobol sequence was used to sample the input space, with the samples scaled to match the specified ranges of the input variables. This method allowed the Monte Carlo analysis to generate a distribution of possible outcomes by repeatedly sampling from the input distributions, providing a comprehensive understanding of the range and likelihood of various model responses. The surrogate model was employed in place of the original numerical simulations to efficiently generate deterministic data points for variance-based sensitivity analysis and reducing computational demands.

Sensitivity analysis was based on Pareto plots to identify the input parameters with the most significant impact on the uncertainty of the model outputs. The total effect index (TEI) quantified each input parameter contribution to the overall output variance, with the results visually represented in Pareto plots for interpretation of the most influential input parameters on the quantities of interest.

The TEI for a given input parameter $X_i$ is computed as

$$\mathit{TEI}\left(X_i\right)=1-\frac{\mathit{\text{Var}}\left(Y|X_i\right)}{\mathit{\text{Var}}\left(Y\right)},$$ (2)

where $\mathit{\text{Var}}\left(Y\right)$ is the total variance of the output $Y$, $\mathit{\text{Var}}\left(Y|X_i\right)$ is the conditional variance of $Y$ when all parameters except $X_i$ are fixed.

For validation, the cumulative distribution function (CDF) of the surrogate-derived parameters was calculated for each quantity of interest estimated by both the in silico model and the in vivo estimates. The CDF represents the probability that a variable will take a value less than or equal to a specified threshold. To assess the agreement between the clinical and in silico estimates, the area under the surrogate-derived and clinical CDFs (referred to as the area metric) was computed. The percentage difference between these areas was then calculated to evaluate model accuracy.

SUPPLEMENTARY MATERIAL

See the supplementary material of the article presents a detailed analysis of the parameters influencing the behavior of the S3 device in in silico models. Figure 1S displays Latin hypercube sampling of model inputs, highlighting interactions between parameters such as aortic wall thickness, device stiffness, and calcification properties, along with hemodynamic variables like heart rate and systolic/diastolic pressure. Figure 2S shows a Pareto plot illustrating the sensitivity of input parameters on the device diameter for the 23 and 26 mm S3 models, analyzed across different cross sections. Figure 3S extends the sensitivity analysis to hemodynamic outputs, including EOA and TPG, again for both device sizes. Figures 4S and 5S compare clinical and in silico data using CDF curves, highlighting the area metric enclosed between the curves (gray area) for device diameter, EOA, and TPG. These analyses provide a quantitative measure of agreement between simulations and clinical observations, supporting model validation and identifying the most influential parameters affecting device performance.

AUTHOR DECLARATIONS

Conflict of Interest

The authors have no conflicts to disclose.

Ethics Approval

Ethics approval for experiments reported in the submitted manuscript on animal or human subjects was granted. This study was approved by the IRCCS ISMETT Ethics Committee (Approval No. IRRB04/04). All participants provided written informed consent prior to enrollment in the study.

Author Contributions

Roberta Scuoppo: Data curation (equal); Formal analysis (equal); Writing – original draft (equal); Writing – review & editing (equal). Chiara Catalano: Conceptualization (equal); Data curation (equal); Writing – original draft (equal); Writing – review & editing (equal). Tahir Turgut: Formal analysis (equal); Investigation (equal); Software (equal). Nils Götzen: Conceptualization (equal); Funding acquisition (equal); Methodology (equal). Stefano Cannata: Data curation (equal); Validation (equal). Giovanni Gentile: Formal analysis (equal); Supervision (equal). Caterina Gandolfo: Formal analysis (equal). Salvatore Pasta: Conceptualization (equal); Funding acquisition (equal); Writing – original draft (equal); Writing – review & editing (equal).

DATA AVAILABILITY

The data that support the findings of this study are available from the corresponding author upon reasonable request.