Validation of the Obesity Lifecycle Model Built to Assess the Cost Effectiveness of Novel Therapies in the Management of Obesity

Abstract

Background

Economic models support healthcare decision makers to efficiently assess value and allocate resources; formal validation is critical to ensure confidence in model outputs. The Obesity Lifecycle Model is a patient-level simulation model capturing natural history, clinical complications, quality-of-life and economic outcomes associated with obesity; however, it has yet to undergo formal external and cross-validation.

Objective

The aim of this study was to validate the Obesity Lifecyle Model as per best practice guidelines from the Professional Society for Health Economics and Outcomes Research (ISPOR) and the Society for Medical Decision Making (SMDM) Task Force.

Methods

Relevant data sources and outcomes were identified for all validations. Selected validation studies informed the model to simulate study outcomes. The model was populated with study characteristics to reproduce studies used in model development (dependent validation), studies not used in model development (independent validation), or other published models (cross-validation). Accuracy between predicted and observed outcomes was assessed using standard statistical methods and mean error calculations.

Results

The model demonstrated overall concordance with observed outcomes, supported by coefficient of determination (R²) and ordinary least squares linear regression line (OLS LRL) estimates generally close to 1.0. Independent validation showed an underprediction for cardiovascular disease (CVD) and mortality with an OLS LRL of 0.9309 and an R² of 0.8984 across all populations (normoglycaemic/prediabetic populations: OLS LRL = 0.9485, R² = 0.8854; T2DM populations: OLS LRL = 0.9208, R² = 0.9068).

Conclusions

The Obesity Lifecycle Model demonstrates favourable concordance with observed clinical and quality-of-life outcomes, supporting its use for evaluating obesity-related complications and informing healthcare decision making.

Supplementary Information

The online version contains supplementary material available at 10.1007/s41669-026-00661-y.

Key Points for Decision Makers

The rising global prevalence of obesity places increasing strain on healthcare systems, highlighting the need for well-validated economic models to support reimbursement decisions and resource planning.

To address this need, we conducted external validation and cross-validation of the Obesity Lifecycle Model, following best-practice guidelines from the Professional Society for Health Economics and Outcomes Research (ISPOR) and the Society for Medical Decision Making (SMDM).

Results indicated alignment between the model’s predictions and observed outcomes, demonstrating the potential practical value of the Obesity Lifecycle Model for informing healthcare decision making in obesity management.

Introduction

Obesity is a multifactorial chronic condition characterised by excessive fat accumulation and a body mass index (BMI) of ≥30 kg/m² [1]. In 2022, the number of adults living with obesity surpassed 890 million, accounting for ~16% of the global adult population, and its prevalence continues to increase [1]. As an established risk factor for several chronic illnesses, such as type 2 diabetes mellitus (T2DM), cardiovascular disease (CVD) and chronic kidney disease (CKD), obesity is associated with long-term clinical complications, leading to high morbidity, impaired health-related quality of life (HRQoL) and increased mortality [2] compared with individuals of healthy weight (BMI: 18.5–24.9 kg/m²) [2, 3]. Obesity represents a major public health challenge [3], placing considerable financial burden on individuals and healthcare budgets, driven mainly by direct healthcare costs for the treatment of obesity-related complications [4], as well as indirect costs caused by loss of productivity and work absenteeism [5].

Ensuring effective implementation of efficacious and safe weight management strategies is necessary to prevent further exacerbation of obesity-related burden. The increasing demand on healthcare resources worldwide, coupled with the growing prevalence of obesity, signifies the need for policy and reimbursement decision making to include economic evaluations. To support decision makers in determining optimal resource allocation and spending, economic models provide important insights into the long-term clinical and economic implications of obesity and facilitate comparisons of weight management strategies [6]. The Professional Society for Health Economics and Outcomes Research (ISPOR) and the Society for Medical Decision Making (SMDM) Task Force advocate that decision makers must have trust and confidence in the accuracy of a model’s results for it to be appropriate for use, and have issued ‘best practice’ guidelines for assessing model quality [7]. An integral component of these recommendations is the requirement for transparent model validation, a technique to determine the accuracy of model predictions compared with observed outcomes in clinical practice [7].

According to ISPOR/SMDM best practice guidelines, model validation comprises five core elements: face validity, verification, cross-validity, external validity and predictive validity [7]. Briefly, face validity assesses the degree to which a model and its components (e.g., model structure, assumptions, outputs, etc.) align with scientific consensus about the disease and its natural progression. Verification assesses whether a model and its calculations perform correctly and align with expectations. Cross-validation involves comparing a particular model against other models to evaluate how well the results align. For external validity, a model is used to simulate an existing scenario and results are compared to determine how well the predicted outcomes align with those observed in the evidence. Meanwhile, predictive validity quantifies the accuracy of a model’s future projections [7].

Few published obesity models have undergone formal validation, meaning the predictive accuracy of most obesity models is largely unknown and/or not reported [8]. However, a recent external validation of the Core Obesity Model (a closed-cohort Markov model designed to predict the economic impact of obesity-related complications) demonstrated favourable alignment between the Core Obesity Model predictions and the results from relevant published studies [9]. This validation process demonstrated the Core Obesity Model’s prediction accuracy and identified areas for further development for future models, such as capturing the impact of existing CVD risk and mortality.

Different models can capture varying perspectives, assumptions, patient profiles and clinical outcomes associated with a disease. Therefore, there remains a need to validate other obesity models to ensure a comprehensive and robust understanding of the obesity-related complications among a diverse patient population. The Obesity Lifecycle Model is a patient-level simulation model designed to predict the natural history, clinical complications, quality of life (QoL) impact and economic outcomes associated with obesity, encompassing a broad range of obesity and diabetes-related complications. As a patient-level simulation model, the Obesity Lifecycle Model accounts for the heterogeneity and complex interdependent risk factors in people with obesity, providing insights into the long-term trajectory of patients [10].

The Obesity Lifecycle Model underwent face validity and verification checks during its development. To further assess whether the Obesity Lifecycle Model is a valuable tool to support decision makers, this study conducted a formal validation (focusing on dependent and independent external validation, and cross-validation) of the model and its predictions for CVD and mortality outcomes, T2DM incidence, life years (LYs) and quality-adjusted life years (QALYs) against relevant published data, in line with ISPOR/SMDM best practice guidelines.

Methods

Model Overview

The Obesity Lifecycle Model is a fixed (annual) time-increment patient-level simulation model designed to simulate the natural history, clinical complications, QoL impact and economic implications of obesity. The model was built in C++ with Microsoft Excel^® interface. Patients enter the model based on sampled baseline characteristics and are divided into three subpopulations according to the patient’s glycaemic status—normoglycaemic, prediabetic, or established T2DM. A summary schematic of the model and its simulation flow is presented in Figure 1. The costs and disutilities of common complications and comorbidities associated with obesity are accrued in annual cycles up to the lifetime horizon of the model. In the base case, the model employs a UK perspective and is built to compare novel interventions for obesity versus existing comparators.

Fig. 1.

The Obesity Lifecycle Model schematic. *CVD events are simulated as a composite event including unstable angina, myocardial infarction, stroke and transient ischaemic attack. Note: a simplifying assumption was applied that each patient could experience a maximum of two CVD events (irrespective of glycaemic status) to avoid unnecessary model complexity and to align with structural modelling assumptions observed in literature. CVD cardiovascular disease, ESRD end-stage renal disease, PSA probabilistic sensitivity analysis, T2DM type 2 diabetes mellitus

At the beginning of a simulation, patient profiles are defined in terms of baseline demographics and characteristics (e.g., age, sex, ethnicity, glycaemic status, comorbidity history, smoking status, etc.) which are updated annually over the simulation period. The model was designed to allow patient risk factor progression (e.g., weight, systolic and diastolic blood pressure, total cholesterol, HbA1c, high-density and low-density lipoprotein cholesterol) to be modelled at a constant rate over time (i.e., linear slopes). During each annual cycle, the occurrence of fatal or non-fatal events is assessed using a Monte Carlo simulation approach [11], as described next. Risk equations are applied to estimate the probability of an event within each cycle. Each probability is compared with a uniformly distributed random number ranging from 0 to 1; an event is considered to occur if the random number is less than the calculated probability. Case-fatality proportions were applied uniformly across all ages. If a patient experiences an event, their risk profile is updated and subsequent risks are recalculated based on their revised health state, thus accounting for potential competing risks.

Risk equations previously validated in large and diverse populations were selected based on their compatibility with the model’s structure and data requirements. For people entering the Obesity Lifecycle Model without T2DM (i.e., normoglycaemic and prediabetic), the Qdiabetes-2018 risk model is used to estimate the probability of T2DM onset [12]. The QRISK3 risk model is employed to estimate the probability of primary CVD events in patients without T2DM [12], while the Recurring Coronary Heart Disease Risk Model from the Framingham Heart Study is used for secondary CVD events in patients without T2DM [13, 14]. Notably, the QRISK3 risk prediction algorithm generated composite CVD risk scores, which were subsequently split into individual CVD component risk scores using CVD proportions derived from the Framingham Heart Study [13, 15, 16]. For individuals entering the Obesity Lifecycle Model with established T2DM, the UK Prospective Diabetes Study (UKPDS) 82 risk equations are employed to estimate the probability of primary and recurrent CVD events and T2DM-related complications, including amputation, blindness, end-stage renal disease and ulceration [17]. Both the QRISK3 and UKPDS 82 risk equations incorporate age as a predictor of overall CVD risk. Risk equations used to inform the probability of other clinical events (i.e., knee replacement, osteoarthritis, asthma, gallstones, hypertension, colorectal cancer, endometrial cancer and obstructive sleep apnoea) are summarised in Supplementary Table S1 (see electronic supplementary material [ESM]); however, these events are not a focus of the validation exercises conducted. The associated risk of all-cause mortality (ACM; based on comorbidity-adjusted UK life tables, with BMI and post-event adjustments) and clinical event mortality is modelled per cycle.

Validation Approach

As previously noted, model validation consists of five key elements: face validity, verification, cross-validity, external validity and predictive validity (Supplementary Table S2, see ESM) [7]. While each type of validation contributes to the assessment of the robustness of economic models, ISPOR/SMDM states external validity is one of “the strongest form(s) of validation” [7]. Consequently, this study primarily focused on the external (dependent and independent) validation of the Obesity Lifecycle Model. To further assess model credibility, the current version of the Obesity Lifecycle Model was also subjected to cross-validation. These validations build upon the face validity and verification tests that were undertaken during development of the Obesity Lifecycle Model (ESM).

Dependent and Independent External Validation

Based on ISPOR/SMDM best practice recommendations for conducting an external validation, three main methodological phases were conducted: Phase 1 identifying data sources to simulate, Phase 2 simulating the identified sources and Phase 3 comparing and assessing correlations.

Phase 1: Identifying Data Sources to Simulate

An external validation is classed as dependent when the same data sources used to estimate model parameters are used to validate the model. Accordingly, the data sources that informed the development of the Obesity Lifecycle Model were selected for dependent external validation (Table 1).

Table 1.

Summary of studies/data sources for dependent external validation

Study	Endpoint	Risk equation	Treatments/sub-groups included in validation
Hippisley-Cox et al. [54]	Cardiovascular disease, defined as a composite of coronary heart disease, ischaemic stroke, or transient ischaemic attack	QRISK3	No treatment was considered
UKPDS 80 [17, 32]	MI, stroke	UKPDS 82	Metformin group, sulfonylurea-insulin group
Hippisley-Cox and Coupland [12]	T2DM incidence	QDiabetes-2018	No treatment was considered

MI myocardial infarction, T2DM type 2 diabetes mellitus, UKPDS UK Prospective Diabetes Study

On the other hand, an external validation is independent if no information from the data source was used when building the model. According to ISPOR/SMDM recommendations, data sources for independent external validation should provide sufficient detail to enable replication of the design and progression of the study or data source. To identify the sources for independent external validation, a published external validation of the Core Obesity Model was first consulted, given its alignment with the objectives of this analysis, and a hand search of the citations was undertaken [9]. As the Core Obesity Model validation was published in 2020, a subsequent targeted literature search was conducted in Ovid MEDLINE, PubMed and Google Scholar from 2019 to November 2024 using combinations of free-text keywords (Supplementary Table S3, ESM) to capture any additional potentially relevant studies published thereafter. Given that the Obesity Lifecycle Model was developed to assess the long-term clinical and economic implications associated with obesity, publications including overweight or obese populations, patients with diabetes and those with CVD, as well as a sufficient length of follow-up to allow the assessment of CVD outcomes and mortality, were prioritised. Several inclusion criteria were also applied to ensure the most relevant studies for independent external validation were selected (Supplementary Table S4, ESM). First, the studies were required to have CVD and mortality endpoints (e.g., myocardial infarction [MI], stroke, angina or unstable angina, heart failure [HF], CV death and ACM) as event rates or total number of observed events. Second, studies required sufficient description of the baseline characteristics and long-term risk factor trajectories to facilitate replication in the modelling exercise. Last, studies required a follow-up of ≥2 years and a patient population ≥1000. Notably, studies from non-representative geographical regions or with non-generalisable patient profiles (e.g., cohorts with extreme age restrictions, specialised clinical cohorts, or regions with atypical epidemiology) were excluded. During the targeted literature search, we identified a pivotal “Call for Action” publication summarising the evidence base for therapies reducing cardiovascular and renal risk in clinical practice, predominantly in patients with T2DM, which appeared relevant to the populations in the model [18]. Therefore, we conducted a pragmatic hand search of the citations to further supplement our initial search findings.

Briefly, the independent external validation of CVD outcomes and mortality in normoglycaemic or prediabetic populations was performed using data derived from six randomised controlled trials (RCTs) [19–25] and six observational studies [26–33]. An additional nine studies were identified to inform the independent external validation of CVD outcomes and mortality in T2DM populations [34–44]. Four studies were selected from the literature searches to facilitate the independent external validation of T2DM incidence [19, 22–24, 45] (Supplementary Table S5, ESM).

Life expectancy predictions were validated against UK life table data from 2020 to 2022 [46].

Phase 2: Simulating the Identified Sources

Endpoints derived from the selected validation studies were incorporated into the analysis only if their clinical definitions matched or closely resembled those utilised in the Obesity Lifecycle Model, ensuring that significant discrepancies were eliminated and meaningful comparisons could be conducted. Where studies reported composite endpoints, the feasibility of aggregating the individual endpoint components of the Obesity Lifecycle Model to estimate the composite endpoint was evaluated. To determine external validity, the following clinical events were assessed: onset of T2DM, fatal or non-fatal MI, fatal or non-fatal stroke, fatal or non-fatal HF, fatal or non-fatal angina, CV death (composite of fatal MI, fatal angina, fatal HF and fatal stroke), ACM and life expectancy. External validations against these clinical events were subsequently grouped into three main categories for the purpose of data visualisation: CVD and mortality, T2DM incidence and life expectancy. To enable a more descriptive presentation of external validation findings, CVD and mortality validation outcomes were further stratified by study type (RCTs or observational studies), baseline diabetes status (normoglycaemic/prediabetic or established T2DM) and by individual CVD outcomes (MI, stroke, HF, angina, CV death, ACM).

For each external validation exercise, the Obesity Lifecyle Model was populated using the baseline clinical characteristics and demographic data reported for the respective treatment arms of the validation study. When specific baseline parameters were not available, data from an alternative study (within the previously selected validation set) whose population was deemed most comparable, based on factors such as age distribution, prevalence and duration of T2DM and comorbidity profile, were used to fill the data gaps (Supplementary Worksheet). The Obesity Lifecycle Model incorporated short-term treatment effects (Year 1) and long-term risk factor trajectories (Year 2 onwards), where such data were available from the relevant validation study. When long-term trajectories were not reported, the model assumed no further change in the parameter over time. An exception was HbA1c, which in the absence of long-term data, was modelled using a natural progression approach based on the random effects panel equation from the UKPDS 68 study [47]. This exception was applied to ensure the model captured the well-established trend for HbA1c to rise over time in T2DM [48–50], unlike other risk factors whose long-term trajectories are less pronounced.

The model time horizon was adapted to match the mean or median follow-up duration of each relevant validation study. When comparing the Obesity Lifecycle Model (which uses annual cycles) with validation studies that reported non-integer follow-up durations, the model used upper and lower integer years and applied linear interpolation to match the prediction timeframe of the study follow-up. The number of patients included in the simulation was also adapted to match the cohort size of each individual validation study and run over 1000 simulations to stabilise estimates. For comparisons between predicted and observed outcomes, an average of the 1000 deterministic simulations (i.e., all inputs fixed at their mean values, with random seed differing between runs) was evaluated against the respective validation studies. The previously defined risk equations (see Model Overview) were applied consistently across all validation exercises without modification.

Phase 3: Comparing and Assessing Correlations

While there are a number of statistical tests appropriate for comparing model predictions with observed outcomes, there is a lack of consensus on the optimal approach for health economic modelling. This study adopts an approach in accordance with a prior validation of the Economic and Health Outcomes Model of T2DM (ECHO-T2DM) [51]. First, the model fit was visually inspected by plotting observed study endpoints (X-axis) against cumulative incidences of predicted outcomes (Y-axis). Perfect concordance would place points along the 45° identity line (IL). Overprediction would result in many points above the IL, while underprediction would place most points below the IL. Second, an ordinary least squares linear regression line (OLS LRL) was fitted to the data to quantify how the model’s predicted outcomes diverged from the observed outcomes, as per other published model validations [9, 51]. The OLS LRL was constrained to pass through the origin (intercept = 0) to facilitate direct interpretation of the OLS LRL slope. An OLS LRL slope below 1.0 indicated that the model tended to underpredict observed values, while a slope above 1.0 indicated overprediction by the model. Third, to quantify the magnitude of discordance of individual validation outcomes relative to the OLS LRL, the coefficient of determination (R²) was calculated for all results. The R² was interpreted alongside a visual inspection of the data, the fitted OLS LRL slope and the IL. For example, if the OLS LRL was identical to the IL (slope = 1.0), an R² value close to 1.0 indicated minimal discordance between predicted and observed data. Conversely, when the OLS LRL slope substantially differed from 1.0, an R² value close to 1.0 suggested a strong association between predicted and observed outcomes, but with an over- or underestimation trend. Together, the OLS LRL and R² offer intuitive and transparent measures of concordance between predicted and observed values and were selected to assess model performance as they are well-established statistical methods widely adopted in other validation studies [9, 51].

To evaluate model fit and quantify discrepancies between predicted and observed outcomes, several mean error metrics were calculated, including the mean absolute percentage error (MAPE), root mean squared percentage error (RMSPE), mean squared log of accuracy ratio (MSLAR) and mean squared root logit error (MSLE), following the methodology outlined by Willis et al. [51]. These metrics provide complementary perspectives on prediction accuracy. A value of zero indicates perfect concordance between predicted and observed outcomes, while increasing values reflect greater deviation and reduced model fit.

Cross-Validation

Cross-validation refers to the process of simulating the same decision problem using different models and comparing their predictions. The methodologies for cross-validation and external validation are largely comparable, though details specific to cross-validation and key differences are outlined in this section.

The Obesity Lifecycle Model underwent cross-validation as per ISPOR/SMDM recommendations, which consisted of a comparison between predicted model outputs and those reported in the National Institute for Health and Care Excellence (NICE) model informing the semaglutide technology appraisal [ID3850] [52]. Simulation of the validation source followed the same approach used for external validation, where the Obesity Lifecycle Model was populated with the baseline patient characteristics, treatment effects (changes in BMI, systolic blood pressure, total and high-density lipoprotein cholesterol and HbA1c) and assumptions of the population used in the NICE semaglutide model. As some data were not available in the NICE submission documentation, evidence gaps were filled by reverting to the pivotal clinical trial publications. As the NICE semaglutide technology appraisal [ID3850] conducted a subgroup analysis comparing semaglutide versus liraglutide in patients with a BMI of ≥35 kg/m² with non-diabetic hyperglycaemia and a high risk of CVD, submission documentation for liraglutide was also reviewed for relevant data to inform the cross-validation exercise. The costs, and subsequently the incremental cost-effectiveness ratio (ICER), with the list price for semaglutide, were redacted in all submission documents. As a result, the cross-validation focused solely on comparisons between the predicted QALY and LY gains from the Obesity Lifecycle Model versus the respective outcomes available from the NICE submission documents. Given that QALYs and LYs were calculated based on the intermediary clinical outcomes described earlier, their validation was also sought as a reflection of the validity of the model’s previous modelling steps. To assess concordance between the predicted and observed outcomes, total and incremental QALYs were plotted and compared.

To assess the consistency of cross-validation findings across different model settings and inputs, QALY and LY predictions from the Obesity Lifecycle Model were further validated against a published cost-effectiveness analysis of semaglutide 2.4 mg in adult patients with obesity or overweight with one or more weight-related comorbidities from a US payer perspective [53], using the same methodology described herein.

Scenario Analysis

Given the cross-validation exercises focused on comparisons between predicted and observed QALY gains, several scenario analyses were conducted to explore the impact of alternative methods for modelling the relationship between BMI and HRQoL, to assess whether predictions against the NICE semaglutide technology appraisal [ID3850] [52] were robust across different methodological approaches (ESM).

Results

Dependent External Validation

CVD and Mortality

The dependent external validations for CVD and mortality were performed against the QRISK3 cohort study and the UKPDS 80 [32, 54]. Overall, CVD and mortality was underestimated by the Obesity Lifecycle Model, with an estimated OLS LRL of 0.8998 (Fig. 2a). An R² value of 0.9874 indicates that model predictions explain a high proportion of the variance in observed outcomes, consistent with a strong association between predicted and observed values (Fig. 2a, Supplementary Table S6, ESM).

Fig. 2.

Dependent external validation outcomes of a CVD and mortality, b T2DM incidence. The dotted line is the identity line and the solid line is the OLS LRL. The shaded area around the solid line represents the 95% confidence interval of the OLS LRL. CV cardiovascular, MI myocardial infarction, OLS LRL ordinary least-squares linear regression line, T2DM type 2 diabetes mellitus

T2DM Incidence

A retrospective database analysis, QDiabetes-2018 [12], was used to inform dependent external validation of T2DM incidence. Overall, the incidence of T2DM was underestimated by the Obesity Lifecycle Model, with an estimated OLS LRL of 0.8200 (Fig. 2b, Supplementary Table S6, ESM). This was accompanied by an R² value of 0.9824, indicating that predicted and observed outcomes are strongly associated and the pattern of variation across study outcomes is well captured by the model predictions.

Mean Error Estimates

Table 2 summarises the mean error estimates from the dependent external validations of the Obesity Lifecycle Model. The model demonstrated best fit for CVD and mortality outcomes, with MAPE, RMSPE, MSLAR and MSLE values of 14.4, 19.5, 3.7 and 1.8%, respectively. In comparison, mean error estimates for T2DM incidence were 20.4, 22.0, 6.3 and 4.4%, respectively.

Table 2.

Summary of external validation mean error estimates

	MAPE (%)	RMSPE (%)	MSLAR (%)	MSLE (%)	R2	OLS LRL
Dependent validation
CVD and mortality	14.4	19.5	3.7	1.8	0.9874	0.8998
T2DM incidence	20.4	22.0	6.3	4.4	0.9824	0.8200
Independent validation
CVD and mortality	50.8	83.4	27.5	16.6	0.8984	0.9309
T2DM incidence	15.2	21.5	5.3	4.9	0.8768	0.7004

CVD cardiovascular disease, MAPE mean absolute percentage error, MSLAR mean squared log of accuracy ratio, MSLE mean squared root logit error, OLS LRL ordinary least squares linear regression line, R² coefficient of determination, RMSPE root mean squared percentage error, T2DM type 2 diabetes mellitus

Independent External Validation

CVD and Mortality

Independent external validations for CVD and mortality were conducted for a total of 284 validation endpoints across treatment arms of 21 studies, including 12 studies in normoglycaemic and prediabetic populations (164 endpoints) and nine studies in T2DM populations (120 endpoints). In normoglycaemic and prediabetic populations, there were six RCTs comparing blood pressure-lowering or lipid-lowering interventions (93 endpoints) and six observational cohort studies (71 endpoints). For T2DM populations, eight RCTs comparing blood pressure–lowering or lipid-lowering interventions (116 endpoints) and one observational cohort study (4 endpoints) were used for independent external validations. Across all 21 validation studies, including T2DM, normoglycaemic and prediabetic populations, the incidence of study outcomes was underestimated by the Obesity Lifecyle Model (OLS LRL = 0.9309 and R² = 0.8984, Fig. 3a).

Fig. 3.

Independent external validation outcomes of a CVD and mortality, b T2DM incidence. The dotted line is the identity line and the solid line is the OLS LRL. The shaded area around the solid line represents the 95% confidence interval of the OLS LRL. ACM all-cause mortality, CV cardiovascular, HF heart failure, MI myocardial infarction, OLS LRL ordinary least-squares linear regression line, T2DM type 2 diabetes mellitus

Stratification by diabetes status demonstrated an underprediction in both normoglycaemic/prediabetic populations (OLS LRL = 0.9485) and T2DM populations (OLS LRL = 0.9208), with R² values of 0.8854 and 0.9068 for the two groups, respectively (Supplementary Fig. 1, Supplementary Tables S7–S8, ESM). For normoglycaemic or prediabetic populations, validation against RCTs was associated with an overprediction, accompanied by an OLS LRL of 1.1092, in comparison to an underprediction against observational cohort studies with an OLS LRL slope of 0.7625 (Supplementary Fig. 2, ESM). Analysis of individual CVD outcomes indicates an underprediction for angina (OLS RLR = 0.7952), HF (OLS RLR = 0.4712), stroke (OLS RLR = 0.9332), CV death (OLS RLR = 0.9208), MI (OLS RLR = 0.9819) and ACM (OLS LRL = 0.9725), with favourable R² statistics (range: 0.7427–0.9275) for all endpoints (Supplementary Fig. 3, ESM).

T2DM Incidence

Independent external validations for T2DM incidence were based on four studies. The observed versus predicted validation outcomes are presented in Fig. 3b (Supplementary Table S9, ESM), demonstrating an underprediction compared with published study outcomes, accompanied by an OLS LRL slope of 0.7004 and an R² of 0.8768.

Life Expectancy

Independent external validation of life expectancy predictions from the Obesity Lifecycle Model was compared with age and sex-specific UK life table data. Results demonstrated there was an underestimation between the observed and predicted outcomes for life expectancy, with an OLS LRL slope of 0.9762 and an R² of 0.9998.

Mean Error Estimates

Mean error rate outcomes for the independent external validations of the Obesity Lifecycle Model are summarised in Table 2. Overall, the model demonstrated better predictive accuracy for T2DM incidence, with MAPE, RMSPE, MSLAR and MSLE values of 15.2, 21.5, 5.3 and 4.9%, respectively. In comparison, mean error estimates for CVD and mortality were 50.8, 83.4, 27.5 and 16.6%, respectively.

Cross-Validation

Cross-validation of QALY and LY predictions from the Obesity Lifecycle Model was conducted using observed outcomes from the NICE model that informed the semaglutide technology appraisal [ID3850] [52]. Under base-case settings, the model predictions generally aligned with the outcomes reported in the NICE submission (Supplementary Fig. 4, ESM). Scenario analyses explored different methods for modelling the relationship between BMI and HRQoL. The scenario using BMI-related utility adjustments from Søltoft et al. [55] most closely replicated the QALY outcomes reported in the NICE submission. In contrast, the scenario applying a less conservative fixed BMI utility decrement from Dixon et al. [56] produced the greatest divergence between predicted and observed outcomes (Supplementary Fig. 4, ESM).

An additional cross-validation exercise was performed using subgroup data from the NICE submission comparing semaglutide versus liraglutide. Results showed that using the Søltoft et al. [55] baseline utility approach most closely replicated the observed QALYs, while using the Dixon et al. [56] fixed BMI utility decrement resulted in the largest deviation from observed outcomes (Supplementary Fig. 5).

The Obesity Lifecycle Model’s QALY and LY predictions were generally consistent with observed outcomes from a published cost-effectiveness analysis by Kim et al. [53], though model predictions were higher under base-case conditions (Supplementary Fig. 6, ESM). For comparisons of semaglutide 2.4 mg versus diet and exercise, the Obesity Lifecycle Model predicted incremental gains of 0.205 QALYs and 0.102 LYs, while Kim et al. [53] reported incremental gains of 0.181 QALYs and 0.061 LYs.

Discussion

The clinical and economic burden of obesity causes significant challenges to healthcare systems globally, with rising prevalence rates contributing to increased resource demands [57]. For health economic models to effectively support healthcare decision making and resource allocation, they must undergo formal validation to ensure credibility and accuracy [7, 58]. Therefore, this study presents the external validation (dependent and independent) and cross-validation of the Obesity Lifecycle Model, a patient-level simulation model designed to predict the clinical and quality-of-life impacts of obesity and its complications.

Results of the dependent external validation showed good concordance with comparator studies, indicating model predictions were generally consistent and accurate. The slope of the OLS LRL indicated an underprediction for CVD and mortality (OLS LRL: 0.8998) and T2DM (OLS LRL: 0.8200). The underprediction for CVD and mortality was primarily driven by an underestimation of MI and stroke outcomes when compared with the UKPDS 80 study [17, 32]. Conversely, the underprediction for T2DM incidence likely reflects methodological constraints in reconstructing the original derivation cohort for the Qdiabetes-2018 risk model, including limitations in replicating ethnicity-specific baseline characteristics and the need to make assumptions due to missing data. Notably, since the validation considered a 20-year diabetes duration and included some elements excluded from UKPDS, such as mortality risk, the model was anticipated to show some underprediction. The predictive accuracy of the Obesity Lifecycle Model was further demonstrated by favourable mean error estimates, particularly the MAPE (14.4% for CVD and mortality; 20.4% for T2DM incidence), which are comparable with statistics reported by other validated models [9, 59].

Overall, independent external validations of the Obesity Lifecyle Model demonstrated there was a level of underprediction for CVD and mortality outcomes, as well as T2DM incidence, which was comparable with trends observed in other model validations; for example, an independent external validation of the Core Obesity Model demonstrated an underprediction of CVD and mortality outcomes, with an OLS LRL of 0.811 and an R² of 0.819 [9]. Stratifying validation outcomes by individual CVD endpoints showed that our underprediction was driven primarily by HF and angina events. Of note, the number of data points from studies reporting HF and angina events was lower compared with other events, increasing the relative influence of outliers on the overall findings. For example, excluding two angina outliers improved the slope estimate (OLS LRL: 1.1443; R²: 0.8042); however, this analysis was solely conducted to illustrate the strong influence of outliers on these outcomes, and all data points were included in the validation to avoid bias. The underprediction of HF events may be due to the use of a non-HF-specific risk equation (QRISK3) in the model. The QRISK3 algorithm predicts the risk of CVD, whose association with BMI is more modest than the strong risk association between BMI and HF, thus may have contributed to the underestimation observed by the model. This is a potential limitation that could be addressed in future iterations of the Obesity Lifecycle Model by utilising an HF-specific risk equation, such as the HUNT 3 risk equation [60]. Furthermore, most studies reporting HF and angina were designed to research patients with hypertension or T2DM, with only one study explicitly reporting overweight or obesity as inclusion criteria, thus potentially affecting generalisability.

Stratifying independent external validation outcomes by study design demonstrated an overprediction trend for RCT populations, but an underprediction trend in observational cohorts. This discrepancy likely reflects the model’s use of risk equations derived from large observational datasets, which are more representative of clinical practice compared with the more controlled settings of RCTs. Compared with clinical practice, patients enrolled in RCTs typically receive more regular monitoring and more intensive management, meaning event rates are often lower than those observed in routine care. Therefore, models calibrated to risk equations derived from real-world data, such as the Obesity Lifecycle Model, may tend to overpredict outcomes in RCT populations. With these contextual differences in mind, this discrepancy between validation outcomes by study design is somewhat anticipated.

Notably, the external validation outcomes presented here are broadly consistent with other published obesity model validations, such as the external validation of the Core Obesity Model [9]. While the general similarity between results provides some confidence in our findings, it is important to recognise that the Obesity Lifecyle Model is a microsimulation model, which differs in several key aspects that may have influenced model predictions, thereby impacting validation outcomes. Furthermore, the Obesity Lifecycle Model incorporates a broad range of obesity-related complications and further predicts diabetes-related complications for patients who progress to T2DM. While these additions provide a more comprehensive representation of the potential health burden associated with obesity, this may contribute to some differences observed in the findings between the Obesity Lifecycle Model and other published obesity model validations.

Cross-validation of QALY and LY predictions against the NICE semaglutide appraisal [ID3850] [52] showed good overall concordance under base-case assumptions. Considering the lack of standardised guidelines for modelling the association between BMI and HRQoL, scenario analyses explored different methods for modelling this relationship. The polynomial approach from Søltoft et al. [55] yielded QALY predictions most consistent with NICE outcomes, while applying the linear decrement reported in Dixon et al. [56] produced less accurate predictions. These findings are consistent with expectations, given the use of the Søltoft et al. [55] method in the NICE submission. Additional cross-validation against Kim et al. [53] further supported the robustness of QALY predictions.

There are several strengths of these validations, which add robustness to the conclusions drawn from this study. First, both external and cross-validation exercises were conducted in accordance with established best-practice guidelines from ISPOR/SMDM [7]. Second, in recognition of the lack of consensus on optimal methods for assessing model concordance, multiple goodness-of-fit measures were employed, including visual inspection, R² statistics, IL and OLS LRL. Third, the independent external validation of CVD and mortality was conducted across an extensive range of publications, covering 284 validation endpoints from 21 studies, including 12 studies in normoglycaemic and prediabetic populations (164 endpoints) and nine studies in T2DM populations (120 endpoints). These studies included both RCTs and observational cohorts across several countries and time periods, suggesting a level of generalisability to our findings.

As with all validation studies, there are some limitations to be acknowledged. One limitation relates to matching model endpoints with the outcomes reported in the validation studies. Some endpoints were excluded from the validation due to their absence in the model; however, other endpoints with imperfect matches were incorporated in the analyses, which may introduce some discrepancies if the validation is repeated. Furthermore, the Obesity Lifecycle Model used population averages for input parameters, which may not reflect the heterogeneity of real-world populations. Simplifying patient heterogeneity and its associated non-linear effects may result in an underestimation of event rates, which may explain some of the differences between the model’s predictions and observed outcomes [61]. Additionally, inconsistencies in the reporting of baseline characteristics and risk factor progression across validation studies limited the ability to replicate study populations precisely. Where necessary, missing data were supplemented using studies with similar populations, though some discrepancies likely remain. Moreover, predictive validity (i.e., assessing the accuracy of future forecasts) was not evaluated as part of this analysis due to the time required for long-term outcome observation. In addition, while the model incorporated risk equations for other clinical events (e.g., knee replacement, osteoarthritis, asthma, etc.; see Supplementary Table S1, ESM), these were not the focus of the validation exercise. This analysis prioritised CVD and mortality outcomes as they are important drivers of the clinical and economic burden associated with obesity. Therefore, future validation of these additional endpoints may represent a logical next step for subsequent research.

It is important to note that this study represents the first iteration of the Obesity Lifecycle Model. Based on the findings of this validation exercise, future versions of the model may consider alternative approaches for predicting T2DM incidence, given the lower predictive accuracy observed for this outcome. Additionally, integrating cardiovascular risk prediction tools, such as SCORE2 [62] or PREVENT [63], may enhance model applicability across diverse geographic settings and represents an interesting area for further investigation. Similarly, future iterations may benefit from incorporating additional obesity-related comorbidities as data emerge, such as non-alcoholic fatty liver disease and mental health conditions.

Conclusion

The comprehensive validation exercises presented in this study demonstrate a favourable concordance between the Obesity Lifecycle Model’s predictions and observed outcomes. While independent validation showed an underprediction for CVD and mortality and T2DM incidence, these trends were comparable with levels observed in other published model validations. These findings add credibility to the Obesity Lifecyle Model, suggesting it is a valid tool for predicting the natural history, clinical implications, life expectancy and quality of life impacts associated with obesity and its comorbidities. This validation study indicates that the Obesity Lifecycle Model may be a valuable resource to support healthcare decision making and policy development for obesity.

Supplementary Information

Below is the link to the electronic supplementary material.

Funding

This work was funded by Boehringer Ingelheim.

Declarations

Conflict of interest

AS is an employee of Boehringer Ingelheim. BM has received consulting honoraria in relation to the conceptualisation and design of the model and the conduct of this study.  SS, CS and MA are employees of Health Economics and Outcomes Research Ltd, who received funding from Boehringer Ingelheim in relation to the conduct of this study. ME has received honoraria grants from Novo Nordisk, Boehringer Ingelheim, Eli Lilly and Daiichi Sankyo.

Data availability statement

Data supporting the findings of this study are available within the paper and its supplementary materials.

Ethical approval

Not applicable.

Code availability

A simplified version of the model was provided to the manuscript reviewers during the peer review process; however, the model and any associated code are proprietary and will not be available in the public domain.

Consent to participate

Not applicable.

Consent for publication

Not applicable.

Author contributions

AS and BM conceptualised and designed the study. All authors contributed to data analysis, interpretation of the results and preparation and review of the manuscript. All authors read and approved the final version of the manuscript for publication.