Beyond numbers: the missing conceptual foundation in evaluating Mexico's health system performance

We read with great interest the recent article “23 years of public policy towards universal health coverage in Mexico. A cross-sectional time-series analysis using routinely collected health data, 2000–2022” by Alvarez et al.¹ We commend the authors for assembling two decades of administrative series. However, several conceptual and methodological choices limit the strength and policy relevance of their claims. Below, we outline four interconnected concerns regarding the article's conceptual basis and its implications for health systems and policy evaluation.

Firstly, the paper lacks a solid conceptual framework guiding the analytical strategy. Several strong, validated, and well-established health system performance assessment frameworks could have been used to evaluate the reforms implemented in Mexico over the past 20 years.2, 3, 4 However, the authors opted for a simplistic inputs/processes/outputs/impacts approach, which is not clearly articulated and is poorly operationalised. This mechanistic framing assumes near-linear links from spending/resources to mortality without accounting for mediators such as institutional designs, allocative efficiency, service quality or governance. This is further complicated by the alignment of spline knots with presidential periods, rather than policy implementation milestones, which weakens causal interpretation.

Second, given the limitations mentioned above, the authors chose to use a poor set of indicators for their assessment of Mexico's recent policy reforms, most notably public expenditure and mortality. Premature mortality is a distal outcome, shaped by demographic, social, and epidemiological transitions accumulated over decades. It does not isolate the institutional, fiscal, or operational effects of abrupt policy changes.^2,5 For this reason, it could not capture the short-term effects of the 2019–2023 health system reorganization—particularly the transition from Seguro Popular (SP) to INSABI and the subsequent creation of IMSS-Bienestar.⁶

Thirdly, this analysis lacks indicators related to health system intermediate and intrinsic objectives—apart from health conditions—, such as coverage, quality, and financial risk protection indicators. It does not mention, for example, changes in coverage and prevalence of catastrophic expenditure, which improved significantly during the operation of SP and worsened considerably during the operation of INSABI and the initial years of IMSS-Bienestar.6, 7, 8, 9 We and others have documented that the dismantling of SP during the COVID-19 pandemic was linked to setbacks in effective coverage and financial protection. National surveys report: (i) an approximately 17% decrease in insurance coverage between 2018 and 2020; (ii) a sharp rise in private healthcare usage, reaching unprecedented levels by 2022; and (iii) a notable increase in catastrophic health expenditure, especially among the uninsured.6, 7, 8, 9, 10

Finally, the “Evidence before this study” section of this article shows an incomplete literature search, overlooking the extensive peer-reviewed research that has already explored Mexico's reforms using theory-informed frameworks and causal evaluation methods.¹⁰ Consequently, the article reiterates findings already reported in the literature and would benefit from explicit integration with the substantial conceptual and methodological advances achieved over the past two decades.

Underlying these limitations is a broader epistemological concern: the rise of theory-free empiricism in health policy research—the application of sophisticated statistical methods to administrative datasets without adequate conceptual grounding. While the dataset assembled for this paper is valuable, the absence of theoretical guidance limits the analysis to a pattern description that is detached from causal logic.

We also identified several statements in the article that may benefit from clarification or additional context, as outlined below:

• •“This is the first comprehensive nationwide analysis of Mexico's health system reforms from 2000 to 2022 and its association to populationś health employing a logical framework.” There is an extensive and robust body of pre- and post-2019 evaluations that have documented changes in coverage, service use, OOPE/CHE, and other intermediate and final objectives of health systems, including health conditions^6,10 related to Mexico's recent health reforms. Some of these pieces have explicitly utilised known assessment frameworks to study these reforms.^6,11
• •“Evidence of progress under SP is scarce or weak.” National agencies (e.g., CONEVAL, INSP) and an extensive peer-reviewed literature have robustly documented the expansion of insurance coverage and improvements in financial protection during the period of SP's operation, followed by reversals after its dismantling.^6,12,13 Interestingly, two papers that strongly support the progress made by SP were co-authored by one of the authors of this paper.^8,14

In conclusion, Álvarez-Aceves et al. provide extensive descriptive evidence on Mexico's health sector; however, their analysis remains superficial. The absence of a solid health system performance framework leaves their findings disconnected from the institutional and policy mechanisms that define reform outcomes. As a result, the study risks being misinterpreted as evidence of policy failure, when, in reality, the problem lies in its overly simplistic analytical approach. More substantial evidence is required before concluding that the investment did not yield health benefits and that the reforms failed. Recent evidence indicates that institutional fragmentation, inconsistent financing, and political volatility remain the primary barriers to UHC in Mexico.^6,8

It is imperative to refocus assessment efforts on system performance and financial protection. Only through such an approach can Mexico—and other middle-income countries—build more resilient and equitable health systems.

Contributors

HAO generated the first draft of the comment. DCL, ESM, MT, and OGD provided critical feedback throughout various rounds of revision. All authors reviewed, revised, and approved the final version of the comment.

Declaration of interests

Octavio Gómez-Dantés was Director General for Performance Evaluation at the Ministry of Health of Mexico during the period 2002–2006, when Seguro Popular was launched.