Dynamic alignment of large language models for evidence-grounded heart failure decision support

Abstract

Objectives

Large language models (LLMs) are increasingly studied for clinical decision support, but high-risk cardiology exposes persistent weaknesses in hallucination control, guideline adherence, and medication-safety reasoning. Heart failure with reduced ejection fraction (HFrEF) is a demanding test case because safe care requires structured guideline-directed therapy, comorbidity-aware monitoring, and reliable risk warnings.

Methods

We developed a dynamic alignment framework using 1087 retrospective HFrEF cases from Affiliated Zhongshan Hospital of Dalian University. An open-source LLaMA-3.1 backbone was optimized through four sequential stages: continual pre-training for heart-failure domain adaptation, supervised fine-tuning for structured clinical responses, reinforcement policy optimization for safety-oriented alignment, and retrieval-augmented generation for guideline grounding. Models were assessed with dual-track clinical and linguistic metrics.

Results

LLaMA-3.1 was the strongest supervised baseline, but supervised fine-tuning alone did not fully resolve guideline-adherence limitations. Staged alignment produced a measurable Alignment Tax: the final retrieval-grounded variant improved the Clinical Score from 0.716 to 0.864 and reached a Guideline Score of 0.881, while BLEU-4 decreased from 0.371 to 0.272. The decline in surface overlap coincided with stronger risk safety, stricter structure, and more guideline-directed outputs.

Conclusions

Dynamic alignment shifted the model from linguistic mimicry toward clinically constrained HFrEF decision support. These findings suggest that staged optimization with policy alignment and retrieval grounding can improve evidence-based recommendations, while conventional language-overlap metrics may underestimate clinically safer generation.

Highlights

• • A four-stage dynamic alignment framework (CPT → SFT → GRPO → RAG) is proposed for evidence-grounded HFrEF decision support.

• • The framework shifts LLM behaviour from linguistic mimicry toward clinically constrained, safety-aware generation.

• • Staged alignment produces a measurable Alignment Tax: BLEU-4 decreases from 0.371 to 0.272 while Clinical Score rises from 0.716 to 0.864.

• • A dual-track evaluation framework separates clinical adequacy from linguistic fidelity, addressing limitations of conventional NLP metrics in high-risk medical domains.

• • The study demonstrates that retrieval-augmented grounding yields the largest improvement in guideline consistency (Guideline Score 0.479 → 0.881).

1. Introduction

Heart failure with reduced ejection fraction (HFrEF) remains one of the most clinically demanding cardiovascular syndromes because its definition is simple while its management is intrinsically multidimensional. Contemporary Chinese and European guidelines define HFrEF by a left ventricular ejection fraction of 40% or lower, but this threshold captures a heterogeneous population with different aetiologies, comorbidity burdens, haemodynamic states, and risks of decompensation.^1,2 Globally, heart failure continues to impose substantial mortality, hospitalization, and health-system burden, with ageing, hypertension, diabetes, coronary artery disease, renal dysfunction, and multimorbidity contributing to persistent growth in prevalence and complexity. ³ In China, this burden is amplified by population ageing and the high prevalence of cardiometabolic risk factors, making standardized and individualized HFrEF management a public-health and health-informatics priority.^1–3 The clinical pathway is not restricted to diagnosis or risk classification; it includes acute stabilization, volume assessment, guideline-directed pharmacotherapy, device or revascularization consideration when indicated, long-term surveillance, rehabilitation, and repeated adjustment of treatment intensity. ⁴ HFrEF therefore represents a stringent test case for clinical language modelling because safe output requires a coherent synthesis of symptoms, imaging, biomarkers, comorbidities, haemodynamic constraints, and therapeutic priorities rather than isolated factual recall.

The management burden of HFrEF arises from the density of guideline-directed medical therapy (GDMT) and from the need to adapt that therapy to unstable real-world conditions, where implementation barriers can prevent evidence-based recommendations from reaching eligible patients. ⁵ Imaging, natriuretic peptide testing, cardiac troponin interpretation, electrocardiography, coronary evaluation, and congestion assessment are often required to place a patient on the correct diagnostic and therapeutic trajectory. ⁶ Current recommendations and contemporary HFrEF reviews require timely initiation and titration of foundational therapies, including renin-angiotensin system inhibition or angiotensin receptor-neprilysin inhibition, evidence-based beta-blockade, mineralocorticoid receptor antagonism, sodium-glucose cotransporter 2 inhibition, and diuretics for congestion when appropriate.^7–9 These therapies must be selected against a background of blood pressure, renal function, potassium level, diabetes, atrial fibrillation, ischaemic heart disease, pulmonary disease, medication tolerance, and adherence risk. Polypharmacy is not a peripheral problem in this setting; it is a structural property of contemporary HFrEF care, and expert consensus increasingly treats medication burden, interactions, deprescribing decisions, and sequencing as core clinical tasks. ⁸ These factors make HFrEF a domain in which decision support must combine guideline adherence with patient-specific safety constraints rather than merely produce plausible explanatory text.

Artificial intelligence has been studied extensively in heart failure, but much of the prior work has focused on circumscribed prediction or classification tasks. Machine-learning systems have been evaluated for diagnosis, early detection, readmission prediction, imaging-based phenotyping, electrocardiographic interpretation, risk stratification, and extraction of functional status from clinical documentation.^10–18 These applications demonstrate that computational models can identify clinically relevant signal from imaging, electrocardiography, structured records, and free text, yet they generally operate on a narrow segment of the care pathway. Recent generative models and large language models (LLMs) extend this paradigm by producing natural-language responses, retrieving biomedical evidence, summarizing records, and answering clinical questions.^{15,16,19–22} In cardiovascular disease, LLMs have been evaluated for patient education, knowledge assessment, documentation, decision support, and guideline-related question answering.^21,22 This shift is important because HFrEF management is expressed largely through clinical language: medication plans, monitoring instructions, risk warnings, and follow-up decisions are written as structured recommendations rather than as scalar predictions. At the same time, a language model that performs well on knowledge questions or documentation tasks cannot be assumed to generate safe HFrEF management plans. The relevant clinical target is not conversational fluency, but the consistent production of complete, guideline-concordant, risk-aware, and locally grounded recommendations.

Generic LLMs and simply fine-tuned models therefore face a distinct reliability problem in high-risk medical environments. Clinical deployment requires data privacy, institutional control, local validation, auditability, measurable safety behaviour, and explicit guideline incorporation, which has motivated renewed interest in open-source models that can be adapted within hospital infrastructure.^23,24 Even when LLMs perform competitively on clinical decision benchmarks, their outputs remain vulnerable to hallucination, omission, overconfident phrasing, and insufficient contextualization for local practice.^24,25 Retrieval-augmented generation has emerged as one response to this limitation because it allows the model to condition generation on external evidence instead of relying solely on parametric memory.^19,23,26 In cardiology, guideline-embedded retrieval has been shown to improve responses to acute coronary syndrome guideline questions, reinforcing the principle that specialty recommendations should be surfaced at inference time when clinical decisions depend on current guideline content. ²⁶ Evidence from human-LLM collaboration studies also indicates that benefit is context-dependent rather than automatic; documentation quality, diagnostic reasoning, management accuracy, and safety outcomes may move in different directions depending on task design, clinician interaction, and verification workflows.^25,27 Realistic health-model benchmarks similarly show that open-ended clinical evaluation requires physician-derived rubrics rather than examination-style accuracy alone. ²⁸ These findings support a conservative premise for HFrEF modelling: a clinical LLM must be evaluated as an aligned decision-support system with explicit safeguards, not as a general text generator.

An additional methodological difficulty is that conventional language-model optimization does not necessarily align with clinical adequacy. Supervised fine-tuning rewards similarity to reference answers and can improve response structure, but it may preserve a preference for fluent, loosely phrased, and lexically familiar output even when a clinical task demands rigid medication classes, safety warnings, and guideline language. Standard NLP metrics such as BLEU or ROUGE are useful for measuring surface concordance, yet they can undervalue clinically appropriate reformulation and overvalue unsafe textual mimicry.^29–31 This creates the conceptual basis for the alignment tax: as a model is constrained toward evidence-grounded and safety-sensitive clinical behaviour, it may sacrifice some surface similarity to reference prose. In HFrEF, such a tradeoff is clinically acceptable only if the loss of linguistic overlap is accompanied by stronger recognition of drug completeness, haemodynamic risk, contraindication-sensitive reasoning, monitoring requirements, and guideline compliance. The central methodological question is therefore not whether an LLM can imitate cardiology language, but whether its optimization target can be shifted from linguistic resemblance toward clinical sufficiency.

Despite the growing interest in medical LLMs, a critical gap remains in translating general-purpose language capabilities into domain-specific clinical decision support that satisfies the safety and guideline-adherence requirements of high-risk specialties such as cardiology. Traditional machine learning approaches have been applied to heart disease diagnosis using feature selection and optimized classifiers, ³² but these methods focus on classification rather than generating structured clinical recommendations. The broader trend toward data-driven decision-making spans diverse fields, from materials characterization ³³ and environmental monitoring ³⁴ to consumer analytics, ³⁵ underscoring the growing reliance on AI-mediated interpretation of complex data across domains. Existing approaches to clinical LLM adaptation have primarily relied on supervised fine-tuning or prompt engineering, which improve surface fluency but do not explicitly enforce medication safety, contraindication awareness, or guideline-directed therapy completeness. Furthermore, conventional evaluation frameworks that rely on lexical overlap metrics (e.g., BLEU, ROUGE) may reward textual mimicry rather than clinically appropriate reformulation, potentially masking unsafe omissions or guideline deviations. The motivation for the present study therefore arises from three interrelated challenges: (1) the need to move beyond linguistic imitation toward clinically weighted adequacy in HFrEF decision support; (2) the need for an evaluation framework that separates clinical safety from surface textual similarity; and (3) the need to demonstrate whether staged alignment can systematically redirect an open-source LLM toward evidence-grounded, safety-aware, and guideline-concordant HFrEF management recommendations. The principal contributions of this study are as follows: (1) we propose a four-stage dynamic alignment framework that progressively shifts LLM behaviour from linguistic mimicry toward clinically constrained generation; (2) we introduce a dual-track evaluation framework that separates clinical adequacy from linguistic fidelity, revealing an Alignment Tax that is clinically informative rather than a performance defect; (3) we demonstrate that retrieval-augmented grounding produces the largest improvement in guideline consistency; and (4) we provide qualitative and quantitative evidence that the trade-off between surface overlap and clinical safety is both systematic and clinically defensible in HFrEF decision support.

To address this problem, the present study proposes a four-stage dynamic alignment framework for HFrEF clinical language modelling. The framework begins with continual pre-training to adapt the base model to heart-failure terminology and clinical discourse, followed by supervised instruction tuning to teach a structured five-part response schema for diet, monitoring, diagnostic evaluation, medication planning, and follow-up warnings. A reinforcement-based policy optimization stage is then introduced to prioritize format adherence, medication completeness, safety recognition, and guideline-oriented reasoning. Finally, retrieval-augmented grounding is used to inject local heart-failure guideline evidence into generation, reducing dependence on parametric memory alone.^19,23,26 This design explicitly separates domain adaptation, structured clinical imitation, safety-oriented policy alignment, and evidence retrieval, allowing each component to address a different failure mode in HFrEF decision support. In this retrospective benchmark study, we evaluated whether dynamic alignment of an open-source LLaMA-3.1-8B backbone using 1087 de-identified HFrEF cases from Affiliated Zhongshan Hospital of Dalian University could shift model behaviour from surface-level linguistic mimicry toward clinically weighted adequacy, safety recognition, guideline consistency, and structured GDMT compliance.

2. Methods

2.1. Clinical cohort task formalization

This study was designed as a retrospective, single-centre clinical language modelling benchmark for heart failure with reduced ejection fraction (HFrEF). The source corpus comprised 1087 de-identified retrospective HFrEF cases from Affiliated Zhongshan Hospital of Dalian University. The dataset was constructed from electronic medical records, clinical narratives, examination summaries, echocardiographic reports, laboratory information, medication records, discharge recommendations, and clinician-authored treatment plans. The study was approved by the Ethics Committee of Affiliated Zhongshan Hospital of Dalian University [approval number: KYS2026-011] and was conducted in accordance with the Declaration of Helsinki. The requirement for written informed consent was waived by the ethics committee because of the retrospective design and use of de-identified data. All patient-identifying information was removed before model development and evaluation. The retrospective case collection period spanned January 2019 to October 2024, encompassing both inpatient and outpatient HFrEF encounters. All clinical data were sourced from the cardiology department of the study institution, and the model development and evaluation were conducted between November 2024 and March 2025. This study is reported in accordance with the TRIPOD-LLM reporting guideline, ³⁶ and the completed TRIPOD-LLM checklist is provided as a supplementary file.

Eligible records were required to meet prespecified clinical and documentation criteria. Inclusion criteria were age ≥18 years, a clinical diagnosis of HFrEF with left ventricular ejection fraction (LVEF) ≤40% on transthoracic echocardiography, availability of a complete clinical narrative describing symptoms and cardiopulmonary status, and sufficient treatment documentation to construct an expert reference response. Records were also required to contain the minimum information needed for HFrEF-oriented reasoning, including at least one measure of cardiac structure or function, cardiovascular symptom description, blood pressure or heart-rate information, and medication or management recommendations. Exclusion criteria were missing or indeterminate LVEF, preserved or mildly reduced ejection fraction, paediatric records, duplicate encounters from the same clinical episode, records dominated by non-cardiac terminal illness, acute surgical or interventional records without longitudinal heart-failure management content, incomplete source text preventing expert annotation, and records with identifiers that could not be reliably de-identified.

To prevent information leakage across training, validation, testing, and retrieval stages, the 1087 cases were partitioned at the patient level using an approximately 70/15/15 stratified split, yielding 765 training cases, 161 validation cases, and 161 held-out test cases. To ensure patient-level isolation, all encounters from the same patient were assigned to the same split, preventing any overlap of patient data between training, validation, and test partitions. Stratification was performed across LVEF strata (≤20%, 21-30%, 31-35%, 36-40%) and comorbidity profiles to ensure balanced representation of clinical heterogeneity in each subset. The continual pre-training corpus was constructed exclusively from published heart-failure textbooks and guideline-derived materials and did not contain any patient records from the clinical dataset. The supervised fine-tuning and GRPO alignment stages used only the training partition (n=765), while the validation partition (n=161) was reserved for early stopping, hyperparameter selection, and reward weight calibration. The held-out test partition (n=161) was used exclusively for final dual-track evaluation and was never exposed during any training or alignment stage. The retrieval-augmented generation knowledge base was constructed from local guideline documents that were segregated from both the expert reference answers and the test-set reference responses, ensuring that retrieved evidence passages could not introduce information from the evaluation ground truth. This strict separation across training, validation, testing, and retrieval corpora was designed to minimize the risk of data leakage and to provide an unbiased estimate of model performance on unseen clinical cases.

The eligible record set was then formalized as a conditional generation task in which each source record supplied both a clinical state and an expert management target. Each clinical instance was represented as a paired sample $\left(x_i,y_i\right)$ , where $x_i$ denotes the de-identified patient state and $y_i$ denotes the expert reference management response. The patient state was formalized as $x_i=\left\{s_i,h_i,v_i,l_i,e_i,c_i,r_i\right\}$ , in which $s_i$ represents presenting symptoms such as dyspnoea, orthopnoea, chest tightness, oedema, cough, or reduced exercise tolerance; $h_i$ represents cardiovascular history and comorbidities; $v_i$ represents haemodynamic observations such as blood pressure and heart rate; $l_i$ represents laboratory variables such as B-type natriuretic peptide, cardiac troponin, renal function, electrolytes, or glucose-related indices when available; $e_i$ represents electrocardiographic and echocardiographic findings including LVEF; $c_i$ represents clinically relevant constraints such as hypotension, renal impairment, arrhythmia, chronic obstructive pulmonary disease, diabetes, prior myocardial infarction, or nephrotic syndrome; and $r_i$ represents the local clinical context encoded in the reference treatment plan. This notation was used to define the modelling problem, not to convert the record into a fixed-width tabular feature matrix.

The target output space was defined as a structured clinical response space $\mathcal{Y}={\mathcal{Y}}_{\text{diet}}\times {\mathcal{Y}}_{\text{monitor}}\times {\mathcal{Y}}_{\text{exam}}\times {\mathcal{Y}}_{\text{drug}}\times {\mathcal{Y}}_{\text{follow}}$ , corresponding to dietary management, monitoring or nursing level, further diagnostic evaluation, medication recommendations, and follow-up or emergency warning instructions. The model therefore estimated a conditional generation function $f_{\theta }:\mathcal{X}\to \mathcal{Y}$ , or equivalently $p_{\theta }\left(y_i\mid x_i\right)$ under a constrained clinical output schema. The formulation required the generated response to preserve five forms of clinical adequacy: recognition of congestion and haemodynamic risk, completeness of guideline-directed medical therapy, adjustment for comorbidities, coverage of diagnostic next steps, and explicit follow-up safety warnings. This structure was intended to separate clinically meaningful adequacy from surface-level textual similarity.

2.2. Supervised domain adaptation architecture

The full modelling strategy was organized as a four-stage dynamic alignment architecture, with each stage addressing a distinct failure mode in HFrEF clinical generation. The overall study workflow, including patient data inputs, staged alignment, and dual-track evaluation, is illustrated in Figure 1. The parameter-efficient adaptation and evidence-grounded generation architecture is detailed in Figure 2. The clinical objective of each optimization stage is summarized in Table 1. Let ${\theta }_0$ denote the parameters of the selected base model. The alignment process can be written as a staged transformation from ${\theta }_0$ to ${\theta }_1=T_{\text{CPT}}\left({\theta }_0\right)$ , ${\theta }_2=T_{\text{SFT}}\left({\theta }_1\right)$ , and ${\theta }_3=T_{\text{GRPO}}\left({\theta }_2\right)$ , followed by retrieval-grounded inference ${\widehat{y}}_i=g_{{\theta }_3}\left(x_i,R\left(x_i\right)\right)$ . Here, $T_{\text{CPT}}$ , $T_{\text{SFT}}$ , and $T_{\text{GRPO}}$ denote continual pre-training, supervised fine-tuning, and reinforcement-based clinical alignment, respectively, and $R\left(x_i\right)$ denotes guideline evidence retrieved for the patient state.

Figure 1.

Four-Stage Dynamic Alignment Framework for Evidence-Grounded HFrEF Decision Support. The embedded label “NLP Score” denotes the Linguistic Fidelity Score used in the revised manuscript.

Figure 2.

Parameter-efficient alignment and evidence-grounded generation architecture. The frozen LLaMA-3.1-8B backbone was adapted with trainable LoRA modules through continual pre-training, supervised fine-tuning, and GRPO-based clinical alignment. Retrieval-augmented generation incorporated local guideline evidence at inference, and the resulting HFrEF outputs were evaluated using separate clinical adequacy and linguistic fidelity metrics. The embedded label “NLP Score” denotes the Linguistic Fidelity Score used in the revised manuscript.

Table 1.

The four-stage dynamic alignment framework for HFrEF clinical modelling.

Optimization stage	Clinical objective	Data source	Algorithmic strategy
Continual pre-training	Expand heart-failure domain knowledge	Specialist textbooks and guideline literature	Domain-adaptive language modelling
Supervised fine-tuning	Learn structured clinical response patterns	Expert-curated HFrEF question-answer pairs	Instruction tuning with clinical references
GRPO alignment	Strengthen safety and guideline-oriented reasoning	Clinical prompts with reward-based feedback	Reinforcement alignment using dual rewards
Retrieval-augmented generation	Ground recommendations in current evidence	Local heart-failure guideline knowledge base	Dense retrieval with evidence injection

The initial model-selection phase compared five open-source foundation models with comparable deployment feasibility in a local hospital environment: Baichuan2-7B, DeepSeekLLM-7B, GLM-4-9B, LLaMA-3.1-8B, and Qwen3-8B. Each candidate model was adapted to the same HFrEF instruction format and evaluated under the dual-track metric framework described below. LLaMA-3.1-8B was selected as the optimization backbone because it achieved the strongest combined performance after supervised fine-tuning, with the highest observed balance between clinical adequacy and linguistic fidelity across the candidate set. The choice of LLaMA-3.1 reflected the availability of a modern open-weight model family with documented multilingual, reasoning, and safety-oriented post-training characteristics. ³⁷ This selection strategy treated the base model as a clinically constrained generative prior rather than as a final decision-support system.

Parameter-efficient fine-tuning was used throughout the adaptation pipeline to reduce memory pressure while preserving sufficient task plasticity. The model was optimized with PEFT-LoRA using rank r=8 and scaling factor alpha=32 under bfloat16 precision, following the low-rank adaptation principle of updating trainable decomposition matrices while freezing the base model weights. ³⁸ The primary learning rate was set to $1\times {10}^{-4}$ for domain adaptation and instruction tuning. Low-rank updates were applied to linear projection modules so that the trainable parameter set $\mathrm{\Delta }\theta$ represented a constrained update to the frozen backbone parameters ${\theta }_0$ , yielding adapted parameters $\theta ={\theta }_0+\mathrm{\Delta }\theta$ . The transformer blocks in Figure 2 are shown schematically to indicate the frozen backbone and trainable LoRA adapters rather than layer-specific full-parameter updates. The main configuration of model adaptation and retrieval-grounded inference is summarized in Table 2.

Table 2.

Configuration of model adaptation and retrieval-grounded inference.

Component	Parameter	Configuration
Backbone	Base model	LLaMA-3.1-8B
Backbone	Optimization status	Frozen backbone with trainable LoRA adapters
LoRA adaptation	Method	PEFT-LoRA
LoRA adaptation	Rank and scaling factor	r = 8, α = 32
LoRA adaptation	Target modules	Linear projection modules
LoRA adaptation	Precision	bfloat16
Continual pre-training	Data source	Heart-failure textbooks, guideline-derived materials, and unlabelled clinical literature
Continual pre-training	Batch size	8
Supervised fine-tuning	Data source	Expert-curated HFrEF question-answer pairs
Supervised fine-tuning	Output schema	Five-section clinical response
Supervised fine-tuning	Epochs and batch size	8 epochs; batch size 4
GRPO alignment	Initialization	Supervised fine-tuned model
GRPO alignment	Reward design	Format reward and HFrEF clinical reward
GRPO alignment	Clinical reward components	Medication completeness, risk safety, and guideline consistency
GRPO alignment	Sequence length	Maximum input length 2048; maximum completion length 1024
Retrieval	Evidence source	Local heart-failure guideline knowledge base
Retrieval	Embedding model and vector store	BAAI/bge-m3; ChromaDB
Retrieval	Passage segmentation	Chunk size 500; overlap 50
Retrieval	Evidence use	Retrieved evidence injected into the generation context
Evaluation	Clinical metrics	Drug recall, risk safety, guideline score, and section completeness
Evaluation	Linguistic metrics	BLEU-4, ROUGE-L, semantic similarity, and Linguistic Fidelity Score

Several measures were implemented to mitigate overfitting and selection bias. First, parameter-efficient LoRA adaptation (rank r=8) inherently constrains the trainable parameter space, reducing the risk of memorizing individual training examples relative to full-parameter fine-tuning. Second, early stopping was applied during supervised fine-tuning using validation loss on the held-out validation partition (n=161); training was terminated when validation loss failed to improve for three consecutive epochs. Third, a dropout rate of 0.1 was applied to the LoRA adapter modules to regularize intermediate representations. Fourth, the KL divergence penalty in the GRPO objective served as an implicit regularizer, anchoring the optimized policy to the supervised reference distribution and preventing excessive deviation that could lead to reward hacking. Fifth, the patient-level data split ensured that no patient appeared in both training and test sets, eliminating a common source of information leakage in medical NLP studies. Regarding dataset balancing, the natural clinical distribution of HFrEF cases was preserved rather than artificially rebalanced, because artificial resampling could distort the comorbidity and severity profile that the model needs to learn. Instead, stratified sampling across LVEF strata and comorbidity categories was used during data partitioning to ensure that each subset reflected the heterogeneity of the source cohort. Feature selection was not performed in the traditional tabular sense; rather, clinical feature engineering was achieved through the structured patient-state representation x_i, which organizes clinically relevant variables into physiologically meaningful categories (symptoms, history, haemodynamics, laboratory values, imaging findings, and clinical constraints). This representation ensures that the model processes all clinically salient information while maintaining interpretability of the input space.

Continual pre-training was used to adapt the selected backbone to the lexical distribution, clinical reasoning patterns, and guideline terminology of HFrEF. Let ${\mathcal{D}}_{\text{CPT}}={\left\{z_j\right\}}_{\hspace{0.17em}\mathrm{j}=1}^{\mathrm{N}}$ denote the domain corpus constructed from heart-failure textbooks, guideline-derived materials, and unlabelled clinical literature. The objective followed standard autoregressive language modelling:

$${\mathcal{L}}_{CPT}\left(\theta \right)=-{\displaystyle \sum }_{j=1}^N{\displaystyle \sum }_{t=1}^{\mid z_j\mid }\log \hspace{0.17em}{p}_{\theta }\left(z_{j,t}\mid z_{j,<t}\right)$$ (1)

This stage produced an intermediate parameter state ${\theta }_1=T_{CPT}\left({\theta }_0\right)$ , where $T_{CPT}$ denotes domain-adaptive language modelling before instruction learning. Training used bfloat16 precision, LoRA rank $r=8$ , learning rate $1\times {10}^{-4}$ , and batch size 8. The purpose of this stage was not to teach the model the final response schema, but to reduce domain mismatch before exposure to structured HFrEF question-answer pairs.

After domain adaptation, supervised instruction tuning mapped de-identified HFrEF patient states to expert reference responses under the five-part clinical output schema. For a supervised dataset ${\mathcal{D}}_{\text{SFT}}={\left\{(\text{xi},\text{yi}\hspace{0.17em})\right\}}_{\mathrm{i}=1}^{\mathrm{M}}$ , the objective minimized the negative log-likelihood of the reference response:

$${\mathcal{L}}_{SFT}\left(\theta \right)=-{\displaystyle \sum }_{i=1}^M{\displaystyle \sum }_{t=1}^{\mid y_i\mid }\log \hspace{0.17em}{p}_{\theta }\left(y_{i,t}\mid x_i,y_{i,<t}\right)$$ (2)

The supervised stage yielded ${\theta }_2=T_{SFT}\left({\theta }_1\right)$ and was trained for 8 epochs using bfloat16 precision, PEFT-LoRA rank $r=8$ , LoRA scaling factor $\alpha =32$ , learning rate $1\times {10}^{-4}$ , and a supervised batch size of 4. The instruction targets required generation of dietary advice, monitoring level, further examination recommendations, medication strategy, and follow-up warnings. This stage established the structured response pattern for HFrEF management, while later stages refined safety weighting and evidence consistency.

2.3. Evidence-grounded policy optimization

After supervised training, additional constraints were required to align generation with safety-sensitive clinical behaviours and to condition final recommendations on external guideline evidence. GRPO-based policy optimization was applied after supervised tuning to reinforce clinical behaviours that are insufficiently captured by token-level likelihood. This stage followed the policy-regularization logic of proximal policy optimization and the critic-free group-relative formulation used in recent reasoning-model training.^39–41 The supervised model ${\pi }_{{\theta }_2}$ served as the initialization policy, and the optimized policy ${\pi }_{{\theta }_3}$ was trained against reward signals defined over generated responses ${\widehat{y}}_i\sim {\pi }_{\theta }\left(·\mid x_i\right)$ . The total reward combined a structural format component and a heart-failure clinical adequacy component:

$$R_{\text{total}}\left(x_i,{\widehat{y}}_i\right)={\lambda }_1{R}_{\text{format}}\left(x_i,{\widehat{y}}_i\right)+{\lambda }_2{R}_{\text{HF}\_\text{fusion}}\left(x_i,{\widehat{y}}_i\right)$$ (3)

with ${\lambda }_1=0.3$ and ${\lambda }_2=0.7$ . The format reward assessed adherence to the five-part clinical response structure. The HFrEF fusion reward encoded clinically weighted criteria for medication completeness, safety recognition, and guideline consistency:

$$R_{\text{HF}\_\text{fusion}}={\omega }_1{R}_{\text{drug}}+{\omega }_2{R}_{\text{safety}}+{\omega }_3{R}_{\text{guideline}}$$ (4)

where ${\omega }_1=0.4$ , ${\omega }_2=0.4$ , and ${\omega }_3=0.2$ . The drug component emphasized completeness of guideline-directed medical therapy, the safety component emphasized red-flag recognition and contraindication awareness, and the guideline component rewarded agreement with HFrEF clinical recommendations. To examine the robustness of the clinical reward to weight specification, a sensitivity analysis was conducted by systematically varying the HFrEF fusion reward weights. Four alternative weight configurations were evaluated: (i) ω₁=0.3, ω₂=0.5, ω₃=0.2 (safety-prioritized); (ii) ω₁=0.5, ω₂=0.3, ω₃=0.2 (drug-completeness-prioritized); (iii) ω₁=0.3, ω₂=0.3, ω₃=0.4 (guideline-prioritized); and (iv) ω₁=0.4, ω₂=0.3, ω₃=0.3 (balanced safety-guideline). The default weights (ω₁=0.4, ω₂=0.4, ω₃=0.2) were derived through iterative piloting with three board-certified cardiologists, who initially assigned equal weights and then adjusted the configuration based on the clinical priority of medication completeness and safety relative to guideline consistency. The default weights were pre-determined on the validation set through expert piloting and were not re-calibrated using the test partition. The sensitivity analysis assessed whether the final Clinical Score, Risk Safety, and Alignment Tax magnitude changed substantially under alternative weightings; the held-out test partition was used solely to evaluate the robustness of the pre-determined default configuration, not to select among alternatives. It should be noted that the GRPO reward weights (ω₁=0.4, ω₂=0.4, ω₃=0.2) used during policy optimization are distinct from the composite Clinical Score component weights (0.30/0.30/0.20/0.20) used for evaluation; the former guide the reinforcement learning signal, while the latter summarize clinical performance for reporting.

To prevent excessive divergence from the reference policy, the policy objective incorporated a Kullback-Leibler regularization term. The optimized objective can be expressed as:

$$\underset{\theta }{\max \hspace{0.17em}}{\mathbb{E}}_{\widehat{y}\sim {\pi }_{\theta }\left(·\mid x\right)}\left[R_{\text{total}}\left(x,\widehat{y}\right)-\beta D_{KL}\left({\pi }_{\theta }\left(·\mid x\right)\parallel {\pi }_{\text{ref}}\left(·\mid x\right)\right)\right]$$ (5)

where ${\pi }_{\text{ref}}$ denotes the frozen reference policy and $\beta$ controls the strength of policy anchoring. This penalty constrains updates that improve reward at the cost of unstable generation, thereby reducing the risk of policy collapse or loss of the supervised clinical response format. GRPO training used four-card parallel execution with maximum input length 2048 and maximum completion length 1024. For local inference experiments, vLLM was configured with a GPU utilization cap of 0.85, and PyTorch expandable memory segments were enabled to reduce memory allocation fragmentation. ⁴² Inference latency was measured empirically rather than characterized asymptotically.

Retrieval-augmented generation was introduced as the final grounding layer to reduce reliance on parametric memory for guideline-sensitive recommendations. This design followed the general RAG principle of coupling a generative model with external non-parametric evidence retrieval. ⁴³ The local guideline corpus was segmented into overlapping passages using chunk size 500 and overlap 50. Each passage $d_k$ was embedded into the BAAI/bge-m3 vector space as $v_k=E\left(d_k\right)$ , and each patient query $x_i$ was mapped to a query vector $q_i=E\left(x_i\right)$ . Candidate evidence passages were ranked by cosine similarity:

$$s\left(q_i,v_k\right)=\frac{q_i^{\top }{v}_k}{\parallel q_i{\parallel }_2\parallel v_k{\parallel }_2}$$ (6)

The retrieval function selected the top-ranked evidence set $R\left(x_i\right)={\text{TopK}}_k\left\{s\left(q_i,v_k\right)\right\}$ from the ChromaDB vector store, yielding evidence snippets $R\left(x_i\right)=\left\{r_{i1},...,r_{ik}\right\}$ that were injected into the generation context before producing ${\widehat{y}}_i$ . The final response was generated as ${\widehat{y}}_i=g_{{\theta }_3}\left(x_i,R\left(x_i\right)\right)$ , where the retrieval term constrained recommendations toward locally indexed evidence while preserving the structured response schema learned during supervised and reinforcement alignment.^43,44 This stage defined the retrieval-grounded inference branch shown in Figure 2, in which locally indexed evidence was introduced before generation rather than used only for post hoc interpretation.

2.4. Dual-track evaluation framework

Model performance was assessed with the dual-track clinical and linguistic evaluation framework summarized in Table 3. As shown in Table 4, The clinical track quantified whether generated outputs preserved HFrEF management adequacy, including drug recall, diet accuracy, medication-key ${\mathrm{F}}_1$ , risk safety, guideline consistency, and section completeness. A composite clinical score was calculated from clinically weighted components, with emphasis on medication completeness, risk recognition, guideline agreement, and output coverage. These measures were selected because HFrEF decision support requires safe therapy selection and recognition of comorbidity-sensitive constraints rather than only fluent text generation.

Table 3.

The dual-track clinical and linguistic evaluation framework.

Evaluation domain	Metric	Clinical relevance
Clinical	Drug Recall	Completeness of recommended cardiovascular therapies
Clinical	Risk Safety	Recognition of high-risk clinical warnings
Clinical	Guideline Score	Consistency with heart-failure guideline recommendations
Clinical	Section Completeness	Coverage of required clinical response modules
Linguistic	BLEU-4	Surface overlap with expert-written references
Linguistic	ROUGE-L	Sequence-level textual concordance
Linguistic	Semantic Similarity	Meaning-level agreement with reference answers
Linguistic	Linguistic Fidelity Score	Aggregate linguistic quality across text metrics

Table 4.

Components of the composite Clinical Score.

Component	Scoring range	Weight	Example scoring rule
Drug Recall	0-1 (proportion)	0.30	Proportion of required GDMT classes (ARNI/ACEI/ARB, beta-blocker, MRA, SGLT2i, diuretic) correctly recommended
Risk Safety	0-1 (binary composite)	0.30	1 if all red-flag warnings present (hypotension, hyperkalaemia, renal impairment, contraindications); 0 otherwise
Guideline Score	0-1 (ordinal)	0.20	Degree of concordance with local HFrEF guideline recommendations, assessed by expert comparison
Section Completeness	0-1 (proportion)	0.20	Proportion of five response sections (diet, monitoring, examination, medication, follow-up) present and clinically meaningful
Diet Accuracy	0-1 (binary)	Included in completeness	1 if sodium/fluid restriction advice is appropriate for congestion status
Medication-key F1	0-1 (F1 score)	Included in drug recall	Harmonic mean of precision and recall for key medication terms relative to reference answer

Note. Clinical Score = 0.30 × Drug Recall + 0.30 × Risk Safety + 0.20 × Guideline Score + 0.20 × Section Completeness. Diet Accuracy and Medication-key F1 are sub-components of Section Completeness and Drug Recall, respectively, and are not separately weighted. Guideline Score is rated on a 0–1 ordinal scale anchored as follows: 0 = contradicts guideline recommendations; 0.5 = partially concordant (correct class but suboptimal agent, dose, or monitoring); 1 = fully concordant with locally endorsed HFrEF guideline. GDMT = guideline-directed medical therapy; ARNI = angiotensin receptor–neprilysin inhibitor; ACEI = angiotensin-converting enzyme inhibitor; MRA = mineralocorticoid receptor antagonist.

Clinical evaluation of model outputs was conducted by a panel of three board-certified cardiologists (each with ≥ 5 years of HFrEF management experience) from the study institution who were independent of the model development team and did not participate in manuscript authorship. The cardiologists were blinded to the model stage (i.e., whether an output originated from the SFT baseline, CPT, GRPO, or RAG variant) during scoring to minimize assessment bias. Each reviewer independently scored the generated responses from the held-out test set using the dual-track scoring rubric. Inter-rater agreement was assessed using Fleiss’ κ for the binary adequate/inadequate classification and the intraclass correlation coefficient (ICC) for the continuous composite Clinical Score, yielding a Fleiss’ κ of 0.82 (95% CI: 0.76–0.87) and an ICC of 0.85 (95% CI: 0.79–0.90), both indicating substantial agreement. The mean pairwise Cohen’s κ was also reported to facilitate comparison with prior clinical NLP studies (mean Cohen’s κ = 0.82, 95% CI: 0.76–0.87). Disagreements in individual metric scores were resolved through consensus discussion; if consensus could not be reached after discussion, the median of the three scores was used. The scoring rubric was pilot-tested on 30 cases before formal evaluation to calibrate rating consistency and to refine the operational definitions of each clinical metric. The finalized rubric defined explicit scoring anchors for each component: drug recall was scored as the proportion of required GDMT drug classes correctly identified; risk safety was scored as a binary assessment of whether all clinically relevant red-flag warnings were present; guideline score was assessed by comparing recommendations against the local guideline reference; and section completeness was scored as the proportion of the five required response sections that were present and clinically meaningful.

The linguistic track quantified overlap and semantic concordance with expert reference responses using BLEU-4, ROUGE-L, semantic similarity, section completeness, and an aggregate Linguistic Fidelity Score. These metrics were interpreted as measures of textual fidelity, not as standalone evidence of clinical safety. The separation of clinical and linguistic tracks allowed the evaluation to detect alignment-induced tradeoffs, particularly cases in which guideline consistency and safety improved while surface lexical overlap declined. Base-model comparisons, stage-wise ablations, radial profiles, standardized heatmaps, and GRPO optimization curves were used to characterize model behaviour across the full alignment pipeline. A representative de-identified HFrEF case was additionally used for qualitative visualization of output differences between the supervised baseline and the final aligned model.

3. Results

3.1. Clinical heterogeneity of the retrospective HFrEF benchmarking cohort

The benchmark dataset comprised 1087 retrospective HFrEF cases from Affiliated Zhongshan Hospital of Dalian University. The cases were derived from de-identified clinical records and treatment plans and were used to construct a generation task centred on structured HFrEF management rather than free-form medical dialogue. Across the available question-answer examples, the clinical inputs reflected substantial heterogeneity in systolic dysfunction severity, symptom burden, comorbidity profile, and treatment constraints. Echocardiographic LVEF values spanned markedly reduced ranges, with examples including LVEF values from the mid-20% range to values near the upper boundary of HFrEF eligibility. Presenting features included dyspnoea, orthopnoea, paroxysmal nocturnal dyspnoea, chest tightness, oedema, cough, reduced exercise tolerance, oliguria, and gastrointestinal symptoms. The examples also contained clinically relevant comorbidities and risk modifiers, including hypertension, type 2 diabetes mellitus, previous myocardial infarction with stent implantation, atrial fibrillation, chronic obstructive pulmonary disease, nephrotic syndrome, renal vulnerability, and suspected acute coronary syndrome.

The source records also showed variation in biomarkers and physiological context, including elevated B-type natriuretic peptide, variable cardiac troponin values, blood-pressure heterogeneity, heart-rate differences, and pulmonary congestion signs on physical examination. These features created a task in which the model was required to identify congestion, haemodynamic risk, coronary risk, arrhythmia-related anticoagulation needs, renal and electrolyte constraints, and comorbidity-specific medication considerations. Because this study was designed as a language-modelling benchmark rather than an epidemiological cohort analysis, the dataset was used to evaluate clinically structured generation and alignment behaviour rather than to estimate population-level associations.

To provide a transparent breakdown of dataset heterogeneity and support assessment of generalizability, the baseline clinical and demographic characteristics of the 1087-case HFrEF cohort are summarized in Table 5. The cohort comprised 723 unique patients (some with multiple encounters), with a mean age of 67.2 ± 12.6 years and a male predominance of 75.8%. The mean LVEF was 31.9 ± 6.8%, with 37.7% of cases in the 36-40% stratum and 7.8% with severely reduced LVEF (≤20%). Hypertension was the most prevalent comorbidity (60.3%), followed by prior myocardial infarction (50.2%) and angina (37.5%). Atrial fibrillation or flutter was present in 27.2% of cases, while valvular disease and cardiomyopathy were less common. Echocardiographic parameters reflected the expected pattern of left ventricular dilation (mean LVEDD 58.2 ± 8.9 mm) and left atrial enlargement (mean LAD 46.3 ± 7.1 mm). These distributions confirm that the benchmark captures a clinically heterogeneous HFrEF population spanning a range of systolic dysfunction severity, comorbidity burden, and haemodynamic profiles.

Table 5.

Baseline clinical and demographic characteristics of the HFrEF cohort.

Variable	Value (n=1087)
Age, years, mean ± SD	67.2 ± 12.6
Age categories, n (%)
< 50 years	86 (7.9)
50-64 years	310 (28.5)
65-74 years	410 (37.7)
75-84 years	200 (18.4)
≥ 85 years	81 (7.5)
Male sex, n (%)	824 (75.8)
LVEF, %, mean ± SD	31.9 ± 6.8
LVEF strata, n (%)
≤ 20%	85 (7.8)
21-30%	352 (32.4)
31-35%	240 (22.1)
36-40%	410 (37.7)
LVEDD, mm, mean ± SD	58.2 ± 8.9
LAD, mm, mean ± SD	46.3 ± 7.1
BMI, kg/m2, mean ± SD	25.6 ± 10.9
Comorbidities, n (%)
Hypertension	656 (60.3)
Myocardial infarction	546 (50.2)
Angina	408 (37.5)
Atrial flutter/fibrillation	296 (27.2)
Mitral valve disease	88 (8.1)
Cardiomyopathy	82 (7.5)
Aortic valve disease	67 (6.2)
Conduction block	63 (5.8)
Paroxysmal supraventricular tachycardia	5 (0.5)
NT-proBNP, pg/mL, median (IQR)	2297 (435-9075)
cTnI, ng/mL, median (IQR)	0.052 (0.025-3.8)
Diabetes mellitus	382 (35.1)
Chronic kidney disease	218 (20.1)
COPD	96 (8.8)
Stroke	76 (7.0)
Baseline medications at admission, n (%)
ARNI (sacubitril/valsartan)	87 (8.0)
ACE inhibitor	326 (30.0)
ARB	294 (27.0)
Beta-blocker	793 (73.0)
MRA (spironolactone)	435 (40.0)
SGLT2 inhibitor	131 (12.0)
Loop diuretic	902 (83.0)
Statin	546 (50.2)

Note. Values are presented as n (%) unless otherwise indicated. The cohort comprised 1087 hospital encounters from 723 unique patients; all characteristics are reported at the encounter level, as each case represents a separate admission or outpatient visit. Duplicate encounters from the same patient were retained where they represented clinically distinct episodes, but all encounters from the same patient were assigned to the same data split to prevent information leakage. LVEF = left ventricular ejection fraction; LVEDD = left ventricular end-diastolic diameter; LAD = left atrial diameter; BMI = body mass index; NT-proBNP = N-terminal pro-B-type natriuretic peptide; cTnI = cardiac troponin I; CKD = chronic kidney disease; COPD = chronic obstructive pulmonary disease; ARNI = angiotensin receptor–neprilysin inhibitor; ACEI = angiotensin-converting enzyme inhibitor; ARB = angiotensin receptor blocker; MRA = mineralocorticoid receptor antagonist; SGLT2 = sodium–glucose cotransporter-2; SD = standard deviation; IQR = interquartile range.

3.2. Base-model benchmark and the limitations of supervised adaptation

After supervised fine-tuning, the five candidate base models showed measurable separation in their combined clinical and linguistic profiles. In the composite benchmark, LLaMA-3.1-8B achieved the strongest combined performance, with a Clinical Score of 0.716 and a Linguistic Fidelity Score of 0.669, exceeding Baichuan2-7B, DeepSeekLLM-7B, GLM-4-9B, and Qwen3-8B in the integrated comparison (Figure 3(a)). This result supported the selection of LLaMA-3.1-8B as the backbone for subsequent staged alignment. The supervised benchmark also showed that model selection based on a single domain would be insufficient, because the clinically weighted and linguistic scores were related but not interchangeable.

Figure 3.

Base-model benchmark after supervised fine-tuning. (a) Composite clinical and Linguistic Fidelity scores across Baichuan2-7B, DeepSeekLLM-7B, GLM-4-9B, LLaMA-3.1-8B and Qwen3-8B. (b) Clinical-domain radial profile across drug recall, diet accuracy, risk safety, medication-key F1, guideline score and section completeness. (c) NLP-domain radial profile across BLEU-4, ROUGE-L, semantic similarity and section completeness. (d) Heatmap of standardized evaluation metrics across candidate models. In the figure, the label “NLP_Score” in the legend corresponds to the Linguistic Fidelity Score.

The clinical-domain radial profile showed that supervised fine-tuning produced structured responses with variable adequacy across individual clinical dimensions (Figure 3(b)). LLaMA-3.1-8B reached comparatively strong values for drug recall and risk safety, but guideline consistency remained limited. This pattern was not specific to a single model. Guideline Score was low across all supervised baselines, with LLaMA-3.1-8B reaching only 0.479 despite having the strongest overall combined profile. The corresponding NLP-domain radial profile showed that LLaMA-3.1-8B also had the most favourable linguistic profile after supervised tuning, including the highest BLEU-4 value among the candidate models (Figure 3(c)). These linguistic gains did not remove the residual clinical gap in guideline adherence.

The standardized heatmap further confirmed that supervised fine-tuning alone did not yield uniformly adequate clinical behaviour across the candidate models (Figure 3(d)). LLaMA-3.1-8B provided the best starting point because it combined the highest Clinical Score with the highest Linguistic Fidelity Score, but all models retained weak guideline-oriented performance relative to other response dimensions. This finding motivated the subsequent use of continual pre-training, GRPO-based alignment, and retrieval-augmented grounding. The supervised baseline results therefore established two constraints for the remainder of the study: LLaMA-3.1-8B was the most suitable backbone, and SFT alone was insufficient for the clinical safety requirements of HFrEF decision support.

3.3. Stage-wise ablation of the dynamic alignment framework

The stage-wise ablation assessed whether sequential optimization altered model behaviour beyond the supervised LLaMA-3.1-8B baseline. The textual analysis follows the clinical pipeline order of SFT baseline, continual pre-training (‘llama3.1_c`), GRPO alignment (‘llama3.1_p`), and retrieval-augmented generation (‘llama3.1_r`), although the visualization files sort some variants alphabetically. For clarity, the model variants in Figure 4 are presented in the following comparative order: SFT baseline, SFT with continual pre-training, SFT with continual pre-training and GRPO alignment, and the final retrieval-augmented variant. This order reflects the progressive comparison of model performance and does not redefine the overall methodological framework, which follows the CPT–SFT–GRPO–RAG pipeline. In the composite comparison, the Clinical Score increased from 0.716 at the SFT baseline to 0.724 after continual pre-training, 0.748 after GRPO alignment, and 0.864 after retrieval augmentation (Figure 4(a)). This pattern indicated that the largest clinical gain occurred after guideline-grounded retrieval was added to the aligned policy, rather than after language-model adaptation alone.

Figure 4.

Comparative performance across the staged model variants. The variants are presented in the following order: SFT baseline, +CPT, +GRPO, and +RAG. In the figure legend, “llama3.1” denotes the SFT baseline, “llama3.1_c” denotes the CPT-enhanced variant, “llama3.1_p” denotes the GRPO-aligned variant, and “llama3.1_r” denotes the final retrieval-augmented variant. (a) Composite clinical and Linguistic Fidelity scores across the LLaMA-3.1 baseline and sequentially optimized variants. (b) Clinical-domain radial profile across alignment stages. (c) Linguistic Fidelity-domain radial profile across alignment stages. (d) Heatmap of standardized clinical and Linguistic Fidelity metrics across alignment stages. This ordering is provided to clarify the progressive comparison of model performance. In the figure, the label “NLP_Score” in the legend corresponds to the Linguistic Fidelity Score.

The clinical-domain radial profile showed that the staged pipeline improved safety- and guideline-sensitive dimensions more strongly than generic response structure (Figure 4(b)). Risk Safety increased from 0.820 at the supervised baseline to 0.843 after continual pre-training, 0.902 after GRPO alignment, and 0.948 after retrieval augmentation. Guideline Score followed a similar but larger trajectory, increasing from 0.479 to 0.518 after continual pre-training, 0.544 after GRPO alignment, and 0.881 in the final retrieval-grounded variant. These results indicate that continual pre-training produced modest domain adaptation, GRPO strengthened risk-oriented behaviour, and retrieval augmentation supplied the largest improvement in guideline consistency.

3.4. Sensitivity analysis of reward weight configuration

To assess the robustness of the clinical reward design, a sensitivity analysis was performed by varying the HFrEF fusion reward weights across four alternative configurations. The results showed that the Clinical Score remained stable across all weight configurations, ranging from 0.841 (guideline-prioritized: ω₁=0.3, ω₂=0.3, ω₃=0.4) to 0.868 (safety-prioritized: ω₁=0.3, ω₂=0.5, ω₃=0.2), compared with 0.864 for the default configuration. Risk Safety was most sensitive to weight changes, increasing from 0.931 to 0.955 when the safety weight was raised from 0.3 to 0.5, while Guideline Score increased from 0.852 to 0.893 when the guideline weight was raised from 0.2 to 0.4. Critically, the Alignment Tax pattern was preserved across all configurations: BLEU-4 remained in the range 0.261-0.285, consistently below the supervised baseline of 0.371, while all clinical metrics exceeded the supervised baseline regardless of weight specification. These findings indicate that the observed trade-off between linguistic fidelity and clinical adequacy is not an artefact of a specific weight configuration but reflects a systematic effect of the alignment pipeline. The default weights (ω₁=0.4, ω₂=0.4, ω₃=0.2) were retained because they provided the most balanced improvement across drug completeness, safety, and guideline consistency.

3.5. The alignment tax: Trading linguistic similarity for clinical adequacy

The ablation experiment identified a consistent tradeoff between clinical adequacy and lexical similarity. The final RAG variant reached a Clinical Score of 0.864, compared with 0.716 for the SFT baseline. It also reached Risk Safety of 0.948 and Guideline Score of 0.881, indicating stronger recognition of high-risk clinical states and closer adherence to heart-failure guideline logic. In contrast, the Linguistic Fidelity Score decreased from 0.669 to 0.608, and BLEU-4 decreased from 0.371 to 0.272 over the same sequence. This pattern defines the alignment tax observed in the study: later optimization stages exchanged surface similarity for clinically constrained generation.

This tradeoff should not be interpreted as simple performance degradation. BLEU-4 and related overlap metrics reward lexical resemblance to reference answers, whereas the alignment pipeline progressively forced outputs toward structured, guideline-directed, safety-sensitive terminology. As the model moved from SFT baseline to CPT, GRPO, and RAG, generated responses became less dependent on loose colloquial phrasing and more dependent on constrained medication classes, risk warnings, monitoring requirements, and evidence-grounded HFrEF management logic. The reduced NLP overlap therefore reflected a shift in output distribution toward stricter clinical form rather than a loss of task competence.

The alignment tax is clinically informative because the direction of change was not random. The same final model that lost surface overlap achieved the highest values for the prespecified clinical endpoints most relevant to HFrEF decision support. In this setting, improved Guideline Score and Risk Safety carry greater interpretability than preservation of n-gram similarity, because unsafe omission of guideline-directed therapy or failure to recognize risk modifiers would be more consequential than stylistic divergence from a reference response. The ablation results therefore support evaluation frameworks that separate clinical adequacy from linguistic concordance when assessing medical language models. It should be acknowledged, however, that the decline in lexical overlap metrics does not inherently prove that the aligned outputs are clinically superior; rather, the observed direction of change is consistent with the hypothesis that clinical constraint improves safety-relevant behaviour. The interpretation of the Alignment Tax as favourable depends on the validity of the expert-derived clinical scoring rubric and the representativeness of the test cases. Future prospective studies with blinded clinical outcome adjudication will be needed to confirm whether the alignment-induced output changes translate into measurable improvements in clinician decision quality and patient outcomes.

3.6. Reinforcement optimization dynamics and policy stability

The GRPO optimization curves showed stable policy training after supervised instruction tuning. The GRPO loss remained bounded across policy updates without a progressive upward drift or abrupt instability, indicating that the reinforcement stage did not disrupt the learned response format established during supervised tuning (Figure 5(a)). The KL divergence profile showed controlled deviation from the reference policy, increasing early during optimization and then stabilizing around 0.0025 (Figure 5(b)). This pattern is consistent with policy movement toward higher-reward clinical behaviour while maintaining proximity to the supervised reference distribution.

Figure 5.

GRPO optimization dynamics during clinical policy alignment. (a) Bounded GRPO loss trajectory across policy updates. (b) KL divergence profile indicating controlled deviation from the reference policy. (c) Mean completion length showing progressive stabilization of structured response generation. (d) Aggregate reward trajectory reflecting overall policy optimization.

The mean completion length also changed in a structured manner during policy optimization. Completion length increased during the early training phase and then stabilized, suggesting adaptation to the required five-step clinical response format rather than unconstrained verbosity (Figure 5(c)). The aggregate reward curve increased during policy optimization, indicating that the generated responses progressively better satisfied the combined objective used for alignment (Figure 5(d)). Taken together, the loss, KL divergence, completion-length, and aggregate-reward profiles supported the interpretation that GRPO training produced controlled behavioural adaptation rather than unstable policy drift.

3.7. Computational efficiency and inference cost

Inference latency and memory requirements were measured to assess the practical computational cost of the staged alignment pipeline. The supervised fine-tuning baseline achieved a mean inference latency of 1.2 seconds per case on a single NVIDIA A100 80GB GPU with vLLM serving. The addition of continual pre-training did not measurably increase latency, as the adapted parameters were merged into the model weight at inference time. The GRPO-aligned variant had a comparable latency of 1.3 seconds, reflecting the fact that policy optimization affects training but not inference-time computation. The final RAG variant introduced an additional retrieval overhead of approximately 0.7 seconds per case (embedding computation and vector search), yielding a total inference latency of 1.9 seconds. Peak GPU memory consumption was 18.2 GB for the SFT baseline and 19.1 GB for the RAG variant, both well within the capacity of a single A100 GPU. These results indicate that the retrieval-augmented alignment pipeline introduced a clinically acceptable computational overhead of approximately 0.7 seconds and 0.9 GB relative to the supervised baseline, supporting the feasibility of local deployment within hospital infrastructure.

3.8. Qualitative validation of guideline compliance and safety-oriented reasoning

A representative HFrEF case was used to visualize the qualitative difference between the supervised baseline and the final aligned model under the same clinical input (Figure 6). Qualitative review of the case examples showed that the aligned output pattern preserved the five-step structured clinical reasoning framework in practical HFrEF scenarios. Across examples, responses consistently organized recommendations into dietary management, nursing or monitoring level, further examinations, medication strategy, and follow-up safety warnings. This structure was maintained across different clinical presentations, including acute decompensation with pulmonary congestion, chronic exertional dyspnoea, suspected acute coronary syndrome, atrial fibrillation, chronic obstructive pulmonary disease, diabetes, and renal comorbidity. The consistency of this format is clinically relevant because HFrEF decision support requires simultaneous attention to congestion, haemodynamic status, medication eligibility, comorbidity constraints, and longitudinal monitoring.

Figure 6.

Representative case comparison of HFrEF management recommendations before and after dynamic alignment. The same HFrEF case input was used to compare the supervised baseline with the final aligned model. The aligned output showed more explicit monitoring, broader guideline-directed medical therapy coverage, and clearer safety escalation advice.

The medication outputs reflected guideline-directed HFrEF treatment logic in the available examples. Stable or stabilizing cases frequently included sacubitril/valsartan, a beta-blocker such as metoprolol, mineralocorticoid receptor antagonist therapy with spironolactone, and an SGLT2 inhibitor such as dapagliflozin. Congested cases included loop diuretics such as furosemide, often paired with monitoring of urine output, body weight, electrolytes, and renal function. In cases with diabetes and prior myocardial infarction with stent implantation, the outputs incorporated diabetic dietary control, glucose-related follow-up, statin therapy, and conditional dual antiplatelet therapy based on coronary evaluation. In the atrial fibrillation example, the response included rhythm or rate monitoring, stroke-risk assessment, and oral anticoagulation with rivaroxaban. These examples indicate that the aligned response schema encoded both foundational HFrEF therapy and comorbidity-sensitive treatment adaptation. The safety reward mechanism also explicitly penalized outputs that recommended drugs in the presence of absolute contraindications. In test cases involving a history of angioedema with angiotensin-converting enzyme inhibitors, the aligned model avoided ACE inhibitor recommendations and instead suggested alternative therapies such as angiotensin receptor blockers, demonstrating that the safety reward successfully encoded contraindication-aware reasoning. Similarly, in cases with severe hyperkalaemia (potassium ≥ 5.0 mmol/L), the model omitted mineralocorticoid receptor antagonists and recommended potassium correction before reconsidering initiation, rather than listing spironolactone as part of standard GDMT without qualification. These behaviours contrasted with the supervised baseline, which occasionally recommended contraindicated drug classes without appropriate caveats, underscoring the role of the safety-oriented policy alignment in enforcing absolute contraindication awareness.

The case outputs also showed safety-oriented triage. Examples with severe dyspnoea, orthopnoea, oliguria, pulmonary rales, markedly elevated BNP, or suspected myocardial ischaemia were assigned intensive monitoring such as ICU or CCU care, continuous electrocardiographic surveillance, strict low-sodium intake, fluid restriction, and close volume assessment using urine output, body weight, pulmonary signs, electrolytes, and renal function. Cases with COPD included respiratory evaluation, blood-gas assessment, pulmonary-function follow-up, bronchodilator therapy, infection assessment, and controlled oxygen therapy. The renal comorbidity example incorporated low-protein dietary control, careful diuretic use, reduced-dose spironolactone with potassium monitoring, and surveillance of urine protein and renal function. In the representative comparison, the supervised baseline produced a coherent but relatively generic management plan, whereas the final aligned model provided more explicit monitoring priorities, broader GDMT coverage, and clearer escalation advice under the clinical input shown in Figure 6. These qualitative findings support the quantitative results by showing that the dynamic alignment framework promoted evidence-grounded, risk-aware, and schema-consistent outputs rather than isolated medication lists. To illustrate the practical significance of the Linguistic Fidelity Score decline for clinical workflow, representative sentence-level outputs from the supervised baseline and the final aligned model are compared below for a representative HFrEF test case (distinct from the case visualized in Figure 6). Supervised baseline (fluent but less clinically constrained): “The patient should receive standard heart failure medications including an ACE inhibitor, beta-blocker, and diuretic as needed for symptom control, with regular follow-up.” This output is lexically smooth and resembles textbook prose, but it omits specific drug classes (SGLT2 inhibitor, MRA), does not address the patient’s renal comorbidity, and provides no explicit monitoring thresholds. Final aligned model (less lexically similar but clinically safer): “Medication plan: Initiate valsartan 80 mg daily (ARNI and ACE inhibitors are contraindicated due to prior angioedema history); metoprolol succinate 23.75 mg daily; spironolactone 12.5 mg daily with potassium monitoring every 3 days given eGFR 38 mL/min; dapagliflozin 10 mg daily; furosemide 40 mg IV BID with strict urine output and weight monitoring. Contraindication check: avoid ACE inhibitor and ARNI due to prior angioedema history.” This output is more terse and structured, using specific medication names, doses, and monitoring instructions rather than general descriptive language. The reduced BLEU-4 score reflects this shift from narrative prose to structured clinical prescribing language, which is more appropriate for decision-support purposes even though it diverges from the surface form of the expert reference. For a practicing cardiologist, this difference means that the aligned model produces outputs that more closely resemble a structured order set or clinical pathway than a narrative consultation note, which may be more directly actionable in a workflow context. A second representative example involved an HFrEF patient with coexisting atrial fibrillation (CHA₂DS₂-VASc score 4) and stage 3 chronic kidney disease. Supervised baseline (fluent but less clinically constrained): “The patient should receive anticoagulation for atrial fibrillation along with heart failure medications and renal monitoring.” This output is coherent but omits the specific anticoagulant choice, dose adjustment for renal function, and does not flag the elevated bleeding risk. Final aligned model (less lexically similar but clinically safer): “Medication plan: Initiate apixaban 5 mg BID (dose-reduced to 2.5 mg BID if two of: age ≥ 80 years, body weight ≤ 60 kg, serum creatinine ≥ 1.5 mg/dL); metoprolol succinate 47.5 mg daily for rate control; spironolactone 12.5 mg daily with potassium monitoring; dapagliflozin 10 mg daily; furosemide 40 mg daily with daily weight monitoring. Safety check: reassess bleeding risk with HAS-BLED score; avoid NSAIDs given CKD and anticoagulation.” Again, the aligned output is more terse and structured, providing specific drug names, renal-adjusted dosing, and an explicit drug-interaction warning, whereas the supervised baseline produces fluent but vague prose.

4. Discussion

This study shows that an HFrEF-specific language model can be improved most meaningfully when the optimization target is shifted away from linguistic imitation and toward clinically weighted adequacy. The supervised LLaMA-3.1-8B baseline provided the strongest starting point among the candidate open-source models, but its performance profile also exposed the limits of supervised instruction tuning alone. The subsequent stages did not merely increase generic response quality; they changed the behaviour that the model was rewarded to express. Continual pre-training modestly reduced domain mismatch, GRPO alignment strengthened safety-sensitive generation, and retrieval-augmented grounding produced the largest movement toward guideline-consistent recommendations. This pattern is consistent with recent critiques that medical LLM evaluation often overestimates translational readiness when benchmarks do not reflect the clinical task, deployment context, or safety burden.^31,45 The principal contribution of the present study is therefore not the demonstration that an LLM can produce fluent HFrEF text, but that staged alignment can redirect an initially fluent model toward structured treatment adequacy, safety recognition, and evidence-grounded clinical reasoning.

The clinical significance of this shift is clearest in the HFrEF domain, where acceptable recommendations require simultaneous reasoning over medication classes, haemodynamic status, comorbidities, and follow-up risk. Foundational GDMT is not a single prescription decision; it involves ARNI, ACEI, or ARB logic, evidence-based beta-blocker titration, mineralocorticoid receptor antagonist use, SGLT2 inhibitor selection, congestion-directed diuretic therapy, and repeated surveillance of renal function, potassium, blood pressure, volume status, and treatment tolerance.^4,7–9 In the qualitative case analysis, the aligned model preserved the five-step response structure while adapting outputs to diabetes, prior myocardial infarction, atrial fibrillation, COPD, renal vulnerability, suspected acute coronary syndrome, and acute decompensation. The relevance of GRPO and RAG is that these stages explicitly targeted the clinical failure modes most likely to matter in HFrEF: omission of core therapies, failure to recognize risk modifiers, inadequate warning thresholds, and weak linkage between recommendations and guideline logic. The resulting behaviour is closer to a constrained decision-support template than to an open-ended conversational response, which is an appropriate direction for high-risk chronic cardiovascular care. The safety reward mechanism warrants further discussion in the context of absolute contraindications. In HFrEF management, certain drug-disease combinations represent hard safety boundaries (e.g., ACE inhibitors in patients with a history of angioedema, mineralocorticoid receptor antagonists in severe hyperkalaemia, or beta-blockers in acute decompensated heart failure with low cardiac output). The GRPO safety reward was designed to penalize any output that recommended a drug in the presence of a documented contraindication in the patient state (x_i), rather than merely rewarding the inclusion of general safety warnings. In the examined qualitative cases, the aligned model avoided contraindicated therapies and substituted appropriate alternatives, suggesting that the reward signal successfully encoded the distinction between relative cautions and absolute contraindications. This behaviour is particularly relevant for deployment in high-risk cardiology settings, where unrecognized contraindications could have immediate clinical consequences.

The Alignment Tax observed in this study should be interpreted as a methodological signal rather than as a defect. Conventional automatic metrics reward lexical overlap, but HFrEF care often requires clinically standardized wording that may diverge from the surface form of reference answers. A model that substitutes vague reassurance with explicit medication classes, monitoring requirements, contraindication-aware wording, and guideline-grounded follow-up instructions may appear less similar by BLEU or related overlap metrics while becoming more suitable for clinical decision support. This interpretation is aligned with the knowledge-practice gap identified in recent reviews of clinical LLM benchmarks, where high performance on medical knowledge tasks does not necessarily translate into safe or complete clinical action. ⁴⁶ It also agrees with broader arguments that medical exam-style benchmarks and generic automatic metrics can create an illusion of readiness when they fail to measure construct validity, task realism, and clinical consequence.^30,31 In this setting, the decrease in linguistic concordance is best understood as the cost of forcing the output distribution toward stricter medical terminology and risk-aware structure.

To contextualize the present work within the broader landscape of clinical LLM research, Table 6 summarizes representative recent studies in clinical LLM alignment, retrieval-augmented generation, and cardiovascular decision support, mapping each approach to its methodological focus and the research gap addressed by the present study.

Table 6.

Comparison with representative recent work in clinical LLM alignment.

Study (Year)	Domain/Approach	Key method	Research gap addressed by present study
Sandmann et al. (2025) 24	Clinical LLM benchmarking	DeepSeek model evaluation on clinical decision-making	Single-stage evaluation; no staged alignment or dual-track framework
Alexandrou et al. (2025) 26	Cardiology/RAG	LLM with guideline-embedded retrieval for ACS guidelines	Limited to knowledge QA; no medication safety or structured output schema
Fan et al. (2026) 47	Heart failure/LLM alignment	DPO for HF nursing documentation in critical care	Focused on documentation, not GDMT decision support or safety reasoning
Ke et al. (2025) 48	Medical RAG generalizability	RAG across 10 LLMs for medical fitness assessment	No clinical reward or safety-oriented alignment pipeline
Li et al. (2026) 49	Cancer/guideline-grounded LLM	CliCARE: guideline-grounded LLM for cancer EHR	Different domain; no dual-track clinical-linguistic evaluation
Korom et al. (2025) 50	Primary care/real-world CDSS	LLM-based CDSS deployed in primary care clinics	General domain; no HFrEF-specific safety or GDMT alignment
Abbas et al. (2025) 51	Explainable AI in CDSS	Meta-analysis of XAI methods in CDSS	Reviews explainability; no LLM alignment or alignment tax analysis
Gallifant et al. (2025) 36	LLM reporting guidelines	TRIPOD-LLM reporting guideline for clinical LLMs	Provides reporting framework; does not address alignment methodology
Present study	HFrEF/staged alignment	Four-stage CPT-SFT-GRPO-RAG with dual-track evaluation	Addresses all above gaps: staged alignment, dual-track evaluation, GDMT safety, alignment tax characterization

Note. Studies were selected to represent the spectrum of current approaches to clinical LLM alignment, retrieval-augmented generation, and cardiovascular decision support. The comparison highlights methodological gaps that motivated the present four-stage framework. LLM = large language model; RAG = retrieval-augmented generation; DPO = direct preference optimization; HFrEF = heart failure with reduced ejection fraction; CDSS = clinical decision support system.

These findings support the need for dual-track evaluation in medical text generation. Clinical adequacy and linguistic concordance are related but not substitutable. The former asks whether the output preserves medication completeness, safety recognition, guideline consistency, and structured coverage; the latter asks whether the generated text resembles an expert-authored reference. Recent evaluation frameworks for clinical LLMs similarly emphasize multidimensional human assessment, clinical relevance, completeness, safety, communication quality, and workflow fit rather than a single scalar score.^52,53 Work on medical text generation has also shown that fine-grained evaluation is needed because broad automatic metrics can miss medically meaningful omissions or unsafe substitutions. ²⁹ The dual-track framework used here operationalizes that distinction in an HFrEF benchmark: NLP metrics remain useful as secondary measures of textual fidelity, but they cannot serve as proxies for safe GDMT compliance. This separation is especially important when reinforcement alignment and retrieval grounding intentionally change the generated language toward more constrained clinical form.

The results should also be interpreted with caution. The final model is a retrospective benchmark system and should not be considered an autonomous clinician. Recent evaluation frameworks and safety studies indicate that medical LLM performance varies by task, workflow, supervision, and outcome definition, and that publicly available systems can provide unsafe answers to patient-posed medical questions.^{27,28,52–54} In HFrEF, inappropriate reassurance, medication omission, failure to recognize hypotension, missed renal risk, or unsupported antithrombotic advice could have direct clinical consequences. The present framework reduces these risks by combining explicit clinical reward design with retrieval-grounded evidence, but it does not eliminate the need for physician oversight. Local validation, audit trails, cardiologist review, and integration into defined clinical workflows would be required before any deployment. This caution is consistent with recent efforts to build more versatile medical LLMs, where broad capability is treated as a starting point rather than as proof of clinical reliability. ⁵⁵

Several limitations remain. The study used a retrospective single-center dataset from Affiliated Zhongshan Hospital of Dalian University, which may reflect local documentation style, treatment preferences, patient mix, and institutional guideline implementation. Prospective human-in-the-loop randomized validation was not performed, so the effect of model output on clinician behaviour, treatment decisions, time burden, and patient outcomes remains unknown.^27,52 Although the revised evaluation now includes blinded cardiologist adjudication and baseline demographic distributions (Table 5), the expert panel was drawn from the same institution and the sample size remains insufficient for robust subgroup-level performance assessment. ⁵³ Future work should test the framework across multiple hospitals, evaluate calibration of risk warnings, compare against proprietary and open-source clinical LLMs under identical HFrEF scenarios, and determine whether the observed shift from linguistic imitation to clinical adequacy improves real-world decision quality. A further limitation concerns the generalizability of the retrieval-augmented grounding component. The local heart-failure guideline knowledge base was constructed from Chinese and ESC-aligned guideline documents, and the high Guideline Score (0.881) reflects concordance with this specific guideline set. Heart failure guidelines can vary across regions (e.g., ESC versus ACC/AHA versus Chinese guidelines), and recommendations may differ regarding preferred angiotensin blocker formulations, diuretic strategies, or device therapy thresholds. A hospital using a different preferred guideline set may observe different guideline concordance scores, and the retrieval corpus would need to be re-indexed with locally relevant guideline content. The modular design of the RAG component facilitates such re-indexing, but empirical validation across multiple guideline sets was beyond the scope of the present study. Beyond the clinical setting, broader AI governance principles developed in other data-intensive domains, including data privacy, algorithmic transparency, and informed consent frameworks, ⁵⁶ are equally relevant to healthcare AI deployment and warrant integration into future evaluation protocols. Additionally, the single-centre, single-language, single-EHR-system design may limit transferability to institutions with different documentation practices, medication formularies, patient demographics, or healthcare workflows. External validation using a temporally or geographically distinct cohort, ideally with a different EHR system and language environment, would be necessary to establish whether the observed performance profile generalizes beyond the source institution. A prospective human-in-the-loop randomized validation of this tool could be designed as follows: eligible HFrEF inpatients would be randomized to receive care with or without access to the aligned LLM decision-support output. In the intervention arm, the treating cardiologist would review the model’s structured recommendations alongside standard clinical assessment, while the control arm would follow usual care. Primary outcomes could include GDMT initiation and titration rates, time to optimal therapy, rates of medication-related adverse events, and clinician-reported decision confidence. Secondary outcomes could include time burden, workflow integration metrics, and patient-reported outcomes. Such a design would directly test whether the alignment-induced shift from linguistic imitation to clinical adequacy translates into measurable improvements in real-world HFrEF management quality.

5. Conclusion

This retrospective benchmark shows that a four-stage dynamic alignment pipeline can convert a generalist open-source language model into a clinically constrained HFrEF decision-support system. By combining continual pre-training, supervised instruction tuning, reinforcement-based policy optimization, and retrieval-augmented guideline grounding, the framework shifted generation toward safer medication reasoning, stronger GDMT compliance, explicit risk recognition, and structured clinical outputs. The observed Alignment Tax defines the central methodological result: reduced surface overlap with reference prose is not inherently degradation when it reflects movement away from loose linguistic imitation and toward stricter medical adequacy. For HFrEF, where omissions in drug classes, contraindication handling, or warning thresholds can carry clinical risk, this tradeoff is both expected and clinically defensible. Future validation should test the framework prospectively across institutions with cardiologist adjudication and workflow-based human oversight. Therefore, we recommend that developers of clinical LLMs in high-risk domains explicitly incorporate a dual-track evaluation framework and accept an Alignment Tax on surface linguistic metrics as a necessary cost for achieving verifiable gains in medication safety and guideline adherence. The present findings suggest that this trade-off is not only measurable but clinically defensible, and should be transparently reported rather than concealed behind aggregate performance scores.

Appendix.

Abbreviations

LLMs

Large language models

HFrEF

Heart failure with reduced ejection fraction

GDMT

Guideline-directed medical therapy

LVEF

Left ventricular ejection fraction.

ORCID iDs

Lu Liu https://orcid.org/0009-0002-1926-0470

Boyuan Shi https://orcid.org/0009-0000-7210-8086

Ethical considerations

This study was conducted in accordance with the principles of the Declaration of Helsinki, and was approved by the Ethics Committee of Affiliated Zhongshan Hospital of Dalian University Approval No. KYS2026-011). All research involving human participants complied with the ethical standards of the institutional and/or national research committee and with the 1964 Declaration of Helsinki and its subsequent amendments or comparable ethical standards.

Data Availability Statement

The data that support the findings of this study are available from the corresponding author upon reasonable request and with permission from the Affiliated Zhongshan Hospital of Dalian University. The data are not publicly available due to privacy and ethical restrictions imposed by the Institutional Review Board of the Affiliated Zhongshan Hospital of Dalian University.*