Skip to main content
NCBI home page
PLOS Digital Health logoLink to PLOS Digital Health
. 2023 Feb 8;2(2):e0000190. doi: 10.1371/journal.pdig.0000190

Optimizing cardiovascular risk assessment and registration in a developing cardiovascular learning health care system: Women benefit most

T Katrien J Groenhof 1, Saskia Haitjema 2, A Titia Lely 3, Diederick E Grobbee 1, Folkert W Asselbergs 4,5,6, Michiel L Bots 1,*; on behalf of the UCC-CVRM and UPOD Study groups
Editor: Leo Anthony Celi7
PMCID: PMC9931327  PMID: 36812613

Abstract

Since 2015 we organized a uniform, structured collection of a fixed set of cardiovascular risk factors according the (inter)national guidelines on cardiovascular risk management. We evaluated the current state of a developing cardiovascular towards learning healthcare system–the Utrecht Cardiovascular Cohort Cardiovascular Risk Management (UCC-CVRM)—and its potential effect on guideline adherence in cardiovascular risk management. We conducted a before-after study comparing data from patients included in UCC-CVRM (2015–2018) and patients treated in our center before UCC-CVRM (2013–2015) who would have been eligible for UCC-CVRM using the Utrecht Patient Oriented Database (UPOD). Proportions of cardiovascular risk factor measurement before and after UCC-CVRM initiation were compared, as were proportions of patients that required (change of) blood pressure, lipid, or blood glucose lowering treatment. We estimated the likelihood to miss patients with hypertension, dyslipidemia, and elevated HbA1c before UCC-CVRM for the whole cohort and stratified for sex. In the present study, patients included up to October 2018 (n = 1904) were matched with 7195 UPOD patients with similar age, sex, department of referral and diagnose description. Completeness of risk factor measurement increased, ranging from 0% -77% before to 82%-94% after UCC-CVRM initiation. Before UCC-CVRM, we found more unmeasured risk factors in women compared to men. This sex-gap resolved in UCC-CVRM. The likelihood to miss hypertension, dyslipidemia, and elevated HbA1c was reduced by 67%, 75% and 90%, respectively, after UCC-CVRM initiation. A finding more pronounced in women compared to men. In conclusion, a systematic registration of the cardiovascular risk profile substantially improves guideline adherent assessment and decreases the risk of missing patients with elevated levels with an indication for treatment. The sex-gap disappeared after UCC-CVRM initiation. Thus, an LHS approach contributes to a more inclusive insight into quality of care and prevention of cardiovascular disease (progression).

Author summary

The (inter)national guidelines for management of cardiovascular risk state that for all patients who come to a health care provider for the evaluation of their cardiovascular risk or symptoms a certain set of cardiovascular risk factors should be measured. In order to facilitate this recommendation across all hospital specialisms that take care of patients who come for the evaluation of their cardiovascular risk or symptoms, we organized in our hospital a uniform, structured collection of a fixed set of cardiovascular risk factors (the UCC-CVRM initiative) as part of a learning health care system (LHS). In the present report, we evaluated the effect of this approach on adherence to the guideline by comparing the guideline adherence before and after UCC-CVRM. We found that the adherence to the guideline improved considerably, more pronounced in women than in men. Furthermore, UCC-CVRM lead to a reduction of the risk of missing patients with elevated levels with an indication for treatment. We conclude that such an LHS approach contributes to a more inclusive insight into quality of care and prevention of cardiovascular disease (progression).

Introduction

Healthcare is challenged by a growth in patient numbers. Furthermore, these patients are of higher age, suffer from multiple diseases–including a higher portion of chronic diseases-, and use more medications simultaneously [1]. Traditionally, evidence that forms the basis of care is derived from trials and cohorts. Yet, insights from trials are not always compatible with routine care. This science to care gap can in part be explained by differences in patient characteristics due to strict selection criteria or differences in care setting [2]. For example women are underrepresented in cardiovascular studies and have shown to receive substandard care in terms of risk management [3,4]. Altogether, the science to care gap results in modest use of evidence based findings in clinical practice, increased off label treatment, and non-adherence to guidelines. In return, the potential of routine care data is not captured fully, potentially resulting in a loss of valuable information on the course of disease and treatment in real life. This sparked the interest for a learning healthcare system (LHS).

The LHS was first described in 2007 as a system where routine care delivery and evidence synthesis are connected in a cycle [5]. In a LHS, routine clinical data is used to evaluate quality and safety of care and to generate evidence and drive discoveries, that in turn can be implemented back into routine care when using the LHS cycle (Fig 1). Because the cycle starts at routine care and uses data that has already been collected for care purposes, learning health care systems have the potential to generate evidence more efficient in terms of quality, time and costs [1]. Transformation from a traditional healthcare system to such a cyclic LHS approach requires efforts from all stakeholders in care and science, starting at generating awareness and adoption of evidence based medicine and generating sufficient quality input data [6]. Maturation of a LHS involves a shift from a more passive disconnected evaluation of care, to full integration of research into routine care [6]. Since 2007, many initiatives around the LHS arose, yet remained mostly described in theory [7]. A great contrast with the essence of the LHS theorem: “learning by doing” [5].

Fig 1. Learning healthcare system cycle.

Fig 1

Open in a new tab

In 2016, the Center for Circulatory health of the University Medical Centre Utrecht, Utrecht, The Netherlands, started the Utrecht Cardiovascular Cohort Cardiovascular Risk Management (UCC-CVRM) initiative: a cardiovascular LHS [8]. Cardiovascular disease (CVD) (risk) management is notoriously difficult because of multi-morbidity and different phenotypes and severities of disease. Guidelines provide support and advocate cardiovascular risk factor management in all cardiovascular patients. Still, prior research has shown differences in completeness of CVRM measurement between clinical specialties and patient groups: a missed opportunity prevent disease [8,9]. In the cardiovascular field, other attempts for LHS are described by Maddox et al [1] in the domains of “science and informatics”, “patient-clinician partnerships”, “incentives”, and “development of a continuous learning culture”. Most initiatives focus on either of the domains, using for example EHR data for predictive modelling [amarasingham et al. med care 2010; 48:981–988], using EHR data for effective cohort identification or eCRF extraction [33248277], identifying incentives that are associated with clinical improvement, using the LHS as a culture shift for continuous learning [1]. The UCC-CVRM created an infrastructure for uniform, structured collection of cardiovascular risk profile in routine clinical care of all departments treating patients with cardiovascular disease(s). This enables to provide feedback to physicians on quality of care, and generate new evidence on etiology, diagnostics, prognosis, and therapy, including efficacy, safety and cost-effectiveness for cardiovascular disease (risk). Thus, UCC was designed to be able to facilitate all domains, catalyzed by the basis of solid cardiovascular risk assessment. Furthermore, we included all patients visiting any department with a link to/dedicated to cardiovascular disease, connecting 8 departments in our hospital (and primary care) and thus creating a synergetic basis for interdisciplinary collaborations.

Here we evaluated the current state of this developing cardiovascular learning healthcare system and it potential effect on guideline adherence in cardiovascular risk management.

Methods

Study design and population

All data from the EHRs of the UMC Utrecht is extracted to the Utrecht Patient Oriented Database (UPOD). In short, UPOD comprises all digital traces of the UMC Utrecht, e.q. clinical information, demographic data, medication, diagnoses and laboratory measurements. UPOD extracted this data from 2003 onwards, encompassing data of more than 2.3 million individual patients to date [10,11]. A complete description of the UPOD database has been published elsewhere [10].

In May 2015, the first patient was included in the Utrecht Cardiovascular Cohort (UCC-CVRM) in the University Medical Center Utrecht. The UCC-CVRM is a prospective cohort study targeted to uniform assessment and registration of the guideline based cardiovascular risk profile in all patients presenting with a (risk factor for) cardiovascular disease within routine care. A detailed description of the protocol is published elsewhere. All data collected in UCC-CVRM is also captured in the EHR of the UMC Utrecht at the uniform structured locations.

We conducted a before-after study comparing data from patients included in UCC-CVRM and patients treated in the period before UCC-CVRM initiation who would have been eligible for UCC-CVRM from UPOD. We used data from patients visiting outpatient clinics who provided a written informed consent for UCC-CVRM from January 2016 up to December 31th 2018, and UPOD patients from January 1, 2013 to December 31, 2015. UPOD patients were matched to UCC-CVRM patients 1:4 based on age, sex, department of referral and inclusion in UCC-CVRM and diagnosis at inclusion. Diagnosis at inclusion was defined as the diagnosis treatment code registered closest to date of inclusion in UCC-CVRM. For UPOD, a theoretical inclusion date was defined by taking the time difference between inclusion and diagnosis for the UCC-CVRM patient date and applying this difference to the date of diagnose code for the matched UPOD patient. Throughout the manuscript, we will refer to the UPOD population as the “before UCC-CVRM” patients.

Ethics and privacy

UCC-CVRM has been approved by the Institutional Review Board of the UMC Utrecht and all data is handled according to privacy regulations [2]. UPOD, encompassing secondary use of EHR data, is in accordance with Institutional Review Board (IRB) and privacy regulations of the UMC Utrecht: clinical data can be used for scientific purposes if patients cannot be identified directly from the data. All patients are informed through the opt-out procedure, a general UMC Utrecht procedure via which patients can object to use of their clinical data for scientific evaluations.

Data collection

We collected data on sex, age, cardiovascular history and lifestyle factors, physical measurements (BMI and blood pressure), laboratory measurements including total cholesterol, high-density lipoprotein-c (HDL-c), low-density lipoprotein-c (LDL-c), triglycerides, estimated glomerular filtration rate (eGFR) according to the CKD-EPI formula, hemoglobin, and glycated hemoglobin (HbA1c) [12]. An overview of variables and data sources is listed in Table 1. We predefined time windows between inclusion data and variable measurement for data extraction per variable (Table 1).

Table 1. Data sources and time window per variables stratified for study population.

UCC-CVRM Before UCC-CVRM
Variables Data source Data source Time window
Demographics
Age Structured questionnaire General hospital administration
Sex Structured questionnaire General hospital administration
Cardiovascular history
CVD events Structured questionnaire Diagnosis and financial billing codes Before inclusion date*
Risk factor history Structured questionnaire Diagnosis and financial billing codes Inclusion date to 42 days after
Prescription of medication for that risk factor
Lifestyle
Smoking Structured questionnaire Text mining algorithm 365 before to 7 days after
(Un)Structured questionnaires
Free text
Physical activity Structured questionnaire Not extractable Inclusion data to 180 days after N/a
Physical measurements
BMI Structured measurements field EHR Structured measurements field EHR
Blood pressure Structured measurements field EHR Structured measurements field EHR Inclusion date to 365 days after
Laboratory measurements
Hb, total cholesterol, HDL-c, LDL-c, triglycerides, eGFR (CKD EPI formula), creatinine, HbA1c Laboratory measurements Laboratory measurements 60 days before to 60 days after inclusion date
Cardiovascular medication use Structured questionnaire Extracted form electronic prescription system using the ATC codes starting with A10, B01, B02A, and C02 to C10. 90 days before to 90 days after
Open in a new tab

* the date of the matching diagnose code was used as a theoretical inclusion date

Outcome definitions

First, we compared the proportion of measured risk factors before UCC-CVRM and after UCC-CVRM in the full cohort and stratified for sex. We also evaluated factors (UCC-CVRM, sex, interaction of UCC-CVRM and sex, age, history of CVD, history of diabetes) associated with measurement of smoking, SBP, LDL-c, eGFR and HbA1c. Second, we assessed if the patients had an indication for (start or change) of treatment for elevated blood pressure, LDL-c, and HbA1c [13]. The blood pressure target was <140/90mmHg, the LDL-c target was <2.5 mmol/L (at the time the 1.8mmol/L target for the very high risk population had not been adopted yet in The Netherlands), and the HbA1c target was <53mmol/L for patients with type II diabetes. Third, we calculated the proportion of missed uncontrolled hypertension, dyslipidemia, and HbA1c by multiplying the proportion of patients that were not measured by the proportion of patients with an off target measurement. Then, we calculated the likelihood to miss patients with uncontrolled hypertension, dyslipidemia, and elevated HbA1c before UCC-CVRM by dividing the proportions missed before and after UCC-CVRM initiation. Fourth, we evaluated time trends in consent consults as a starting point for data collection. Finally, we provided feedback on clinical management of the most extreme measurements to evaluate awareness of guideline adherent CVRM. The most extreme values were defined as: a blood pressure above 180/110mmHg, a total cholesterol >8 mmol/L, a HbA1c level above 48mmol/mol without reported diabetes, an eGFR <30ml/min, and alcohol use >10 units per week [12].

Statistical analysis

Statistical analyses were conducted in R studio (version 3.4.1, Copyright (C) 2017 The R Foundation for Statistical Computing). We used student t- tests to compare normally distributed continuous variables and Pearson chi square or fisher’s exact test where appropriate for proportions. Associations of measurement of risk factors with UCC-CVRM initiation, sex, interaction of UCC-CVRM initation and sex, age, history of CVD, history of diabetes) were analyzed using generalized linear models with binomial family and logit link (logistic regression).

Results

Patient selection and characteristics

Up to December 31st 2018, 1904 out-patient clinic patients were included in UCC-CVRM. We could match these patients to 7195 patients in UPOD (1:3.8 match). Baseline characteristics of both populations are described in S1 Table. Prevalence of current smoking was higher before UCC-CVRM (23%) versus after UCC-CVRM initiation (12%). Other risk factors showed similar distributions before and after in UCC-CVRM initiation.

Guideline adherent cardiovascular risk factor assessment

Completeness of cardiovascular risk factor measurement ranged from 0% to 77% before UCC-CVRM (Fig 2; S1 Table). For all other risk factors, registration increased after UCC-CVRM initiation ranging from +11% (eGFR; from 76% to 87%) to +87% (physical activity; from 0% to 87%). For laboratory measurements, largest increase was seen for HbA1c (from 25% to 82%, +57%). This leads to a completeness of all cardiovascular risk factor measurements ranging from 82% to 84% after UCC-CVRM initiation. Completeness of risk factor measurement was lower in women compared to men before UCC-CVRM for all risk factors (Fig 3, S2 and S3 Tables). After UCC-CVRM initiation, increase in measurement was higher in women, especially for lipids, eGFR and HbA1c, independent of age or co-occurrence of cardiovascular diseases and diabetes (Table 2). This means that the UCC-CVRM organization decreased the sex-gap in terms of a difference in measurement of risk factors in men and women.

Fig 2. Completeness of risk factor measurement before and after UCC-CVRM initiation.

Fig 2

Open in a new tab

Fig 3. Completeness of risk factor measurement before and after UCC-CVRM initiation stratified for sex.

Fig 3

Open in a new tab

Table 2. Factors associated with measurement of risk factors.

Factors Measurement of risk factors
Smoking SBP LDL-c eGFR HbA1c
Intercept 0.98 (0.81–1.18) 0.31 (0.26–0.38) 0.53 (0.44–0.62) 0.62 (0.52–0.75) 0.30 (0.24–0.36)
UCC-CVRM initiation 4.26 (3.19–5.81) 3.68 (2.79–4.94) 6.99 (5.91–8.29) 1.45 (1.18–1.80) 8.73 (7.38–10.4)
Female sex 0.63 (0.56–0.71) 0.74 (0.66–0.84) 0.86 (0.77–0.95) 0.62 (0.55–0.70) 0.75 (0.68–0.84)
UCC-CVRM initiation * Female sex # 1.33 (0.89–1.97) 1.35 (0.92–1.96) 1.99 (1.53–2.59) 2.11 (1.57–2.85) 2.37 (1.83–3.08)
Age (year increase) 1.02 (1.02–1.03) 1.04 (1.04–1.04) 1.00 (1.00–1.00) 1.03 (1.03–1.03) 1.00 (1.00–1.00)
CVD 1.74 (1.50–2.01) 2.94 (2.47–3.54) 1.04 (0.93–1.15) 1.86 (1.61–2.16) 1.11 (0.99–1.24)
DM 1.31 (1.09–1.59) 1.29 (1.06–1.59) 0.98 (0.85–1.13) 1.54 (1.27–1.87) 2.84 (2.46–3.28)
Open in a new tab

UCC-CVRM–Utrecht Cardiovascular Cohort, SBP–Systolic Blood Pressure, LDL-C–Low-Density-Lipoprotein cholesterol, eGFR–estimated Glomerular Filtration Rate, HbA1c –glycated hemoglobin, CVD–history of a cardiovascular event including coronary heart disease/stroke/peripheral artery disease/abdominal aortic aneurysm, DM–history of diabetes mellitus.

#interaction of UCC-CVRM intervention and sex

Detection of hypertension, dyslipidemia, and elevated HbA1c

Blood pressure was measured in 77% of patients before UCC-CVRM and in 93% of UCC-CVRM patients (S1 Table). Hypertension with an indication for treatment (change) was found in 45% of patients before UCC-CVRM and in 48% of UCC-CVRM patients. Combining this with the percentage of unmeasured patients (inverse of measured: 23% before UCC-CVRM and 7% after UCC-CVRM initiation), this leads to a likelihood to miss hypertension of (0.23*0.45)*100% = 10.4% before UCC-CVRM and (0.07*0.48)*100% = 3.4% after UCC-CVRM initiation. This translates to a risk of missing of 3.4/10.4 = 0.33 after UCC-CVRM initiation: a 67% reduction of likelihood to miss uncontrolled hypertension after UCC-CVRM initiation (S3 Table). Similarly, LDL-c was measured in 33% of patients before UCC-CVRM and in 83% of UCC-CVRM patients. Dyslipidemia with an indication for treatment (change) was found in 65% of patients before UCC-CVRM and in 66% of UCC-CVRM patients. This translates to a 75% reduction of likelihood to miss uncontrolled dyslipidemia after UCC-CVRM initiation (S4 Table). Lastly, HbA1c was measured in 25% of patients before UCC-CVRM and in 82% of UCC-CVRM patients. Off target HbA1c with an indication for treatment (change) was found in 26% of patients before UCC-CVRM and in 11% of UCC-CVRM patients. This translates to a 90% reduction of likelihood to miss uncontrolled diabetes after UCC-CVRM initiation (S4 Table). The higher increase in measurement in women after UCC-CVRM initiation also translates to a larger reduction in likelihoods to miss uncontrolled hypertension, dyslipidemia, and diabetes in women (68%, 80%, and 93%, respectively) compared to men (61%, 70%, and 85%, respectively (all p<0.0001)).

Time trends

Data collection depends on inclusion of patients in UCC-CVRM. Times with limited staff availability (holidays, flu period) proved to be challenging to keep invitation and informed consent rates constant. This is reflected in the mean number of inclusion consults per month: 50 for holiday months (May, August, September, and December) and 77 for non-holiday months in 2018. Although UCC-CVRM is phrased and positioned as a LHS and routine clinical care, in the developing phase it was not seen as routine care by supporting staff and was therefore not prioritized during clinic rush hours in times with limited staff.

Feedback: Extreme values

A total of 233 extreme values were checked from 2016 to 2018 for guideline adherent management. From the clinical reports in the EHR, it was unclear if follow-up was provided according to the guidelines for 26 of the extreme values (11%). Of these 26, 11 were HbA1c values (42%) and 7 (27%) systolic blood pressure measurements and 8 other.

Discussion

The development of a cardiovascular LHS through systematic registration of the cardiovascular risk profile in all patients referred for evaluation of an (a)symptomatic cardiovascular disease improves guideline adherence substantially and decreases risk of missing patients with elevated risk factors levels with an indication for treatment. This effect is even more pronounced in women compared to men, indicating that how and LHS approach can resolve selectivity in care and provide and inclusive view on health in the entire–in this case cardiovascular–patient population.

The UCC-CVRM has set up a hospital-wide basis for a cardiovascular LHS starting with data collection in routine clinical care. The level of completeness of the cardiovascular risk assessment was significantly higher in UCC-CVRM compared to what we found before UCC-CVRM. Although guidelines prescribe evaluation of the cardiovascular risk profile in vascular patients referred to a hospital for further evaluation [14], the need for complete CVRM assessment might not have been recognized or prioritized considering this is a tertiary care center with high level complexity of diseases. The prevalence of abnormal HbA1c values was significantly higher before UCC-CVRM than after. Possibly, CVRM was checked by the general practitioner and only high-risk patients were referred, or HbA1c was selectively measured in (per) diabetic patient, both results in an overestimation of prevalence of abnormal value and also missed uncontrolled HbA1c. For hypertension and dyslipidemia this, however, did not seem to hold. The UCC-CVRM approach resulted in a more inclusive population: especially in women the increase in measurement completeness was eminent. In both EUROSPIRE and SWEDEHEART registration studies, women were underrepresented (24% and +/-26%, respectively). Women that were included were undertreated and failed to reach the treatment targets more frequently compared to men [3,4]. Amongst others, failure to recognize the importance of risk factor control and higher rates of reported drug side effects in women may have contributed to this difference [15,16].

When interpreting results on the effect of UCC-CVRM on the registration of risk factors one should take strengths and limitations into account. We proposed matching UCC-CVRM patients 1:4, which was impossible for some patients with outlier age (i.e. from the department of geriatrics) or with a very rare diagnosis. This might have resulted in a decrease of balance, yet we think this does not influence our results to the extent that it will change our main conclusions. Physical activity could not be retrieved for patients before UCC-CVRM. Within our EHR, there is no specific structured field defining physical activity nor do we have a text-mining algorithm to extract this from unstructured text. We therefore might have underestimated the registration of physical activity before UCC-CVRM and overestimated the impact of UCC-CVRM in this respect. Registration of smoking was high compared to other studies, which report approximately 68–94% availability of smoking status in structured EHR cells [17–19]. For our project, we used an in-house developed (free)text-mining algorithm tweaked to our EHR to retrieve smoking information, which is not restricted by structured fields or even specific parts in the EHR and therefore can extract more information from the EHR [20]. This smoking mining algorithm was validated on a UCC-CVRM set before and showed good diagnostic accuracy with exception of short-term current smoking because many patients had quit in the meantime resulting in a lower positive predictive value [19]. Potentially, similar overestimation of current smoking prevalence occurred for before UCC-CVRM patients, explaining the higher prevalence compared to UCC-CVRM. Another limitation of using routine care data for our analysis, as for any routine care data analyses, is the restriction to what is reported in the EHR. We assume here that everything what was measured, was reported in the EHR. As we used mostly lab-based or other very much structured data and a validated algorithm to text-mine smoking information [19], we think that our extractions are as complete as possible. Yet, not testing–thus reporting- a specific risk factor can also be regarded as meaningful as a) the factor was considered to be normal (for example: normal BMI appearance of a patient) and thus b) not relevant to clinical decision making (no proportion attributable risk by BMI for this patient). This missing not at random (MNAR, the choice for or against a reported factor contains meaning about this factor) pattern could have made us overestimate the likelihood of missing a clinically relevant risk factor [20]. Because these likelihoods are based on the assumption of equal distribution of these values among unreported values. Lastly, these data represent only patients visiting the outpatient clinic. UCC-CVRM organization for admitted patients has proven to be more challenging due to limited staff and burden for the patient as these are more diseased compared to outpatient visitors. We are investigating alternative routes to support complete cardiovascular risk factor assessment in these patients, potentially at their follow-up appointment after discharge.

Although the concept of LHS was known, an example of a practical application of it was lacking at the time UCC-CVRM was initiated [7]. We inevitably ran into challenges grown from pioneering. Since the UCC-CVRM was set up prospectively and also contains a biobank, interpretation of current regulation resulted in an informed consent requirement, which might have influenced the inclusiveness of our population. An academic reflection on current ethical and legislative frameworks and how these are compatible with both the aims of a LHS and existing regulations whilst safeguarding patients’ autonomy and wellbeing is now emerging [21,22]. Adoption of UCC-CVRM structure–most importantly the invitation and inclusion processes–required efforts to tailor these processes to department specific organizations and dedicated support from the Center of Circulatory Health Management. The paradigm shift of traditional healthcare system to an LHS requires a collaborative effort from all stakeholders, with strong leadership from stakeholder representatives [23].

UCC-CVRM has shown to promote optimal care, to provide feedback on the quality of care, and facilitate research. Now, sufficient quality and amount of data is available to support automated feedback processes and decision support, for these computerized decision support systems (CDSS) require complete input data [11]. A CDSS was developed to aid guideline adherent cardiovascular risk management [24]. With this tool, clinicians and patients are provided with an overview of their cardiovascular risk factors, a 10-years risk prediction for cardiovascular events, and guideline adherent suggestions for therapy [24]. CDSS can also be used for automated feedback on extreme values via alerts. On a higher level, CDSS for care evaluations, and benchmarking between clinicians or even hospitals. Formal cost-effectiveness evaluations regarding CDSS use for cardiovascular risk management are yet to be conducted.

Conclusion

In a collaborative effort by care and research professionals, the UCC-CVRM has set a benchmark for a cardiovascular LHS. Systematic registration of the cardiovascular risk profile in all vascular patients referred for evaluation impacts tremendously on guideline adherence and risk of missing patients with elevated risk factors levels with an indication for treatment. An LHS approach provides insight into of quality of care within an more inclusive cross-section of cardiovascular population, ultimately leading to improved primary and secondary prevention of cardiovascular disease.

Supporting information

S1 Table. Cardiovascular risk factor measurement and distributions before and after UCC-CVRM initiation.

(DOCX)

Click here for additional data file. (23.4KB, docx)
S2 Table. Presence of risk factor measurement before and after UCC-CVRM initiation, stratified for sex.

(DOCX)

Click here for additional data file. (23KB, docx)
S3 Table. Risk factor distributions before and after UCC-CVRM initiation, stratified for sex.

(DOCX)

Click here for additional data file. (19.8KB, docx)
S4 Table. Likelihood to miss uncontrolled risk factor calculations.

(DOCX)

Click here for additional data file. (18.5KB, docx)

Acknowledgments

Members of the UPOD study group: Wouter van Solinge, Imo Hoefer, Saskia Haitjema, Mark de Groot. Members of the Utrecht Cardiovascular Cohort- CardioVascular Risk Management (UCC- CVRM) Study group: F.W. Asselbergs, Department of Cardiology; G.J. de Borst, Department of Vascular Surgery; M.L. Bots (chair),Julius Center for Health Sciences and Primary Care; S. Dieleman, Division of Vital Functions (anesthesiology and intensive care); M.H. Emmelot, Department of Geriatrics; P.A. de Jong, Department of Radiology; A.T. Lely, Department of Obstetrics/Gynecology; I.E. Hoefer, Laboratory of Clinical Chemistry and Hematology; N.P. van der Kaaij, Department of Cardiothoracic Surgery; Y.M. Ruigrok, Department of Neurology; M.C. Verhaar, Department of Nephrology & Hypertension, F.L.J. Visseren, Department of Vascular Medicine, University Medical Center Utrecht and Utrecht University.

Data Availability

The data are based on information from confidential electronic health records. As such data cannot be made publicly available. Requests for use of the data can be directed to the UCC-CVRM project office (email: ucc@umcutrecht.nl). The request will be taken to the steering committee of UCC-CVRM and discussed further taking Dutch privacy and legal regulations into account. This approach has been detailed in the rationale and design publication of the UCC initiative. Eur J prev Cardiol 2017 by Asselbergs FW et al. (PMID: 28128643).

Funding Statement

The UCC-CVRM is primarily financed by the UMC Utrecht as it is care as usual. UCC-CVRM as a whole, and via MLB as chair of the UCC-CVRM steering committee, was partly supported by a grant from the Netherlands Organization for Health Research and Development (#8480-34001) to develop feedback procedures, not for salary. The funders had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript.

References

  • 1.Maddox TM, Albert NM, Borden WB, Curtis LH, Ferguson TB jr, Kao DP, et al. The Learning Healthcare System and Cardiovascular Care: A Scientific Statement From the American Heart Association. Circulation. 2017; 135(14): e826–e857. doi: 10.1161/CIR.0000000000000480 [DOI] [PubMed] [Google Scholar]
  • 2.Meystre SM, Lovis C, Burkle T, Tognola G, Budrionis A, Lehmann CU. Clinical Data Reuse or Secondary Use: Current Status and Potential Future Progress. Yearb Med Inform. 2017; 26(1): 38–52. doi: 10.15265/IY-2017-007 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 3.Hambraeus K, Tyden P, Lindahl B. Time trends and gender differences in prevention guideline adherence and outcome after myocardial infarction: Data from the SWEDEHEART registry. Eur J Prev Cardiol. 2016; 23(4): 340–348. doi: 10.1177/2047487315585293 [DOI] [PubMed] [Google Scholar]
  • 4.Cooney M, Reiner Z, Sheu W, Ryden L, Sutter J, De Bacquer D, et al. SURF—SUrvey of Risk Factor management: first report of an international audit. Eur J Prev Cardiol. 2014; 21(7): 813–822. doi: 10.1177/2047487312467870 [DOI] [PubMed] [Google Scholar]
  • 5.Institute of Medicine. In: Olsen L, Aisner D and McGinnis JM (eds). The learning healthcare system: workshop summary. Roundtabel on Evidence-Based Medicine. Washington, DC: The National Academies Press; 2007. Available from: https://www.ncbi.nlm.nih.gov/books/NBK53494/. Last accessed: 2019/07/26. [PubMed] [Google Scholar]
  • 6.Wouters RHP, Graaf R, Voest EE, Bredenoord AL. Learning health care systems: Highly needed but challenging. Learning Health Systems. 2020; e10211. doi: 10.1002/lrh2.10211 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 7.Budrionis A, Bellika JG. The Learning Healthcare System: Where are we now? A systematic review. J Biomed Inform. 2016; 64: 87–92. doi: 10.1016/j.jbi.2016.09.018 [DOI] [PubMed] [Google Scholar]
  • 8.Asselbergs FW, Visseren FL, Bots ML, de Borst GJ, Buijsrogge MP, Dieleman JM, et al. Uniform data collection in routine clinical practice in cardiovascular patients for optimal care, quality control and research: The Utrecht Cardiovascular Cohort. Eur J Prev Cardiol. 2017; 24(8): 840–847. doi: 10.1177/2047487317690284 [DOI] [PubMed] [Google Scholar]
  • 9.Kotseva K, Wood D, De Bacquer D, De Backer G, Ryden L, Jennings C, et al. EUROASPIRE IV: A European Society of Cardiology survey on the lifestyle, risk factor and therapeutic management of coronary patients from 24 European countries. Eur J Prev Cardiol. 2016; 23(6): 636–648. doi: 10.1177/2047487315569401 [DOI] [PubMed] [Google Scholar]
  • 10.Ten Berg MJ, Huisman A, Van den Bemt PM, Schobben AF, Egberts AC, Van Solinge WW. Linking laboratory and medication data: new opportunities for pharmacoepidemiological research. Clin Chem Lab Med. 2007; 45(1): 13–19. doi: 10.1515/CCLM.2007.009 [DOI] [PubMed] [Google Scholar]
  • 11.Bezemer T, de Groot MC, Blasse E, Ten Berg MJ, Kappen TH, Bredenoord AL, et al. A Human(e) Factor in Clinical Decision Support Systems. J Med Internet Res. 2019; 21(3): e11732. doi: 10.2196/11732 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 12.Richtlijn Cardiovasculair risicomanagement (CVRM). Nederlands Huisartsen Genootschap, Nederlandse Internisten Vereniging, Nederlandse Vereniging voor Cardiologie, 2019. Available from: https://www.nhg.org/sites/default/files/content/nhg_org/uploads/multidisciplinaire_richtlijn_cardiovaculair_risicomanagement.pdf. Last accessed: 2020/01/17. [Google Scholar]
  • 13.Piepoli MF, Hoes AW, Agewall S, Albus C, Brotons C, Catapano AL, et al. 2016 European Guidelines on cardiovascular disease prevention in clinical practice: The Sixth Joint Task Force of the European Society of Cardiology and Other Societies on Cardiovascular Disease Prevention in Clinical Practice (constituted by representatives of 10 societies and by invited experts)Developed with the special contribution of the European Association for Cardiovascular Prevention & Rehabilitation (EACPR). Eur Heart J. 2016; 37(29): 2315–2381. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 14.Franconi F, Brunelleschi S, Steardo L, Cuomo V. Gender differences in drug responses. Pharmacol Res. 2007; 55(2): 81–95. doi: 10.1016/j.phrs.2006.11.001 [DOI] [PubMed] [Google Scholar]
  • 15.Bots SH, Peters SAE, Woodward M. Sex differences in coronary heart disease and stroke mortality: a global assessment of the effect of ageing between 1980 and 2010. BMJ Global Health. 2017; 2(e000298). doi: 10.1136/bmjgh-2017-000298 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 16.Atkinson MD, Kennedy JI, John A, Lewis KE, Lyons RA, Brophy ST, et al. Development of an algorithm for determining smoking status and behaviour over the life course from UK electronic primary care records. BMC Med Inform Decis Mak. 2017; 17(1): 2. doi: 10.1186/s12911-016-0400-6 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 17.Wu CY, Chang CK, Robson D, Jackson R, Chen SJ, Hayes RD, et al. Evaluation of smoking status identification using electronic health records and open-text information in a large mental health case register. PLoS One. 2013; 8(9): e74262. doi: 10.1371/journal.pone.0074262 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 18.Patel J, Siddiqui Z, Krishnan A, Thyvalikakath TP. Leveraging Electronic Dental Record Data to Classify Patients Based on Their Smoking Intensity. Methods Inf Med. 2018; 57(5–06): e2. [DOI] [PubMed] [Google Scholar]
  • 19.Groenhof TKJ, Koers LR, Blasse E, de Groot M, Grobbee DE, Bots ML, et al. Data mining information from EHRs produced high yield and accuracy for current smoking status. J Clin Epidemiol. 2019; 118: 100–106. [DOI] [PubMed] [Google Scholar]
  • 20.Groenwold RHH. Informative missingness in electronic health record systems: the curse of knowing. Diagn Progn Res. 2020; 4: 8. doi: 10.1186/s41512-020-00077-0 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 21.Faden RR, Kass NE, Goodman SN, Pronovost P, Tunis S, Beauchamp TL. An ethics framework for a learning health care system: a departure from traditional research ethics and clinical ethics. Hastings Cent Rep. 2013; Spec No:S16-27. doi: 10.1002/hast.134 [DOI] [PubMed] [Google Scholar]
  • 22.Cumyn A, Barton A, Dault R, Cloutier AM, Jalbert R, Ethier JF. Informed consent within a learning health system: A scoping review. Learning Health Systems. 2019: e10206. doi: 10.1002/lrh2.10206 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 23.Kuhn TS, Hacking I. The Structure of Scientific Revolutions. Chicago: University of Chicago Press; 2012. [Google Scholar]
  • 24.Groenhof TKJ, Rittersma ZH, Bots ML, Brandjes M, Jacobs JJL, Grobbee DE, et al. A computerised decision support system for cardiovascular risk management ‘live’ in the electronic health record environment: development, validation and implementation-the Utrecht Cardiovascular Cohort Initiative. Neth Heart J. 2019; 27(9): 435–442. [DOI] [PMC free article] [PubMed] [Google Scholar]
PLOS Digit Health. doi: 10.1371/journal.pdig.0000190.r001

Decision Letter 0

Alistair Johnson

14 Nov 2022

PDIG-D-22-00238

Optimizing cardiovascular risk assessment and registration in a developing cardiovascular learning health care system: women benefit most

PLOS Digital Health

Dear Dr. Bots,

Thank you for submitting your manuscript to PLOS Digital Health. After careful consideration, we feel that it has merit but does not fully meet PLOS Digital Health's publication criteria as it currently stands. Therefore, we invite you to submit a revised version of the manuscript that addresses the points raised during the review process.

Please submit your revised manuscript within 30 days Dec 14 2022 11:59PM. If you will need more time than this to complete your revisions, please reply to this message or contact the journal office at digitalhealth@plos.org. When you're ready to submit your revision, log on to https://www.editorialmanager.com/pdig/ and select the 'Submissions Needing Revision' folder to locate your manuscript file.

Please include the following items when submitting your revised manuscript:

* A rebuttal letter that responds to each point raised by the editor and reviewer(s). You should upload this letter as a separate file labeled 'Response to Reviewers'.

* A marked-up copy of your manuscript that highlights changes made to the original version. You should upload this as a separate file labeled 'Revised Manuscript with Track Changes'.

* An unmarked version of your revised paper without tracked changes. You should upload this as a separate file labeled 'Manuscript'.

If you would like to make changes to your financial disclosure, please include your updated statement in your cover letter. Guidelines for resubmitting your figure files are available below the reviewer comments at the end of this letter.

We look forward to receiving your revised manuscript.

Kind regards,

Alistair Johnson

Section Editor

PLOS Digital Health

Journal Requirements:

1. Please review your reference list to ensure that it is complete and correct. If you have cited papers that have been retracted, please include the rationale for doing so in the manuscript text, or remove these references and replace them with relevant current references. Any changes to the reference list should be mentioned in the rebuttal letter that accompanies your revised manuscript. If you need to cite a retracted article, indicate the article's retracted status in the References list and also include a citation and full reference for the retraction notice.

2. Please amend your detailed Financial Disclosure statement. This is published with the article. It must therefore be completed in full sentences and contain the exact wording you wish to be published.

a State the initials, alongside each funding source, of each author to receive each grant.

b. State what role the funders took in the study. If the funders had no role in your study, please state: “The funders had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript.”

c. If any authors received a salary from any of your funders, please state which authors and which funders.

If you did not receive any funding for this study, please simply state: “The authors received no specific funding for this work.”

3. Please provide separate figure files in .tif or .eps format only and remove any figures embedded in your manuscript file. Please also ensure that all files are under our size limit of 10MB.

For more information about figure files please see our guidelines:

https://journals.plos.org/digitalhealth/s/figures

https://journals.plos.org/digitalhealth/s/figures#loc-file-requirements

4. We noticed that you used "data not shown" in the manuscript. We do not allow these references, as the PLOS data access policy requires that all data be either published with the manuscript or made available in a publicly accessible database. Please amend the supplementary material to include the referenced data or remove the references.

5. We notice that your supplementary tables are included in the manuscript file. Please remove them and upload them with the file type 'Supporting Information'. Please ensure that each Supporting Information file has a legend listed in the manuscript after the references list.

Additional Editor Comments (if provided):

Apologies for the delay in reviewing this manuscript. Upon receipt of the review I am recommending minor revision.

[Note: HTML markup is below. Please do not edit.]

Reviewers' comments:

Reviewer's Responses to Questions

Comments to the Author

1. Does this manuscript meet PLOS Digital Health’s publication criteria? Is the manuscript technically sound, and do the data support the conclusions? The manuscript must describe methodologically and ethically rigorous research with conclusions that are appropriately drawn based on the data presented.

Reviewer #1: Yes

--------------------

2. Has the statistical analysis been performed appropriately and rigorously?

Reviewer #1: Yes

--------------------

3. Have the authors made all data underlying the findings in their manuscript fully available (please refer to the Data Availability Statement at the start of the manuscript PDF file)?

The PLOS Data policy requires authors to make all data underlying the findings described in their manuscript fully available without restriction, with rare exception. The data should be provided as part of the manuscript or its supporting information, or deposited to a public repository. For example, in addition to summary statistics, the data points behind means, medians and variance measures should be available. If there are restrictions on publicly sharing data—e.g. participant privacy or use of data from a third party—those must be specified.

Reviewer #1: No

--------------------

4. Is the manuscript presented in an intelligible fashion and written in standard English?

PLOS Digital Health does not copyedit accepted manuscripts, so the language in submitted articles must be clear, correct, and unambiguous. Any typographical or grammatical errors should be corrected at revision, so please note any specific errors here.

Reviewer #1: Yes

--------------------

5. Review Comments to the Author

Please use the space provided to explain your answers to the questions above. You may also include additional comments for the author, including concerns about dual publication, research ethics, or publication ethics. (Please upload your review as an attachment if it exceeds 20,000 characters)

Reviewer #1: This is an interesting manuscript that demonstrates a real-world application of the learning healthcare system (LHS) applied in the context of cardiovascular risk assessment and management. Grohenhof et al. clearly state the problem definition. Notably, they present LHS as a solution to what they define as the science-to-care gap.

The authors conducted a prospective cohort study comparing the measurement of cardiovascular risk factors before and after the implementation of the UCC-CVRM, an infrastructure implemented in the EHR allowing for standardized collection of cardiovascular-related health data. To demonstrate the benefits of this infrastructure, the authors compared the proportion of measured risk factors before and after its implementation. The other outcomes measured by the authors were the proportion of patients with an indication of treatment for a given cardiovascular condition and an estimation of the proportion of patients with potentially missed severe cardiovascular-related conditions.

The results are interesting as they have shown an increase in the registration of all risk factors after implementing UCC-CVRM (see S1 Table and S2 Table). The authors extrapolate their results a little bit further by estimating the likelihood of missing uncontrolled risk factor in patients who would have otherwise required treatment (S4 Table).

I have the following comments :

Major issues

Abstract and Introduction

1) No explicit mention of similar LHS initiatives in the cardiovascular space. Notably, there seem to have other examples in the literature that have implemented LHS in this space, and it would be essential to cite and explain how your initiative differs. In the cited article of Maddox et al., multiple examples are discussed.

Methods

2) Please add p-values to show the significance of the multiple statistical tests conducted (S1 Table, S2 Table

3) I don’t agree with the calculation of the likelihood of missing hypertension: in the context of EHR documentation, not being reported does not equate not being measured. Moreover, your calculation implies a uniform distribution among unmeasured variables, which is also incorrect. I like the idea you explored with those calculations, but I would like to see a few sentenced conceding the limitation of the methodology.

Figures

4) Figure 5 and 6 do not allow for easy comparison of the pre/post CVRM implementation. I would suggest using side-by-side bar chart or use the S1 and S2 tables instead.

Minor issues

1) Minor grammar issues and sentence structures could benefit from some touch-up.

i.e. lines 48-50 could be rewritten for simplicity «Healthcare is challenged by a growth in patient numbers, patients of higher age, patients with multiple diseases and medications, and a higher level of chronicity of disease. »

2) Clarification of the statement line 275. Please clarify how CDSS relates or complements the work you presented in this manuscript. The last paragraph was not so clear to me.

3) Add a discussion regarding the interaction of CVRM initiation and female sex. Notably, elaborate on the potential reasons for which the results are significant for eGFR, A1c and not for SBP and smoking.

--------------------

6. PLOS authors have the option to publish the peer review history of their article (what does this mean?). If published, this will include your full peer review and any attached files.

Do you want your identity to be public for this peer review? If you choose “no”, your identity will remain anonymous but your review may still be made public.

For information about this choice, including consent withdrawal, please see our Privacy Policy.

Reviewer #1: Yes: Eric Yamga

--------------------

[NOTE: If reviewer comments were submitted as an attachment file, they will be attached to this email and accessible via the submission site. Please log into your account, locate the manuscript record, and check for the action link "View Attachments". If this link does not appear, there are no attachment files.]

While revising your submission, please upload your figure files to the Preflight Analysis and Conversion Engine (PACE) digital diagnostic tool, https://pacev2.apexcovantage.com/. PACE helps ensure that figures meet PLOS requirements. To use PACE, you must first register as a user. Registration is free. Then, login and navigate to the UPLOAD tab, where you will find detailed instructions on how to use the tool. If you encounter any issues or have any questions when using PACE, please email PLOS at figures@plos.org. Please note that Supporting Information files do not need this step.

PLOS Digit Health. doi: 10.1371/journal.pdig.0000190.r003

Decision Letter 1

Leo Anthony Celi

30 Dec 2022

Optimizing cardiovascular risk assessment and registration in a developing cardiovascular learning health care system: women benefit most

PDIG-D-22-00238R1

Dear prof Bots,

We are pleased to inform you that your manuscript 'Optimizing cardiovascular risk assessment and registration in a developing cardiovascular learning health care system: women benefit most' has been provisionally accepted for publication in PLOS Digital Health.

Before your manuscript can be formally accepted you will need to complete some formatting changes, which you will receive in a follow-up email from a member of our team. 

Please note that your manuscript will not be scheduled for publication until you have made the required changes, so a swift response is appreciated.

IMPORTANT: The editorial review process is now complete. PLOS will only permit corrections to spelling, formatting or significant scientific errors from this point onwards. Requests for major changes, or any which affect the scientific understanding of your work, will cause delays to the publication date of your manuscript.

If your institution or institutions have a press office, please notify them about your upcoming paper to help maximize its impact. If they'll be preparing press materials, please inform our press team as soon as possible -- no later than 48 hours after receiving the formal acceptance. Your manuscript will remain under strict press embargo until 2 pm Eastern Time on the date of publication. For more information, please contact digitalhealth@plos.org.

Thank you again for supporting Open Access publishing; we are looking forward to publishing your work in PLOS Digital Health.

Best regards,

Leo Anthony Celi, MD MS MPH

Editor-In-Chief

PLOS Digital Health

***********************************************************

Reviewer Comments (if any, and for reference):

Associated Data

    This section collects any data citations, data availability statements, or supplementary materials included in this article.

    Supplementary Materials

    S1 Table. Cardiovascular risk factor measurement and distributions before and after UCC-CVRM initiation.

    (DOCX)

    Click here for additional data file. (23.4KB, docx)
    S2 Table. Presence of risk factor measurement before and after UCC-CVRM initiation, stratified for sex.

    (DOCX)

    Click here for additional data file. (23KB, docx)
    S3 Table. Risk factor distributions before and after UCC-CVRM initiation, stratified for sex.

    (DOCX)

    Click here for additional data file. (19.8KB, docx)
    S4 Table. Likelihood to miss uncontrolled risk factor calculations.

    (DOCX)

    Click here for additional data file. (18.5KB, docx)
    Attachment

    Submitted filename: PDIG-D-22-00238 Response to reviewers.docx

    Click here for additional data file. (32KB, docx)

    Data Availability Statement

    The data are based on information from confidential electronic health records. As such data cannot be made publicly available. Requests for use of the data can be directed to the UCC-CVRM project office (email: ucc@umcutrecht.nl). The request will be taken to the steering committee of UCC-CVRM and discussed further taking Dutch privacy and legal regulations into account. This approach has been detailed in the rationale and design publication of the UCC initiative. Eur J prev Cardiol 2017 by Asselbergs FW et al. (PMID: 28128643).

    Articles from PLOS Digital Health are provided here courtesy of PLOS

    RESOURCES