The implementation of a value-based learning health system for preventative care in Ontario, Canada

Abstract

Objective: While value-based learning health systems may address challenges associated with the integrative delivery of therapeutic lifestyle management in usual care, the extent to which they have been evaluated in real-world settings have remained limited. Methods: To explore the feasibility and user-experiences, associated with the first-year implementation of a preventative Learning Health System (LHS), consecutive patients were evaluated following referral from primary and/or specialty care providers from the Halton and Greater Toronto Area in Ontario, Canada, between December 2020 and December 2021. The integration of a LHS into medical care was facilitated using a digital e-learning platform, and consisted of exercise, lifestyle, and disease-management counselling. The dynamic monitoring of user-data allowed patients and providers to modify goals, treatment plans, and care-delivery in real-time in accordance with patient engagement, weekly exercise, and risk-factor targets. All program costs were covered by the public-payer health care system using a physician fee-for-service payment model. Descriptive statistics evaluated attendance to prescheduled visits, drop-out rates, changes in self-reported weekly Metabolic Expenditure Task-Minutes (MET-MINUTES), perceived changes in health knowledge, lifestyle behaviours, health status, satisfaction with care, and programmatic costs. Results: 378 of 437 patients (86.5%) enrolled in the 6-month program; The average age of patients was 61.2 ± 12.2, 156 (41.3%) of which were female and 140 (37.0%) with established coronary disease. After 1 year, 15.6% dropped out of the program. On average, weekly MET-MINUTES rose by 191.1 throughout the program (95% CI [331.82, 57.96], P=0.007), with increases most prominent among sedentary populations. Participants reported significant improvements in perceived health status and health knowledge, at a total health-care delivery cost of $517.70 per patient for a completed program. Conclusion: The implementation of an integrative preventative learning health system was feasible, with high patient engagement and favourable user-experiences. Further research is required to compare health outcomes against usual care.

Introduction

Available evidence has demonstrated that the uptake and sustainability of healthy lifestyles are best achieved when integrated into health care [1,2]. Yet, the real-world feasibility associated with integrative care-delivery models for lifestyle modification have been shown to be undermined by factors including physician time, expertise, financial reimbursement, and platforms that allow for monitoring progression of health behaviours [3-5]. As a result, patient uptake, engagement, and sustainability of attaining minimum recommended levels of physical activity and other lifestyle factors in the population remain suboptimal [6-9].

Value-based learning health systems (LHS) have been proposed as a novel approach to enhance the quality, safety, effectiveness, and efficiency of complex medical care delivery [10]. An LHS is a type of ecosystem that brings together a variety of different elements, such as scientific research, social factors, technology, policy, legal considerations, and ethical principles to create a cycle of continuous learning and improvement. This approach allows for regular updates and advancements to be made throughout the entire system. Such systems are designed to continuously improve patient care and outcomes by using data in real time to inform medical decision-making and self-management behaviour. This is accomplished by collecting and analyzing data on factors, such as, patient outcomes, satisfaction, and costs, with goal of improving the quality of patient care and reducing healthcare expenditure. The framework for developing value-based LHSs consists of the following four components: 1) core values, 2) pillars and accelerators, 3) processes, and 4) outcomes. Core values govern the vision, mission, and operations of any LHS. Pillars and accelerators refer to the infrastructure, systems, and resources providing foundational supports, and generally consist of internal and/or external representation from scientific, social, technology, policy, legal, and ethical stakeholders. Outcomes may incorporate user-experiences, health indicators, and costs [11-13].

Processes inherent to any LHS focus on the dynamic use of care-delivery, data and knowledge to inform medical decision-making at both the individual-level, and at the programmatic level. In regards to the latter, continuous programmatic quality improvement relies on stakeholder pillars to evaluate the care-quality, performance indicators, and outcomes, in order to make iterative recommendations to operational flow, priorities, and care-delivery. LHSs have been used as an implementation strategy for a wide variety of clinical applications including, but not limited to, implementation and optimization of electronic health records, systematic improvement of clinical care in hospitals, improving the integration of precision medicine into clinical medicine, and mental health strategies [12,13]. There is evidence that value-based learning health systems can effectively improve healthcare quality, patient satisfaction, and costs [14]. However, the extent to which the implementation of a preventative-based learning health systems could improve the engagement and adherence of patient lifestyle modification remains unclear.

My Heart Fitness (MHF) is a multidisciplinary preventative care program that operates under value-based LHS. MHF’s core values incorporate accessibility, adaptability, and open-innovation. The overarching goals of the MHF LHS are two-fold: first, to provide individuals with highly accessible preventative care to improve population outcomes; and second, to implement a rapidly adaptive, highly responsive and affordable integrative health care solution, that leverages open innovation to account for the dynamic nature of lifestyle behaviour change in populations. The MHF LHS platform was created in large part using open-source coding, and underwent extensive pilot testing between May 2017 and Nov 2020. On December 15, 2020, the MHF LHS platform was officially launched in two selected regions in Ontario. The objective of this study was to examine the feasibility and user experiences associated with the first year of the MHF LHS implementation.

Methods

Overview of the MHF LHS

The MHF LHS consisted of two components: (1) a digital learning platform; (2) an agnostic health care integration implementation process. The digital learning management platform was comprised of educational videos, podcasts, and/or infographics, and health literacy quizzes that were organized within a series of educational learning modules. The platform tracked the engagement and progress of patients’ prescribed self-management activities (e.g., the review of education content, and the completion of self-monitoring tasks) through the platform’s digital analytics. The integration of the digital platform into medical care was facilitated by physicians and allied health professionals via telemedicine clinics. The MHF LHS pillars consisted of a variety of internal and external stakeholders, including medical care professionals (cardiologists, internists, family physicians, kinesiologists, dietitians, and nurse practitioners), 4 research scientists (population health, clinical epidemiology, health systems, and clinical research), as well as representation from medical education, research ethics, health informatics, and health policy. Anonymous feedback was regularly solicited from the program’s pillars to gather insights and suggestions for quality improvement. This feedback was discussed in weekly programmatic team meetings to facilitate continuous quality improvement to ensure alignment with core-values and priorities.

MHF LHS health care agnostic system integration began with a patient digital referral from a primary or specialty care physician, which facilitated enrollment and participation into the program. Referral indications were kept broad for both primary prevention (at-risk) and secondary prevention (those with established cardiovascular diseases) given the care needs for exercise and/or lifestyle modification counselling among such populations. Given core values of accessibility and adaptability, and open-innovation, the program was modified to provide physical activity prescriptions to populations regardless of physical disability. Accordingly, no exclusion criteria beyond age criteria (non-adult populations, 17 years of age or younger) were imposed.

Each patient referred into the MHF LHS participated in a 6-month program consisting of an educational and telemedicine clinic curriculum that taught patients about their risks, risk-factor management, smart goals, key concepts around the mechanisms involved in coronary artery disease, aging, and plaque rupture, and several aspects related to exercise (safety, goal-setting, monitoring, progression), healthy weight management, Canada’s Food Guide and the Mediterranean diets, and smoking-cessation. Each patient received baseline and exit report cards, and health (i.e., exercise and dietetic) prescriptions. Referring physicians received progress reports after each telemedicine visit encounter (Supplementary Appendix). The six-month telemedicine program began with a baseline comprehensive health risk assessment at programmatic entry/intake to address individualized health needs and goals and finished with an exit questionnaire to determine how health behaviours and perceptions have changed over the course of the program. Each patient received a total of 11 prescheduled telemedicine appointments with physicians, kinesiologists, and a dietitian, as well as individualized exercise and nutrition prescriptions, which were interspersed in approximate equal intervals throughout the six-month period. Each clinic visit allowed for providers and patients to review pre-assigned educational content, establish or re-establish patient goals, follow up on and monitor progress, identify new or pre-existing barriers, review patient symptoms and exercise safety parameters, and evaluate selective risk-factors (in accordance with the prespecified clinic visit content). Patients were given auto-reminder texts and/or emails prior to each prescheduled telemedicine visit. To mitigate care-fragmentation when the referring and attending MHF physicians differed, MHF attending physicians only recommended modifications to medication management, rather than prescribe medications where applicable. Correspondence between MHF attending and referring physicians were always facilitated through consultation and clinical progress notes, following each telemedicine MHF visit.

The central goals of the multidisciplinary telemedicine clinics were twofold: (1) To facilitate the monitoring and modification of treatment in response to patient progress in attaining lifestyle and/or risk-factor targets; (2) To oversee the counseling of personalized educational content between health-care providers and patients. Integration into clinical care facilitated a dynamic action plan that evaluated patient progress, and responded to patient progress, based on the anthropometric and self-management data provided by the patient. In so doing, health care professionals could modify exercise and dietary prescriptions, educational curricula, patients’ lifestyle goals, pharmacological management, and safety parameters (e.g., exercise intensity) where appropriate. Dynamic medical decision-making in response to patients’ self-reported behaviours and health status, was designed to emphasize the adaptability and responsiveness to patient data, in accordance with the design of learning health systems. In short, the patient goals, educational curriculum, action plans, and medical decision-making processes were iterative, customized, and responsive to behavioural progression (or lack thereof).

To help standardize interactions between MHF attending physicians, patients, and allied health professionals, a 4-week training program was initiated for all healthcare professionals, which consisted of a series of online training videos detailing key objectives and processes for each clinic, as well observation, instructions, and feedback on cognitive interviewing strategies (i.e., focusing on self-regulatory self-management) and motivational interviewing. Goals were established collectively between patients and MHF care providers to ensure shared accountability for lifestyle modification treatment plans.

The MHF LHS was entirely integrated within Ontario’s publicly funded health care system. To optimize accessibility, patients were not charged user fees or out-of-pocket payments for program participation. Each telemedicine visit was covered by physicians’ fees as reimbursed by the Ontario Health Insurance Plan (OHIP) using a fee-for-service model. To fund additional services not covered by the Ontario Insurance Plan (e.g., educational digital platform, kinesiologists, dietitians), a proportion of physician fees were retained by the MHF LHS to cover expenses. The MHF LHS received Independent Research Ethics approval as a continuous quality improvement initiative, thereby waiving the need for patient consent.

Data sources

Data was obtained from self-administered questionnaires and patient interviews as part of the clinical care delivery pathway. Self-administered questionnaires at program intake were captured electronically through OCEAN^TM and used to identify patient demographics, clinical factors (e.g., cardiovascular risk), behaviours (e.g., exercise, diet, alcohol, smoking), perceived health status, mental health, medication use, and anthropometric data. Such information was then uploaded into the patient’s electronic medical record. Additional data identified through patient interview at the time of each telemedicine clinical visit encounter were also uploaded into the patients’ electronic health record, and included such measures as height, weight, waist circumference, blood pressure, whether required educational material was reviewed, and weekly MET-MINUTES. Programmatic exit questionnaires were administered through Google^TM survey forms which sought patient perspectives involving changes in health status, health knowledge, health behaviours and care satisfaction. For the purposes of this study, most data was collated within Accuro EMR™. Patient access to, and review of, educational materials were tracked through electronic clicks/impressions on the MHF e-learning web platform. To ensure that patient data was usable for shared medical decision-making in real-time for clinical applicability purposes, such information was made available to patients and providers through the electronic medical records, clinical notes and/or patient progress report cards at each telemedicine visit encounter. All data platforms were HIPAA and PHIPA-compliant. Each patient was given a unique study identification number; no forms or questionnaires contained patient personal identifiers.

Feasibility

Feasibility was explored through various patient engagement metrics, which consisted of programmatic enrollment, attendance to prescheduled telemedicine visits, programmatic drop-out rates, as well as patient clicks/impressions within the digital e-platform. Attendance to prescheduled telemedicine visits were tracked at each encounter. Any no-shows (patients who did not attend a prescheduled visit) were contacted by the MHF administrative to reschedule. Patients could also voluntarily drop out of the program at any time. If no-shows could not be contacted after three attempts, had not responded to voice messages, text messages, or emails, and had no log-in activity on their e-platform, the MHF LHS adjudicated such individuals as having dropped out of the program.

User experiences

As with LHS frameworks, the priority for MHF was to utilize data in real-time to inform patient self-management, the provision of care, and programmatic quality control. The identification of minimal data variables that could serve multiple purposes necessitated various literature searches, environmental scans, and feasibility deployment evaluations. Emphasis was given to those data variables that had clinical utility, self-monitoring application, and outcome correlation. One such example was self-reported weekly Metabolic Expenditure Task-Minutes (MET-MINUTES), which served as a primary LHS data point for several reasons: First, it is an ideal exercise behavioural metric, given the incorporation of both exercise intensity (MET or Metabolic Expenditure Task) and duration. Second, publicly accessible materials, such as the Compendium of Physical Activity Guidelines, provide comprehensive sets of physical activities with their corresponding MET levels. Such materials have been incorporated into the MHF e-learning platform and serve as an ideal knowledge-translation tool for patients. Third, weekly MET-MINUTES is computationally straightforward to calculate for both patients and healthcare providers, by simply multiplying the MET associated with the patient’s preferred exercise activity by the number of minutes they performed that exercise activity per week. Such computational feasibility had applicability to kinesiologists who created personalized exercise prescriptions according to patients’ exercise preferences and weekly MET-MINUTES goals. Fourth, weekly MET-MINUTES have been widely used in research settings and have been shown to be inversely correlated with adverse health outcomes such as mortality and hospitalizations [13]. Finally, weekly MET-MINUTES was ideally suited for an LHS, given its potential application for data feedback reflecting patient behaviour. Failure to attain weekly MET-MINUTES goals helped inform educational counselling, prescribing, and provision of clinical care to patients in real time. Self-reported weekly MET-MINUTES were collected at every clinic visit throughout the 6-month program.

Additional user experiences were sought at the programmatic exit, and incorporated questionnaires related to perceptual changes in health, health knowledge, health behaviours and care satisfaction. These questionnaires underwent content validity testing including domain determination sampling, and instrument formation. Domains of interest were identified by conducting a literature review of relevant areas such as perceptions of health status, health knowledge, health behaviours and health-care satisfaction. Content and face validity of all questions was conducted internally by the MHF LHS clinical and research pillars. This expert panel consisted of 4 medical doctors (three cardiologists and one primary care provider), 1 nurse practitioner, 2 medical students, 2 kinesiologists, 1 dietitian, 1 pharmacist, and a community volunteer. Each measure was evaluated for relevance and clarity, and modifications were made to questionnaire items until unanimity among panellists was achieved. The questionnaires were then tested on a convenience sample of 10 patients to evaluate the feasibility of administration and the receiving of additional feedback.

Programmatic delivery costs

Programmatic delivery costs were entirely subsidized through publicly funded fee-for-service physician billings (in accordance with the Ontario Health Insurance Plan, OHIP). A component of the fee-for-service billing was retained by the program to help offset other programmatic operating costs not covered by OHIP (e.g., dietitian visits, kinesiologist visits, and administrative staff). Given that one of MHF LHS’ key performance indicators was the total cost to third-party payers, calculations did not separate components retained by physicians vs. those retained by the program.

Data analyses

Given that referrals and staggered enrollment throughout the first year of implementation, data analyses focussed on only those patients who had been pre-scheduled to have completed their 6-month program within the first year of implementation (the Learning knowledge cycle between December 15^th, 2020 and December 15^th, 2021). Descriptive statistics of key performance indicators incorporated t-test or Wilcoxon sign-rank (where applicable) for numerical data. To explore whether weekly self-reported exercise MET-MINUTES increased over time, one-way ANOVA testing explored weekly exercise MET-MINUTES over time, across the entire cohort, and within two prespecified subgroups: (1) non-exercising individuals reporting zero MET-MINUTES at program baseline (herein termed zero Weekly MET-MINUTES); (2) individuals who reported at least some exercise at program baseline (herein termed nonzero Weekly MET-MINUTES). Two-sample t-tests were used to determine how pre-post programmatic changes in weekly MET-MINUTES differed according to prespecified subgroups of age, gender, and the presence or absence of coronary artery disease. User experiences were evaluated using binomial tests for categorical data (which compared the distribution of self-reported responses against those expected assuming the ‘null hypothesis’ for programmatic efficacy). Statistical significance was defined as 2-tailed alpha <0.05. All analyses were conducted using R^TM.

Results

Baseline characteristics

Between December 2020 and December 2021, 437 patients were referred to the program through referrals from community physicians. In total, 378 of the 437 referred patients enrolled and participated in one or more visits during the study period (86.5% enrollment rate). The average age of patients was 61.18 ± 12.22, 156 (41.27%) of which were female, The average Framingham risk score was 18.59% ± 10.84%, 140 (37.04%) patients had established coronary disease, 170 (45.0%) dyslipidemia, 168 (44.4%) hypertension, 77 (20.4%) diabetes, and 153 (40.48%) patients had obesity. Average baseline MET-Minutes was reported at 585.74 ± 731.21 and 26 (6.88%) were current smokers, and 8 (2.12%) drank alcohol beyond recommended guidelines. Patients’ self-perceived health at baseline was 5.02 ± 2.10 on an ordinal scale from 1 (poorest health) and 10 (best health) (Table 1).

Table 1.

Baseline characteristics of patient participants enrolled between Dec 15, 2021 and Dec 16, 2022

Baseline Characteristics (Mean ± SD)	All Patients (N=378)
Demographics
Age
Median - yr	61.18 ± 12.22
Age >60 yr - no. (%)	200 (52.91%)
Female sex - no. (%)	156 (41.27%)
Race or ethnic group - no. (%)
White	93 (24.60%)
African Heritage	7 (1.85%)
Latin American	7 (1.85%)
South Asian	27 (7.14%)
Southeast Asian	17 (4.50%)
West Asian	2 (0.53%)
Mixed race	5 (1.32%)
East Asian	3 (0.79%)
Other	9 (2.38%)
Not Reported	208 (55.03%)
Clinical Characteristics
Body weight (kg)	85.12 ± 21.88
Waist circumference (cm)	98.66 ± 15.68
Body Mass Index - BMI (kg/m2)	29.59 ± 6.38
Underweight ≤18.5 - no. (%)	2 (0.53%)
Normal Weight: 18.5-24.9 - no. (%)	81 (21.43%)
Overweight: 25.0-29.9 - no. (%)	141 (37.30%)
Obese class: >29.9 - no. (%)	153 (55.04%)
Blood pressure - Systolic	125.11 ± 13.98
Blood pressure - Diastolic	77.32 ± 9.17
Heart age (years)	63.34 ± 13.83
Cholesterol (mmol/L)
LDL Cholesterol	2.20 ± 0.95
NON HDL Cholesterol	2.86 ± 1.03
HDL Cholesterol	1.29 ± 0.35
Cholesterol/HDL ratio	3.37 ± 1.06
Total Cholesterol	4.14 ± 1.13
Framingham Risk Score (%)	18.59 ± 10.84
Presence of previous cardiac history - no. (%)
Coronary Artery Disease	140 (37.04%)
Congestive Heart Failure	5 (1.32%)
Arrhythmic Disease	12 (3.17%)
Cardiomyopathy	25 (6.61%)
Valve-related	12 (3.17%)
Familial Hypercholesterolemia	1 (0.26%)
Presence of previous medical conditions - no. (%)
Pre-Diabetes	33 (8.73%)
Diabetes	77 (20.37%)
Dyslipidemia	170 (44.97%)
Hypertension	168 (44.44%)
Behavioural metrics
Exercise (Total MET-minutes)	585.74 ± 731.21
Alcohol consumption - no. (%)
Consumption Within Guidelines	130 (34.39%)
Consumption Beyond Guidelines	8 (2.1%)
Does not consume	82 (21.69%)
Not Reported	158 (41.80%)
Smoking status
Current smoker	26 (6.88%)
Former smoker	109 (28.84%)
No smoking history	243 (64.29%)
Barriers to Healthy Lifestyles
Lack of Funds	29 (7.67%)
Lack of expertise or skills	75 (19.84%)
Health status
Self-perceived health status	5.02 ± 2.10
Excellent (10) - no. (%)	7 (1.85%)
Very good (7.5) - no. (%)	97 (25.66%)
Good (5) - no. (%)	172 (45.50%)
Fair (2.5) - no. (%)	80 (21.16%)
Poor (0) - no. (%)	14 (3.70%)
Not Reported - no. (%)	8 (2.12%)

Feasibility

By December 15, 2021, 366 of 378 enrolled patients had been given one or more prescheduled telemedicine visits (3,055 prescheduled telemedicine visits scheduled). Overall attendance to prescheduled telemedicine visits was 2,933/3,055 (96%), with rates varying between 91% and 100% depending on the type of telemedicine visit scheduled (Figure 1). By December 15, 2021, 59 patients (15.6% of the total cohort) had prematurely dropped out of the program, while 192 patients completed the program in its entirety. Most of those patients who dropped out (n=31) did so within the first 90 days of the program. Baseline characteristics among completers did not significantly vary when compared to the entire sample. Attendance to prescheduled telemedicine visits did not significantly vary according to the provider (i.e., physicians, kinesiologists, dietitians) or accordingly to visit number (Figure 2A, 2B).

Figure 1.

Referral, enrollment, and focus of analysis for current paper.

Figure 2.

Attendance and drop-out to program. From December 2020 to December 2021, number of appointments to virtual visits, attendance, and drop-out were tracked for our preventative care program. A. Has the number of appointments and attendance graphed by sequential visits, total number of patients 378. B. Represents the amount of days between intake and dropouts (i.e., how many drop-outs between day 1 through 30), with a total of 59 total dropouts.

During the study period, there were 6,385 views on our MHF e-knowledge platform. Viewers on average spent 5 minutes, 7 seconds on the website. Our most viewed video had 526 views and the most popular podcast episode had 161 views. The average time spent on a video or podcast page per patient was 5 minutes, 38 seconds and 7 minutes, 40 seconds, respectively, which approximates the duration of the content.

User experiences

Average weekly MET-MINUTES significantly rose among patients who completed the program (Pre-post programmatic change in weekly average MET-MINUTE was 191.06, P=0.007). The rise in average weekly MET-MINUTES was most pronounced among non-exercisers who reported zero MET-MINUTES at baseline (Pre-post programmatic change in weekly MET-MINUTES was 491.9, P<0.001). Among patients who reported some exercise at baseline, the change in average weekly MET-MINUTES was more modest and did not attain statistical significance (Pre-post programmatic change in average weekly MET-MINUTES was 125.7, P=0.13) (Figure 3). Pre-post changes in average weekly MET-MINUTES increased significantly more among males than females (Pre-post programmatic changes in average weekly MET-MINUTES was 298.9 vs. 39.8 for males vs. females respectively, P=0.03), but did not significantly differ between older vs. younger patients (Pre-post programmatic changes in average weekly MET-MINUTES, was 230.8 vs. 150.9 for individuals ages 66+ years vs. those <65 years old, P=0.56), or among those with coronary artery disease at baseline as compared to those without (Pre-post programmatic changes in average weekly MET-MINUTES was 219.4 vs. 169.0 respectively, P=0.71).

Figure 3.

Average MET-Minutes according to Visit Number. From December 2020 to December 2021, average MET-Minutes were tracked for our virtual preventative care program. Subgroup analysis was performed wherein patients reporting 0 MET-Minutes at their first visit was compared to those who reported more than 0 MET-Minutes at baseline and those who reported any MET-Minutes as baseline (represented in red, blue, and yellow respectively). Data is presented segmented by visit number. Error bars represents 95% confidence intervals for average mean per specific visit. Single factor ANOVA identified a significant difference between subgroups (P<.001). Post-hoc Tukey-Kramer test found significant difference between nonzero vs. zero (P<.001), nonzero vs. any (P=.16) and zero vs. any METs (P<.001) at baseline.

At program exit, 93.6% of patients reported that their health knowledge improved, and 86.5% of patients reported their overall health status improved because of the program. In addition, 90.32% felt their exercise habits improved, 67.21% felt their knowledge and receptivity to cholesterol medication changed, 63.44% of patients felt their mental health improved, 61.5% felt their nutrition improved and (Table 2). In total, 173 (93%) participants reported that they were either very satisfied or satisfied with the program, with 174 (93.5%) participants reporting that they were either very likely or likely to refer their friends or family to the program.

Table 2.

Self-perceived changes in perceived health status, knowledge, and behaviours at program exit

Change in Self-Perceived Health Measures	Significantly Improved n (%)	Improved, n (%)	No change or worse, n (%)	P value
Change in self-reported overall health compared to intake survey (n=186)	12 (6.45%)	53 (28.49%)	121 (65.05%)	<0.001
To what extent has overall health changed as a result of the program (n=186)	28 (15.05%)	132 (70.97%)	26 (13.98%)	<0.001
To what extent has mental health changed as a result of the program (n=186)	14 (7.53%)	104 (55.91%)	68 (36.56%)	<0.001
To what extent has health knowledge changed as a result of the program (n=186)	64 (34.41%)	110 (59.14%)	12 (6.45%)	<0.001
To what extent has exercise habits changed as a result of the program (n=186)	77 (41.40%)	91 (48.92%)	18 (9.68%)	<0.001
To what extent has consumption of fruit and vegetables changed as a result of the program (n=186)	42 (22.58%)	83 (44.62%)	61 (32.80%)	<0.001
To what extent has consumption of whole grains changed as a result of the program (n=186)	32 (17.20%)	70 (37.63%)	84 (45.16%)	<0.001
To what extent has consumption of plant-based vegetables changed as a result of the program (n=186)	26 (13.98%)	90 (48.39%)	70 (37.63%)	<0.001
To what extent has cholesterol medication knowledge changed as a result of the program (n=182)	69 (37.92%)	90 (49.45%)	23 (12.64%)	<0.001
To what extent has cholesterol medication receptivity changed as a result of the program (n=184)	32 (17.39%)	55 (29.89%)	97 (52.72%)	<0.001

Programmatic delivery costs

Among 378 patients over a one-year period, there were 2987 total visits, of which 1642 were for physician telemedicine clinics, eligible for the fee-for-service reimbursement. The total amount of physician fee reimbursement during this period was $141,676.80 (averaging $86.28 per eligible fee reimbursement visit). The total cost to third-party payers for the 6-month program was $517.70 per patient.

Discussion

Our study explored the feasibility, user experiences, and costs associated with an integrative multidisciplinary preventative learning health system that was operationalized within Ontario’s publicly funded universal health care system. Using a one-year real-world implementation testing period, our results demonstrated that enrollment rates and attendance to prescheduled telemedicine visits exceeded 85%, with only 15.6% of patients prematurely dropped out of the program prior to completion. User experiences demonstrated significant increases in self-reported weekly MET-MINUTES particularly for those most sedentary at baseline, as well as improvements in perceived health status, and health knowledge.

Despite the established benefits associated with integrative preventative care for lifestyle modification [3,4], the effectiveness of implementing such solutions in real-world settings remained undermined by several factors including physician constraints in time, expertise, monitoring systems and financial reimbursement to modify and improve patients’ exercise and lifestyle management behaviour [9]. Value-based learning health systems have several conceptual advantages over usual care. First, such approaches synthesize data obtained from clinical care, to rapidly inform the care delivery of participants continuously and dynamically [12,13]. Second, learning health systems’ pillars help inform user outcomes to ensure the quality, safety, and efficacy of programmatic delivery [10]. Third, the ability to evaluate learning health systems within predefined knowledge cycles allows for continuous quality improvement within time intervals not otherwise achievable using more conventional cohort studies [16]. To the best of our knowledge, MHF is the first real-world example of a preventative value-based learning health system implemented in Canada. Core values of MHF’s LHS necessitated that implementation as an integrative preventative health care solution was accessible, adaptable, and cost-efficient. While this study only evaluated implementation over a one-year time interval, such early findings serve as a proof of concept which justifies further comparative evaluations against other models of care.

Cardiac rehabilitation serves as an example of one such alternative model against which a preventative learning health system could be compared. Cardiac rehabilitation is an evidence-based multidisciplinary program of preventative care that delivers exercise and lifestyle management to patients who are at risk for, or who have established, cardiovascular disease [17,18]. Given their associated improvements in mortality and morbidity, cardiac rehabilitation is considered by many to be an evidence-based gold-standard model of preventative care [4,17,19]. However, unlike the learning health system examined in this study, cardiac rehabilitation delivers such care predominantly by allied health professionals, with care-delivery distinct from patients’ own physicians and/or their ambulatory-care environment [20-23]. Available evidence has also demonstrated that cardiac rehabilitation is associated with high behavioural attrition [4,24,25]. For example, only a minority of eligible populations are referred into cardiac rehabilitation [24,26-29]; among those referred, studies have demonstrated that fewer than half enroll into the program [30,31]; among those who enroll, studies have demonstrated that approximately only half may complete the program with further behavioural attrition in exercise adherence evident thereafter [30,31]. Moreover, average cost associated with the delivery of cardiac rehabilitation in high-income countries may approach or exceed $1,500 per patient [32-34]. In contrast, the behavioural attrition observed within the MHF LHS was markedly less. For example, only 85% of patients referred by their physicians enrolled into the program; only 15% of those who enrolled prematurely dropped out of the program. The programmatic costs per patient completer was $517.70 - approximately one-quarter those associated with a similar duration of cardiac rehabilitation delivered within the same jurisdiction as that associated with the MHF LHS.

The public health implications associated with integrating exercise and lifestyle management into usual ambulatory-based care is non-trivial. Available evidence has demonstrated that the prevalence of physical inactivity in the population may approach 85% [35]. Moreover, the prevalence of sedentary behaviour and physical inactivity in the Westernized societies have materially changed over the years, despite improvements and advancements in wearable technologies [36,37]. Our study demonstrated that an integrative preventative learning health system solution resulted in marked increases in self-reported weekly MET-MINUTES among sedentary high-risk participants, many of whom achieved recommended minimum targets of physical activity (i.e., 500 or more MET-MINUTES per week). Such increases in physical activity could have significant health economic ramifications. For example, at least one Canadian study demonstrated a $150 million savings in direct health-care expenditures for every 10% increase in the number of attaining minimum recommended physical activity targets in Canada [38]. Such estimated economic advantages may even be expected to rise further with improvements in the adoption of healthy dietary choices, and smoking cessation. In sum, the implementation of a learning health system that educates, monitors, and integrates exercise and lifestyle management within medical care may have transformational population health implications with potential cost savings to system payers.

As the MHF program operates under a value-based LHS framework, it was crucial to evaluate its effectiveness in real-time and provide feedback to our core pillars and internal stakeholders. Accordingly, confidential feedback was collected from our team and several themes were identified that address the clinical and programmatic implications associated with the results of this study (Table 3). One key theme that emerged was the need for the development, implementation, and evaluation of additional novel performance metrics that could be applied to track real time programmatic responsiveness. Examples include serial health status measurements to identify changes in self-perceived health over time, and food-plate technologies which could better track nutrition and dietetic needs over time. Another important theme pertained to the need for more efficient electronic alert systems which would facilitate the identification of a patient’s behavioural attrition to exercise and self-learning activities. Doing so might facilitate a more personalized risk-stratified approach to behavioural interventions dynamically over time, to those in greatest need prior to programmatic drop-out. Such examples could include the implementation of e-alerts to downward trends in self-report, self-learning, and/or weekly MET-MINUTES. Other important themes included the need for wearable technology implementation and comparative effectiveness studies against other health system gold-standards (e.g., cardiac rehabilitation).

Table 3.

Themes emerged utilizing confidential feedback from core pillars

1. Novel performance metrics	2. Behavioural attrition
● Food plate technologies	● System-alerts in response to downward trends in weekly MET-MINUTES & Portal digital engagement
● Health status
3. Wearable technologies	4. Comparative effectiveness
● Accessible technologies that provide continuous data capture	● Pragmatic trials against cardiac rehabilitation and other models of care

There are several noteworthy study limitations that deserve consideration. First, by design, our study was intended to only serve as a descriptive one-year case study; there were no control groups against which MHF LHS experiences could be compared. Second, user experiences, including weekly MET-MINUTES were based on self-report, which may over-estimate physical activity when compared to estimated derived using objectively measured accelerometer/wearable-based technology [39]. Moreover, user experiences obtained at programmatic exit may have been limited by recall and social desirability biases [40]. Third, the sample size and geographical implementation regions were limited in size and scope. The extent to which such results are generalizable to larger sample sizes and in other jurisdictions remains unclear. Organizational differences in the health system payment models, physician reimbursement fees, and payers also vary widely across jurisdictions and countries. Therefore, the feasibility, deployment, efficacy, effectiveness, and cost-effectiveness of a learning health system may depend on the local health system within which implementation occurs. In sum, we are not advocating for the MHF LHS to serve as the solution for all preventative care, but rather as a proof-of-concept application for learning health systems in general.

In conclusion, our study suggests that the implementation of an integrative preventative care learning health system-based solution in Ontario is feasible. The extent to which such findings are generalizable and result in health outcome advantages over and beyond usual-care alternatives, necessitates further evaluation.

Disclosure of conflict of interest

None.