Patient information portal for congenital aortic and pulmonary valve disease: a stepped-wedge cluster randomised trial

Abstract

Background

In response to an increased need for patient information in congenital heart disease, we previously developed an online, evidence-based information portal for patients with congenital aortic and pulmonary valve disease. To assess its effectiveness, a stepped-wedge cluster randomised trial was conducted.

Methods

Adult patients and caregivers of paediatric patients with congenital aortic and/or pulmonary valve disease and/or tetralogy of Fallot who visited the outpatient clinic at any of the four participating centres in the Netherlands between 1 March 2016–1 July 2017 were prospectively included. The intervention (information portal) was introduced in the outpatient clinic according to a stepped-wedge randomised design. One month after outpatient clinic visit, each participant completed a questionnaire on disease-specific knowledge, anxiety, depression, mental quality of life, involvement and opinion/attitude concerning patient information and involvement.

Results

343 participants were included (221 control, 122 intervention). Cardiac diagnosis (p=0.873), educational level (p=0.153) and sex (p=0.603) were comparable between the two groups. All outcomes were comparable between groups in the intention-to-treat analyses. However, only 51.6% of subjects in the intervention group (n=63) reported actually visiting the portal. Among these subjects (as-treated), disease-specific knowledge (p=0.041) and mental health (p=0.039) were significantly better than in control subjects, while other baseline and outcome variables were comparable.

Conclusion

Even after being invited by their cardiologists, only half of the participants actually visited the information portal. Only in those participants that actually visited the portal, knowledge of disease and mental health were significantly better. This underlines the importance of effective implementation of online evidence-based patient information portals in clinical practice.

Key questions.

What is already known about this subject?

• Patient information in congenital heart disease has been demonstrated to be suboptimal in the Netherlands, comparable to prior findings in other countries. Better informed patients have been previously found to be associated with improved quality of life, treatment adherence, health behaviour and clinical outcome, but also with more efficient healthcare utilisation and lower healthcare costs. In response, we developed an online evidence-based information portal for patients with congenital aortic and pulmonary valve disease in the Netherlands. However, its effectiveness remained to be investigated.

What does this study add?

• Our findings demonstrate the potential effectiveness of an online evidence-based patient information portal in improving knowledge in patients with congenital heart disease, but also underline the crucial importance of effective implementation and active use of the portal.

How might this impact on clinical practice?

• Our findings advance our knowledge and skills on how we can better inform patients, provide the basis for interventions aimed at supporting us therein, and are essential in guiding further efforts aimed at improving patient information and subsequently improving knowledge, decision-making, outcome, quality of life and healthcare utilisation.

Introduction

Thanks to major advances in the treatment of congenital heart disease over the past decades, approximately 90% of patients now reach adulthood.¹ However, this has made congenital heart disease a chronic illness that represents a growing health burden among children and adults. For example, as of 2010 there were an estimated 2.4 million people living with congenital heart disease in the USA alone among a total population of approximately 309 million.²

The consequences of congenital heart disease for the individual patient are complex, time-varying and dependent on the specific defect(s), individual patient-related factors and treatment options and decisions. These consequences will have a significant impact on many aspects of patients’ lives, both physical and psychosocial. Therefore, informing patients and their relatives in a complete, objective and understandable manner is essential and may optimise patient quality of life, lifestyle, health behaviour, treatment adherence, involvement and healthcare utilisation.^3–14

In response to an increased need for patient information in congenital heart disease in the Netherlands,¹⁵ where an estimated 65 000 people live with congenital heart disease, we previously developed a patient information portal for congenital heart disease in a nationwide initiative, starting with a pilot project aimed at a subgroup of congenital heart disease patients with aortic or pulmonary valve disease, including tetralogy of Fallot.^{15 16}

To assess the effectiveness of this information portal, we conducted a stepped-wedge cluster randomised trial in four congenital heart disease centres in the Netherlands among (parents of) patients with congenital aortic or pulmonary valve disease, including tetralogy of Fallot.

Methods

This study was registered in the Netherlands Trial Register (NTR6805) and written informed consent was obtained from all participants. The study is reported in accordance with the Consolidated Standards of Reporting Trials guidelines.¹⁷

Participants

Participants were recruited from four congenital heart disease centres in the Netherlands, namely Erasmus University Medical Center (Rotterdam), Leiden University Medical Center (Leiden), Academic Medical Center (Amsterdam) and Radboudumc (Nijmegen).

All patients aged between 18 and 40 years and parents/caregivers of patients <18 years of age with congenital aortic stenosis/regurgitation, congenital pulmonary stenosis/regurgitation and/or tetralogy of Fallot who visited the paediatric or adult cardiology outpatient clinic at one of the participating centres during the study period were considered for inclusion. Subjects were only included if their aortic and/or pulmonary valve disease was of at least moderate haemodynamic severity (peak Doppler gradient ≥36 mm Hg and/or ≥moderate regurgitation).^{18 19} Mentally incompetent subjects and subjects that could not read or write Dutch were excluded. Subjects could only participate once, and were thus not recruited again at subsequent outpatient clinic visits after prior inclusion (no repeated measures). There were no restrictions on the moment during follow-up at which subjects could be included (eg, at diagnosis, routine check-up, preoperative, postoperative, etc).

Intervention

The intervention consisted of access to a previously developed evidence-based online patient information portal. The development of this portal has been previously described.¹⁶ Practical introduction of the information portal in the outpatient clinic was tailored to the workflow at each participating department and all participating physicians and support staff were trained in its use. After introduction, subjects in the intervention group were invited to visit the portal by their treating paediatric or adult congenital cardiologist during the outpatient clinic consultation.

Subjects in the control group received standard care, without access to the information portal.

Trial design

The trial was conducted according to a prospective stepped-wedge cluster randomised design from 1 March 2016 to 1 July 2017 (figure 1).²⁰ All four centres started in the control phase, in which enrolled subjects did not receive the intervention. Subsequently, each of the participating centres transitioned to the intervention phase at a different time point, according to a stepped-wedge randomised design.²⁰ All subjects enrolled during the intervention phase were invited to visit the information portal by their treating paediatric or adult congenital cardiologist during the outpatient clinic consultation. To ensure the accrual of sufficient control subjects, the first centre transitioned to the intervention phase after at least 80% of the required total control group sample size had been accrued (1 September 2016). The dates for transition from control to intervention at each centre were 2 months apart (1 September 2016, 1 November 2016, 1 January 2017 and 1 March 2017) and each of the four participating centres were randomly allocated to one of these four starting dates. Randomisation was performed by an independent researcher by randomly drawing four cards listing the names of each of the four centres, with the order of the draw corresponding with the order in time of transition to intervention. Allocation concealment was achieved by placing each of the four cards in an opaque unmarked sealed envelope by a different independent researcher before random draw. Because of the nature of the intervention, it was not possible to blind investigators or participants to the allocation.

Figure 1.

Stepped-wedge cluster randomised trial design. The four participating centres were randomly allocated as centres 1 through four as depicted in this figure and described in the Methods section.

Outcomes

All participants completed a questionnaire 1 month after outpatient clinic visit. Age, sex and educational level were recorded as demographics in the questionnaire and diagnosis was extracted from the patient’s medical record.

Primary outcome: disease-specific knowledge

Disease-specific knowledge was assessed using a questionnaire developed specifically for the purpose of this study in a multidisciplinary working group consisting of a paediatric cardiologist (RAB), adult congenital cardiologist (APJvD), congenital cardiac surgeon (JK), patient (EvG), clinical psychologist (EMWJU) and epidemiologists (JRGE and JJMT). This questionnaire (online supplement 2) consisted of seven multiple choice questions that test the subjects’ knowledge of what their own (child’s) personal condition is (two questions), the implications of heart valve disease for lifetime risk of an operation (one question), daily functioning (one question) and work/career (one question), the purpose of their regular check-ups (one question) and symptoms that may indicate deterioration of their condition (one question).

Secondary outcomes

Subjects’ feeling about how well informed they were, experiences with patient information, preferences for involvement, anxiety, depression, health-related mental quality of life and satisfaction with the information portal (only intervention group) were assessed as secondary outcomes (further details in online supplement 1).

Sample size

We based our sample size calculations on data from a prior study by Korteland et al on a population of adult patients with heart valve disease who were facing heart valve replacement surgery.²¹ Because this study did not assess our primary outcome using comparable methods (nor any other study to our knowledge), we based our calculations on the secondary outcome measure that was assessed in both studies, namely the Hospital Anxiety and Depression Scale (HADS). Based on the results of Korteland et al, we assumed a mean HADS of 10.5±7.9 in the control group and 7.7±6.7 in the intervention group. As there were no data available on intracluster correlation, we chose to take its possible effect on outcome into account by overpowering our study and thus basing our sample size on a power of 0.85 instead of 0.80. At a power of 0.85 and a 0.05 significance level, this led us to an estimated required sample size of 244 patients at a 1:1 sampling ratio (122 in the control arm and 122 in the intervention arm).

Statistical analysis

All outcomes were analysed according to both the intention-to-treat and the as-treated principles.^{20 22} Analyses were performed in the R statistical software (V.3.3.3, R Development Core Team, R Foundation for Statistical Computing, Vienna, Austria). Continuous data are presented as mean±SD or median (range) and categorical data (including Likert scales) are presented as proportions and/or counts. Comparison of baseline characteristics between groups was done using the Student’s t-test and χ² test where appropriate. For comparison of outcome measures between groups, linear regression models were used to analyse continuous outcomes (including summary scores) and ordinal regression models for ordinal outcomes (single Likert scales and Control Preferences Scale). All analyses of outcomes were adjusted for centre and time effects using mixed models (random effect for centre and fixed effect for calendar time).²⁰

Results

Between 1 March 2016 and 1 July 2017, 962 eligible subjects were asked to participate (542 control phase, 420 intervention phase), of which 343 gave written informed consent and filled out the questionnaire (35.7% inclusion rate), 221 in the control group (standard care) and 122 in the intervention group (standard care + access to information portal) (figure 2). Only 63 of the subjects in the intervention group (51.6%) reported actually visiting the information portal (as-treated intervention group). Baseline characteristics were comparable between the control group and both the intention-to-treat and as-treated intervention groups (table 1). There were also no significant baseline differences between the subjects in the intervention group that visited the information portal (as-treated intervention group) and those who did not (online supplement 3).

Figure 2.

Flow diagram of inclusion.

Table 1.

Baseline characteristics of the included subjects

	Control	Intervention		P value
		Intention-to-treat	As-treated	Intention-to-treat	As-treated
	n=221	n=122	n=63
Age group				0.395	0.984
Children	48.9% (108)	43.4% (53)	46.0% (29)
Adults	51.1% (113)	56.6% (69)	54.0% (34)
Male sex*	35.7% (79)	40.2% (49)	31.7% (20)	0.603	0.481
Diagnosis				0.873	0.438
PV disease	67.0% (148)	65.6% (80)	73.0% (46)
ToF	46.2% (102)	38.5% (47)	47.6% (30)
AV disease	29.9% (66)	32.0% (39)	25.4% (16)
PV+AV disease	3.2% (7)	2.5% (3)	1.6% (1)
Educational level*				0.153	0.613
Elementary	0.5% (1)	1.7% (2)	0.0% (0)
Lower vocational	3.7% (8)	7.7% (9)	4.8% (3)
Lower secondary	3.7% (8)	6.0% (7)	3.2% (2)
Intermediate vocational	32% (70)	33.3% (39)	33.9% (21)
Higher secondary	8.2% (18)	10.3% (12)	6.5% (4)
Higher vocational	28.8% (63)	28.2% (33)	37.1% (23)
University	23.3% (51)	12.8% (15)	14.5% (9)

Data presented as ‘proportion (count)’.

*In the case of paediatric patients, sex and educational level relate to the parent that participated in the study.

AV, aortic valve; PV, pulmonary valve; ToF, tetralogy of Fallot.

Disease-Specific knowledge

Disease-specific knowledge among the control and intervention groups are presented in figure 3. All subjects answered at least 2 of the 7 disease-specific knowledge questions correctly. In the intention-to-treat analysis, there was no significant difference in disease-specific knowledge between the control and intervention groups (p=0.891). When only considering the 63 subjects that actually visited the information portal as the intervention group (as-treated analysis), disease-specific knowledge was significantly better in these subjects than in control subjects (p=0.041).

Figure 3.

Disease-Specific knowledge in the control and intervention groups (according to both the intention-to-treat and as-treated principles). All subjects answered at least two of the seven disease-specific knowledge questions correctly. All significance tests were adjusted for centre and time effects using mixed-regression models (random effect for centre and fixed effect for calendar time).

There were significant interactions between the intervention (intention to treat) and diagnosis (the positive effect of the intervention on disease-specific knowledge was greater in pulmonary valve disease/tetralogy of Fallot compared with aortic valve disease, p=0.009) and age group (greater positive effect among parents of paediatric patients compared with adult patients, p=0.009), but not with educational level (p=0.655), sex (p=0.189) or centre (p=0.472).

Secondary outcomes

Subjects in the intervention group did not feel more informed than control subjects in neither the intention-to-treat nor the as-treated analyses (figure 4).

Figure 4.

How well-informed subjects felt in the control and intervention groups (according to both the intention-to-treat and as-treated principles). The graphs are centred on the response category ‘neutral’ (vertical grey line in the centre of the graph). All significance tests were adjusted for centre and time effects using mixed regression models (random effect for centre and fixed effect for calendar time).

There was no association between how well informed subjects felt and their objective knowledge (β=0.137, p=0.083, adjusted for intervention (as-treated), time and centre).

Contradictions in the information received from various sources were experienced by 14% of the control group (‘agree’ 9%, ‘strongly agree’ 5%), which was comparable to the intervention group in both the intention-to-treat (12%; ‘agree’ 7%, ‘strongly agree’ 5%; p=0.241) and as-treated (15%; ‘agree’ 7%, ‘strongly agree’ 8%; p=0.928) analyses.

Anxiety, depression and preferences for involvement in own care and decision-making were comparable between control and intervention subjects in both the intention-to-treat and as-treated analyses (table 2).

Table 2.

Autonomy preference, anxiety and depression, mental quality of life and control preferences

	Control	Intervention		P value*
		Intention-to-treat	As-treated	Intention-to-treat	As-treated
	n=221	n=122	n=63
API	77.5±8.1	78.0±7.7	78.0±7.9	0.594	0.815
Information seeking	88.7±8.8	89.7±7.7	90.4±7.6	0.850	0.422
Decision-making	62.4±13.7	62.8±14.5	62.0±14.8	0.250	0.970
HADS	7.4±5.8	7.3±5.9	7.6±6.1	0.954	0.561
Anxiety	4.9±3.4	4.6±3.5	5.0±3.9	0.962	0.225
Depression	2.5±2.9	2.6±2.9	2.5±2.6	0.887	0.740
SF-36 MCS	75.5±16.0	75.4±16.9	75.8±15.8	0.346	0.482
Vitality	65.3±18.3	67.2±19.3	67.0±17.6	0.066	0.455
Social functioning	84.4±20.4	83.0±20.3	83.5±18.9	0.663	0.657
Role-emotional	78.5±23.0	76.1±24.1	74.5±24.5	0.953	0.444
Mental health	78.1±16.2	78.9±16.4	80.6±15.4	0.160	0.039
CPS (The final treatment decision should be made by)				0.829	0.738
Physician	0.5% (1)	0.0% (0)	0.0% (0)
Physician, after considering patient opinion	14.3% (31)	12.2% (14)	14.3% (9)
Physician and patient together	82.0% (178)	80.9% (93)	79.4% (50)
Patient, after considering physician opinion	2.8% (6)	5.2% (6)	4.8% (3)
Patient	0.5% (1)	1.7% (2)	1.6% (1)

Data presented as ‘mean±SD’ or ‘proportion (count)’.

*All significance tests were adjusted for centre and time effects using mixed-regression models (random effect for centre and fixed effect for calendar time).

API, Autonomy Preference Index; CPS, Control Preferences Scale; HADS, Hospital Anxiety and Depression Scale; MCS, Mental Component Scale; SF-36, Short Form 36 Health Survey,.

The total Mental Component Summary score of the Short Form 36 Health Survey was also comparable between control and intervention subjects in both the intention-to-treat and as-treated analyses (table 2). However, in the Mental Health subscale, intervention subjects reported significantly better quality of life than control subjects in the as-treated analysis (p=0.039).

The information portal received high ratings from the 63 subjects that visited it, for both contents (median rating on a 1–10 scale: 8, IQR 7–8) and design (median rating on a 1–10 scale: 8, IQR 7–8).

Discussion

After the introduction of an information portal among patients with congenital aortic or pulmonary valve disease, including tetralogy of Fallot, only half of the participants invited by their cardiologist to visit the information portal actually visited the portal. Among those subjects that actually visited the information portal, disease-specific knowledge and mental health were significantly better at 1 month after outpatient clinic visit, while baseline characteristics and all other outcomes were comparable to control subjects and to intervention subjects that chose not to visit the portal. These findings demonstrate the potential effectiveness of an online evidence-based patient information portal in improving knowledge in patients with congenital heart disease, but also underline the crucial importance of effective implementation and active use of the portal.

Patients, parents and physicians alike have been previously demonstrated to experience substantial shortcomings in the way that patients and their parents are currently informed and involved.^{15 16} Subsequently, patient/parent knowledge is limited, leading to suboptimal patient/parent involvement and substantial decisional conflict and valve-related anxiety.^{15 16} In light of these shortcomings, our findings demonstrate the potential effectiveness of an online evidence-based patient information portal in improving patient information, as subjects that used the information portal had significantly improved disease-specific knowledge and mental health.

However, our results also underline the importance of careful and effective implementation of such interventions, as only half of the subjects invited to use the information portal actually did so and an effect could only be demonstrated in those who did. The usage rate of our information portal (52%) is substantially higher than previously reported for patient information and decision support interventions (25%–35%).²³ However, it remains suboptimal as it still leaves a large proportion of patients inadequately informed, as evidenced by their limited disease-specific knowledge. It remains unclear why one half of participants in the intervention group chose not to visit the information portal and the other half did, as we did not find any differences between these two groups in baseline characteristics and outcome measures other than knowledge and mental health. The observed lack of an association between how informed patients felt and their objective knowledge level may indicate that many patients may be unaware of their knowledge deficits and, thus, do not see the need to seek additional information. Interventions aimed at helping such ‘unconsciously uninformed’ patients gain insight into their own knowledge level may allow these patients to more reliably estimate how well informed they are and subsequently motivate them to seek additional information if they are inadequately informed. For instance, a short list of essential knowledge items (ie, ‘What you should know about your heart defect’) can be provided to patients or patients can be asked to take a short knowledge test before outpatient clinic visit, the results of which can then be reviewed together with their physician. Timing of information provision may be another important factor, as we included participants at all points during clinical follow-up (eg, at diagnosis, at routine check-up, preoperative, postoperative, etc.). Providing information to patients at the right time when their information need is highest, for instance at diagnosis or surrounding interventions, may improve active use of the portal. Other patient barriers such as limited numeracy, anxiety, cultural factors and language barriers should also be considered in the conception, design and implementation of patient information interventions.15 16 21 24

Physician and healthcare system barriers should also be taken into account. A systematic review on the implementation of patient decision support interventions reports lack of physician training, disagreement with the contents of the intervention, physician views on patient involvement and time pressure as important barriers for physicians to motivate active use of the interventions among their patients.²³ We addressed many of these barriers during the introduction of the information portal with our extensive site initiation visits and center-tailored implementation plans, which may explain in part our relatively high usage rate compared with those previously described in the literature.²³ However, implementation may be further improved by more actively involving paramedical staff such as nurse practitioners, integration into the electronic patient record, employing waiting room tools such as computers or tablets and improving ease of use of the portal in the consulting room with the physician.

Furthermore, we found that the information portal was less effective among adult patients than among parents of paediatric patients. Informing, engaging and involving adolescents and young adults with congenital heart disease is a well described challenge in current practice, which often leads to suboptimal knowledge, poor health behaviour and substantial loss to follow-up (up to 50% loss to follow-up during the transition from paediatric to adult care).^{11 25 26} In this light, our findings may advocate a fundamentally different approach in informing adolescents and young adults. Although the language and contents of our information portal were tailored specifically to the needs of each age group (parents of paediatric patients, teenagers and young adults), the overall design and format of the portal were generally the same.¹⁶ Employing innovative formats such as video/animation, virtual reality, three-dimensional modelling and serious gaming principles may prove more effective in engaging and informing adolescents and young adults and support successful transition from paediatric to adult care.

With regard to secondary effects of improved patient information and knowledge, in this study, we found significantly improved mental health after use of the information portal, however we found no effect on other psychosocial outcomes. Our short follow-up duration (1 month) should be taken into account in the interpretation of these findings, as a longer follow-up or a longer exposure to the intervention may be required for a measurable effect on psychosocial outcomes to manifest. In prior studies, better informed and more activated patients have been found to be associated with improved quality of life, treatment adherence, health behaviour and clinical outcome, but also with more efficient healthcare utilisation and lower healthcare costs.3–14 21 27 However, the relationship between improved patient knowledge of disease and anxiety remains unclear. Improving knowledge may not necessarily reduce anxiety, because while patients may find reassurance in knowing more about their condition, it may also give them more to worry about. Furthermore, besides the effect of improved knowledge, the sole availability of reliable information that patients trust, the format and design of the information and framing may each also have a direct effect on anxiety. This complex relationship is evidenced by inconsistent findings in prior studies regarding the effect of information portals and decision aids on anxiety.^{21 28 29} Further investigation may provide insight into how we may best inform patients/parents to improve their knowledge and simultaneously reduce anxiety. Lastly, how improved patient information and knowledge relates to patient activation, involvement and concordance of treatment decisions with patient values remains to be elucidated.

Limitations

Although substantially higher than previously reported for comparable interventions, the limited usage rate of our information portal may have affected outcome and led to a limited sample size of our as-treated analyses. This study represents Dutch clinical practice and possible international differences in medical practice, culture and language should be taken into consideration. Results may differ for disease states other than aortic and pulmonary valve disease, which should be taken into account when interpreting our results. Although we found no centre effect, the possible influence of any unobserved interprovider differences in patient information should be taken into consideration. As this was a stepped-wedge cluster randomised study the inherent limitations of this study design, such as possible intracluster correlation, should be taken into account.²⁰

Conclusions

After the introduction of an information portal among patients with congenital aortic or pulmonary valve disease, including tetralogy of Fallot, only half of the participants invited by their cardiologist to visit the information portal actually visited the portal. Among those subjects that actually visited the information portal, disease-specific knowledge and mental health were significantly better at 1 month after outpatient clinic visit, while baseline characteristics and all other outcomes were comparable. Thus, an online evidence-based patient information portal is potentially effective in improving knowledge in patients with congenital heart disease, although active use of the portal is crucial. There is an urgent need for efforts aimed at supporting effective implementation and use of information portals.