Proof-of-Concept of Online-Only Enrollment for Collecting Longitudinal Patient-Reported Outcomes in Young Individuals With Congenital Heart Disease

Abstract

Background

The importance of patient-reported outcomes (PRO) in research in congenital heart disease (CHD) is being increasingly recognized. The resources needed for direct, in-clinic enrollment of study participants are important barriers. Social media enable new ways to interact with potential participants. We aimed to test the feasibility of collecting PRO without direct contact by the research team, with subsequent linkage with the Quebec CHD Registry.

Methods

We targeted all patients with CHD aged ≥14 years and parent of a child with CHD aged 5-17 years who received cardiac care in Québec. We disseminated a link to an online survey consisting of 4 PRO instruments on the internet and by displaying in pediatric cardiology waiting rooms. No direct contact by a study team was done. The data were subsequently linked to a clinical registry of >50,000 patients.

Results

A total of 160 people accessed the study, of whom 68 signed the consent form. A plateau of enrollment was quickly reached despite reminders. A total of 44 participants provided information for linkage to the clinical registry. The study had an over-representation of severe disease cases and lacked ethnic diversity. The retention rate at 1 year was 25%. Health-related quality of life was consistent with other CHD cohorts, and participants reported good satisfaction with life.

Conclusion

Recruiting participants with CHD without direct contact was ineffective at population level. Several factors may explain these results, including a weaker community network among participants than anticipated, apprehension about sharing personal information online, and difficulty effectively targeting participants on the internet.

Most children born with congenital heart disease (CHD) now reach adulthood, and long-term outcomes have greatly improved, especially among those with complex defects.^1,2 However, individuals with CHD may continue to experience long-term complications, such as heart failure, arrhythmias, exercise intolerance, and pulmonary arterial hypertension.³ Mortality, malignant arrhythmia, and hospitalization—the so-called hard outcomes—have often been used to quantify morbidity in individuals with CHD. However, these hard outcomes are infrequent and represent a very incomplete picture of the overall health of patients with CHD.^4,5

Health is defined as “a state of complete physical, mental, and social well-being” by the World Health Organisation.⁶ There has been increased interest in assessing patient experience, which led to a greater emphasis on patient-reported outcomes (PRO). PRO are patient health information that provides a direct insight into the patient’s perspective on their overall well-being, including global health indicators such as quality of life and functional status. A 2013 statement by the American Heart Association recommended PRO as primary indicators of cardiovascular health.⁷

Several studies on the quality of life in people with CHD have been published. Systematic reviews reported heterogeneous methodology and small sample sizes, resulting in conflicting results.^5,8, 9, 10 The included studies tended to have a cross-sectional design, which did not enable in-depth assessment of PRO across the lifespan. Efforts to harmonize the methodology and collect large-scale and longitudinal data are therefore necessary. However, enrolling large numbers of patients and following them in time can be costly and resource intensive.

In recent years, the use of social media to enroll study participants has increased, and some have reported an interesting snowball effect that allowed reaching more participants with rare diseases, therefore reducing costs.¹¹ However, it remains unclear whether these alternative approaches are sustainable for collecting population-scale data in a longitudinal manner.

We assessed whether PRO gathered through an online enrollment approach could be effectively linked to the Quebec Congenital Heart Disease Registry (QCHDR), a population database that includes over 50,000 individuals with CHD in Québec, Canada.¹² The aim of this study was to test the feasibility of an alternative self-enrollment via social media, without direct in-person involvement by the study team, to gather PRO in children and young adults with CHD.

Methods

Study design and enrollment method

The QUALITY study is a proof-of-concept prospective cohort study. We designed a fully online questionnaire and tested the feasibility of gathering PRO by disseminating information on the study using social media, without direct or in-person contact by the research staff. We targeted children aged 5-17 years and their parents, as well as adults with any type of CHD receiving congenital cardiology care in the province of Québec, Canada.

The online questionnaires were first piloted on a subgroup of a known cohort of participants with tetralogy of Fallot¹³ in November 2021. We sent the study information and questionnaire link to 63 patients by mail or email to assess their understanding of the content and the ease of navigating the questionnaire. Based on inputs from these participants, minor adjustments were made before deploying the study more widely.

All participants provided informed consent or assent, depending on their age. This study was reviewed and approved by the research ethics board of the Centre Hospitalier Universitaire de Sherbrooke and by all participating institutions.

Social media sharing and internet outreach

Figure 1 shows the main enrollment campaigns through the study period. We listed organizations and groups that regularly interact with our target population and maintain an online presence on social media. This list included hospitals, advocacy organizations, foundations, and patient support groups. We subsequently reached out to them directly through social media and email, whenever possible, to explain the study and ask them to share the study link. Not all organizations agreed to participate. Notably, all hospitals specialized in CHD (pediatric and adult) mentioned that it was not in their mandate to promote this study online. Their reasons were generally related to high volumes of solicitation, concerns about fairness to other research teams, or a lack of alignment with the mission of their social media networking.

Figure 1.

Temporal pattern of patient recruitment highlighted by key events. CCPCRN, Canadian Congenital & Pediatric Cardiology Research Network.

Ultimately, the study link was shared on the Facebook pages of the Canadian Congenital & Pediatric Cardiology Research Network (CCPCRN), Fondation En Coeur, a parent support group, and a hospital patient partners group. Facebook was the primary social media platform used by most of these groups, except for Fondation En Coeur, which also shared the link on Instagram. The CCPCRN page shares news about ongoing studies and recent advances in congenital cardiology. The Fondation En Coeur is a nonprofit organization whose mission is to help children with CHD and their families navigating through illness. En Coeur’s Facebook page had 6000 followers at the time of the study, most of whom are persons living with CHD and their families. The posts were sponsored twice on the page of the Fondation En Coeur and the page of the CCPCRN. The parent support group page was an active Facebook page run by parents of children with heart defects who share and discuss how they cope with heart defects (>800 members). Members of the page accepted to let us post an advertisement presenting the study and reshared it 2 months after our initial publication. The patient-partner page is a discussion platform for patients collaborating with researchers throughout the research design and process at a specific hospital. A targeted advertisement for the study was also shared on this platform to reach potential participants.

Furthermore, we identified potential social media ambassadors by consulting 4 pediatric cardiology centers and advocacy foundations in Québec to find well-connected patients. The clinic’s nurse or a representative from the foundation first reached out to gauge their interest in becoming ambassadors. If they agreed, we held a brief virtual meeting to explain the project and provide promotional materials. We asked them to share the study at least once on their social media platforms. We followed up 2-6 weeks later to document their sharing activities and any successes or obstacles they encountered. The ambassadors did not receive any financial compensation for their participation.

Two patient advocacy foundations, En Cœur (targeting children) and Heart and Stroke (targeting adults), featured the study in their newsletters twice (see Fig. 1). In addition, the study was prominently displayed on the websites of both the CCPCRN and Heart and Stroke.

Display in waiting rooms

Posters and flyers were regularly displayed in the 4 Québec pediatric cardiology centers for the duration of the study. These materials were strategically placed in waiting areas and echocardiography rooms, allowing patients to scan a QR code and directly access the study on their mobile devices. Information about the study was also displayed on screens in the cafeteria of one of the 4 centers.

Online participation and data collection

Potential participants accessed the online questionnaires via a computer or a mobile device. A screening question determined if the participant was eligible. After providing informed consent and assent, eligible participants were directed to the PRO questionnaires. These questionnaires measured health-related quality of life and functional status (2 instruments), as well as satisfaction with life (2 instruments). The choice of instruments was based on the International Consortium for Health Outcome Measurement (ICHOM) and on a literature review of health-related quality of life (HRQoL) instruments in CHD. ICHOM recommends sets of instruments for specific health conditions to standardize the assessment of PRO that are significant to patients to be included in future registries,¹⁴ including for CHD. The 3 instruments recommended by ICHOM for the assessment of quality of life and perceived health status were the Satisfaction With Life Scale (SWLS), Pediatric Cardiac Life Inventory (PCQLI), and Patient Reported Outcomes Measurement Information System (PROMIS). A linear analog scale (LAS) was added on review of its psychometric properties in adults with CHD. All instruments have been validated in both pediatric and adult populations, with the exception of the PCQLI, which is specific to children, and the LAS, which, although not specifically validated in children, has been validated in adults with CHD and was deemed appropriate for children due to its simplicity.15, 16, 17, 18, 19, 20, 21, 22

The PROMIS and the PCQLI are patient-reported health status and HRQoL instruments,^19,23 whereas the SWLS and the LAS evaluate satisfaction with life.^15,22 The LAS consists of a slider and a vertical scale graduated from 0 (worst quality of life imaginable) to 100 (best quality of life imaginable).

Participants completed questionnaires tailored to their age group: adults (18+ years), adolescents (14-17 years), and parents of children aged 5-17 years. Children younger than 14 years participated indirectly through a survey included in the parent/proxy questionnaire. Adolescents and their parents had the option to complete their respective questionnaires independently, although their respective deidentified data remained linked.

After completing the PRO questionnaires, participants were asked to fill out a demographic survey, which included their postal code for estimating a material deprivation quintile,²⁴ identifying data for linkage with the QCHDR, and 4 assessment questions regarding the survey’s structure and content. They were also asked how they heard about the study.

The time necessary to complete the survey was approximately 10 minutes. Questionnaires were available in English and French. Automatic email reminders were planned for years 1, 2, and 3. All data were collected and stored on the REDCap web-based data collection tool.²⁵

Outcomes

The metrics used to assess feasibility included the number of participants who completed the questionnaires, the demographic and clinical representativeness of the sample compared with the QCHDR, the match success rate with the QCHDR, the effectiveness of various enrollment methods, and acceptability of the study. In addition, we assessed the retention rate of participants during the first follow-up year.

Linkage with the QCHDR

Linkage of the participants data with the QCHDR was done using a probabilistic algorithm that enabled matching between subjects using unique identifiers (health insurance number) in addition to a combination of personal identifiers (first name, surname, and date of birth). Deterministic matching was used for unique identifiers, and probabilistic matching was used for names and date of birth. This algorithm was adapted from the previously published algorithm used for indexing participants in the QCHDR.¹²

Statistical analysis

Descriptive statistics were used to summarize the results. Numbers and proportions are presented as percentages, and continuous variables are presented as medians and interquartile ranges (IQR).

Results

After the pilot phase in November 2021, the social media and internet campaign was launched in October 2022. Despite our efforts to engage the CHD population, and despite the high number of potentially eligible participants, the number of enrolled participants was very low. After 18 months of active enrollment, the number of participants plateaued, and it was decided to stop the study. There were 160 people who accessed the questionnaires. Of these, 43 (26.9%) failed the screening and were not eligible. Of the remaining potentially eligible participants, 68 of 117 (58.1%) electronically signed the consent form.

Of the 68 consented participants, only 39 (57.4%) completed the demographic questionnaire. The demographics of the participants are shown in Table 1. Results of the 2021 Canadian global census for the province of Québec are provided for comparison.²⁶

Table 1.

Demographic characteristics of the participants of QUALITY

Demographic	N (%)	% in the general population
Ethnicity
Not a visible minority	39 (100)	83.9
South Asian	0	1.5
Chinese	0	1.4
Black	0	5.1
Filipino	0	0.5
Arab	0	3.4
Latin American	0	2.1
Southeast Asian	0	0.8
West Asian	0	0.5
Korean	0	0.1
Japanese	0	0.1
Other group	0	0.5
Prefer not to answer	0	–
Indigenous identity	1 (2.5)	2.5
Sex assigned at birth
Male	11 (28.2)	49.4
Female	28 (71.8)	50.6
Other	0	Unavailable
Unsure	0	–
Prefer not to answer	0	–
Gender
Male	11 (28.2)	49.2
Female	27 (69.2)	50.5
Other	0	0.23
Unsure	1 (0.03)	–
Prefer not to answer	0	–

The majority of participants were assigned female at birth (28 of 39, 71.8%), and all participants identified as White. One participant also identified as Indigenous. For the 27 participants who provided a postal code, the material deprivation index was fairly evenly distributed across quintiles, with 11% in the most privileged quintile and 15% in the least privileged quintile.

Figure 1 illustrates the daily enrollment trends throughout the study period. The peak enrollment period coincided with the release of publications by both the Heart and Stroke Foundation and the Fondation En Coeur, the former being published 3 days earlier than the latter. A subsequent round of sharing by En Coeur proved less successful than the first publication. Subsequently, sharing the study within a parent-run group led to the enrollment of only 4 participants in July 2023 and 7 participants in October 2023. Enrollment was mostly observed in the few days after a publication or a post. Between these, new participants’ enrollment remained low, underscoring the ongoing need for team interventions.

Because of the small sample size, it was difficult to draw definitive conclusions on the most effective enrollment strategy. However, preliminary observations suggest that social media was the most successful method, with 20 of 68 respondents (29.4%) recruited through this channel (Fig. 2). In all, 8 potential ambassadors were contacted, 6 accepted to share the study, and 5 confirmed having shared the study at least once.

Figure 2.

Absolute number of participants recruited by the method.

Completion of the study process and longitudinal follow-up of participants

A total of 68 of 117 (42%) potentially eligible participants did not sign the consent form after accessing the study. Another 20% of the participants stopped before completing the quality-of-life questionnaires. Finally, only 33% of the participants who signed the consent form completed all of the study questionnaires, including demographic and identifiers. After the study was initiated, the protocol was modified to include a more extensive demographic questionnaire. Consequently, 7 early participants did not complete this questionnaire. Longitudinal retention was poor; of the 68 participants who completed the initial survey, only 45 (66.2%) had reached the 1-year mark before this analysis and were eligible to complete the 1-year follow-up survey. Six of 45 (8.8%) completed it after receiving the automatic email. A reminder from the research team was sent to eligible participants, increasing the number to 17 of 45 (25%).

Match success rate and types of malformations

Twenty-six participants provided a health insurance number, and 22 provided a date of birth and a name without a health insurance number, allowing us to potentially match 44 participants with the QCHDR. All participants with a medical insurance number were successfully matched, whereas 12 of the 22 participants without a health insurance number found a match in the registry, resulting in a match rate of 81% (38 of 44), therefore establishing a primary diagnosis for all 38 participants. Tetralogy of Fallot was the most frequently identified malformation (16 of 38, 42.1%). The majority of participants had a complex CHD, including cyanotic heart defects (16 tetralogy of Fallot, 2 transpositions of the great arteries, 47.4%) and single ventricle physiology (4/38, 10.5%). Figure 3 presents the number of cases by CHD in the current study compared with the distribution in the QCHDR (in blue).

Figure 3.

Distribution of patients in the Quebec Congenital Heart Disease Registry (QCHDR) and in the QUALITY cohort. CHD, congenital heart disease; DORV, double outlet right ventricle; VA, ventriculo-auricular.

Quality-of-life scores

Detailed results for each instrument are provided in Table 2. In general, participants reported a good HRQoL. On the PROMIS scale, adults had a median physical health score of 44.9 (IQR: 37.4-54.1) and a median mental health score of 48.3 (IQR: 43.5-59.0). For children, the median global health score was 47.5 (IQR: 42.1-49.2), whereas parents reported a median score of 37.9 (IQR: 32.9-45.4) when evaluating their child’s global health. The average score for this instrument in the general population is 50, with a standard deviation of 10. The median PCQLI score was 72.2 (IQR: 63.1-86.6) for adults, 64.8 (IQR: 59.4-66.4) for children and adolescents, and 57.6 (IQR: 36.4-67.6) for proxy-reported disease-specific mean HRQoL. The mean scores for the reference population with mild CHD were approximately 80 when evaluated by children and 85 when evaluated by parents, whereas the mean score for children with severe disease was around 65 in both case.¹⁸

Table 2.

Quality-of-life scores

	SWLS score, median (IQR)	Linear analog scale score, median (IQR)	PROMIS global health score, median (IQR)	Pediatric cardiac quality of life inventory, median (IQR)
Adults (n = 32)	29 (21-32)†	85 (60-90)	Physical 44.9 (37.5-54.1) Mental 48.3 (43.5-59.0)	72.2 (63.1-86.6)
Children and adolescents (n = 16)∗	19 (12-23)†	80 (73-92)	Global 47.5 (42.1-49.2) Fatigue 52.9 (46.4-52.9) Pain 50.3 (42.6-54.7)	64.8 (59.4-66.4)
Parents (n = 32)	NA	76 (70-85)	Global 37.9 (32.9-45.4) Fatigue 56.1 (56.1-62.5) Pain 53.1 (43.3-58.5)	57.6 (36.4-67.6)

IQR, interquartile range; NA, not applicable; PROMIS, Patient Reported Outcomes Measurement Information System; SWLS, Satisfaction With Life Scale.

^∗Four adolescents participated directly, and 12 children completed a questionnaire with their parents.

^†Total score of the instrument is on 35 for adults and 25 for children.

Life satisfaction was also good, with adults reporting a median LAS score of 85 (IQR: 60-90), children and adolescents reporting a median of 80 (IQR: 73-92), and parents reporting a median of 76 (IQR: 70–85). Finally, adults reported high life satisfaction on the SWLS (35-point scale), with a median score of 29 (IQR: 21-32). Children reported a median score of 19 (IQR: 12-23) on the child version of the SWLS, which is a 25-point scale.

Evaluation of the questionnaire by the participants

Forty-six of 68 (67.6%) participants completed the appreciation survey. Of those, 93% could easily understand all the questions and found the questionnaire easy to complete. Ninety-five percent said that the questionnaires took less than 10 minutes to complete. A large majority (95%) would recommend a participant with a heart defect to take part in this study.

Discussion

Expert societies are increasingly recognizing the importance of considering patients’ experience while studying the impact of a disease.⁷ Access to PRO data, which is essential to meeting this need, is complicated by its absence from traditional databases and by the need to access patients directly to collect it.

Our experience with QUALITY has shown that the use of social media to collect PRO from the CHD population, while cheaper and easier to implement than traditional recruitment methods, is not sustainable in collecting population and longitudinal data. There were only 68 participants successfully enrolled in the study despite more than 18 months of active recruitment and despite several thousand potentially eligible participants.

This result was surprising given that recent reviews concluded that most studies using digital approach or comparing it with more traditional methods of recruitment found it to be effective and cost-efficient.^27,28 It must be noted that these reviews included studies that tended to target the general population or prevalent health problems, over large geographical areas. In some cases, authors successfully engaged small, closely knit communities of patients with rare diseases. In our study, we failed at specifically targeting the CHD population through algorithms currently available on social media. One hypothesis is that because congenital heart defects are heterogeneous, patients may be less connected with each other on social media than what was initially anticipated. In addition, current social media algorithms may not effectively target users with specific cardiovascular health-related interests. Only pediatric centers displayed posters about the study in their clinics, and some methods used to reach participants were performed via foundations that cater mostly to pediatric patients and their parents. Hence, our conclusions could differ between pediatric and adult population, and more studies may be needed in the adult CHD population. However, we believe that adults with CHD could be reached through social media advertisements and in collaboration with the Heart and Stroke Foundation, which were the most efficient methods for pediatric participants.

Retention of participants throughout the different steps of the survey and through longitudinal follow-up was low. A significant number of participants dropped out when presented with the informed consent form, despite our efforts to shorten it as much as possible. In terms of longitudinal retention, only 25% of participants remained in the study after 1 year, which was lower than anticipated. In typical longitudinal studies, retention rates of around 70% across multiple cycles are often expected.²⁹ However, maintaining such levels of participation generally requires sustained active and direct engagement by a dedicated research team—a resource-intensive approach that was not in place in this study. Our findings suggest that relying exclusively on digital means of communication for retention may not be as effective as initially hoped.

Our sample was not representative of the population of CHD in Québec either demographically or in terms of disease severity. Consequently, quality-of-life score presented in the Results section are likely not representative of the overall Québec CHD population. Another issue was the near absence of minorities and the over-representation of females and White people. The over-representation of White females with higher education has been raised in several studies using online recruitment methods.^28,30,31 Furthermore, a high proportion of participants had severe malformations. The over-representation of participants with tetralogy of Fallot can be partially explained by the initial direct mail solicitation to pilot the platform before its deployment, though they remain over-represented even when accounting for this factor. We hypothesize that participants whose defects significantly impacted their lives would be more inclined to engage with such a study, thus introducing a selection bias.

Even though the concepts evaluated in this study did not yield the expected results, our study provides some important clues for improving the collection and reporting of PRO for congenital cardiology research. Although potential participants were difficult to reach and engage, our fully automated survey platform used for consent and data collection was efficient and convenient for study participants. The linkage with an existing patient registry was possible and useful to effortlessly link PRO with clinical data. As social media landscape is becoming more fragmented, it is likely that online only enrollment will not be sufficient, but it could likely remain a complement to other methods. Whether dedicating significant financial resources in promoting and sponsoring the study on social media would have made a difference remains to be investigated (some authors have mentioned cost per participants enrolled to be as high as CAD$700).³² Finally, as the instruments used in this study were selected rigorously with a focus on meeting the needs of the community affected by CHD, and as there is a consensus in using these metrics to further the care of patients with CHD, we believe that there is an argument to be made for incorporating PRO in routine clinical assessments using a similar online platform. This would provide a useful tool for tailored patient follow-up while increasing the availability of systematically collected PRO for research use.

Conclusion

Collection of large-scale PRO in patients with CHD remains challenging. Longitudinal data are becoming increasingly necessary, and efficient ways to incorporate PRO to research and clinical registry are needed. We found that solely using online methods, without a direct approach by a study team member, was insufficient for recruiting a representative sample of participants with CHD. The success in linking participants’ PRO with registry-based research data demonstrated that well-integrated research efforts with a facilitated access to research data may decrease the redundancy of data collection. Complementary approaches combining online and traditional recruitment strategy should be considered.