Feasibility of evaluating AI-enabled digital symptom monitoring in metastatic patients with NSCLC receiving pembrolizumab therapy: A German single-arm observational pilot study

Abstract

Introduction

Close symptom monitoring can benefit patients with metastatic nonsmall cell lung cancer (mNSCLC) receiving first-line therapy. Remote patient monitoring technologies, like the artificial intelligence (AI)-enabled Kaiku^® Health platform that allows oncology patients to report their health status in real-time to healthcare providers, may enhance patients’ treatment experience.

Methodology

The lung artificial intelligence-enabled digital solution pilot study (“Lung AID”) assessed the feasibility of future studies on Kaiku^® Health in patients with mNSCLC receiving first-line pembrolizumab in Germany. Patient engagement with Kaiku^® Health and practicality of collecting patient-reported outcomes (PROs) via the separate Lung AID EDC system were assessed by platform access rates. Kaiku^® Health access required one login, while Lung AID EDC access required submission of ≥1 PRO questionnaire. Post hoc analyses explored access by site experience with Kaiku^® Health.

Results

Over a 17-month enrollment period, 47 of 100 planned patients were enrolled in the study. Kaiku^® Health was accessed by 85.1% of patients, with higher engagement at experienced sites (96.2%). Only 38.3% accessed the Lung AID EDC system; 31.9% used both systems.

Discussion

High Kaiku^® Health access rates imply patient interest in remote digital monitoring for mNSCLC. However, recruitment challenges and use of a separate system to collect PRO data demonstrated difficulties in assessing the feasibility of these technologies in real-world settings. Our results highlight the need for streamlined patient monitoring tools and enhanced site and patient engagement strategies.

Conclusion

While definitive conclusions on future studies cannot be drawn, the study offers key insights into challenges that should be considered in future research.

Introduction

Symptom monitoring is crucial in the clinical care of patients undergoing anticancer treatment, yet many symptoms remain unreported. Basch et al. ¹ found that clinicians miss symptoms in advanced cancer patients up to half the time. Real-time symptom monitoring allows providers to respond more quickly to adverse events, improving symptom management and potentially enhancing patient-reported outcomes (PROs). Technological systems that collect information on health changes, flag issues, and alert care teams can facilitate early intervention, potentially improving both clinical outcomes and patient wellbeing.^1–17 In Germany, there is limited data on the use and benefits of digital patient monitoring tools for patients with metastatic nonsmall cell lung cancer (mNSCLC) receiving anticancer treatment.

NSCLC is the most common type of lung cancer, accounting for approximately 85% of cases.^18–20 Pembrolizumab is an immune checkpoint inhibitor commonly used as a first-line treatment for mNSCLC. It works by targeting and inhibiting the PD-1/PD-L1 pathway, boosting the immune system's ability to fight cancer cells.^21–25

Kaiku^® Health is a Conformité Européene (CE) marked Class IIa medical device and an artificial intelligence (AI)-enabled digital healthcare platform. It allows oncology patients to report their health status in real-time, enabling timely healthcare provider (HCP) responses. Its machine learning extreme gradient boosting algorithm (XGBoost) ²⁶ predicts probabilities for symptom onset or continuation based on three previous symptom reports, time from reports, and basic patient information (age, sex, and time to treatment initiation). Patients are provided with personalized symptom management instructions, based on the severity of the symptoms they report. Given the increasing integration of digital health technologies in Germany, there is a need for real-world data on their implementation and impact, including the use of Kaiku^® Health for patients with mNSCLC receiving first-line pembrolizumab therapy.

The Lung AI-enabled digital solution pilot study (“Lung AID”) assessed the feasibility of conducting future evaluation studies on Kaiku^® Health patients with in mNSCLC receiving first-line pembrolizumab therapy in Germany. Specifically, the study evaluated patient engagement with Kaiku^® Health and the practicality of collecting PROs in a separate data platform (also referred to as the Lung AID EDC system). The objective of the study was to describe how many patients accessed Kaiku^® Health during the study period (as an indicator of engagement) and how many accessed the Lung AID EDC system at least once after their initial study visit (baseline visit). Additionally, sociodemographic and clinical characteristics of patients who accessed Kaiku^® Health were compared with those who did not, to identify potential factors influencing platform engagement. Post hoc analyses further explored patient access to both platforms, Kaiku^® Health and the Lung AID EDC system, separately for sites with and without Kaiku^® Health experience.

Methodology

The manuscript was prepared in accordance with the STROBE (Strengthening the Reporting of Observational Studies in Epidemiology) guidelines. ²⁷

Study design and setting

Lung AID was a prospective, observational, single-arm, multicenter pilot study conducted at 10 cancer centers, including clinic and practice-based centers in both rural and urban areas across Germany. Site selection was based on availability of patients and site interest. The study was approved by the ethics committee of the University of Lübeck, and by the ethics committees/institutional review boards of all other participating sites.

Patients and eligibility criteria

Patients with mNSCLC, aged 18 years or older, who were either starting or had already started a first-line pembrolizumab-based regimen per usual care within the past 6 months prior to enrollment (as per European Summary of Product Characteristics (SmPC)) were eligible for the study. Additionally, they needed an Eastern Cooperative Oncology Group (ECOG) performance status of 0–2, ²⁸ regular access to and use of a web-connected device (smartphone, tablet, or computer) with internet at home, fluency in written and spoken German, and had to provide written informed consent.

Enrollment and sample size

Enrollment for the study began on 26 January 2022 (first patient in, FPI), with an initial target of enrolling 100 patients. However, enrollment closed on 30 June 2023 regardless of the achieved number of patients. The sample size was determined based on feasibility considerations, including study duration and available resources. Patients were identified consecutively by HCPs to minimize selection bias and were invited for an informed consent discussion before study inclusion. Patients were followed for 9 months after their first pembrolizumab administration after study inclusion or until the study follow-up period and data collection ended on 31 January 2024 (last patient last visit, LPLV).

Data sources and measurement

All eligible patients who participated in the study were granted access to Kaiku^® Health, received an e-mail invitation from their healthcare personnel to use the platform, and were given flyers with information about the Kaiku^® Health registration process. After completing registration, patients were prompted weekly to report their health status and treatment-related symptoms. During routine care visits, patients were reminded as needed by study site personnel to use Kaiku^® Health for symptom monitoring. Patients answered predefined, structured, disease-specific questions, and if symptoms were reported or predicted, follow-up questions were triggered (Supplemental Material S1). HCPs received daily e-mail summaries of new patient inputs and immediate alerts for severe symptoms, urging patients to contact their HCP. The system also tracked HCP actions in response to alerts. This solution helps oncology patients self-report symptoms while enabling HCPs to monitor symptom severity, compare patient data, and collaborate with each other on care through a web portal.

Upon registration by site personnel, patients had also access to the Lung AID EDC system (web-based validated software AlcedisTRIAL version 1.8.0.1.3). Patients were given information and instructions on how to access the Lung AID EDC system via flyers. After logging in, patients could use a module within the Lung AID EDC system to complete standardized study-related PRO questionnaires (EQ-5D-5L, ²⁹ EORTC-QLQ-C30, ³⁰ SF-36 ³¹ ) and other surveys, including baseline questionnaires (demographic and baseline information, Technology Readiness Index (TRI) 2.0, ³² and Patient Health Questionnaire (PHQ-4) ³³ ), as well as questionnaires on Kaiku^® Health user satisfaction, usefulness of Kaiku^® Health, System Usability Scale (SUS), ³⁴ and early withdrawal. The respective questionnaires were activated at specific time points, with patients receiving Short Message Service (SMS) reminders on their smartphones to complete them (Supplemental Material S2). The study site personnel were not explicitly asked to remind patients to complete questionnaires in the Lung AID EDC system. HCP questionnaires were also distributed by e-mail to physicians to collect information on the patient, provider, and site Clinical information (i.e. height, weight, tumor anamnesis, tumor staging, molecular biomarker data, concomitant diseases, and tumor therapy) was abstracted from patient charts at study inclusion. HCPs assessed and documented the ECOG status during patient screening for study eligibility.

Statistical analysis

At study completion, pseudonymized data on Kaiku^® Health usage was transferred to the study database. Access to Kaiku^® Health was defined as at least one successful login during the study, while access to the Lung AID EDC system required at least one submission of any of the three PRO questionnaires after the initial study visit, without needing all questions to be answered. Post hoc analyses described patient access to Kaiku^® Health and the Lung AID EDC system, categorized by site experience. Six sites were classified as inexperienced sites, as they had no prior experience with Kaiku^® Health, while four sites were considered experienced, having had any amount of prior experience with Kaiku^® Health.

Descriptive statistics were used for the analysis of this pilot study. Categorical variables were summarized using counts and percentages, while continuous variables were described using means and standard deviations (SD). All analyses were conducted using SAS software, version 9.4 (SAS Institute Inc., Cary, NC, USA).

Results

Patient profiles of the total study population at baseline

A total of 47 out of the planned 100 patients were enrolled. All 47 patients who were invited to discuss informed consent agreed to participate in the study. Consenting patients were followed for a mean ± SD of 7.0 ± 3.1 months. A total of 15 patients prematurely discontinued the study (6 due to death, 3 withdrew consent, and 6 discontinued for unknown reasons). Table 1 presents patients’ demographics, educational status, ECOG status, and concomitant diseases of the total study population at baseline.

Table 1.

Sociodemographic and clinical profiles of patients within the total study population (N = 47).

	Total study population (N = 47)
General demographics
Age, mean ± SD	65.8 ± 7.7 years
Sex (male), N (%)	24 (51.1)
BMI, mean ± SD	25.2 ± 4.6 kg/m2
School degree, N (%)
Lower secondary school 1	14 (29.8)
Secondary school 2	22 (46.8)
High school 3	9 (19.2)
No degree	2 (4.3)
ECOG status, N (%)
ECOG status 0	25 (53.2)
ECOG status 1	14 (29.8)
ECOG status 2	8 (17.0)
Presence of concomitant diseases at enrollment, N (%)
Yes	36 (76.6)
No	11 (23.4)
Concomitant diseases (multiple answers possible), N (%)
Chronic pulmonary disease	3 (6.4)
COPD	17 (36.2)
Other chronic lung disease	1 (2.1)
Diabetes mellitus without chronic complications	4 (8.5)
Rheumatic inflammatory disease	1 (2.1)
Renal disease	5 (10.6)
Metastatic secondary malignancy	2 (4.3)
Arterial hypertension	27 (57.5)
Coronary heart disease	2 (4.3)
Heart insufficiency	2 (4.3)

¹Refers to either “Volksschulabschluss” (completion of 8th grade) or “Hauptschulabschluss” (completion of 9th or 10th grade) or completion of the 8th or 9th grade at a “Polytechnische Oberschule (POS).”

²Refers to “Realschulabschluss / Mittlere Reife“ (completion of 10th grade), or the completion of 10th grade at a POS.

³Refers to the “Abitur” (either a general or subject-specific higher education entrance qualification), awarded after completing either the 12th or 13th grade.

BMI: body mass index; COPD: chronic obstructive pulmonary disease; ECOG: Eastern Cooperative Oncology Group.

Patient access to Kaiku^® Health and the Lung AID EDC system

Most patients (85.1%) accessed Kaiku^® Health at least once during the study period. However, after the initial study visit only 38.3% of all patients logged in to the Lung AID EDC system and successfully submitted at least one of the three PRO questionnaires. Just under a third of all patients accessed both, Kaiku^® Health and the Lung AID EDC system (31.9%) (Table 2).

Table 2.

Patient access to Kaiku® Health, the Lung AID EDC system, and both platforms, categorized by recruitment from either all, inexperienced, or experienced sites.

	Number of patients	Percentage
All recruited patients	47	100.0
Patient access to Kaiku® Health	40	85.1
Patient access to the Lung AID EDC system	18	38.3
Patient access to both platforms	15	31.9
Patients recruited by inexperienced1 site	21	100.0
Patient access to Kaiku® Health	15	71.4
Patient access to the Lung AID EDC system	11	52.4
Patient access to both platforms	8	38.1
Patients recruited by experienced2 site	26	100.0
Patient access to Kaiku® Health	25	96.2
Patient access to the Lung AID EDC system	7	26.9
Patient access to both platforms	7	26.9

¹Site has 0 months of experience with Kaiku^® Health.

²Site has >0 months of experience with Kaiku^® Health.

When categorizing patient access by site experience, it was found that all patients, except one, recruited by an experienced site accessed Kaiku^® Health (96.2%). Compared to this, patient access to Kaiku^® Health was lower in patients who were recruited by an inexperienced site (71.4%). The Lung AID EDC system was accessed by just over a fourth of patients from experienced sites (26.9%) and by approximately half of patients recruited from inexperienced sites (52.4%). Access to both platforms was achieved by 26.9% of patients from experienced sites and by 38.1% from inexperienced sites (Table 2).

Patient profiles of Kaiku^® Health users and nonusers at baseline

The 40 out of 47 patients who accessed Kaiku^® Health at least once during the study period formed the Kaiku^® Health Set (KHS). Patient demographics, including age, sex, and body mass index (BMI), were similar between the KHS and the seven patients who did not access Kaiku^® Health during the study (Non-Kaiku^® Health Set, NKHS) (Table 3).

Table 3.

Sociodemographic and clinical profiles of patients, categorized into those using Kaiku® Health (Kaiku® Health set, KHS) versus those not using it (Non-Kaiku® Health set, NKHS).

	KHS (N = 40)	NKHS (N = 7)
General demographics
Age, mean ± SD	65.8 ± 8.2 years	65.4 ± 3.8 years
Sex (male), N (%)	20 (50.0)	4 (57.1)
BMI, mean ± SD	25.3 ± 4.8 kg/m2	24.7 ± 3.1 kg/m2
School degree, N (%)
Lower secondary school 1	13 (32.5)	1 (14.3)
Secondary school 2	17 (42.5)	5 (71.4)
High school 3	8 (20.0)	1 (14.3)
No degree	2 (5.0)	0 (0.0)
ECOG status, N (%)
ECOG status 0	23 (57.5)	2 (28.6)
ECOG status 1	11 (27.5)	3 (42.9)
ECOG status 2	6 (15.0)	2 (28.6)
Presence of concomitant diseases at enrollment, N (%)
Yes	30 (75.0)	6 (85.7)
No	10 (25.0)	1 (14.3)
Concomitant diseases (multiple answers possible), N (%)
Chronic pulmonary disease	3 (7.5)	0 (0.0)
COPD	12 (30.0)	5 (71.4)
Other chronic lung disease	1 (2.5)	0 (0.0)
Diabetes mellitus without chronic complications	3 (7.5)	1 (14.3)
Rheumatic inflammatory disease	1 (2.5)	0 (0.0)
Renal disease	4 (10.0)	1 (14.3)
Metastatic secondary malignancy	2 (5.0)	0 (0.0)
Arterial hypertension	21 (52.5)	6 (85.7)
Coronary heart disease	2 (5.0)	0 (0.0)
Heart insufficiency	0 (0.0)	2 (28.6)

¹Refers to either “Volksschulabschluss” (completion of 8th grade) or “Hauptschulabschluss” (completion of 9th or 10th grade) or completion of the 8th or 9th grade at a “Polytechnische Oberschule (POS).”

²Refers to “Realschulabschluss/Mittlere Reife” (completion of 10th grade), or the completion of 10th grade at a POS.

³Refers to the “Abitur” (either a general or subject-specific higher education entrance qualification), awarded after completing either the 12th or 13th grade.

BMI: body mass index; COPD: chronic obstructive pulmonary disease; ECOG: Eastern Cooperative Oncology Group.

Several differences were observed between the KHS and NKHS. For instance, NKHS patients had a higher percentage of concomitant diseases at enrollment. The educational backgrounds varied both within and between the two analysis sets. With 71.4% in the NKHS and 42.5% in the KHS, secondary school had the highest graduation rates in both sets compared to lower secondary school and high school. Regarding the patient's level of functioning, more NKHS patients were classified as having an ECOG status of 1 (restricted in physically strenuous activity but ambulatory and able to carry out work of a light or sedentary nature, e.g., light house work, office work) or 2 (ambulatory and capable of all self-care but unable to carry out any work activities; up and about more than 50% of waking hours) compared to KHS patients. In contrast, a higher proportion of KHS patients had an ECOG status of 0 (fully active, able to carry on all predisease performance without restriction) (Table 3). ²⁸

Discussion

Kaiku^® Health facilitates real-time health status reporting by oncology patients to their HCPs, enabling more timely and appropriate interventions. This pilot study assessed the feasibility of future evaluation studies on Kaiku^® Health in patients with mNSCLC receiving first-line pembrolizumab therapy in Germany. Patient engagement with Kaiku^® Health and the practicality of collecting PRO data in the separate Lung AID EDC system were investigated by assessing patient access to these platforms, without analyzing reported symptoms and PRO instrument responses.

Most patients accessed Kaiku^® Health at least once during the study (Table 2). The higher usage of Kaiku^® Health at experienced sites may be attributed to several factors, including better familiarity of the HCPs with the platform, more effective patient education and onboarding processes, and a greater overall integration of digital tools into the clinical workflow. Experienced sites might also have more effective support systems available to assist patients with technical issues. Moreover, these sites are likely more experienced at identifying and addressing potential barriers to patient interaction with digital tools, such as patients’ lack of digital literacy or concerns about data privacy and security. Consequently, patients recruited from experienced sites may feel more supported and encouraged to engage with digital patient monitoring tools like Kaiku^® Health.

The lower patient access to Kaiku^® Health at inexperienced sites compared to experienced sites emphasizes the importance of provider training and technology experience. To address this in future studies, more robust support for less experienced sites should include more tailored training programs that focus on both the operational and technical aspects of the study. In addition, implementing ongoing mentorship or a peer-support system, where experienced sites guide less experienced ones, could promote knowledge sharing and enhance interaction with digital tools. Frequent monitoring and feedback on the digital technologies are important to identify challenges early, providing real-time support to reduce performance disparities and improve patient engagement with digital tools.

Inexperienced sites, despite having lower initial access rates, still achieved significant patient interaction with Kaiku^® Health (Table 2). This indicates that even in settings where familiarity with the platform is initially limited, patients are willing and able to engage with AI-enabled health solutions. The overall high access rates for Kaiku^® Health suggest patient interest in AI-enabled solutions within the first-line metastatic NSCLC treatment setting. More personalized care, the convenience of remote monitoring, and the potential for better management of symptoms and side effects might be reasons for patients’ interest in Kaiku^® Health.

Only a minority of patients logged in to the separate Lung AID EDC system and successfully submitted at least one PRO questionnaire after their initial study visit (Table 2). The discrepancy between access rates of Kaiku^® Health and the Lung AID EDC system may be explained by the fact that, while study site personnel reminded patients to use Kaiku^® Health during routine clinical care visits, they did not specifically remind them to complete the questionnaires in the EDC system. Nevertheless, the low access rate to the Lung AID EDC system reveals challenges of assessing study-related patient-reported outcomes in real-world settings using a separate system. Future research should focus on improving evaluation tools to better capture and interpret patient data and develop strategies to enhance both patient and site engagement with these tools.

NKHS patients, who have a higher prevalence of concomitant diseases and worse ECOG statuses, exhibit a greater overall disease burden compared to KHS patients (Table 3). ²⁸ Especially those with higher ECOG scores, indicating greater disability, could particularly benefit from remote monitoring and easy access to medical support through a digital platform. However, these patients experience limitations in their daily functioning that could make using digital tools more challenging. Hence, they may require additional assistance in using digital tools, such as simplified instructions, regular follow ups, and seamless connectivity with healthcare professionals. Additionally, the observed differences in educational background underscore the importance of personalized care approaches in designing and implementing digital patient monitoring tools. By accounting for varying levels of education and health status, digital platforms can be made more accessible, user-friendly, and effective for all patients. Digital tools that meet the specific needs of diverse patient populations likely ensure that every patient can use these technologies to manage their health effectively, leading to better engagement and sustained use of the tools.

During this study, recruitment was challenging as only 47 of the targeted 100 patients were enrolled in the study. Sites were included based on their availability of patients and interest, resulting in a potential site selection bias and a limited number of recruiting sites. This likely resulted in slower recruitment rates and an overall smaller pool of potential participants, especially in a specialized field like first-line metastatic NSCLC. Consequently, the results may not be generalizable to other centers within or outside of Germany.

Through the implementation of targeted measures, the challenges in recruitment were effectively addressed. For instance, the study design was adapted by expanding the eligibility criteria, adding two new sites with expanded pre-implementation support, and enhancing patient education materials. ³⁵

The use of two separate digital systems, Kaiku^® Health and the Lung AID EDC system, potentially led to additional challenges. Each system required specific integration and adaptation processes at the sites. Additionally, patients’ willingness to engage with not just one, but two new systems likely affected their decision to participate in the study. To address this in future studies, consolidating both systems into a unified platform would be beneficial. For instance, integrating the PRO data collection functionality into Kaiku^® Health would simplify the user experience, making it easier for patients to participate and reducing the burden on healthcare staff. This streamlined approach could improve efficiency in setup, maintenance, and monitoring, likely improving both participation rates and data quality.

Improving regular usage of digital tools postrecruitment in the real-world may require enhancing strategies to increase patient adherence to study procedures and interaction with related platforms. Effective approaches may include using high usability tools and enhanced onboarding and training (such as personalized tutorials and interactive demonstrations), additional technical support (such as through e-mail support, helpdesks, or troubleshooting guides) and the provision of personalized feedback and progress reports on health status. ³⁶ Introducing gamification elements (e.g. points, badges, and levels) have been shown to improve patient adherence and interaction with digital solutions.^37,38 Collecting feedback from patients and healthcare professionals about their experiences with digital platforms, along with involving them early in the development stages of digital tools, can make these tools more user-friendly and increases the likelihood of sustained engagement. ³⁹ Moreover, robust evidence to elucidate the benefits for patients, healthcare professionals, and payers is needed. Such evidence may also offer insights into the successful integration of digital solutions within routine care potentially leading to further enhancement in patient usage and interaction. This study underscores challenges in evaluating digital patient monitoring tools in real-world contexts.

Conclusion

The overall high Kaiku^® Health access rates observed in this study underscore the interest of patients toward digital patient monitoring tools in the first-line mNSCLC treatment setting. However, recruitment was challenging, and the target of 100 participants could not be reached within the specified time frame. The low access rates of patients using both systems prevented further analyses from providing meaningful conclusions regarding the feasibility of conducting future evaluation studies using Kaiku^® Health. While the pilot study suggests an opportunity for HCPs to use digital tools to improve patient care, it also demonstrates the complexities involved in evaluating associated patient outcomes in the real-world. Future studies should improve patient and site engagement strategies to enhance interaction with digital tools. While definitive conclusions on future evaluations cannot be drawn, the study offers valuable insights into the challenges that should be considered in future research.