Defining dyadic cancer pain concordance using participant-initiated interactions with a remote health monitoring system

Abstract

Background

Studies on symptom concordance between patients and their caregivers often use cross-sectional designs, which may fail to capture the longitudinal, dynamic symptom experience. The Behavioral and Environmental Sensing and Intervention for Cancer (BESI-C) is a remote health monitoring system that utilizes smartwatches and ecological momentary assessments (EMAs) to empower patients and caregivers to monitor and manage cancer pain at home. BESI-C collects real-time symptom data in naturalistic settings, enabling longitudinal tracking and analysis of symptom patterns over time.

Objective

To define and examine dyadic concordance using participant-initiated symptom reports collected via remote health monitoring.

Methods

Dyads of patients with advanced cancer and their family caregivers were recruited to use BESI-C for 2 weeks, reporting pain in real time through EMAs. We used Bangdiwala’s B statistic to determine the concordance of patient-reported pain and caregiver-reported perceived patient pain under different contextual criteria (eg, co-location of participants; user engagement with BESI-C) that we hypothesized would impact concordance. We also explored a hypothesis that concordance would improve between study week 1 versus week 2.

Results

Data from 21 patient-caregiver dyads were used for analysis. The reporting of pain events was highly variable between patients and their caregivers. Concordance of pain reporting improved when patients and caregivers were co-located and both wearing their BESI-C smartwatches. We did not observe consistent patterns in patient-caregiver concordance between week 1 and week 2.

Conclusion

We propose an analytical approach to define and evaluate concordance between patients’ and caregivers’ real-time symptom reports that can be applied to dyadic, longitudinal symptom data collected using remote health monitoring. Future work should examine the relationship between patient-caregiver symptom concordance with key quality-of-life metrics and sociodemographic factors that impact participant engagement with remote health monitoring technologies.

Background

Cancer pain is often managed at home by patients and their families.^1,2 For patients with advanced cancer, family members play a significant role in managing pain, such as administering medications or other pain-alleviating therapies.^3,4 Managing cancer pain can be challenging and cause emotional distress, anxiety, and depression for both patients and their family caregivers.^5,6 Implementing a dyadic approach in pain assessment and management—one that considers a patient and their caregiver as a unit of care and incorporates both perspectives—can encourage collaborative decision-making, improve communication,^7,8 and optimize health.⁹

Patients and their family caregivers often have different perceptions of a patient’s pain which can impact symptom management.^10–12 Previous studies have assessed concordance between patient-reported and caregiver-reported symptoms using cross-sectional approaches that may not effectively capture the dynamic symptom experience.^13–15 To design tailored interventions for both patients and their caregivers, longitudinal studies are needed to understand the dynamic symptom experiences at the dyadic level within the context of everyday life.^10,16,17 Remote health monitoring systems (RHMS) can enhance the dyadic pain management approach by empowering patients and their caregivers to track symptoms, such as pain, in real-time in their daily lives for a more accurate and longitudinal depiction of their experiences.¹⁸

In this study, we propose an analytical approach to determine and analyze the concordance between patient-reported pain and caregiver-reported perceived patient pain by leveraging real-time data collected using the Behavioral and Environmental Sensing and Intervention for Cancer (BESI-C). BESI-C is an innovative example of RHMS designed to improve cancer pain management in the home setting. BESI-C consists of several components, including smartwatches worn by both patients and caregivers programmed with a custom application to facilitate the reporting of pain events using on-demand ecological momentary assessments (EMAs)—repeated brief assessments in real-time¹⁹ (Figure 1).

Figure 1.

(Left) BESI-C environmental sensors. (Right) Examples of the BESI-C smartwatch EMA interface to record pain events.

Participants mark patient pain as it occurs using an on-demand EMA and then receive an automatically generated follow-up EMA 40 minutes later to assess any changes in the pain or any interventions utilized to reduce the pain. Participants also receive a daily, scheduled EMA that assesses other quality of life factors, such as the amount of time patients and caregivers spent together over the past 24 hours. The smartwatches also collect physiological data (eg, heart rate) and activity data (eg, accelerometer). BESI-C also includes environmental sensors that collect ambient environmental data (eg, temperature) and interact with the smartwatches to determine the location of the patient or caregiver in their home when a pain event is recorded. Details of BESI-C design, feasibility testing, and early pilot results have been previously reported.^20–23

This study fills a gap in the literature by examining the longitudinal, temporal concordance between patient-reported pain and caregiver-reported perceived patient pain. The proposed analytical approach can be applied to other symptom data collected using RHMS.

Methods

Study design and participants

Approval was granted by the University of Virginia Health Sciences IRB (#21017), and all participants signed informed consent before data collection. Dyads of patients with advanced cancer (ie, locally or metastatic cancer, any type, and estimated prognosis of <12 months) and their primary family caregivers were recruited from an outpatient palliative care clinic and a home hospice program. Patients and their caregivers used BESI-C smartwatches to record and characterize cancer-related pain as it occurred over 2 weeks. Participants could use on-demand EMAs to report patient cancer-related pain on their respective BESI-C smartwatch regardless of their location (inside or outside their homes).

We posed 2 primary hypotheses for this study: (1) patient-caregiver concordance of pain reports would increase under different contextual criteria related to engagement with BESI-C and co-location of participants (see “Determining concordance and testing hypotheses” section for specific contextual criteria), and (2) patient-caregiver concordance of pain reports would increase between weeks 1 and 2 as participants became more comfortable using the BESI-C system.

Study variables

We made a series of decisions regarding the operationalization of key variables, such as defining pain events, clustering and overlapping rules, and the concordance time window. We cannot overemphasize the importance of ensuring clarity with variable definitions when working with complex, longitudinal, dyadic RHMS symptom data. We considered patient-reported pain events (ie, on-demand EMAs submitted via the smartwatch application) as the anchor/reference and clustered patient-reported pain events occurred within a 40-minute window. Caregiver-reported perceived patient pain events (EMAs) within a 40-minute window were also clustered.

We chose a 40-minute window as 40 minutes after each initial pain event recorded by the patient/caregiver, the BESI-C application sends a follow-up EMA to re-assess the pain severity and any pharmacological or non-pharmacological actions taken to mitigate the pain. During this 40-minute window, participants can record additional pain events, for example, due to escalating pain. If a pain event was recorded within a 40-minute window of more than one cluster, it was grouped with the first/earlier cluster. We defined an “overlap” as when a caregiver-reported perceived patient pain event was recorded within 30 minutes before or after a patient-reported pain event or cluster. Operational definitions of pain events, clusters, and overlapping events are summarized and illustrated with diagram examples in Table 1.

Table 1.

Definition of pain events, clusters, and overlapping events with diagram examples.

	Definition	Diagram representation
Event	A discrete episode of pain recorded by a patient or caregiver on their respective smartwatch using an on-demand EMA.
Cluster	Pain events recorded by the patient or caregiver within a 40-minute window. The time window starts from the timestamp of the first pain event and ends 40 minutes later.
Overlap	Any caregiver-recorded pain event or cluster recorded within a 30-minute window before or after a patient-recorded pain event or cluster. *Note: Patient-recorded pain events are used as the reference/anchor to determine overlap.

CT, caregiver; EMA, ecological momentary assessment; PT, patient.

Table 2 describes the different overlap statuses we considered for patient and caregiver-reported pain events. The overlap status of 1, 2, and 3 were used in a contingency table to calculate the Bangdiwala’s B statistic²⁴ for symptom concordance (see Determining concordance and testing hypotheses section).

Table 2.

Overlap statuses used to code pain events and clusters with diagram examples.

Overlap status	Event/Cluster code	Diagram representation
1) Patient pain event or cluster with no caregiver event or cluster 30-minutes before or after.	PTa – yes, CGa – no (no overlap)
2) Caregiver pain event or cluster, with no patient event or cluster 30 minutes before or after.	PT – no, CG – yes (no overlap)
3) Caregiver pain event or cluster 30- minutes before or after a patient event or cluster.	PT – yes, CG – yes (yes overlap)
4) No patient or caregiver pain event or cluster marked in a 30- minute window	PT – no, CG – no (no overlap)

^aCG, caregiver; PT, patient. The dashed ovals represent the overlap status being defined.

Determining concordance and testing hypotheses

We determined concordance by calculating: (1) the Bangdiwala’s B statistic and (2) the proportion of “pt—yes, cg—yes” (overlap status 3) divided by the total number of cases across overlap statuses 1, 2, and 3, which we termed “consensus” (see Table 2). The Bangdiwala’s B statistic is a quantitative measure to assess inter-rater reliability between 2 raters (ie, patient and caregiver),²⁴ which is more suitable for data with unbalanced marginal distributions than Cohen’s d.²⁵ B statistic can range from 0 to +1. A value of 0-0.05 corresponds to poor agreement; 0.05-0.15 corresponds to fair; 0.15-0.35 corresponds to moderate; 0.35-0.65 is considered substantial; values ≥ 0.65 show perfect agreement between 2 raters.²⁶

We calculated consensus and B statistic under 3 contextual criteria to investigate our hypothesis that concordance would increase given changes in patient and caregiver co-location and engagement with the system: criterion 1 (C1): without consideration of patient/caregiver locations or smartwatch use; criterion 2 (C2): patient and caregiver both at home, but without consideration of smartwatch use; and criterion 3 (C3): patient and caregiver both at home and wearing study smartwatches. To examine participant location (“at home” or “not at home”), we first checked the self-reported location on the pain EMA and if that was not available, we checked the localization data captured by the interaction between environmental sensors and the smartwatches. We utilized the presence of accelerometer and heart rate data on the smartwatches to confirm patients and caregivers were wearing their smartwatches. To test our second hypothesis, we used Wilcoxon signed-rank test²⁷ to examine if concordance for each dyad improved from study week 1 to week 2 (eg, comparing consensus [the percentage of pt—yes, cg—yes events] in week 1 versus week 2 from the same dyad). All data analysis was done using R version 4.4.0 (2024-04-24).²⁸

Contextualizing concordance

To further contextualize our concordance results, we examined participants’ responses to selected items on a post-deployment survey that inquired whether they recorded pain events independently or in consultation with their partner (note: before the deployment, participants are encouraged to enter data independently so we can best assess their individual experience, but this is not required). We also examined the patient’s self-reported Eastern Cooperative Oncology Group (ECOG) Performance Status collected from the baseline pre-deployment survey to understand their physical capability for recording pain events.²⁹

Additionally, we analyzed responses to questions about participant employment status (from the baseline pre-deployment survey) and how much time patients and caregivers reported spending together (from the end-of-day scheduled EMAs) to understand how these factors related to patient and caregiver physical proximity might impact the reporting of pain events and subsequent concordance.

Results

Data from 21 patient-caregiver dyads (42 participants) were collected between February 2022 and October 2024. The average age of patients was 63 (SD= ± 13) and the average age of caregivers was 57 (SD= ±15). The majority of patients were male (n = 11) while the majority of caregivers were female (n = 14). Most caregivers were patients’ spouses or domestic partners (n = 15). Table 3 summarizes the sociodemographic characteristics of the study participants.

Table 3.

Sociodemographic characteristics of study participants, overall sample, and by patients and caregivers.

	Total, n (%)	Patients, n (%)	Caregivers, n (%)
	42 (100%)	21 (50%)	21 (50%)
Column percentages
Age band
20-40	4 (10%)	1 (5%)	3 (14%)
41-60	11 (26%)	5 (24%)	6 (29%)
61-80	27 (64%)	15 (71%)	12 (57%)
Gender
Female	24 (57%)	10 (48%)	14 (67%)
Male	18 (43%)	11 (52%)	7 (33%)
Race
Other	1 (2%)	0 (0)	1 (5%)
Black/African American	2 (5%)	1 (5%)	1(5%)
White	39 (93%)	20 (95%)	19 (90%)
Ethnicity
Latino/Hispanic	1 (2%)	0 (0)	1 (5%)
Non-Latino/Hispanic	41 (98%)	21 (100%)	20 (95%)
Highest education level
Less than high school	1 (2%)	0 (0)	1 (5%)
High school graduate	18 (43%)	11 (52%)	7 (33%)
Some college/college degree	18 (43%)	8 (38%)	10 (48%)
Professional/graduate degree	4 (10%)	1 (5%)	3 (14%)
Missing	1(2%)	1 (5%)
Current employment
Employed full-time	9 (21%)	2 (10%)	7 (33%)
Employed part-time	5 (12%)	1 (5%)	4 (19%)
Retired	21 (50%)	13 (61%)	8 (38%)
Other (disabled/unemployed)	7 (17%)	5 (24%)	2 (10%)
Relationship to patient
Spouse/partner		N/A	15 (71%)
Parent			1 (5%)
Child			3 (14%)
Friend			1 (5%)
Other family			1(5%)
Primary cancer diagnosis
Hematological malignancies		5 (24%)	NA
Solid tumors		16 (76%)
Self-reported ECOG a ,29
I am fully active		1 (5%)
I can’t do heavy work, but I can do some light work		9 (43%)
I can’t do any work, but I can care for myself		6 (28%)
I need some help caring for myself, and I spend most of the day in bed or in a chair		4 (19%)
I need much help caring for myself and I spend nearly all day in bed or in a chair.		1(5%)

^aEastern Cooperative Oncology Group, commonly used scale to assess patient’s functional status.

Bangdiwala’s B statistic between patient and caregiver-reported pain based on contextual criteria 1, 2, and 3 are presented in Table 4. For example, dyad 8, under criterion 1, had 31 patient-reported pain events, 38 caregiver-reported perceived patient pain events, and 9 overlapping events, resulting in 12% consensus (9/78). Under criterion 2, there were 4 patient-reported pain events, 8 caregiver-reported pain events, and 9 overlapping pain events, yielding 43% in consensus (9/21). Under criterion 3, there were 4 patient-reported events, 7 caregiver-reported events, and 9 overlapping events, leading to 45% in consensus (9/20). The consensus range for criterion 1 spans from 0% to 67%, while for criteria 2 and 3, it ranges from 0% to 92%. The B statistic and consensus under criteria 2 and 3 were equal for all dyads except dyads 2, 8, 13,14, and 20. All dyads had better concordance (higher B statistics and consensus) under criteria 2 and 3 versus criterion 1. Only 4 dyads (dyads 3, 10, 11, and 21) had poor concordance (B statistic < 0.05) under all 3 criteria. Bolded dyad numbers represent dyads with substantial to perfect concordance across all criteria (Table 4).

Table 4.

B statistics, and percent consensus between patient and caregiver reported pain events, per dyad, for criterion 1, 2, and 3.

Dyad	PT—yes, CG-no (no overlap), status 1			PT—no, CG-yes (no overlap), status 2			PT—yes, CG-yes (overlapping), status 3			Consensus	B	Consensus	B	Consensus	B
	C1 a	C2	C3	C1	C2	C3	C1	C2	C3	C1	C1	C2	C2	C3	C3
Dyad 1	28	1	1	2	1	1	1	1	1	3% (1/31)	0.01	33% (1/3)	0.2	33% (1/3)	0.2
Dyad 2	18	7	4	1	0	0	8	5	2	30% (8/27)	0.25	42% (5/12)	0.42	33% (2/6)	0.33
Dyad 3	18	12	12	15	7	7	3	3	3	8% (3/36)	0.01	13% (3/22)	0.04	13% (3/22)	0.04
Dyad 4	70	20	20	23	9	9	26	22	22	21% (26/119)	0.11	43% (22/51)	0.33	43% (22/51)	0.33
Dyad 5	39	27	27	29	22	22	14	13	13	17% (14/82)	0.06	20% (13/62)	0.08	20% (13/62)	0.08
Dyad 6 b	5	0	0	3	2	2	12	12	12	60% (12/20)	0.53	86% (12/14)	0.85	86% (12/14)	0.85
Dyad 7b	17	0	0	34	5	5	5	5	5	9% (5/56)	0.017	50% (5/10)	0.5	50% (5/10)	0.5
Dyad 8	31	4	4	38	8	7	9	9	9	12% (9/78)	0.03	43% (9/21)	0.32	45% (9/20)	0.34
Dyad 9	12	11	11	12	5	5	8	8	8	25% (8/32)	0.12	33% (8/24)	0.21	33% (8/24)	0.21
Dyad 10a	4	3	3	21	9	9	0	0	0	0	0	0	0	0	0
Dyad11	36	24	24	15	13	13	4	4	4	7% (4/55)	0.012	10% (4/41)	0.02	10% (4/41)	0.02
Dyad 12a,b	5	1	1	6	0	0	7	7	7	39% (7/18)	0.26	88% (7/8)	0.87	88% (7/8)	0.87
Dyad 13	22	12	12	30	28	28	9	9	8	15% (9/61)	0.04	18% (9/49)	0.07	17% (8/48)	0.06
Dyad 14	55	17	17	6	4	4	9	9	7	13% (9/70)	0.06	30% (9/30)	0.2	25% (7/28)	0.15
Dyad 15	23	17	17	24	21	21	12	11	11	20% (12/59)	0.08	22% (11/49)	0.09	22% (11/49)	0.09
Dyad 16 b	12	2	2	0	0	0	24	24	22	67% (24/36)	0.67	92% (24/26)	0.92	92% (22/24)	0.92
Dyad 17a,b	8	2	2	3	1	1	5	5	5	31% (5/16)	0.19	62% (5/8)	0.57	62% (5/8)	0.57
Dyad 18	37	32	32	6	6	6	11	11	11	20% (11/54)	0.12	22% (11/49)	0.13	22% (11/49)	0.13
Dyad 19a,b	13	5	5	12	5	5	16	10	10	39% (16/41)	0.26	50% (10/20)	0.4	50% (10/20)	0.4
Dyad 20	30	16	16	31	21	21	14	13	12	18% (14/75)	0.06	26% (13/50)	0.13	24% (12/49)	0.11
Dyad 21	49	25	25	10	5	5	3	3	3	4% (3/62)	0.01	9% (3/33)	0.03	9% (3/33)	0.03

^aDyads recruited from hospice.

^bDyads with substantial or perfect concordance under criteria 2 and 3; Bold dyad numbers represent dyads with substantial to perfect concordance across all criteria. Under C1, we did not consider patient or caregiver location or watch-wearing. Under C2, we only considered pain events that were reported when both patient and caregiver were at home and did not consider watch-wearing. Under C3, we considered pain events reported when both patient and caregiver were at home, and both were wearing their smartwatches.

Figure 2 presents the B statistics across all dyads and how they changed under the 3 criteria (see Methods, Determining concordance and testing hypotheses section). The B statistic values improved (ie, approached 1) under criterion 2 (when both patients and caregivers were wearing their smartwatches) and further improved under criterion 3 (when patients and caregivers were both at home and wearing their smartwatches.)

Figure 2.

B statistic across all dyads based on the 3 contextual criteria described in the Methods section. Under criterion 1 (C1), we did not consider patient or caregiver location or watch-wearing. Under criterion 2 (C2), we only considered pain events that were reported when both patient and caregiver were at home and did not consider watch-wearing. Under criterion 3 (C3), we considered pain events reported when both patient and caregiver were at home, and both were wearing their smartwatches.

The amount of time that the patient and caregiver spent together during the study period is summarized in Table S1 to further contextualize concordance between patient and caregiver symptom reporting. Patient and caregiver “time spent together” is based on their responses to a question in the scheduled end-of-day EMAs (“Time spent with the patient/caregiver?”) and their self-reported employment status. For example, Table S1 distinguishes between caregivers who are working full-time and those who are retired, potentially allowing them to spend more time with the patient. For instance, dyad 19 caregiver and patient were both retired and indicated that they spent more than 4 hours together during most of the study period. Dyad 4 patient and caregiver were both retired and stated that they did not answer the pain questions independently; however, their concordance was fair under criterion 1 and moderate under criteria 2 and 3. Interestingly, we did not observe any differences in concordance of pain reports within dyads where caregivers were employed and those with caregivers who were unemployed/retired, despite the impact we anticipated this would have on pain reporting (ie, employment status may change the amount of time patient and caregiver spend together in the home, thus impacting likelihood of concordant pain reporting).

Table S2 describes the percentage of overlapping patient and caregiver-reported pain events (see Table 1 for how overlapping pain events were determined) for each dyad in week 1 compared to week 2 of the study period. For example, dyad 2 had 8 overlapping pain events out of 27, 4 of which occurred in week 1 and the other 4 in week 2. The Wilcoxon signed rank test P-value was only significant under criterion 2 (P-value < .05), suggesting a statistically significant difference in the percentage of pt—yes, cg—yes events between week 1 and week 2, when both patient and caregiver were at home (C2).

Discussion

In this study, we propose an analytical approach to define and analyze the concordance of dyadic pain reporting between patients with advanced cancer and their primary family caregivers, using real-time, longitudinal data collected through BESI-C, an innovative RHMS that facilitates cancer pain tracking and management in the home setting.²² Our study makes an important contribution to the symptom science literature by offering an approach that can be helpful to others engaged in similar research that utilizes longitudinal, dyadic symptom reporting data. Analytical techniques to analyze complex symptom data are particularly needed as the use of RHMSs become more prevalent.³⁰

Overall, our results demonstrate that the reporting of pain events can be highly variable between patients and their caregivers. As noted in previous studies, family caregivers often experience difficulties in detecting and describing patient symptoms, such as pain, due to their subjective nature.³¹ Caregivers may rely on different sources of information, including their previous knowledge about the patient’s typical behavior and expressions of pain, which can lead to discrepancies between patient and caregiver perceptions of the symptom experience?³²

We examined concordance for pain events using the Bangdiwala’s B²⁴ under 3 criteria to further contextualize the pain experience. Not surprisingly, when we refined our criteria to include only pain events where both the patient and caregiver were at home and wearing their smartwatches (criterion 3), the B statistic values improved (approached one). This suggests that the physical proximity/co-location of both parties and using the monitoring devices might reduce discrepancies (eg, improve concordance) in pain reporting. In other words, being in the same location and consistent engagement with RHMSs (eg, wearing the BESI-C smartwatches) helped ensure both parties were actively engaged in symptom monitoring.

Across all dyads, a third of our sample (10/21; almost 50%) had moderate to perfect concordance under criteria 2 (both at home) and 3 (both at home and wearing smartwatch), and the rest had poor to moderate concordance under the same criteria. We also qualitatively examined the sociodemographic characteristics of dyads (eg, age, gender) with substantial to perfect concordance to identify any specific patterns or traits that set them apart from others. However, we did not find any notable differences. The various symptom experiences across dyads underscore the complexity of pain perception and highlight the need for tailored interventions that can bridge the perceptual gap between patients and caregivers.^33,34

We hypothesized that concordance would improve from week 1 to week 2 as participants gained more familiarity with the BESI-C system. We found significant variability in the patient-caregiver consensus (the percentage of pt—yes, cg—yes events) by week; some dyads showed higher consensus in week 1 and others in week 2. Additionally, when we analyzed concordance among dyads with varying amounts of time spent together (as detailed in Table S1), we observed variability in concordance levels. This variability may be attributed to different levels of engagement with the RHMS among participants which may be influenced by sociodemographic variables, an important factor we plan to investigate further in future research.

Limitations

The results presented in this study are based on data collected from patients with advanced cancer and their family caregivers. While this work focuses on a specific disease and symptom (cancer pain), the overall analytic approach is applicable to understanding the concordance of symptom reports from other patient populations and diseases. Researchers using different RHMS or data collection procedures may need to modify the approach to suit their specific contexts. Study data were collected from healthcare institutions in Central Virginia with predominantly White, non-Hispanic populations which limited racial and ethnic diversity of the study sample. The lack of diversity may affect the generalizability of findings with respect to factors such as racial and ethnic differences.³⁵ Future studies should aim to include a more diverse sample to ensure the broader applicability of the findings. To define concordance, we used patient-reported pain events as the anchor/reference. However, if we had considered caregiver-reported perceived patient pain as the reference, the results might have differed. These results reflect an interim analysis of prospectively collected data; results may change once the entire final data set is available and analyzed.

Future directions

Future work should investigate what classifies as “high” or “low” symptom concordance, and how it correlates with patient and caregiver quality-of-life metrics. For example, does being “in sync” with your partner regarding symptom reporting have a positive impact on metrics, such as overall distress levels or mood? In this foundational work using longitudinal, dyadic symptom report data, we sought to answer the question: “what is concordance and how should we quantify it?” Future work will extend this analysis by exploring “how much does concordance matter?” Additionally, we plan to further investigate how patients and caregivers characterized pain events/clusters by exploring differences in reported pain severity and distress levels.

Conclusion

Our study addresses a critical gap in symptom science by proposing an analytical approach to assess concordance between real-time patient and caregiver symptom reports using data from RHMS. Developing standardized methods that are appropriate for complex, longitudinal symptom data collected in real-time is essential for optimizing pain management strategies. Future research should build on this framework to explore additional symptom domains and the broader impact of concordance on the quality of life for both patients and their caregivers.

Author contributions

Mina Ostovari (Conceptualization, Formal analysis, Investigation, Methodology, Visualization, Writing—original draft), Natalie Crimp (Conceptualization, Data curation, Methodology, Visualization, Writing—review & editing), Sarah J. Ratcliffe (Conceptualization, Funding acquisition, Methodology, Supervision, Validation, Writing—review & editing), and Virginia LeBaron (Conceptualization, Funding acquisition, Methodology, Project administration, Supervision, Writing—review & editing)

Supplementary material

Supplementary material is available at JAMIA Open online.

Funding

This work was supported by the National Institute of Nursing Research of the National Institutes of Health under award number R01NR019639. The content is solely the responsibility of the authors and does not necessarily represent the official views of the National Institutes of Health.

Conflicts of interest

The authors have no conflicts of interest to declare.

Data availability

The study data cannot be shared publicly due to privacy and ethical restrictions, as they include sensitive personal health information. Data may be shared upon reasonable requests to the corresponding author, subject to institutional review board approval and a data use agreement.

Sanction

• Are you or any of your co-authors named on the following list relating to trade sanctions: the OFSI Consolidated List, the US Specially Designated Nationals or Blocked Persons list, and/or the EU Consolidated List? No

• Are you or any of your co-authors employed by an institution or entity which is named on the following lists relating to trade sanctions: the OFSI Consolidated List, the US Specially Designated Nationals or Blocked Persons list, and/or the EU Consolidated List? No

• Does your article contain military or defense-related or other controlled technical information which is restricted from publication by applicable export control laws? No