Informing the Design of Individualized Self-Management Regimens from the Human, Data, and Machine Learning Perspectives

Abstract

Intelligent systems for self-management can help patients and improve quality of life. However, designing AI-based systems is challenging because designers need to account not only for user needs, but also for capabilities and practical constraints of underlying algorithms. We propose and implement a human-centered AI framework to align human and technological requirements and constraints that can guide design of intelligent systems for personal health. We use concepts from a machine learning technique, reinforcement learning, to elicit user needs, through directed content analysis of user interviews, and uncover practical data constraints, through analysis of “in the wild” user engagement logs from a self-monitoring app. We gather and triangulate human-machine-data requirements for a self-management tool for individuals with endometriosis — a poorly understood, complex chronic condition with no reliable treatment. We present recommendations for developing a system that aligns with needs, capabilities, and constraints from human user, data, and machine learning perspectives.

1. INTRODUCTION

Care for chronic conditions is a major health priority globally [100]. Self-management — the day-to-day activities individuals undertake outside of the clinic to cope with their chronic illness — plays a critical role in managing and preventing the progression of disease [21, 22, 39, 40, 73, 152]. However, establishing a self-management care regimen can be a major hurdle. Faced with often generic guidance, individuals are left with the burden of translating this information into their day-to-day lives [30, 57, 102, 105, 111]. For instance, the recommendation to “engage in regular exercise” is left up to individuals to determine how to implement (e.g., which exercises would make sense for them, how often, and how intense). Furthermore, it is not known a-priori which strategies will be successful for a given person. Thus, individuals have to experiment through trial-and-error, which can be a lengthy process. This approach becomes even more complex when individuals have to choose among multiple strategies that can be combined into a regimen. Finally, in conditions with limited scientific knowledge and no established self-management guidelines, there is an additional burden on the individual to identify candidate strategies [36, 158, 161] and effective personalized regimens [36, 48].

Personal health informatics solutions have been proposed to support self-management for individuals [35, 42, 52, 71, 78, 93, 121, 131], scaffold for problem solving [95, 96], and promote experimentation to identify triggers of disease flares [44, 45, 81]. Tools for self-tracking and reflection have helped in multiple contexts like migraine [128], IBS [37, 80], autism spectrum disorder [82], HIV [28], diabetes [27, 115], and for those with multiple chronic conditions [20, 91]. However, most solutions still leave a lot of the analytical work to individuals, e.g., which strategies to experiment with (either alone or in concert of each other), how to go about experimenting with them, and determining if they work. As such, designing intelligent systems to support management in the context of complex chronic illness represents an example of Ackerman’s sociotechnical gap [2] — i.e., there is a known discrepancy between the nuanced, flexible, and contextual real-world task of self-management, and the rigid and brittle capabilities of technology.

In prior research with endometriosis patients [120], we documented a need for solutions that can provide individualized self-management recommendations and help to identify strategies with a positive health impact. These recommendations should take individual factors and context into consideration, help discover and select strategies to try out, and learn which ones are effective for each person. Given the structure of the problem identified in this work, a promising research direction for individualized, adaptive recommendations for self-management is the use of artificial intelligence (AI) methods in general, and reinforcement learning (RL), in particular. RL is unique in its ability to make sequential recommendations that adapt to changes in a complex environment. With RL, an agent learns a policy, i.e., a mapping from states to recommended actions (e.g., for a person living with diabetes, blood glucose level measurements mapped to recommended insulin doses), to maximize a pre-defined reward (e.g., decrease in HbA1c levels in a person living with diabetes) while simultaneously adapting to changes in the state, i.e., the environment resulting from the actions (e.g., the health condition of the person with diabetes). This ability to adapt in decision-making under uncertainty makes RL a plausible candidate to power an intelligent personal informatics system to support the trial-and-error process of self-management, by providing individualized recommendations of strategies for users to try and evaluating their success — over time and through interactions with the system — for learning individualized self-management regimens that work for each person [67]. However, RL also has a somewhat rigid conceptual model and requires structuring of a problem space into its framework of action space, state space, reward, and agent/policy. Furthermore, RL is notoriously data hungry and requires large training datasets to enable learning. As a result, while RL has many unique benefits, the introduction of RL may widen the sociotechnical gap [2] that already exists in intelligent solutions for personal health.

In order to design intelligent, personalized systems that meet the real-world needs of users, a human-centered approach is necessary [18]. User-centered and participatory design have been established as critical pathways towards developing technological solutions, which have been increasingly applied in AI [72, 108, 153]. However, traditional user-centered design methods often prioritize user needs over technical capabilities and limitations [14]. While this approach has been shown to be widely successful in traditional software development [34, 50, 56, 84, 147], it may have limitations when applied to machine learning (ML) and AI. Given the need for human-centered design and an uptick in interactive systems that incorporate ML/AI [31, 33, 150], there is a need for new design methods that balance human needs with the technical capabilities of these systems.

Various human-centered AI (HAI) principles have been put forward to address this gap [66]. Chancellor [32] argues that human-centered machine learning practices must be applied throughout the whole ML pipeline of problem brainstorming, development, and deployment. Human-Centered Algorithm Design proposed by Baumer [18] outlines design strategies across theoretical, speculative, and participatory processes, with the focus on incorporating social interpretations into the design approach. Value-Sensitive Algorithm Design [162] focuses on the early elicitation of human insights to guide the abstract and analytical creation of algorithms, aiming to mitigate bias and avoid compromising user values; however, there is no way to explicitly account for the demands and specifications of specific ML techniques, and it does not account for the data perspective. By contrast, Stakeholder-Centered AI Design [160] accounts for both human and data perspectives, using co-design with stakeholders’ own data to prototype with AI, but only addresses generic algorithm considerations. While these approaches include user, data, and/or ML perspectives, they do not account for all of these perspectives simultaneously and have no way to design for a specific ML algorithm. In fact, to a large degree, these new efforts continue to prioritize user needs and values and use them as a blueprint for designing new technologies. Since AI-enabled technologies often have unique capabilities and hard to change restrictions that must be considered when designing user-facing systems [25], neither the traditional user-centered design, nor the newer user-centered methods for AI account for these constraints. While these approaches offer some directions, a gap remains in practices to design HAI technologies, particularly for a given ML solution [127]. This requires new design approaches that place both users and AI on the same level and enables negotiation between them.

In this paper, we propose and implement an HAI framework for designing intelligent personal informatics systems — Multi-Perspective Directed Analysis (MPDA) — that accounts for human, data, and machine learning requirements and constraints concurrently. MPDA uses constructs extracted from an ML approach, RL, to elicit both user needs, through directed content analysis of user interviews, and practical data constrains, critical for ML and AI-driven systems, through analysis of user engagement logs with an app for collecting self-monitoring data. We apply the proposed framework to gathering and triangulating human-machine-data requirements for a self-management tool for individuals with endometriosis — a poorly understood, complex chronic condition with no cure or reliable treatment. In this study, we ask the following research question: What insights at the intersection of human needs and values, human self-tracking behaviors as evidenced by “in the wild” self-tracking data, and capabilities and constraints of RL, can inform the design of RL-based intelligent systems for self-management of endometriosis?

The work presented here makes these specific contributions:

• First, we propose (and provide an example of) a novel human-centered AI framework — Multi-Perspective Directed Analysis — that maps the human, data, and machine learning requirements and constraints within a complex real-world scenario for the design of an intelligent interactive system. We identify high-level concepts from RL to use as organizing principles for analyzing and synthesizing both qualitative and quantitative data. We use the MPDA framework to answer the research question asked in this study, which allows us to synthesize the human, data, and ML perspectives, evaluate trade-offs, and generate recommendations for design.

• Second, we identify promising directions for an RL-based intelligent interactive system for management of complex chronic illness, and elaborate several sociotechnical gaps [2]. In simultaneously attending to human, data, and ML requirements, we are able to surface recommendations to guide the design of an RL-enabled system to support users in leveraging their personal health data to experiment with self-management regimens in the context of endometriosis. Such a system for developing individualized self-management regimens in the case of endometriosis holds significant promise in improving care and management of this burdensome condition.

2. BACKGROUND

2.1. Self-Management of Endometriosis

We investigate the specific research question of this study in the context of endometriosis, where the need for individualized recommendations is particularly acute. Endometriosis is a chronic condition defined by the presence of a tissue similar to uterine endometrium located in physiologically inappropriate body locations, leading to chronic, cyclic, and persistent or progressive symptoms [6, 164]. These symptoms (e.g., pain, gastro-urinary symptoms, and fatigue) can be debilitating and impact activities of daily living [126]. Diagnosis and treatment remain problematic since the disease is poorly understood scientifically, and while surgery to remove diseased tissue may alleviate symptoms, no cure exists and patients actively engage in ongoing care and management [1, 26, 68, 117]. Endometriosis is an illness that has many uncertainties, and wide heterogeneity in disease presentation and in response to treatment and self-management [148].

2.2. The Phendo App for Self-Tracking Endometriosis

The Phendo app [60, 61] is a mobile research app that was developed in partnership with endometriosis patients to capture the real-world experience of the disease by allowing users to catalog the day-to-day signs and symptoms, self-management activities, and other lived experiences of endometriosis outside of the clinic. A broader goal is to leverage citizen science to facilitate ML from the collected data to discover new insights about the disease and support individuals in the care and management of their illness. A series of prior studies explored users’ motivations for tracking and important dimensions to track, via interviews and focus groups [99], and further elicited variables (e.g., symptoms, self-management strategies) that people with endometriosis find important via online surveys and content analysis of an online endometriosis community [98]. This self-tracking app was designed alongside end-users — individuals with endometriosis — so that the collected data reflect the illness experience of individuals from the patient perspective.

Users of the Phendo app self-track a wide range of information about the different dimensions of their illness — screenshots from the Phendo app are shown in Figure 1. Table 2 provides the full list of Phendo questions, along with the vocabulary type (pre-set, customized, or free-text) of the available answers. The community of Phendo users is active and engaged with self-tracking activities that provide a rich day-to-day picture of the disease [64]. Our analysis of Phendo app usage patterns showed that long-term users of the app are more likely than short-term users to self-track their self-management activities [62].

Fig. 1.

Screenshots of the Phendo app

Table 2.

Description of relevant questions in the Phendo app, the vocabulary type (pre-set, customized, or free-text), and the available answers.

Questions related to self-management strategies – action space
Phendo Question	Answer Type	Examples
What did you do to self-manage? *	Pre-set: 14 multiple choice items	heat pack, massage, talk therapy
Did you do any of these exercises that hurt? *	User-specified multiple choice	running, situps, lunges, kickboxing
Did you do any of these exercises that help? *	User-specified multiple choice	yoga, pilates, swimming
Did you eat any foods that worsen symptoms? *	User-specified multiple choice	sugar, gluten, white flour, beer
Did you eat any foods that improve symptoms? *	User-specified multiple choice	fresh veggies, lean meat, nuts
Take any supplements?	User-specified multiple choice	CBD oil (15 mg), magnesium (500mg)
Take any hormones?	User-specified multiple choice	progestin(implant), microgestin (1.5 mg)
Take any medication?	User-specified multiple choice	Percocet (10mg), Oxycodone (7mg)
Questions related to the personal experience of illness – state space
Phendo Question	Answer Type	Examples
How was your day?	Pre-set: 5 single-choice items	good, manageable, bad, unbearable
Do you have your period?	Pre-set: 2 single choice items	yes, no
Are you in pain now? (body location, severity) †	Pre-set: 39 multiple choice items	ovaries; cramping; moderate
Any GI/Urine issues? (description, severity) †	Pre-set: 15 multiple choice items	endo belly, vomiting, constipation; severe
Experiencing something else, other symptoms? (description, severity) †	Pre-set: 21 multiple choice items	fatigue, headache, swelling; mild
How is your mood?	Pre-set: 30 multiple choice items	calm, happy, angry, anxious
Are you bleeding?	Pre-set: 3 multiple choice items	clots, spotting, breakthrough bleeding
Which activities were hard to do?	Pre-set: 20 multiple choice items	sleep, shower, work, sit down, walk
How was sex?	Pre-set: 5 multiple choice items	painful during, painful after, avoided
Daily journal	Free-text
Data related to evaluating the success of self-management strategies – reward
Description		Example
Difference in symptom severity scores, with a focus on: pain, GI issues, and other common endometriosis symptoms – the relevant Phendo questions are shown with †. We quantify how frequently this reward information is tracked before and after self-management activities are logged.		Reduction in pain severity tracked before and after engaging in self-management
Data related to self-management trials – agent/policy
Description		Example
Self-management trials consist of an event (self-management strategy, highlighted above in pink) tracked, along with pre-event and post-event goal data (i.e., self-management strategy, with reward data tracked before and after). Effect estimates are calculated for each trial (the difference between pre- and post-strategy goal response), at both the population-level and for individuals.		Walking to pain trial with a negative effect estimate indicates a reduction in pain between before and after implementing the strategy

Phendo questions related to the the state space (personal experience of illness) are highlighted in taupe, and those related to the action space (self-management strategies) are highlighted in pink.

The analysis for this study includes all users who had tracked at least one instance of self-management marked with an asterisk (*).

Prior work with Phendo data has focused on characterizing the enigmatic illness — an analysis of self-tracked Phendo data has helped to augment what is known about the disease and fill gaps in the medical literature [148]. Other analyses have detailed specific experiences of endometriosis self-management, in particular, the impacts of physical activity [63]. More recent work has sought to understand the needs of individuals and their providers in supporting care and management with personal informatics tools. In a study that we conducted to elicit the needs of patients and providers in caring for endometriosis, all stakeholders conveyed a dire need for support in self-management of this complex condition [120]. Because of the enigmatic and chronic nature of endometriosis, treatments are often ineffective and management regimens are highly personalized, commonly requiring lengthy and involved trial-and-error with various strategies. While patients emphasize that they want low-impact, restorative self-management strategies to help them live with and ameliorate symptoms, they are not well-supported in their experimentation over time and have difficulty identifying strategies, sticking with them, and evaluating if they are working. Participants imagined features of intelligent systems that could help them scaffold this trial-and-error self-experimentation process — helping them figure out what to try, determine if it is working, and structure the data collection process. The current study is situated within this broader research initiative and leverages our previous work. In this study, the data that have been collected serve as the basis of the quantitative analysis while users of the Phendo app are consulted in the qualitative part.

3. RELATED WORK

3.1. Intelligent Systems for Health and Management of Chronic Illness

Intelligent systems for reflection and action.

The Human-Computer Interaction (HCI) community has established a rich body of literature on developing intelligent systems and tools to use personal health data to support individuals in their health-related goals. Personal informatics examines solutions that focus on facilitating individuals’ engagement with personal data. Li et al [88] proposed a stage-based model of personal informatics that outlines various stages of user engagement with their personal data. Many personal informatics solutions have focused on the presentation of personal data to support reflection and improve self-knowledge [28, 37, 82, 103]. These systems have also been augmented with ML to better support reflection and action [94, 104]. Other research has explored the use of ML in sociotechnical systems, for example in clinical medicine [124] and mental health [33, 144].

Personal informatics for self-experimentation.

One common approach to supporting increased self-knowledge using personal health data is through self-experimentation. Traditional personal informatics tools, even without the use of AI or statistical analysis, can help individuals identify trends and patterns in their health data to support personal discovery and behavior change [15, 86, 128]. Personal informatics interventions have also been developed specifically for this purpose. These systems are frequently designed to facilitate conducting n-of-1 trials (also called single case designs), where users act as their own controls, to highlight an individual’s response to a treatment rather than a group’s [44, 81]. While many of these systems have shown promise in supporting experimentation, they have also been constrained by rigid options and limited personalization [45, 46, 80, 86, 143]. But users with complex health conditions are still in need of personalized, customizable solutions to support self-experimentation towards developing effective long-term regimens.

Approaches for automated, individualized self-experimentation.

Some research has explored the use of advanced computational frameworks that use ML to develop more flexible, person-centered systems for self-experimentation [47, 129]. Other research has investigated automated, individualized intervention approaches such as adaptive treatment strategies [8, 104], micro-randomized trials (MRTs) [83], and just-in-time adaptive interventions (JITAIs) [110], for a variety of health-related outcomes [41]. These tools exist in domains that have more well-defined parameters and outcomes, but could provide guidance in the context of complex chronic illness self-management.

Some of these existing approaches to adaptive intervention fit into the RL paradigm [139, 140]: a computational approach to understanding, automating, and optimizing a sequenced set of actions that maximize an outcome of interest. An RL agent learns which intervention is best to suggest from a pre-determined (continuous or discrete) set of actions, given a state, towards maximizing a total reward. JITAIs adjust the type, timing, and framing of support provided based on a user’s dynamic internal and contextual state, aimed at maximizing the positive impact of an intervention. JITAIs, which sometimes use RL approaches to learn the best intervention for users, result in (expert or learned) decision rules that map an individual’s current state to a particular intervention or treatment at each decision time point. Therefore, an RL approach to an intelligent system for supporting the trial-and-error process in self-management of a complex chronic condition is a promising solution to investigate.

3.2. Reinforcement Learning

RL agents make sequential decisions as they interact with the environment, i.e., as the world changes, toward achievement of a specified goal. An RL agent learns how to update its recommendation policy from previous actions and evaluated rewards, based on the variables it observes. The policy dictates the behavior of the agent (system) and uses observations to map from the state space to the action space when in particular observed states. The state, i.e., the information used for individualization, helps decide when and/or how to intervene with particular actions. A reward signal defines the goal of the RL problem, which the agent maximizes, while the value function (which the RL agent updates as it interacts with the environment) quantifies the long-term desirability of states (after taking into account likely subsequent states and corresponding rewards).

RL as a candidate for automated, individualized self-experimentation.

RL offers potential benefits that make it a promising candidate for providing automated, adaptive recommendations for individualized self-management in an illness context with a lot of complexity and uncertainty. Since it uses the results of each iteration to update the policy and inform future algorithmic decisions, it is particularly well-suited to help users self-experiment with self-management strategies, when it is not known beforehand what will be helpful, when, or for whom. In short, RL engages algorithmically in a trial-and-error process similar to the goal-directed, sequential learning that individuals follow when working to develop their own individualized self-management regimens [92]. RL can handle the dynamics of sequential interactions between the user and a system by utilizing feedback from the environment to adjust its actions; RL can account for long-term user engagement with a system; and RL can optimize a policy by sequentially interacting with the environment without requiring explicit user input [4].

In the context of endometriosis, RL-based recommendations offer several advantages. The goal of an RL agent is not only to optimize interactions with the environment, but also to learn insights that allow individualized interventions. Since RL allows for individual-level optimization, rather than focusing on population-level estimates, given that endometriosis shows strong person-to-person variation in symptoms and treatment responses, person-level personalized self-management recommendation is needed. RL is a sequential decision-making algorithm that can leverage multiple correlated time points during learning, making it suitable for learning with correlated longitudinal data. Since RL is able to provide sequential, adaptable, and individualized recommendations, it could be set up to conduct n-of-1 trials to facilitate self-experimentation at the individual-level — and helpful for other chronic diseases beyond endometriosis.

RL has shown exceptional performance in multiple scenarios [106, 137]. However, despite the previous successes of this approach and numerous benefits, there are also critical unresolved impracticalities in the context of this study: (i) successful examples have been limited to highly structured, controlled, and well-defined environments, i.e., they do not translate easily to practical, yet more complex situations; (ii) the state-of-the-art, deep-learning based RL techniques are data-hungry, i.e., they require inordinately large, repeated interactions with the environment to learn effective policies; and (iii) they are black-box models, i.e., it is inherently difficult to interpret the complex features learned by these models and why recommendations are made.

Problem formulation in the RL framework.

The real-world task of figuring out what self-management strategies might work for an individual with an illness like endometriosis is an inherently interactive process that calls for a sequential decision-making process. Thus, it is possible to formulate the problem as a Markov decision process (MDP) and solve it with RL [135, 163]. The fundamental trade-off between exploitation (optimizing recommendations) and exploration (learning insights about the environment it is interacting with) is critical for efficiently updating the RL policies that decide which action to recommend next. To support individuals in developing a personalized management routine, an RL agent must learn a policy that can prescribe, based on observed context (i.e., user state), appropriate self-management strategies (i.e., actions at each interaction) that best improve an individual’s health status (i.e., the reward signal).

In real-world situations, especially when the RL directly interacts with and learns from humans’ actions, there are multiple challenges to consider in designing an RL agent [70]. On one hand, the operational success of an RL policy is constrained by the size of the data it can learn from, the type of information it has access to, the number of interactions it can leverage, and the type of actions it can recommend. On the other, users of the system have their own preferences (e.g., the framing and tone of a recommendation) and values (e.g., preserving agency in their self-management activities).

Misalignment between human and machine can create unhelpful recommendations (e.g., a 20-minute jump rope session for an individual with chronic pelvic pain) and as a consequence, at best break trust and at worst result in harm. Hence, understanding user-centered requirements for RL and its different components is critical to designing human-centered RL systems. In addition, an intelligent system that uses RL may ease the cognitive burden put on individuals to find their optimal self-management regimen. At the same time, this type of system requires delivery as a mobile intervention, which must incorporate human feedback. Delivery via a mobile self-tracking app requires the RL system to be able to determine when and how to best deliver the intervention components at different decision points. An interactive solution that incorporates human feedback in the learning process is necessary [12].

4. METHODS AND MATERIALS

In order to align design requirements of an intelligent system for supporting experimentation with self-management across human, data, and machine learning perspectives, we propose and implement a novel HAI framework to conduct a mixed-methods study. From the ML perspective (described in Section 4.1), we use high-level concepts from RL to conduct directed coding. From the human perspective (described in Section 4.2), we rely on the input of individuals with endometriosis to understand their needs and values when conducting self-management and receiving recommendations. The original high-level RL concepts form the basis for the directed content analysis of the qualitative data. From the data perspective (described in Section 4.3), we conduct analysis of “in the wild” self-tracking data from current users of the Phendo app, who log their experiences in their day-to-day lives (i.e., in the absence of any ML algorithm) to understand patterns of illness experiences and self-management behavior from real-world data. The organizing principles from RL, which are revised according to the empirical data from the qualitative analysis, provide structure to the quantitative analysis of usage logs. Through iterative and ongoing discussions amongst the authors, we triangulate findings from across these three different perspectives to arrive at a set of conclusions and design recommendations. An overview of the organizing principles and data sources is presented in Figure 2.

Fig. 2.

Overview of analytical approach with organizing principles.

Our institutional review board approved all study procedures. This current study is situated within our broader research initiative focused on characterizing the enigmatic disease endometriosis and developing tools to support individuals in managing this complex chronic illness. This study leverages the Phendo app, as described in Section 2.2 — we analyze data that have been self-tracked with the app, re-analyze transcripts from prior focus groups, and conduct interviews with active Phendo users.

4.1. Machine Learning Perspective: RL

We select high-level RL concepts (i.e., action space, state space, reward, agent/policy — identified by Sutton and Barto [139]) as the basis for this analysis. These concepts are presented in the first column of Figure 2. In the context of this study, the action space represents the items available for the intelligent interactive system to recommend — self-management strategies. The state space represents the illness, contextual, and broader environmental variables that the agent will take into account for its recommendation. The reward is the function that will be used to evaluate the success of each recommended strategy. The agent/policy refers to the actual individualized self-management recommendations and behavior of the intelligent interactive system. A mapping between the RL concepts and the corresponding components of an intelligent system for self-management is presented in Figure 3.

Fig. 3.

RL concepts mapped to the corresponding components of an intelligent interactive system to support individualized self-management.

A wide body of literature for RL policy learning exists [5, 87], with recent work in the context of mobile health data [76, 90]. Since the aim in the current study is to support individuals in developing a personalized self-management routine that might evolve over time, we focus our attention on learning individualized sequences of strategies that maximize users’ well-being as they interact with them (i.e., online learning). At the same time, we can leverage available historical data (i.e., offline data) for the design and modeling of the algorithm, inform its value and policy functions, and to avoid the cold-start problem.

An intelligent system that will be used and useful to individuals with endometriosis will have to meet their needs, operate acceptably and within expectations, and fit into their lives and self-management routines. To ensure these requirements are met, a key step undertaken in this study is to map the RL concepts and intervention components to the real-world aspects of human end-users. The design items to consider are: the scope of the action space; how to represent the state space and context; how to model the reward signal; how to balance exploitation vs exploration; what are the decision points (e.g., every hour vs daily opportunities) and decision rules; how much time can users dedicate for each recommendation; how often (dose schedule) and for how long (study duration) are users willing to experiment with the automated recommendation system. We answer these questions based on input from Phendo users and already available Phendo data. Below, we illustrate how our framework can use these RL-specific constructs to structure both qualitative analysis of user interviews and quantitative analysis of Phendo usage logs.

4.2. Human Perspective: Qualitative Analysis

For the qualitative analysis, we analyze transcripts from discussions with people with endometriosis. We use directed content analysis (as described by Hsieh [75]) based around the high-level RL concepts described in the previous section to code, understand, and organize the data into meaningful findings. The way that the RL concepts are applied in this part of the study is depicted in the second column of Figure 2. Note that data from the quantitative analysis also represent the real-world experiences of users, but are discussed separately within the data perspective in the next section.

Data.

We re-analyze focus group transcripts from prior studies and conduct additional interviews with Phendo users with follow-up questions specifically relating to a tool for self-management. In total, we re-analyze 10 transcripts from focus groups with 48 participants (all women with a formal diagnosis of endometriosis) from prior work and data from three follow-up interviews. Focus groups from 2016 (5 groups, n = 27) originally informed the design of the Phendo app. Focus groups from 2019 (5 groups, n = 21) were initially used to elicit design needs for tools to support care and self-management of endometriosis. This secondary analysis of focus group transcripts provides a valuable source of information about Phendo users’ management practices, how people with endometriosis select and evaluate strategies, and where participants have unmet care needs that technology could support. But while these focus group transcripts do directly address self-management practices and technology needs, the discussions do not specifically address an interactive system for experimenting with self-management strategies. To fill this gap and strengthen the evidence collected from the secondary analysis, we conduct in-depth follow-up interviews (n = 3) to ask potential users directly about an interactive system for developing individualized regimens. Interviews probe about participants’ experiences of self-management, the process by which participants have developed their own regimens, and what opportunities and constraints participants foresee in an automated system that provides intelligent recommendations that could support their management. From these data sources, we are able to directly and indirectly elicit constraints and boundaries of an interactive system for self-management. Additional details on these methods, including a description of the study sample and topics covered in the interviews and focus groups, can be found in Appendix A.1.

Analysis.

The qualitative analysis focuses on predetermined high-level RL concepts in order to guide the analysis towards evidence that answers our specific research question. Our inquiry seeks to identify technical requirements and parameters for an intelligent interactive system for self-management, to figure out a range of what users want and are willing to do when using the system, to outline constraints, and to assess potential feasibility of such a system. We use directed content analysis to code the data, starting from the four high-level RL concepts as predetermined codes (i.e., action space, state space, reward, agent/policy), then refine the codes during analysis based on empirical data — the initial codes and final coding categories used as the framework for the study are presented in Table 1. During the analysis, the authors worked together to review, refine, and name the categories and extract key examples to present. Given the reflexive and collaborative nature of the analysis methods, we focused more on exploration of themes and consensus building rather than consistency in coding [29]. The codes generated through the analysis are used solely for conceptual purposes, serving to establish foundations for an RL model rather than for direct use in RL training. Future work developing the RL agent may employ an MPDA-based approach, where coding consistency will be critical.

Table 1.

Coding Categories. The initial categories are shown, along with the corresponding final categories.

Initial coding categories	Final coding categories
(1) Action Space — Self-management strategies available for the intelligent system to recommend	(1a) Broad action space
	(1b) Explore, with user autonomy
(2) State Space — Health status and circumstances considered by the system for recommendations	(2a) Illness, everyday, and context variables
	(2b) Context for relevant recommendations
(3) Reward — Goal used to evaluate the success of strategies	(3) Short-term indicators
(4) Agent/Policy — Behavior of the system when determining individualized self-management recommendations	(4a) Heterogeneity, so individualized policies
	(4b) Explainable recommendations
	(4c) Engagement patterns sufficient for RL

4.3. Data Perspective: Data Collected from a Self-Tracking App

For the quantitative analysis, we leverage data collected from the Phendo app to capture the illness experiences and self-management behaviors of users “in the wild,” i.e., without any intervention or recommendation, how individuals track their illness and what strategies they rely on for self-management. The Phendo data and analyses used for each of the RL concepts can be found in the final column of Figure 2.

Data.

Phendo users self-track a variety of details about their illness (see Table 2). We identify two broad domains of questions from the Phendo app that are relevant for this analysis: those related to the state space, or the personal experience of illness (colored in taupe in Table 2), and those related to the action space, or self-management (colored pink in Table 2). For each of the customizable self-management strategies (e.g., exercise, food), we manually map and normalize all responses to a smaller, coherent set of pre-determined strategies in order to reduce the number of strategies in the dataset and to facilitate a population- vs individual-level analysis. E.g., the user-entered exercises ‘walk,’ ‘hiking,’ and ‘walking dog’ are all harmonized to a ‘walking’ entry, and the curated ‘strength’ strategy includes responses from customized answers like ‘crunches,’ ‘sit ups,’ ‘core exercises,’ ‘squats,’ ‘planks,’ ‘push ups,’ and ‘weightlifting.’

Since this study focuses on self-management, we extract all data for users who have self-tracked at least one of the Phendo self-management questions marked with an asterisk (*) in Table 2. For each participant, we collect their longitudinal self-tracking data as entered in the Phendo app, so that we can quantify both their self-management habits and information on their health status through time. The dataset contains information from 10,463 users, with n = 399,786 self-tracked responses, of which n = 290,503 are self-management instances.

Analysis.

The quantitative analysis was conducted in alignment with the pre-selected coding categories, and iteratively revised alongside the qualitative analysis, as the categories were revised inductively based on empirical data. For the action space, we analyze the answers to the self-management Phendo questions detailed in Table 2 to characterize the breadth of strategies users experiment with, as well as for how long and how often do they engage with each strategy. Similarly, for the state space, we analyze the answers to the questions related to the personal experience of illness, detailed in Table 2. For the analysis related to the reward, we define simple goals related to pain, GI symptoms, and other symptoms. We quantify how frequently users track this information before and after engaging in self-management activities. An example is highlighted in green in Table 2. For the analysis related to the agent/policy concepts of RL, we quantify the effects of strategies through an analysis of self-management trials, and assess engagement based on how many trials users log “in the wild.” We define a self-management trial as an event where a user tracks a self-management strategy, as well as goal information a day before and after the tracked strategy. That is, a trial consists of a triad of pre-management goal information, a self-management instance, and post-management goal information, all contained within a day of the self-tracked strategy. We then quantify effect estimates of each strategy on symptom severity: i.e., we compute the difference between the post- and pre-strategy goal response for each trial. E.g., a self-management pre-post negative pain-severity effect estimate indicates the strategy reduces pain, while a positive pain-severity effect showcases worsening of the experienced outcomes of the self-management strategy. For a given set of per-strategy trial effects, we compute effect estimates both at the population level (by averaging per-strategy effects) and for individuals (by averaging per-individual longitudinal per-strategy effect). An example is shown in blue in Table 2. In investigating if an RL policy is feasible for this task, this component of the analysis can help us understand if data tracking patterns “in the wild” will align with the ML and human requirements and constraints identified in the qualitative analysis.

5. RESULTS

We integrate results from the qualitative and quantitative analysis, with the RL concepts as the guiding framework. In synthesizing these findings, we identify promising circumstances where an intelligent system for self-experimentation with self-management could be successfully deployed; but we also find barriers and challenges that will need to be resolved. We present eight findings across the four RL categories (Figure 4): 1. Action Space — (1a) The space of actions/self-management activities individuals experiment with is large; (1b) Individuals are willing to explore the action space as long as they can set some boundaries; 2. State Space — (2a) The state space, that determines which actions to recommend, is complex and can vary across individuals; (2b) Individuals favor recommendations that are responsive to the user’s current context; 3. Reward — (3) Individuals assess the success of a strategy in the short term; 4_. Agent/Policy — (4a) Success of specific strategies varies across the population, rendering the need for tailored recommendations from individualized policies; (4b) Individuals want to understand why they received recommendations and why they should try recommended strategies; and (4c) “In the wild” user engagement with self-management and self-tracking is frequent enough to satisfy RL data requirements. Illustrative quotes are included with each of the results (where the interviews are labeled “Int” and the focus groups are labeled “FG”), and Recommendations for design are included in Table 3 at the end of the section.

Fig. 4.

Overview of study findings, organized around the high-level RL concepts used as initial directed coding categories, and sub-themes derived empirically through data analysis.

Table 3.

Technical outline of findings and recommendations.

RL	Finding	Human-Data-Machine Value Recommendations
(1) Action Space Self-management strategies	(1a) The scope of the action space is broad across the population of users, and individuals often combine multiple strategies.	Balance diverse and personalized action space, with the need to constrain the set of available actions. Provide wide range of available strategies to users, to enhance control and autonomy, as well as personalization of experimentation. Leverage both pre-set Phendo vocabulary and user-customized strategies. The action space should be discrete, with its cardinality (i.e., the number of possible actions to choose from) determined for each individual. A smaller action set enables the system to more efficiently identify an optimal self-management policy.
	(1b) Users are willing to explore the action space, but control of the action space is important to participants.	Support both experimentation with strategies already identified by users, and discovery of new strategies that may be effective. Facilitate exploration of the action space to support learning, transitioning to exploitation over time as the RL learns an effective regimen. Provide strategy recommendations to experiment with one thing at a time for a particular health state, which can prevent users from getting overwhelmed and also help constrain the action space. Leverage user input to enable control and autonomy, while meaningfully constraining the action space. Facilitate user input, both globally and with each interaction. First, enable input up front to elicit what users want to experiment with and what they are not willing to do based on individual constraints (e.g., remove strategies based on cost, schedule, or health status). Second, enable real-time input as recommendations are given, e.g., to decline a recommendation at that time or remove it from available options. Incorporating human-in-the-loop aspects of interactive RL could offset computational challenges by improving the efficiency of the model, while also resulting in a more personalized system that enhances user control and autonomy.
(2) State Space Health status and circumstances	(2a) Experiences of illness and everyday variables and context are important for characterizing the state space.	Leverage a diversity of user data about individuals’ health and day-to-day lives, to create tractable representations of users and their context that can inform the RL’s learning process. Define personalized user states through careful human-centered ML: address open user modeling questions (e.g., What is a good user-state? How to learn from both data and user input?) and data-driven learning of context (state space representation learning). Determine the appropriate dimensionality of the state space, since the rate for learning optimal policies depends sublinearly on the cardinality of the state space. Use existing computational approaches to enable representation of complex illness experiences in low-dimensional spaces, while innovating methods to represent the very human aspects of illness. Innovate ways to capture, compute, and represent embodied assessments and abstract conceptualizations for characterizing how a user is feeling. Digital phenotyping methods are one potential solution to capturing rich representations while constraining the dimensionality of the state space. Also devise mechanisms to handle these personalized user models as trajectories over time.
	(2b) Users need recommendations that are suited to their current context.	Use historical data, augmented with sensors to provide real-time context. Address data, human, and algorithmic challenges related to capturing and computing meaningful, real-world, real-time context from individuals. Rather than requiring that every user inputs granular data, which is burdensome and unrealistic, expand the data capturing capabilities of Phendo, e.g., with passive sensing, and combine them with state representation learning from historic data. Use existing self-tracked data about the user’s current context alongside more embodied sources of data, while exploring more creative methods to capture embodied experiences of illness. Innovate mechanisms to compute tractable representations of illness that can translate unique, multi-faceted, complex, and often subjective experiences into objective measures that could be quantified, measured, and compared across time. For example, data could be captured via voice recording or artistic expression.
(3) Reward Goal in evaluating the success of self-management	(3) Users are looking for short-term self-management, but even then there are many ways to compute the reward function.	Start with simple, short-term, symptom-based reward functions as the baseline. Focus on short-term metrics for the design of RL reward functions. Conduct experiments to determine whether to use discrete (e.g., binary increase/decrease in pain reports) or continuous (e.g., difference in pain scores) reward functions. Expand to more complex, domain-derived, data-driven, individualized functions. Innovate methods to translate individualized metrics, which may be multi-faceted, complex, and subjective, into metrics that could be used by an intelligent system for evaluating if a recommended strategy worked. Expand beyond simple reward functions, towards domain-derived, data-driven individualized scoring functions (e.g., composite scores combining pain, GI, mood, and symptom self-reports). A data-driven approach would automatically learn the best state/reward function for human-defined goals, however domain expertise and user input are essential to align the reward function with individuals’ notion of success. E.g., for a goal of short-term pain reduction, the agent could focus on the change in self-tracked pain within a specified time window.
(4) Agent/Policy Individualized self-management recommendations	(4a) Heterogeneity in self-management responses calls for tailored recommendations.	Balance individualized with population-based policies — start by augmentation with similar users, then transition to more fully individualized policies over time. Consider the trade-off between providing individualized recommendations (critical to an intelligent system’s success) and the data requirements (long sequence of interactions) to learn individualized policies. Explore modeling and statistical tradeoffs between fully individualized techniques and hierarchical or pooling models to learn from similar (state, action, or effect) evidence. Bayesian mixed model effects and hierarchical models can pull statistical power across population evidence; clustering approaches can help with dimensionality reduction.
	(4b) Users want an intelligent system with explainable recommendations to help understand why they were given particular recommendations and why they should try to implement the strategy.	Focus on explainable models to provide users with information about decision-making and insights into their illness and management. Avoid complex and black-box system based RL solutions (e.g., Deep-RL), and instead rely on explainable options: e.g., model-based, Bayesian sequential decision policies that allow for statistical and explainable modeling of the state to reward functions, facilitating the discovery of individualized insights.
	(4c) Engagement patterns suggest user interactions will be numerous and frequent enough for RL requirements.	Leverage the alignment between system requirements and users’ documented behaviors in self-tracking and self-management, and operationalize the specific system requirements with experiments. Conduct simulated and/or trial experiments to determine the minimum number and periodicity of interactions required to learn an RL policy in the context of endometriosis self-management.

5.1. (Action Space - 1a) The scope of the action space is broad across the population of users, and individuals often combine multiple strategies.

In the context of chronic disease self-management, the action space corresponds to the space of possible self-management actions that an individual can take. As such, we code all participant descriptions of self-management approaches under this category and analyze self-tracking data related to self-management. Participants detail a multitude of different self-management strategies and self-care approaches for coping with symptoms and for promoting health more broadly. Their regimens incorporate strategies that are included in the Phendo self-tracking vocabulary (see Table 2), both pre-set (e.g., “In the middle of the pain, just give me my heating pad and my drugs.” [FG 2]) and user-customized, such as exercise and diet (e.g., “For me — no dairy, no gluten, very low sugar. Minimize raw food intake — mostly cooked vegetables, dried fruits. Fried foods. Preservatives. Processed foods. So, an endo-friendly diet.” [Int 3]). Participants also describe strategies and self-care tactics beyond what is included in Phendo, for example: “I’ve found hobbies that I can do with two hands help me feel better. I crochet, I am kind of a plant lady, I have a garden.” [FG 7]. Strategies like these could be added into Phendo as user-customized vocabulary. Participants report using various strategies in combination and as needed, for example: “I like to take a holistic approach — I put marijuana in tea and I drink it, sometimes I use the CBD pen. I have ginger and turmeric tea on a regular basis. And I try to exercise at least three times a week.” [FG 7].

From existing “in the wild” data, we corroborate these findings: current Phendo users track a wide range of strategies across the population and individual users often track several in combination. First, we showcase in Figure 5 the count of users that have self-tracked at least 10 instances of each strategy within the self-management Phendo questions. These results align with the participant reports of the wide variety of strategies they use (and are willing to use). Second, we illustrate in Figure 6 the count of users that self-track strategies in combination. We notice that, although a lot of Phendo users focus on one or two distinct self-management strategies, there are many Phendo users already tracking regimens in the app that combine several strategies together. From the ML perspective, it is beneficial that users engage with a range of strategies, yet an RL algorithm will benefit from a constrained action space to effectively learn meaningful policies. Design decisions will need to be made in order to satisfy both human user requirements and the RL requirements. Taken together, the data from users and self-tracking data suggests that there is a broad range of strategies used at the population level, but individuals are likely to only experiment with a few strategies at a time.

Fig. 5.

Histogram of Phendo users with self-tracking data (at least 10 instances) for each self-management strategy. We observe that users across the population engage with a broad range of strategies.

Fig. 6.

Histogram of Phendo users that track several strategies in combination. We observe that users often engage with a combination of several self-management strategies within their timeline.

5.2. (Action Space - 1b) Users are willing to explore the action space, but control of the action space is important to participants.

The second code within the category of the action space relates to users’ willingness to explore different strategy recommendations. Here, we code discussions about what participants are willing to try and limits and barriers that constrain what recommendations they are open to. Participants frequently discuss turning to exploration and trial- and-error self-experiments, since individuals managing endometriosis face a lot of uncertainty in care and are often left without effective treatments. Participants tell us they are interested in trying a broad range of self-management options and are willing to try strategies that don’t necessarily make sense to them. When one interview participant was asked about trying strategies the intelligent system might recommend, she reports: “I’ll always give the new thing a try and put the old thing on hold.” [Int 3]. Further, while users are willing to explore a wide variety of strategies and try suggestions that they may not otherwise, they do want an intelligent system that becomes more tailored to them and personalized over time (shifting from explore to exploit in the RL framework). As this same participant explains: “In the beginning I’d be open to explore all options and see what there is to offer. And then, throughout usage it becomes more fine-tailored to me, I think that would be ideal.” [Int 3].

Participants warn that adding too many strategies to someone’s care routine may become overwhelming, so generally limit experiments to a single strategy. At the same time, they describe benefits to engaging in multiple strategies that target different symptoms (e.g., one food strategy, one physical activity strategy, and one pain management strategy for flares). As one participant explains:

One thing, because that’s all I can handle. When I’m trying a new thing, I try to just ‘suss’ that out for a while just to see if it’s working. But at the same time, not like two things to cut out or add to my diet, but maybe something with diet and something with activity or movement — a combination of things.

[Int 3]

While participants are open to exploring different self-management strategies, which is beneficial to the learning process of RL, they are not willing to attempt very exhausting or taxing activities (e.g., “I do think that there’s a healthy limit to pushing yourself a little bit. But anything that causes me more pain is never going to be helpful.” [Int 1]). While participants describe a range of barriers that limit the options that are feasible for them, cost, logistics, and emotional challenges are the most often mentioned. Participants see value in setting hard limits ahead of time about what strategies are recommended. At the same time, they also imagine advantages to receiving recommendations with a variety of novel strategies that they otherwise might not try and to give feedback to the system in real-time as recommendations are given. For example, one interview participant explains:

Based on my previous experience with that type of activity, I already know it doesn’t work for me. […] I would want to tell the app, ‘Please don’t ever recommend that to me again.’ But, I would say not setting all of the limits at the beginning, because there is something about ‘pushing yourself’ a little bit and trying something a little bit new. […] So I actually think it would be helpful to get the things, but there are going to be some things for certain women, that are just going to be a no-go.

[Int 1]

Participants imagine giving input or feedback to the system about what they want to try (or not try), for example, interview participants asked about the intelligent system propose features that could provide options alongside recommendations to see if users are willing to try a strategy, such as: “yes,” “not now,” “not ever.” They also suggest offering different options of strategies across difficulty level (from gentle to rigorous) to choose from when presented with self-management recommendations. These suggestions from participants could enhance user control and integrate human intuition into the decision-making process, while enabling efficient RL learning with constrained action spaces.

5.3. (State Space - 2a) Experiences of illness and everyday variables and context are important for characterizing the state space.

In the case of chronic disease self-management, the state space corresponds to the ways that an individual’s illness state, current context, and the broader environment are represented. In this category, we code all descriptions of how individuals assess their health status and the context and environmental factors that impact their engagement in self-management and analyze self-tracking data related to illness experience and context. Participants describe rich, nuanced accounts of illness and discuss highly personalized and often embodied ways that they characterize and assess changes in their health status.

In detailing the ways that they understand their own disease, participants emphasize the importance of holistic and individualized representations of their illness experiences and talk about factors that are not always deemed clinically relevant. Pain and fatigue are the most frequently discussed indicators, but GI symptoms (e.g., cramping, diarrhea, nausea, bloating), breakthrough bleeding or spotting, skin issues (pimples/acne, rashes, hives), migraines, emotions (e.g., “mood swings,” depression, anxiety), manifestations of pain (pain-somnia, pain-aggression), and other personal signs of inflammation (e.g., in the face or belly) are seen as necessary to provide a holistic and individualized picture of a person’s health status. Participants also distinguish between dealing with typical, day-to-day endometriosis symptoms and the flare-ups that some individuals experience, as one focus group participant explains: “When it comes to endo, you have the pain aspect but then you also have the flare-up aspect and it kind of goes hand in hand and sometimes it doesn’t go hand in hand.” [FG 2].

Some participants report on indicators that suggest they are in a flare or a flare is coming. Participants recount their unique manifestations of “bad days” when they “feel terrible” or “horrible” (even though they often “don’t look sick”). They talk about being “incapacitated” in bed because of pain (sometimes tied to their menstrual cycle, ovulation, or menses) and feeling so ill they can’t move, are exhausted, or “collapse” when they get home. They talk about knowing their bodies, so they can know when something is not right (“You just have to know your body, and you’d be like, ‘Okay, well, something’s off. How can I help it?’ ” [FG 5]).

The analysis of Phendo data corroborates, as shown in Figure 7, that current users self-track the experience of illness via a multitude of dimensions that are available in the app — the list of available Phendo questions directly related to the individual’s disease experience is found in Table 2. These variables align with the description of what users consider important to represent their illness experiences to facilitate holistic state space representations and that can be used to modulate the recommendations from an intelligent system.

Fig. 7.

Histogram of Phendo users tracking questions related to their illness experience.

5.4. (State Space - 2b) Users need recommendations that are suited to their current context.

The second code within the category of the state space relates to the relevant context features where users are open to receiving different recommendations. As such, in this category, we code participant discussions about what details they consider in deciding on self-management strategies to use and analyze contextual data tracked immediately before self-management strategies are logged. Users describe wanting an intelligent system to consider contextual details related to their illness, their environment, and their day-to-day lives when tailoring strategies to recommend — many of which may be subjective, esoteric, and/or highly personalized. Users request for recommendations to be responsive to their current context, as one interview participant explains:

I track emotional moments when things arise for me. So if the Phendo app could know, ‘OK she’s been logging these things, she’s been anxious, or there’s fatigue.’ Then now is not a good time for the app to recommend training for the 5k. But if I log those things and the app suggests meditating today for 30 minutes, or ‘Why don’t you journal?’ — it would be useful if the recommendations coincide with whatever symptoms I’m tracking.

[Int 3]

On top of this request for context-awareness, users also want strategies to tailor for particular symptoms and for daily management compared to a debilitating flare.

Participants discuss fitting strategies into their daily lives, and mention wanting a system to consider their schedules and daily routines and fit strategies into them in a way that makes sense. Participants also see value in a system that could push recommendations for strategies that would be helpful in the moment. Some users are open to pre-scheduled time set aside to do the self-management strategies offered by the system, but users desire a system that can tailor strategies to the particular context of situations as they arise. As one interview participant describes:

There could be benefits to both. I like the push of, ‘Hey stretch right now.’ However I typically implement those types of things when my threshold for pain has gotten so high to the point where I can’t think, and that’s where I would step away and do that, as opposed to just getting home later tonight and stretching.

[Int 1]

Participants imagine personalized recommendations would be responsive to a user’s state, including health status and more broadly how are they feeling. But context about the day-to-day activities of users and their environments is currently infeasible, difficult, or impossible to capture (e.g., “What headspace am I in when that is suggested?” [Int 3]). Nonetheless, potential users want a system to take these aspects of their day-to-day lives into account.

We corroborate in Figure 8 that there are many existing instances of the self-tracked illness experiences discussed as important in Section 5.3, which are tracked by users within a day before they track a self-management strategy, suggesting that the contextual data currently tracked by users in the app can be useful for tailoring recommended actions. We observe that Phendo users are very likely to report information about their experiences with activities of daily living, as well as with information on symptoms they are experiencing the day before and the day after they self-manage. Therefore, there are already existing Phendo answers that can be used to compute a meaningful state space for an intelligent system and facilitate contextually-tailored recommendations, although we would still miss things that are not currently tracked or cannot be tracked (e.g., “current headspace,” “timing is not working out today,” or “pain so bad can’t think”). In addition, there are sparsity and heterogeneity challenges that are raised by the “in the wild” data analysis with current Phendo data: i.e., not all users track the same set of experiences, and they do not track them consistently for every self-management strategy they try.

Fig. 8.

Histogram of instances for questions related to Phendo users’ illness experience, tracked a day before a self-management strategy.

5.5. (Reward - 3) Users are looking for short-term self-management, but even then there are many ways to compute the reward function.

In an intelligent system for chronic disease self-management, the reward corresponds to the signal that will be optimized when suggesting actions and how the success of actions tried will be evaluated. As such, we code participant descriptions of how individuals evaluate if self-management strategies are working under this category and use simple self-tracked data to evaluate the effect of strategies tracked “in the wild.” In endometriosis, symptoms often persist and sometimes progress, since there is no cure, which means that getting rid of symptoms altogether is often impossible. So, people with endometriosis report having to rely on assessing how they are feeling in the moment and trends or deviations from their individualized baseline to figure out if strategies are effective or not.

When asked about assessing the success of self-management strategies and evaluating the progress of their health goals, individuals largely describe short-term symptom or health-related quality of life indicators. While patients often have longer-term goals for their care and management, these are not what participants report relying on for developing their care regimens. Participants talk about having difficulty figuring out if strategies are working and tell us that they generally rely on checking in with how they feel in the moment (“How do I know something is working? I’m not great at figuring that out all the time. […] For me, if the pain goes away in the moment, then I will stick with that.” [FG 6]). They report relying on mind and body pain and sensations, immediately or within a short time-frame. Even still, users are not looking to eliminate pain and symptoms (which is often not possible), and instead focus on evaluating if strategies are helping to improve their symptoms (“That’s the key word, ‘Not as bad.’ So, better.” [FG 1]).

When evaluating if self-management strategies are working, participants all talk about assessing their pain symptoms; GI symptoms are also frequently used as an indicator. Participants also describe clues or indicators that are specific to them, which they use to help them determine if they are feeling better or worse (e.g., skin rashes, acne breakouts, and the ability to sit in a chair without high pain).

In order to measure the success of self-management strategies in a data-driven fashion, we need to determine the effect signal for each individual: i.e., we need to define the reward function that an intelligent system will use as feedback. When looking at existing Phendo data, we observe that pain, GI, and other symptomatic experiences are frequently self-tracked by current Phendo users, both before and after self-tracked self-management strategies. More specifically, in Figure 9 we showcase the abundance of Phendo self-management trials for which there is pain-related pre-post information. These results suggest that Phendo users track sufficient information related to their self-management goals (pain results are shown here, but similar stories hold for GI and other symptoms) for the design of a meaningful reward function. We have used simple, human-driven functions in this analysis, illustrating that an RL algorithm could have sufficient reward signal to detect an effect, based only on “in the wild” data. The reward function is critical for any RL agent, as it provides the signal from which to learn how to find the right exploration-exploitation tradeoff. Although the simple functions suggest sufficient data and effects exist to learn RL policies, the user perspective emphasizes that we will need to expand on these reward functions to represent individualized metrics.

Fig. 9.

Histogram of Phendo self-management to pain trials. Phendo users engage with a wide variety of self-management strategies towards their pain management goal.

5.6. (Agent/Policy - 4a) Heterogeneity in self-management responses calls for tailored recommendations.

In the context of chronic disease self-management, the agent/policy dictates the behavior of the intelligent system. As such, we code participant discussions about developing their own self-management regimens under this category and analyze engagement and existing self-management trials with self-tracking data. This category has been separated into three codes during analysis; conversations discussing the personalized nature of self-management regimens are coded here. Users are enthusiastic about an intelligent system that could customize personalized recommendations based on their prior experiences, rather than suggesting strategies because it worked for someone else who is “similar” to them. They describe their desire for recommendations to be tailored to them based on their own data (“Based on my stuff I logged — the experiences that I’ve had so far.” [FG 2]), since each person is different.

Phendo users emphasize that a personalized approach to self-management will be required to meet their needs, particularly since they are heterogeneous in which strategies individuals use for different symptoms and health states. Across the wide array of self-management strategies, individuals turn to different activities to address particular symptoms. Some examples of strategies users enact based on various symptoms include:

You can pretty much always do deep breathing. I don’t care how much pain you are in, as a default.

[Int 1]

I’ll read and write a lot when I’m having a bad day, because it’s a way to escape and be in a space that’s more enjoyable. When I’m having a good day, I’m more of a physical person. I’ll be like, ‘Oh, I’d love to go kayaking,’ or ‘Oh, let me go work in my garden.’

[FG 7]

I went to acupuncture when I was having really severe pain before my surgery.

[FG 6]

Cooking has been helpful to me. I have a task in front of me, I can get into a flow, and it is also good because it makes me feel like I’m in control of what I’m eating, what I’m creating. I can just close my eyes, deep breath and be distracted by something else.

[FG 6]

When I’m ovulating and menstruating, soups and broths help for that.

[Int 3]

Aligned with participant perspectives, we find that current Phendo users’ responses to the same management strategy is heterogeneous, both at the population and individual levels. We showcase in Figure 10 walking to pain severity self-management pre-post effects for the population of Phendo users and specific individuals: i.e., a negatively skewed pre-post pain severity effect histogram implies that walking mostly reduces pain, while a positively skewed histogram showcases worsening of the experienced pain after walking. In Figure 10a, we observe how, although the effect of walking is null for a vast majority of Phendo users (notice the spike near the histogram origin), many users report both positive and negative effects in their pain severity within a day window. When looking at the individual (i.e., n-of-1) effects, we observe that the effect of the same self-management strategy (e.g., walking) is wildly heterogeneous for different users in the Phendo cohort. We illustrate the wide range of pain severity reported effects within a day of walking in Figure 10b–Figure 10e, where we observe how walking clearly helps reduce pain for the individual depicted in Figure 10e, but hurts the individual in Figure 10b; the same strategy does not have any impact for the individual in Figure 10c, while it is unclear on the effect for the individual in Figure 10d. We also corroborate the heterogeneity in population and individual effects for different strategies, as well as different goals (pain, GI, other symptoms) at the population level. This finding justifies the need for fully personalized self-management recommendation policies: i.e., one strategy does not suit all individuals with endometriosis.

Fig. 10.

Probability distribution of the pre-post effect of walking to pain in the Phendo cohort (a) and different Phendo individuals (b-e). The heterogeneity is evident, as the effect ranges from very hurtful to helpful for different individuals within the Phendo cohort: e.g., mostly positive pain effects in (b), null pain effects in (c), and mostly negative pain effects in (e).

5.7. (Agent/Policy - 4b) Users want an intelligent system with explainable recommendations to help understand why they were given particular recommendations and why they should try to implement the strategy.

The second code within the agent/policy category relates to the desire for explainability of recommendations. We code discussions about information participants want alongside recommendations in this category. Participants are interested in the reasons for trying a particular strategy and want to set their expectations about how long it may take to see an improvement in symptoms. All three participants interviewed about the intelligent system specifically tell us that they want explanations for recommendations provided by an intelligent system. They explain that they want to know why a strategy is being recommended, especially when the agent suggests strategies that have not yet been effective for an individual. Explanations would ideally include information about why the recommendation was chosen or tailored for the symptom or context. Participants suggest that explanations for what is offered and why would be helpful to users, especially when exploring options that may not make sense to the user or if the suggestions conflict with existing strategies that work. One person suggests that explanations might help motivate them to try the suggested strategies: “I would be more prone to try it if there was something behind it.” [Int 2]. These participant perspectives indicate that explanations could also help users build trust with the system.

Participants are interested in implementing strategies that are responsive to their contextual environment, and report that insight from an intelligent system could help their understanding and exploration. One interview participant suggests that she be able to inspect and explore self-management options that are personally tailored for her, so that she may use the system’s insights along with her own to decide on an appropriate action that is “connected to how [I’m] actually feeling” [Int 1].

5.8. (Agent/Policy - 4c) Engagement patterns suggest user interactions will be numerous and frequent enough for RL requirements.

The final code within the category of the agent/policy relates to users’ willingness to engage with self-management and an intelligent system to support them. We code conversations about engaging in self-management and interacting with an intelligent system in this category. Users tell us that they are willing to extensively self-track their illness experiences and understand that an intelligent system for self-management would require active engagement. Participants are not concerned about the burden of using such a system (e.g., “Logging observations is not cumbersome to me.” [Int 1]).

The analysis of current Phendo users’ self-tracking engagement confirms that the frequency at which they track different self-management strategies is high — we find that many users track self-management strategies with associated goals many times within their timeline. In Figure 11, we observe that there are many Phendo users who have tracked up to 10 trials of pain to various strategies: walking (Figure 11a), carbs, grains or gluten based foods (Figure 11b), or talk-therapy (Figure 11c). In Figure 12, we also observe regular engagement: self-management trials occur as often as every day or every other day (see Figure 12a and Figure 12b), but also periodically, as in weekly or bi-weekly for talk-therapy to pain trials (observe the spikes at 8 and 15 days in Figure 12c). The observed “in the wild” number of interactions and frequency provides evidence for the feasibility of an RL-enabled system and the acceptability to users, but the specifics for programming an RL algorithm in practice will need to be determined.

Fig. 11.

Histogram of the number of users per number of trials.

Fig. 12.

Number of days between consecutive trials.

Participants tell us that self-management is already part of their day-to-day routines and that they already dedicate significant time to these tasks, so committing to the self-experiments recommended by the system is seen to fit into their existing care regimens. Interview participants tell us they could easily incorporate 10–30 minutes into their daily routines, but also mention they would be open to recommendations that take one, two, or even three hours if those strategies might help their symptoms. However, participants explain that some strategies could be carried out more frequently than others.

Participants generally agree that they try self-management strategies for about a month before deciding if they work or not, although sometimes they can tell sooner, even immediately. At the longer end of the scale, participants report willingness to experiment with strategies for several months or even up to a year to develop a regimen that works (e.g., “Sometimes it was 60 days, sometimes it was a whole year.” [FG 9]). Further, participants explain that self-management will likely remain part of their ongoing care plan. While some users may use an intelligent system for a short time or on-and-off (e.g., during a flare-up) and then set it aside until the next episode where they need support (e.g., “I would probably just move on until I have another acute pain episode, then I would be likely to use it again. I think, you try a couple of things. Maybe you learned something helpful.” [Int 1]), others envision using a system to support individualized self-management long-term (e.g., “That’s something I’m going to do the rest of my life, I’ve committed to that. […] So, ongoing for sure. I don’t think I’ll ever reach my peak where I don’t need this anymore, no.” [Int 3]). These levels of engagement, while not guaranteed to result in a feasible RL algorithm, are promising from both the human and data perspectives.

Outline of Findings and Recommendations

Here, we lay out guidance for designing an intelligent interactive system that aligns with the human-data-machine insights and requirements elicited in this study. These recommendations address tradeoffs with the capabilities and constraints elaborated in this study and provide a concrete mechanism to convert the ideals articulated in the findings into real-world design decisions. The recommendations presented here provide a workable starting point for the design of an RL-based intelligent system for self-management of endometriosis. They also help to identify places where the sociotechnical gap [2] might be exacerbated by the use of ML or by the complex illness context, which provides guidance for future work and needs for innovation. These recommendations are presented in Table 3.

6. AND IMPLICATIONS FOR DESIGN

Intelligent systems — powered by AI and large volumes of patient-contributed data — have the potential to support humans in managing chronic illness [13]. However, their real potential has not yet been fully realized [10, 38]. Arguably, one barrier is limitations in existing design approaches: user-centered design, while well-established in the context of more traditional software applications, does not account for the unique constrains and inherent structures of AI models and, thus, may lead to requirements not possible to meet with existing AI algorithms [25, 127]. On the other hand, technology-driven approaches to the development of AI may lead to interactive systems inconsistent with user needs and result in limited adoption and unintended harms [18]. To move towards integrating these systems into the management of illness, we propose and implement an HAI framework — which we have termed Multi-Perspective Directed Analysis. We use MPDA to conduct a mixed-methods study to map and synthesize human, data, and ML requirements and constraints to generate design recommendations for an AI-enabled solution in the context of uncertainty in chronic illness. This approach also enables us to identify and elaborate on several sociotechnical gaps [2], where we document a mismatch between the complex, nuanced demands of real-world self-management and the rigid limitations of existing technologies. Here, we discuss implications from synthesizing needs identified by users, “in the wild” self-tracked data, and constraints of an RL approach for management of a complex, poorly understood disease.

6.1. Reflection and Implications

6.1.1. Reflection on the HAI Framework MPDA — Accounting for unique affordances of ML/AI.

Developing AI-enabled systems presents unique challenges compared to traditional software development, since AI models operate within fixed conceptual spaces with specific constraints. Unlike conventional methods, where user needs can directly inform system features, designing AI systems requires adapting to the capabilities of particular AI methods, such as carefully configuring input and output features for neural networks. This highlights the need for approaches that attend to both user needs and technical capabilities of ML.

Sociotechnical approaches, including participatory design, have long emphasized the importance of aligning human and technical factors in system design [58, 109, 130, 146]. However, the complexities introduced by AI require a broader lens to address its more complex sociotechnical dynamics [51], which traditional methods may not fully accommodate [25]. Some human-centered design approaches offer valuable practices — multidisciplinary collaboration, iterative prototyping, and continuous evaluation [9, 79, 112, 113] — to design tools that are both technically feasible while also aligning with user needs. Yet, there is significant value in a framework that explicitly accounts for human, machine, and data perspectives in the design of intelligent systems.

We proposed an HAI framework to synthesize human, data, and ML perspectives — we use high-level concepts of a particular ML approach, RL, as guiding principles to synthesize and triangulate findings across quantitative and qualitative data sources. Each “perspective” informs and complements the others, serving the analysis in important ways. First, the ML perspective allows us to account for the specific requirements and constraints of a particular ML approach, which is a novel aspect of this framework. Second, the human perspective ensures the system is grounded in real-world needs, perspectives, and practices, with qualitative data guiding the development of the final results categories. Third, the data perspective allows us to validate user-reported insights, enhance qualitative findings with a larger sample, and identify gaps in existing data. Self-tracked, “in-the-wild” data offer a unique view of actual user behaviors and practices, informing additional data requirements. This integrated approach provides a comprehensive understanding of both technical and human factors to inform the design of a real-world system.

We demonstrate that the MPDA framework provides a structured approach to the design process, offering a unique method for integrating insights from three distinct perspectives: human, machine, and data. MPDA enables a comprehensive understanding of complex tasks and contexts, such as those related to specific illnesses and care tasks. By organizing and synthesizing these perspectives, MPDA helps to reduce the complexity of the problem space, yielding nuanced insights into human experiences and translating them into actionable design recommendations. While MPDA cannot guarantee the success of an RL algorithm, it offers a systematic way to evaluate its potential by addressing key questions around system requirements, user acceptance, and data feasibility. Additionally, the framework has broader implications for HAI, re-affirming some established principles while also uncovering less explored areas. Below, we describe how MPDA connects to and advances the principles of HAI.

6.1.2. Upholding human-centered AI.

The framework facilitates human-centeredness in two key ways. First, the framework itself explicitly accounts for the perspectives of people, data, and a specific algorithm — relying on human users as experts helps to avoid dehumanizing them or creating harmful technologies [33]. Second, using this framework has enabled us to generate recommendations that foreground the human-centerdness required for a successful real-world intelligent system, rather than allowing the algorithmic perspective to dominate. MPDA introduces a principled way to integrate both human and technological perspectives systematically, ensuring that human-centered principles drive our design recommendations.

• Designing for control and user autonomy. Facilitating human intuition integrated with computational support is a fundamental goal of HAI [136]. Thus, enabling user control and autonomy are key features for designing human-centered AI systems. The MPDA framework helped identify areas where incorporating user input and human-in-the-loop aspects of RL could enhance control and offset computational challenges. In particular, in defining and constraining the action space, users wish to input preferences and limits upfront, as well as provide feedback to the system as recommendations are given (e.g., if they want to engage in a strategy or how they might want to modify it so that it fits their current needs). Existing applications provide examples for how features of interactive RL could accomplish these goals (e.g., guided exploration of the action space, or action advice where the user suggests the action they believe is optimal at a given state) [12], which can work to simultaneously meet the needs of users and the requirements of RL, while advancing principles of HAI.

• Enabling explainability. Explainability can enable personalization [141] and offer insights into managing chronic conditions [23, 157], and is itself central to HAI [59]. Explainability in this context is particularly important, due to illness uncertainty and the high demands of user involvement with an RL, so users must constantly decide when to follow system recommendations. For an explainable RL-based system, designers should avoid black-box algorithms [11, 106, 137], opting instead for model-based techniques like MDP, bandit algorithms [7], or Bayesian methods, e.g., via Thompson sampling based flexible solutions [149] or via hierarchical models that learn both within and across individuals [145]. A small but quickly growing body of literature highlights explainable RL approaches specifically — largely applied in dissimilar contexts like gaming, computer vision, and robotics, but which could be adapted to this personal health context [49, 69, 85, 101, 123, 155].

• The importance of safety, privacy, and trust. Although safety, privacy, and trust are critical principles of HAI systems [116], participants in our study did not identify these as important features. Rather than not being important to users, this likely reflects a dangerous assumption that these aspects are already addressed. The absence of these concerns in our study highlights a limitation of our approach, emphasizing the need for researchers to proactively consider critical HAI elements that may not be identified by users.

Additionally, we identify several key insights into less commonly emphasized aspects of HAI that arose in the context of endometriosis. Taken together, the user needs and design recommendations generated in this study are consequences of the complexity and unpredictability of managing endometriosis. This is not a given — other aspects of the disease emerge as most prominent in different settings. When seeking a diagnosis, the invisible nature of the disease is prominent [77]. In the context of shared decision making, research documents needs related to the stigma of the disease, privacy around disclosures, and managing emotions associated with illness [120]. This study’s findings related to illness complexity and uncertainty resulted in specific requirements and recommendations. We identify opportunities and challenges of using ML in a design space with nuanced, holistic representations of patient data.

• Personalization for individualized, internal context. We document wide variations in illness experiences and identify diverse approaches to self-management. While the need for personalization in chronic disease self-management is well understood [55, 142], it presents a considerable challenge for ML and AI. This study documented needs and opportunities for personalization from the perspective of RL, specifically, the need to individualize the action space, state space, reward, and the agent/policy. Translating complex human experiences into representations that can be used by RL remains challenging, requiring reconciliation between human and algorithmic inputs. However, personalized user modeling has been explored in both HCI [65] and ML research [154]. There are also new ML approaches to guide the development of personalized reward functions [97]. Furthermore, considering the prospect of designing a personalized ML-enabled system from the data perspective suggests that individuals are willing to engage in self-monitoring, which could support the application of RL. At the same time, the study also identified the need to constrain the action space, providing several actionable strategies to address this challenge.

• Embodied health experiences. Individuals’ perceptions of their illness experiences are multi-faceted, complex, unique, and sometimes subjective, often relating to nuanced, embodied sensations. At the same time, computational tools require tractable representations of health status to evaluate if self-management strategies offer improvements. In an RL-enabled system, the state space must account for and represent these complex and subjective health states, while the reward mechanism needs to assess the effectiveness of strategies based on individuals’ unique experiences and evaluations of their health. Thus, interactive systems will need to translate these embodied health experiences into something that could be quantified, measured, and compared across time. Feminist scholars [122] and information theorists [24] have discussed how these properties of data require simplifying reality and imposing classifications that may not align with an individual’s experience. The gap between the messy, embodied tracking needs of menstruators and existing technologies has been previously documented [119]. In Data Feminism, D’Ignazio and Klein [54] explain that data serve to render the health experiences of individuals visible, from the patient perspective, and that to be useful, it is necessary to create machine-readable representations. However, they also warn that care must be taken to avoid flattening experiences or dehumanizing users. They recommend representations move beyond simplistic binaries and categories to capture the rich nuance and complexity of lived health experiences, and to contextualize information, rather than relying on the notion that data can “speak for themselves” [54]. Addressing challenges of creating nuanced representations of complex, embodied data requires creative, human-centered technical innovations, which might involve capturing multi-modal data (e.g., voice, video, or artistic creations) [74, 89] and applying digital phenotyping methods to generate rich health representations [114]. These approaches should actively engage users in categorizing and labeling their data to ensure alignment with their lived experiences.

• Real-time, real-world context. An intelligent interactive system for self-management must adapt to individuals’ needs and preferences, respond to life circumstances and uncertainty, and provide dynamic recommendations that account for real-time context. Thus, representations that account for real-time, real-word context are important for defining the state space and for learning individualized policies — i.e., tailoring recommendations, recognizing triggers or patterns, and enabling explainability about why a recommendation was made [3]. Disregarding context results in fragmented, shallow representations of users, detached from their non-clinical experiences and broader environments [138], which can negatively impact patient outcomes [19]. The importance of incorporating context is well-established. Context-aware solutions that use situational information that are relevant to a user for a particular task [53] can enable personalized recommendations [151] and facilitate behavior change [83]. Context-sensitive recommender systems and personal informatics tools further highlight how integrating real-world circumstances supports tailored interventions [3, 125, 151]. Nevertheless, in the context of RL for a complex chronic illness, it will be challenging to capture, represent, and use complex, real-world, real-time context in ways that are not at odds with an RL’s requirement for a constrained state and action space.

These key challenges represent examples of Ackerman’s sociotechnical gap [2], where the shortcomings of technical systems cannot fully support the flexible, nuanced, and contextualized human experiences and activities required in the task of self-management. People with endometriosis describe complex and unpredictable illness experiences, which are difficult to translate into solutions compatible with existing technical capabilities. In this work, we not only articulate where these gaps arise but also present recommendations for solutions that partially solve these problems with known tradeoffs and provide work-arounds, in particular that enhance human-centered elements of control and autonomy.

6.1.3. Implications for the state of care for endometriosis patients.

Unlike conditions with clearly established and well-defined treatment guidelines, people with endometriosis lack reliable, evidence-based treatments, leaving them to navigate their care through self-management [132, 133, 158, 159]. Yet despite this uncertainty, self-management helps empower individuals with endometriosis in their own care, mitigate their symptoms, and cope with the burden of their illness [120]. Even though recommendations provided by intelligent systems may be limited or ineffective, the trial-and-error process of finding strategies that work for an individual remains an essential aspect of care. As the standard of care advances, patients would benefit from systems that support this exploratory process. Given the documented complexity and uncertainty surrounding endometriosis, technology that can help patients identify patterns and discover new management strategies has significant implications for improving the state of care for endometriosis.

We document that individuals already engage in a complex, personal trial-and-error process and are interested in computational tools to support their self-management. Even in the absence of an AI, individuals already log sufficient self-tracked data to suggest that an RL-enabled system could be effective in assisting endometriosis care. We advocate for solutions that offer computational support for both self-experimentation with strategies already identified (i.e., evaluating if strategies are effective) and for discovery (i.e., recommending new strategies). Such tools could empower individuals in their care, support them in developing effective individualized management regimens, and improve quality of life. And while HAI scholars warn against falling into the “solutionism trap” that frames ML as a simple solution to a difficult problem [107, 134], our research affirms that ML may be an appropriate approach to this complex, unresolved problem.

We also situate our work within a feminist research perspective [16, 17, 54], particularly recognizing that individuals coping with a burdensome women’s health disease like endometriosis must contend with issues of bodily autonomy, health equity, and society’s disregard for women’s health concerns [43, 118, 132, 156]. The deeply personalized and embodied experiences of endometriosis are often minimized and neglected, underscoring the need for solutions that center the highly contextualized and varied symptomatic manifestations of the illness. We call for human-centered technologies that leverage self-tracking data to elevate the perspective of users, enabling complex, contextualized representations of illness that can facilitate solutions to support users in their care tasks. We also emphasize the importance of innovating methods to translate individual, nuanced experiences of illness to tractable representations that can be leveraged in intelligent systems to advance personal health goals.

6.1.4. Promoting generalizability.

We provide an illustrative case of eliciting the needs of users within the constraints of a specific ML approach and its computational requirements and translating these into actionable recommendations for the design of an intelligent system. While this framework has been successful in answering the research question within the context of self-management for a particular complex chronic illness, the same approach could be adapted for other contexts that need to attend to human, data, and ML perspectives at the same time. We expect that MPDA will help to integrate human needs, challenges, and aspirations with ML and computational feasibility across a range of scenarios (i.e., other illness contexts, tasks, and ML paradigms), leading to design requirements that are not only desirable by end-users but also achievable. For example, if we were to follow the same process in a different disease where the set of actions was very well known, then individuals would not need to define their own strategies. Thus, aspects of the RL, including the action space and policy learning, could be population-based. For a different task, the high-level concepts would be operationalized differently. In a different ML model, the high-level concepts used as organizing principles would be different. Our work has illustrated that the framework can also lead to the foregrounding of human-centered principles in the design of AI-enabled systems, which is also generalizable to other contexts. In a different disease, we might find that control and autonomy would need to be augmented in very different ways than is the case in endometriosis, e.g., individual-level learning may not be needed but cultural factors may impact the options that users want to control. MPDA also proved valuable in documenting and elaborating the sociotechnical gaps [2] that arise in the context of designing intelligent systems for a complex human task like self-management, which could be extended to other tasks, diseases, and algorithms. At the same time, we also recognize limitations in this approach, for example the important aspects of safety, privacy, and trust were not elicited from end-users as design requirements.

7. CONCLUSION

This study puts forth a novel HAI framework, MPDA, to design an intelligent system for providing adaptive recommendations to support experimenting with self-management and developing personalized regimens. This framework simultaneously attends to both human and technical requirements and constraints. By mapping the needs of human users, a specific ML approach, and data that have been tracked “in the wild” independently of any algorithm, we uncover important insights and design recommendations that are likely to be feasible in real-world settings.

There is a need to help individuals use their health data to create personalized self-management regimens, especially for chronic conditions with substantial complexity and uncertainty, like endometriosis. Intelligent systems could help fill this gap by providing adaptive recommendations. Our investigation suggests that an RL-powered system is a promising solution for real-world self-management. Findings provide useful guidance for the development and deployment of an RL system for experimenting with endometriosis self-management strategies. Beyond providing interesting insights and design guidance, this study importantly contributes a novel framework that could be adapted by other researchers who need to attend to specific human and technical requirements simultaneously. Future work will address how to operationalize the key parts of the RL agent in the context of an intelligent system, such as data-driven learning of user states and composite reward functions, for developing individualized endometriosis self-management regimens.

CCS Concepts: • Human-centered computing → User studies; • Computing methodologies → Reinforcement learning; • Applied computing → Health informatics.

A. APPENDIX

A.1. Additional Details on the Qualitative Methods

In this study, we conducted interviews with three key informants who were active Phendo users, asking questions specifically relating to the use of an interactive system for self-management. We also re-analyzed focus group transcripts from prior studies. In total, we analyzed transcripts from three in-depth interviews and re-analyzed 10 transcripts from focus groups with 48 participants from prior work. A summary of the methods for each of these qualitative components is included here, but additional details for the focus groups can be found in the relevant publications. The demographics for participants across the interviews and focus groups are presented in Table 4. All interview and focus group guides are included below.

Table 4.

Demographics of all participants

	Interview	2019 Focus Groups	2016 Focus Groups
	n = 3	n = 21	n = 27
Age mean (range)	34 (31–41)	32 (21–41)	38 (27–60)
Race or Ethnicity
White	1 (33%)	14 (67%)	21 (78%)
Black	1 (33%)	6 (29%)	2 (7%)
Latina	1 (33%)	2 (10%)	2 (7%)
Asian		1 (5%)	2 (7%)
Years Diagnosed
Less than 5	1 (33%)	12 (57%)
5 to 10	1 (33%)	6 (29%)
10 or more	1 (33%)	3 (14%)
Age at Diagnosis
mean (range)	26 (15–37)		29 (18–40)

Interviews.

In-depth interviews with key informants were conducted in 2020 to help further illuminate the needs of patients actively managing endometriosis symptoms in their day-to-day lives. Active Phendo users were recruited from social media. Eligibility criteria included adults with recent endometriosis symptoms and use of the Phendo app. The sessions lasted sixty minutes and individuals were compensated with a $25 pre-paid card for participating. In-depth discussions centered around the needs and desires of users for a tool to support self-management of endometriosis, and sought to elicit the constraints and acceptability of an AI-enabled tool for experimenting with and identifying self-management regimens that are personalized and effective for individuals. In total, three participants were interviewed. Participants were all women ranging in age from 31 to 41 (34 years mean). Time since diagnosis ranged from 3 to 16 years (8 years mean).

2019 Focus Groups.

Focus groups that were conducted in 2019 were initially used to elicit design needs for tools to support care and self-management of endometriosis. Current endometriosis patients were recruited using social media and flyers hung near clinics. Eligibility for participation were English-speaking adults with a diagnosis of endometriosis, having experienced endometriosis symptoms in the past three months, and having engaged in care for endometriosis in the past year. The sessions lasted ninety minutes and individuals were compensated with a $25 pre-paid card for participating. Semi-structured focus group discussions centered around the ways that patients assess their own health status, practices around self-managing their condition outside of the clinical context, how they communicate with their care teams, and the ways that they evaluate if they are making progress towards their goals. In total, five groups were conducted with a total of 21 participants. Participants were all women ranging in age from 21 to 41 years old (32 years mean). Time since diagnosis ranged from less than one year to 21 years (5 years mean). Additional details on the primary study, including findings, can be found in the original publication [120].

2016 Focus Groups.

Focus groups that were conducted in 2016 originally informed the design of the Phendo app. Individuals with endometriosis were recruited through convenience sampling through the Endometriosis Foundation of America email listserv, through social media, and through flyers hung near gynecological clinics. Eligibility for participation included adults with an endometriosis diagnosis through laparoscopic surgery. The sessions lasted ninety minutes and individuals were compensated with $25 in cash for participating. In these, semi-structured focus group discussions prompted participants to discuss the domains of health relevant to their illness experience, how they care for their illness and manage their symptoms, and what self-tracking would help them with. In total, five groups were conducted with a total of 27 participants. Participants were all women ranging in age from 27 to 60 years old (38 years mean), and age at diagnosis ranged from 18 to 40 years old (29 years mean). Additional details on the primary study, including findings, can be found in the original publication [98, 99].

The interview and focus group guides for all interviews and focus groups analyzed for this study are included below.

Interview Guide

We have developed the Phendo app, an instrument to help endometriosis patients self-track a range of variables related to their condition. Now, we want to determine what would be useful for patients to use to support self-management.

The purpose of today’s session is to help us develop tools that will work best for you. We will ask you about how you currently self-manage your condition, and what kinds of tools you imagine would be helpful. Remember there are no wrong answers (we just want to hear your experiences and opinion) and you won’t hurt our feelings! Oh, and please make yourself comfortable (stand or sit, snacks, bathroom, etc)…

First we are interested in your current approach to self-management and experimentation… By self-management, we mean all the different actions that you take to help you deal on a day to day basis with your symptoms, whether you know they work for you or you are experimenting with.

1. So, let me ask you: given the definition we just gave, would you say you self-manage your endometriosis?

   • If not, why not? Are you interested in trying different options?
2. Can you think about the last time you tried to self-manage your endometriosis, to improve your symptoms? (examples: diet, exercise, hormones, rest, etc)

   • Can you tell me about the process of deciding to use the strategy, incorporating it in your life, and how you decided if it was working or not?
   • Probes: How did you choose this self-management technique? How many did you experiment with at a time? How many were you considering in your head?
   • How long did you try for?
   • How did you keep track?
   • How did you determine if it works? When did you expect to see any effects? Did you expect to see effects more in the short-term or long-term?
   • When you try a new strategy, are you looking for holistic improvements in your health or specific improvements?
3. What was your approach to developing your own self-management regimen?

   • Has your approach to self-management and/or experimentation changed over the course of your illness? Depending on other factors in your life? How have you adjusted according to changes in your life?

OK now let’s pretend that we’ve developed this cool tool and we’ve asked you to use it…. The app might ask you to log your experiences from a few days to a few weeks or maybe months to learn about you, and then the app would enter into an experimental phase where it gives you strategies to try for a few days, weeks, or months, and then finally the app would know enough about you to give you real suggestions but not until after it gets a chance to know you—so we want to figure out what this app might look like, what you would be willing to do and what would be helpful to you…

1. Imagine a tool for experimenting with self-management has been created like described above (observations -> trying things -> learns recommendations)…. We are going to spend the rest of the interview talking about this, but do you have any first impression? How might this be useful to you?

2. (If described approach to self-management above): Thinking about the self-management strategy you described before… How might this tool change the way you self-manage your condition? (If no strategy described above): How might a tool like this support you in figuring out your own self-management routine?

   Logging & Experimenting

3. How would you feel about logging your data in the app for a while before it gave you useful recommendations? (for example, a meal planning app that uses similar methods requires 40 meals logged before providing first recommendation)

4. How much would you be open to experimenting?

   • One strategy at a time or multiple ones? Why? How would it help you? What types of strategies would you be interested in and why would they make sense together or on their own? Do you want to know about all of the options or just whichever are suggested as they come up?
   • During the exploration phase and for the app to learn about you and how you respond to different self-management strategies, the app might suggest a wide range of strategies that might not make sense to you. By a wide range, I mean for instance when learning about your exercise patterns, would provide recommendations that might seem like they won’t help (2 minutes walking per day) to they are just might not be feasible for you (sprint back and forth for an hour). I’d like to understand from you what types of recommendations you would be open to or feel like you would not carry out at all: go for very fast run after work for 1 hour (why? Is it time, is effort required based on your health status, is it just not motivating?; acupuncture 4 times per week (why? Is it too much time? Too far? Don’t know where to find an acupuncturist? Too expensive?); Marijuana? (why? Legal?)
   • You might have already a routine for your self-management, where you know some specific strategy works for you (for instance, stretching every morning), how would you react to recommendations from the app that disrupt or cause you to deviate from that routine? Why?

5. How much of a commitment are you willing to make for self-management? Every day? A few times a week? 10 mins? 1 hr? (WHY?) How many weeks or months are you willing to try a new strategy? How long would you want to keep trying strategies?

   Recommendations (when, where)

6. Mechanisms underlying adherence and retentions

   • Would it help if you made a commitment with the app to set aside time for self-management, without knowing what the specific strategies will be in advance? Why? How much heads up in time would you need when getting these recommendations (eg, the day before, or a prescription for the week)? Why?
   • What kind of feedback or other messaging would encourage you to stick to this? (positive encouragement when log success, when log failure; social support)… Content, frequency, ways of communicating of message?
   • Would knowing that something worked for other patients similar to you be useful?
   • What would turn you off or make you stop using the app?
   • Mechanisms underlying health conditions
   • How does the way you are feeling would change your response to a recommendation? For instance, if a specific day where you are supposed to try a recommendation you have severe bloating, how would you respond to the recommendation? Why?

   Recommendations (what)

7. What kind of information about the recommendations would you expect to see?

8. How do you imagine this information could be presented? Would you like it to be more personal and encouraging, or more clinical and neutral?

9. Do you want an explanation of why such a message is sent out? Would you feel that be more engaging, encouraging?

   Outcomes/rewards

10. When would you like to provide feedback for a recommendation you receive?

11. How long until you expect to see results? What kind of results are you looking for?

12. What kind of messages would be helpful to keep you on track of a self-management tool?

Additional questions: Before we wrap up, anything else you think we should know? Do you have any worries about using a tool like this?

Focus Group Guide — 2019 Focus Groups

Introductions

You are all here because you have endometriosis and have had symptoms for at least the past three months. We are going to talk about them probably for most of our time here! For now, just as a quick icebreaker, let’s go around the room and share our names, can you tell us how long you’ve had symptoms for, how long you’ve been diagnosed. If you have one fun fact about endo, please feel free to share!

Self-Assessment of Health Status

Our first set of questions to you is about your symptoms and your general health status, and in particular, how you assess how well you are doing beyond day-to-day experiences. Here, and for the rest of the session, we are looking into health status as a whole, not only aspects that are endometriosis-specific. There are two reasons for this. First, we know that many patients, and maybe some of you here, have other chronic conditions in addition to endometriosis, and our goal is to build tools that support you as a whole person. The second reason is that it is hard to know what symptoms or signs are due to endometriosis or something else. So, let’s assume we want to know about your general health status as a patient of endometriosis.

1. I can imagine a provider asking you this question of “how have you been feeling in the past three months,” how often do you ask yourself this question?

   • Daily? Weekly? Some specific event triggers the question? Why or why not?
   • How do you interpret this question of how have you been doing? (overall functionality, emotions, specific symptoms that you know are particularly important for you to monitor)?
   • Typically how far back in time would you go to assess your health status and reflect upon it? (e.g., Would it make more sense for you to ask “in the past week, how have I been feeling?” or rather on the other extreme “in the past year, how have I been feeling?”)
2. What part of that assessment and answering the question of “how have you been feeling” is easy for you? What is hard?

   • Are there specific symptoms that are easier than others for you to assess status? Any that are harder? (e.g., Maybe because of time over which they occur or because they change a lot or some other patterns?)
3. Do you use any technology to help you recall or reflect back on your health status through time?

   • If so, what tools do you use and how do you find them useful? Why? Why not?
   • If not or if you are dissatisfied with current tools, what are the functionalities you’d like to see in technology that could support you in reflecting back on your health and assessing your status through time?

Patient-Provider Communication

For the rest of this session, we are going to focus on you and your care team. By care team, we mean healthcare providers who help manage your endometriosis symptoms. Examples of care team members could be a surgeon or a physical therapist.

• 1
  Who does your endometriosis care team consist of?

  • Do they communicate with each other?
  • What does a typical visit look like?

Now, we are interested in how you and your care team communicate. Because we want to build tools that help with shared decision making, that is coming to a treatment decision that make sense for you as a patient and your provider as well, we are very interested in the kind of role you play in your care and the role your provider has. For instance, some patients see themselves as their own advocates, or the primary communicator between the different providers on your care team, if you have one.

• 2
  What is the process for decision-making around your care?

  • What role do you play in your care? Why?
  • How satisfied are you with your role? Why? Why not?
• 3
  How do you convey your goals to your providers? What about your preferences? An example of goal would be “I want to be pain free for a week.” An example of preference is “I’d rather not go on hormonal therapy because I am afraid of the side effects.”

  • Do you have a specific time when you meet with your provider to go through your goals and preferences? Or do you let it be more organic during encounters? (e.g., “if it comes up” or “if my doctor asks me”)
  • How do you tell if your provider has heard you? Specifically, when it comes to treatments, how do you tell if your provider has taken your preferences and goals into account?
  • Are you satisfied with how you convey your goals and preferences? If not, what would you want to see differently? (e.g., have enough time/opportunity other than during encounter to convey them)
  • How would technology help you with conveying your goals and preferences with your care team? (e.g., check list for patients to review during encounters?)
• 4
  What works and what is missing with your communication with your provider?

  • Does it differ among the different members of your care team?
  • How do you ensure your concerns are heard?

Self-Management of Health

Now, we want to move the discussion away from assessing your health status, and thinking about self-management. By self-management, we mean all the different actions that you take to help you deal on a day to day basis with your symptoms, whether you know they work for you or you are experimenting with.

1. Would you say you self-manage your endometriosis?

   • Why or why not?
   • What factors make you decide to try it or not? Other endometriosis patients’ experiences? Trusted source? Feasibility with your daily life? (i.e., cost, time, effort)
2. How many different self-management strategies would you say you experimented with in the past year?

   • What were they?
   • How do you evaluate if a specific strategy works for you?
   • How long do you typically give a specific strategy a go before deciding whether it is helpful to you or not?
   • How do you stick with them? What are the challenges with doing so? What would help you stick with them better?
   • Thinking about technology, how would you ideally use it to help you evaluate a self-management strategy? How would technology help with this? Think of Phendo or any other self-tracking app, what is missing from it that would help you determine whether a specific self-management strategy work for you?
3. How do you find out about different self-management strategies?

   • Reading? Online? Social media, blogs? Your doctors? Your social network (friends, family)? Other patients?
   • Would you say there are a lot of self-management strategies for endometriosis? And if so, you have no problem identifying new ones, or on the contrary do not know where to find them?

Before we wrap up, is there anything else you think we should know? Anything else we should have asked?

Focus Group Guide — 2016 Focus Groups

Symptom tracking – Our first set of questions to you is about your symptoms of endometriosis and whether you have any method to track them. By tracking we mean, keep recording them say in a notebook, or in your calendar, or using an app, it could even be in your diary. So let’s start. Again, remember, no right or wrong answer!

1. How have you tracked your symptoms in the past? What symptoms have you tracked? Anything other than symptoms per se that you think was relevant to track with respect to endometriosis?

2. If you tracked, what were the reasons you decided to track, what did you like about tracking, what did you not like? How did you track and how often?

3. If you didn’t track, why not? What makes tracking your symptoms difficult?

Endo app.

We are now moving to a set of questions about what this mobile app we are working on would look like and how useful it could be for you.

1. Do you use any health apps? If so, which ones? What do you like or not like about these apps?

   • Do you use any apps to help you with tracking your menstrual cycle?
   • What do you like or not like about these apps?
2. Imagine there is an app that can track your symptoms throughout your cycles. How would a tool for tracking your symptoms of endometriosis be useful to you? (Write categories on flip chart and write down people’s suggestions)

   • What would want out of such a tool?
   • What would you definitely NOT want in the tool we are proposing?
3. Thinking about endometriosis as a disease that is not well understood by the medical world yet, what do you think would be important aspects of your disease that need to be tracked in the app? There is a wide range of ways in which endometriosis manifests itself in women, so please tell us what makes sense for you.

   • Pain? Intensity? nature of pain? Location? (could be abdominal pain, could be pain during sex)
   • Digestive issues? Bloating? And where?
   • Sleep issues?
   • How you feel (physically, mentally, emotionally)?
   • Any “weird” symptoms you don’t usually tell your doctor about but think might be endometriosis related? If so, what are they?
4. Thinking about how you manage your endometriosis symptoms, treatments, and how they affect your life… what features would you like in an application? (Write categories on flip chart and write down people’s suggestions)

   • Educational or information resources

     • Advice? From clinicians, peers?
     • Treatment options
   • Reminders for medications and/or treatments?
   • Methods to self-reflection? Thoughts diary
   • Methods for changing your mood?
   • Experimenting with medications

5. Off all the features we discussed today (make sure list is visible) what are the one or two most important ones an endometriosis app must have?

Contributing to research, citizen science

1. What is your current medical understanding of the disease?

2. Do you feel like you understand what is going on with your disease and why it affects you the way it does?

3. What would you like to know about the disease? Either for yourself, for science.

   • Who is affected and why?
   • What are the different types of endometriosis, and which type am I?
   • What happens to patients like me?
   • What works and what doesn’t work for me and patients like me?

4. What do you think that patients with endometriosis (you) could contribute that is novel information to the scientific community about endometriosis?

5. Some people participate in citizen science, where they partner with researchers to answer challenging questions. Would you be interested in contributing to research on endometriosis, for instance, through working out this app together?

Do you feel there is anything else you would like to discuss?