Prioritized health outcomes for adolescents and young adults with acute lymphoblastic leukemia

Key Points

• •Two multidisciplinary guideline panels prioritized 34 outcomes for 32 research questions related to ALL in AYAs.
• •The prioritized outcomes can inform future primary studies, systematic reviews, and guidelines.

Visual Abstract

Abstract

The American Society of Hematology convened 2 multidisciplinary panels to develop guidelines for the management of acute lymphoblastic leukemia (ALL) among adolescents and young adults (AYAs). The objectives of this study are to (1) describe the process for selecting and prioritizing outcomes for 32 research questions relevant to the guideline development, (2) summarize the prioritized outcomes, and (3) describe the frequency of reporting of those outcomes in existing studies of AYAs with ALL. The panels prioritized outcomes for each of the 19 questions on frontline management, 9 on the management of relapsed/refractory disease, and 4 related to both topics. A 3-step process was used: (1) initial identification of outcomes, (2) online survey of all panel members to rate each outcome’s importance, and (3) iterative discussions among the panels to finalize the prioritized outcomes. We examined the frequency with which each prioritized outcome was reported for systematically reviewed research questions. The panels prioritized 34 unique outcomes across questions (median, 7 outcomes per question). The most common outcomes were overall survival (27 questions [84% of questions]); relapse-, event-, disease-, or progression-free survival/relapse (27 questions [84%]); and quality of life (26 questions [81%]). Across 16 systematic reviews for which we found at least 1 study, each prioritized outcome was reported by a median of 25% (interquartile range, 20-48; range 0-79) of the studies. The outcomes prioritized by multidisciplinary guideline panels can inform future primary studies, systematic reviews, and guidelines on ALL, other blood cancers, and AYAs. Future research should involve the development and adoption of a core outcome set for ALL.

Introduction

Acute lymphoblastic leukemia (ALL) is a heterogeneous group of diseases. The disease processes and treatments can affect a range of patient outcomes, and the importance of each outcome may differ among those with ALL, their families, caregivers, and others, depending on patient age, disease severity, treatments being considered, and numerous other factors. Adolescents and young adults (AYAs) represent a transitional group between pediatric and adult patients, and AYAs may experience a greater incidence of unfavorable cytogenetic abnormalities than children.¹ Therefore, AYAs with ALL represent a group of patients for whom relevant outcomes may be particularly distinct from other subpopulations of patients (eg, children and older adults) with ALL.

In 2023, the American Society of Hematology (ASH) convened 2 multidisciplinary panels to develop evidence-based clinical practice guidelines for the management of ALL among AYAs. The panels addressed a wide range of topics related to either frontline therapy of ALL² or the management of relapsed/refractory ALL.³ During many phases of guideline development, the 2 panels and their respective cochairs regularly met together. To inform the development of these guidelines, ASH contracted experts in evidence synthesis and guideline development methodology to conduct systematic reviews addressing prioritized research questions,⁴ assess the certainty of evidence for prioritized outcomes, and support the guideline development. As part of this process, the guideline development panels and the methods experts engaged in extensive iterative discussions to prioritize outcomes for the systematic reviews.

Objectives

We aim to (1) describe the process used for selecting and prioritizing outcomes for 32 research questions on the management of ALL in AYAs, (2) summarize the prioritized outcomes for these research questions, and (3) describe the frequency of reporting of those outcomes in existing studies on AYAs with ALL.

Methods

Composition and leadership of guideline panels

Panelists were nominated (or self-nominated) by ASH members and were vetted with the aim of forming a diverse panel of experts with broad representation and balance with respect to sex, experience (junior and senior members), practice (inclusive of both pediatric and adult hematologists), and lived experience with ALL. Table 1 summarizes the composition of the 2 guideline panels, which comprised 46 panel members in total. Most panel members (38 [83%]) were hematologists/oncologists, including 25 practitioners focusing on adult patients and 13 practitioners focusing on pediatric patients. The panels also included adult people with lived experience of ALL as AYAs (2 on each panel), clinical pharmacists (1 on each panel), social workers (1 on each panel), and a clinical psychologist (on the relapsed/refractory panel).

Table 1.

Composition of guideline panels

Characteristic	Frontline therapy panel (n = 25)		Relapsed/refractory therapy panel (n = 22)		Both panels (N = 46∗)
	n	%	n	%	n	%
Expertise
Patient representative (lived experience)	2†	8	2	9	4	9
Hematologist/oncologist: adult (primarily)	16†	64	10	46	25	54
Hematologist/oncologist: pediatric (primarily)	6	24	7	32	13	28
Clinical pharmacist	1	4	1	5	2	4
Clinical psychologist	0	0	1	5	1	2
Social worker	1	4	1	5	2	4
Sex
Female	12	48	10	46	22	48
Male	13	52	12	54	24	52
Professional experience (after training, for trained experts)
n	24		20		43∗
1-4 y	6	25	4	20	10	23
5-9 y	7	29	6	30	13	30
10+ y	11	46	10	50	20	47
Geographical region of practice (among trained experts)
n	24		20		43∗
United States, Northeast	2	8	4	20	6	14
United States, Midwest	6	25	3	15	9	21
United States, South	10	42	6	30	16	37
United States, West	1	4	3	15	4	9
Australia	1	4	0	0	1	2
Canada	2	8	2	10	3	7
India	1	4	0	0	1	2
Italy	0	0	1	5	1	2
Saudi Arabia	1	4	1	5	2	5
Type of practice (among trained experts)
n	24		20		43∗
Academic	22	92	18	90	39	91
Community	2	8	2	10	4	9

^∗One adult hematologist/oncologist served as an ex officio member on both panels.

^†One patient representative on the frontline therapy panel is also an adult hematologist/oncologist.

Half (48%) of the panel members were female. Among the 43 trained experts, almost half (47%) had >=10 or 10+ years of work experience after clinical training, and almost one-quarter (23%) had 1 to 4 years of experience. Given that the guidelines were being developed by ASH, most experts (35 [81%]) were practicing in the United States, with the largest representation among these in the South (16 [37%]). Other represented countries included Australia, Canada, India, Italy, and Saudi Arabia. Most experts (39 [91%]) were practicing in academic settings. The overall characteristics of the 2 panels were largely comparable, except that the frontline therapy panel had a larger percentage of adult hematologists/oncologists than did the relapsed/refractory therapy panel (64% vs 46%).

Each panel was chaired by 2 hematologists/oncologists (of the 4 chairs, 3 were based in the United States and 1 in Canada). In addition, 1 hematologist/oncologist (based in Canada) attended the meetings of both panels as an ex officio member and liaison to harmonize activities of the panels, minimize unnecessary duplication, and contribute additional methodological expertise specific to evidence synthesis and guideline development in hematology/oncology.

ASH approach to the management of financial conflicts of interest

Per policy, ASH does not accept direct support from for-profit health care companies for the development, printing, publication, or distribution of its clinical practice guidelines. To manage conflicts of interest among individuals serving on specific panels, individuals are required not to accept direct payments or transfers valued at >$7500 per year or paid speaking (ie, speakers bureau) positions from for-profit health care companies with financial interests in the topics of the guidelines. Before joining the panel and at the beginning of each panel meeting, an ASH representative reminded all panel members of this policy.

Scope of topics addressed

The panels prioritized outcomes for 32 research questions, which included 19 questions related to frontline management only, 9 related to the management of relapsed/refractory disease only, and 4 related to both. Supplemental Table 1 lists all 32 research questions for which outcomes were discussed. In a subsequent guideline development phase, the panels prioritized 17 research questions for full systematic review.

Process for prioritizing outcomes

The panels followed the following 3-step process for prioritizing outcomes.

Step 1: initial identification of outcomes

The panel chairs, ex officio member, and 2 methods experts initially developed lists of outcomes for each research question. This was based on clinical expertise and teleconference discussions, informed by a preliminary long list of outcomes identified by the HARMONY Alliance, whose goal is to develop core outcome sets for 7 hematological malignancies, including ALL. A core outcome set is an agreed standardized minimum set of outcomes that should be measured and reported in a specific area of health or health care.⁵ We are not aware of a HARMONY Alliance or other publication describing a final core outcome set for ALL.

Step 2: survey of panel members

Once the preliminary list of outcomes was developed for each research question, we designed an online survey for members of each panel using QuestionPro (Austin, TX). Panel members rated the importance of each outcome for each research question in their panel on a scale of 1 to 9, with higher scores indicating greater assigned importance. This scoring system is recommended by the Grading of Recommendations Assessment, Development and Evaluation (GRADE) system, in which scores of 7 to 9 denote that the outcome is critical for decision-making, 4 to 6 denote that the outcome is important but not critical, and 1 to 3 denote low importance.⁶ If a panel member did not have an opinion regarding the importance of an outcome for a given research question, they could select “No opinion.” In addition, panel members had the option to provide additional free-text comments or suggestions regarding the clinical questions or the outcomes. The surveys were available for completion over 2 weeks.

For each question, we calculated the median rating (on the scale of 1-9) that each outcome received. Outcomes that fulfilled our predetermined threshold of being critical to decision-making (median rating, 7 or higher) were, according to the GRADE system,⁶ preliminarily categorized as prioritized for a given research question, and outcomes with median ratings of 6 or lower were categorized as preliminarily nonprioritized.

Step 3: iterative discussions to finalize prioritized outcomes for each question

After the survey, each panel met iteratively via teleconference to discuss the survey results and finalize the lists of prioritized outcomes for the panel’s research questions. A methods expert began the discussion about each question by providing the ranked list of outcomes rated as critical to decision-making, along with a listing of the nonprioritized outcomes and other outcomes suggested during the survey. The median score for each outcome, along with the percentage of respondents who rated the outcome as critical, was presented. A discussion ensued for each question, during which panel members could make a case against a preliminarily prioritized outcome or for a preliminarily nonprioritized outcome. During all surveys and panel discussions, the cochairs and ex officio members actively sought input from panelists with lived experiences. As appropriate, and based on consensus, each list of prioritized outcomes was refined by the panels, the chairs, and the ex officio member.

Grouping of outcomes

After the prioritization of all outcomes, we mapped individual outcomes to a taxonomy of outcomes developed by Dodd et al that is recommended by the Core Outcome Measures for Effectiveness Trials Initiative.⁷ This taxonomy groups outcomes into 5 broad core areas: mortality/survival, physiological/clinical, life impact, resource use, and adverse effects.

Frequency of reporting of prioritized outcomes in existing studies in ALL in AYAs

The guideline panels prioritized 17 of the 32 research questions for systematic reviews (11 relevant to frontline therapy and 6 relevant to relapsed/refractory disease therapy). We followed standard methodology for the systematic reviews. We included studies of treatment options that either were evaluated exclusively in AYAs, among samples of patients whose mean or median age fell within the range of AYAs (aged 15-39 years) or near the AYA range (either aged 10-14 or 40-65 years); exact criteria depended on the number of potentially eligible studies for each research question. All research questions were restricted to publications since the year 2000; research questions with more available studies were restricted to more recent publications (since 2014). We examined how frequently the prioritized outcomes were reported by studies included in the reviews.

Results

Outcomes identified

In step 1 (the initial identification of outcomes), we identified 37 unique outcomes across all 32 research questions. By core area, these included 4 mortality/survival outcomes, 17 physiological/clinical outcomes, 5 functional outcomes, 6 resource use outcomes, and 5 adverse effect outcomes. The median number of initially identified outcomes per research question was 10 (interquartile range [IQR], 7.5-13.5; range, 3-18). All question-specific outcomes identified in step 1 were included in step 2 (online survey). In step 3 (iterative discussions), 41 unique outcomes were discussed by the panels across all questions. These include the 37 initially identified outcomes, along with 4 additional outcomes that were proposed during panel discussions.

The final lists of prioritized outcomes (at the end of step 3) included 34 unique outcomes (Supplemental Table 2). By core area, these included 4 mortality/survival outcomes, 16 physiological/clinical outcomes, 6 functional outcomes, 6 resource use outcomes, and 2 adverse effect outcomes. The median number of final prioritized outcomes per question was 7 (IQR, 5-8; range, 2-9).

Common final prioritized outcomes across questions

Among the final prioritized outcomes across all 32 questions, the most common outcomes were overall survival (27 questions [84%]), relapse-free survival/event-free survival/disease-free survival/progression-free survival/relapse (27 questions [84%]), quality of life (26 questions [81%]), short- and long-term toxicity (24 questions [75%]), minimal residual disease–negative status (17 questions [53%]), allogeneic hematopoietic stem cell transplantation (10 questions [31%]), and hospital admission (10 questions [31%]). Sixteen outcomes (eg, sexual health and hyperglycemia) were prioritized for only 1 question each.

Frequency of reporting of prioritized outcomes in existing studies on ALL in AYAs

We conducted 17 systematic reviews, but 1 review did not identify any eligible studies. Across the remaining 16 systematic reviews, the median percentage of included studies that reported each prioritized outcome was 25% (IQR, 20-48; range, 0-79) (Supplemental Table 3). The prioritized outcomes reported most frequently by included studies were overall survival (median, 79%; IQR, 63-85; range, 14-100), relapse-free survival/event-free survival/disease-free survival/progression-free survival/relapse (median, 64%; IQR, 46-82; range, 0-96), and short- and long-term toxicity (median, 60%; IQR, 21-74; range, 0-100). The prioritized outcomes reported least frequently by included studies were fertility (0 studies) and quality of life (median, 0%; IQR, 0-0; range, 0-6; reported for only 2 research questions).

Discussion

In this project, multidisciplinary panels of experts and patients used a rigorous 3-step process to identify and prioritize outcomes for 32 research questions relevant to AYAs with ALL. The final prioritized lists included 34 unique outcomes belonging to 5 broad core areas, with a median of 7 outcomes per research question. In 16 conducted systematic reviews, each outcome was reported by a median of only 25% of included studies.

Potential uses of prioritized outcomes

The outcome prioritization process for the various questions addressed in this project was crucial to identifying the outcomes to be included in the systematic reviews and assessed for certainty of evidence for recommendation development. The methods for the systematic reviews and recommendations are described elsewhere.2, 3, 4 However, the prioritized outcome lists described here are also potentially useful well beyond this project.

Researchers designing future primary studies, conducting systematic reviews, and developing clinical practice guidelines in the field of ALL can benefit from using the lists of prioritized outcomes to improve the impact of their research and guidance. The adoption of core outcome sets has been shown to improve the patient relevance and consistency in outcomes in specific fields.⁸ The 32 research questions for which we prioritized outcomes cover a broad range of relevant issues for AYAs with ALL. The lived experience and multidisciplinary nature of the panels facilitated the representation of various key points of view. The median of 7 outcomes prioritized per research question is also the same as the number of outcomes (ie, 7) that is recommended when conducting a GRADE certainty of evidence assessment in a systematic review.⁹

This work identifies opportunities for improvement in terms of outcomes that should be reported more frequently in research in this population. For example, the panel members, particularly those with lived experience of ALL as an AYA, emphasized the importance of quality of life and, for pertinent research questions, fertility outcomes. However, these outcomes were rarely reported, a limitation in the current evidence that should be addressed by future studies. There may be various reasons, such as limited resources and burden of measurement, why these (and other) outcomes are infrequently reported in research. However, the importance of these outcomes to patients, families, and caregivers suggests that extra efforts should be made to include the outcomes. Such efforts can involve more training of clinicians and researchers on the measurement of patient-reported outcomes, greater involvement of patients in research, and the development and widespread adoption of core outcome sets (discussed subsequently).

The extent to which the outcomes prioritized in this project may be relevant to other age groups of patients with ALL (eg, children and older adults) may depend on the age group, specific outcome, and clinical question being addressed in the study/systematic review/guideline. However, at a minimum, the prioritized outcomes could serve as a starting point for discussions about outcome choice. Analogous considerations will likely have to be made if these outcomes are to be applied to similar efforts regarding other blood cancers.

Another major potential use of the list of outcomes prioritized is as a starting point for core outcome set development in the field of ALL. Although core outcome sets have traditionally been developed for use in clinical trials, they are increasingly being used in other primary research, systematic reviews, clinical practice guidelines, and in health care practice.¹⁰ Although we found a preliminary list of outcomes for ALL through the HARMONY Alliance, we were unable to find a developed core outcome set. We searched the Core Outcome Measures for Effectiveness Trials database (a regularly updated database of core outcome sets across medicine and public health) and found a registered (but withdrawn) core outcome set for pediatric ALL.¹¹ We found related core outcome sets for acute myeloid leukemia,^12,13 chronic lymphocytic leukemia,¹³ and non-Hodgkin lymphoma.¹³ Therefore, there is a research need for the development and dissemination of a core outcome set for ALL broadly and, if needed, a separate one for AYA patients with ALL specifically. The development and adoption of such core outcome sets could go a long way toward the consistent use and reporting of stakeholder-prioritized key outcomes in research that can facilitate evidence synthesis and clinical practice guideline development in this field.

A limitation of this project is that most (88%) of the 38 trained experts were based in North America. Somewhat different sets of outcomes may have been prioritized with greater representation from other regions.

Lessons learned/reinforced

The outcome prioritization and systematic review processes in this project emphasize some important lessons. First, it is crucial to include a diverse set of stakeholders in the process of outcome prioritization. At a minimum, these should include people with lived experience of the disease in question, clinicians with a wide range of clinical expertise, primary researchers, systematic reviewers, and guideline developers.¹⁴ For the current topic of ALL, the clinicians included adult- and pediatric-focused hematologists/oncologists, clinical pharmacists, clinical psychologists, and clinical social workers. It is only through such broad representation that the gamut of key issues in a field is discussed and the key outcomes become clear. Second, when engaging with multidisciplinary stakeholders, it is important to incorporate multiple opportunities for engagement and input. We encouraged more comprehensive and nuanced consensus building through online surveys and iterative teleconference discussions. Third, the rich multistakeholder discussions that occur in the context of the combined processes of evidence synthesis, certainty of evidence assessment, evidence-to-decision framework implementation, and recommendation statement development and grading are similar to discussions that occur in the context of outcome prioritization for core outcome set development as well as outcome choice for specific primary studies. Although recent work has found that core outcome sets are not yet being routinely used to inform the guideline development process,¹⁵ the efforts of guideline development panels can, and should, be used to inform future core outcome set development and primary studies. Core outcome sets, once rigorously developed and widely adopted, can not only help standardize outcome measurement and reporting across studies but also ensure that the standardized outcomes are aligned with patient-centered priorities and guideline development needs. This can enable future guideline development to be more patient centered and practically applicable.

Conclusions

Here, we have reported outcomes prioritized by multidisciplinary panels of experts and patients for 32 research questions relevant to AYAs with ALL. Future research should involve the use of these prioritized outcomes and the development of a core outcome set for ALL.

Conflict-of-interest disclosure: The authors declare no competing financial interests.