(Outcome) measure for (intervention) measures – A guide to choosing the appropriate non-invasive clinical outcome measure for intervention studies in celiac disease

Introduction:

Lifelong strict adherence to a gluten-free diet (GFD) has been the primary treatment for celiac disease (CD) for nearly a century and has remained so even as understanding of the disease has increased^1,2. Despite availability of accurate and available diagnostic tests, the majority of individuals with CD remain undiagnosed, suggesting there is a need for enhanced screening strategies. While diagnosis is a major hurdle in CD treatment, problems can persist after therapy is instituted. An imperfect treatment, the GFD is a practically, psychologically and financially challenging with a patient-reported treatment burden comparable to that of end-stage renal disease³. In addition, restrictive diets, including a GFD, are more likely to be nutritionally imbalanced⁴. Even more problematic, up to 30% of patients with CD on a GFD have ongoing symptoms and/or persistent villus atrophy⁵. For these reasons, dietary and behavioral interventions are needed to improve outcomes for the millions of patients trying to follow a GFD. Despite the relative lack of progress to date, the discovery pipeline is starting to flow with vaccines and other pharmacologic interventions currently in development, some of which have already been tested in phase 1 and 2 clinical trials and will require thoughtful study to assess their impact on symptoms, disease outcomes and quality of life.

Generally, interventional CD studies can be grouped into two main categories: real world trials and clinical trials. Real world trials usually involve either diagnosis or behavioral and dietary approaches to improve dietary adherence, diet quality and/or quality of life on a GFD. Real world trials involve pragmatic changes which patients implement in real life settings, thus allowing for evaluation of the impact of treatment on quality of life, social function and related endpoints which are not feasible in the setting of most randomized clinical trials. Conversely, clinical trials, which are performed in very controlled settings often for assessment of novel biomarkers, pharmacological adjuncts to a GFD or pharmacologic alternatives to a GFD that allow for unrestricted gluten ingestion. In clinical trials, objectives and time frames are discrete and well-defined allowing for use of specific targeted outcomes, such as PRO measured symptoms or intestinal histology, as discussed below. Clinical trials are time efficient for assessment of the effects of interventions on these outcomes but, by definition, are only approximations of the real life experiences of patients with any given disorder. This is especially true in CD with its complex intersection of long term complications, acute and chronic symptoms, and the social, economic and nutritional consequences of the current treatment - a GFD (See Figure 1). For these reasons, assessing the overall human impact of interventions, whether of diagnosis, support or treatment, in CD requires real world studies.

Figure 1. Conceptual model of celiac disease outcome measures for clinical trials and real world settings.

Many factors influence quality of life in celiac disease. Some (yellow shading) are more appropriate for real world setting, whereas others (shaded grey) are more applicable in clinical trial settings.

Selecting appropriate endpoints for intervention studies in CD is especially important given the large number of patients who may potentially benefit. Biopsy confirmed CD has a pooled global prevalence of 0.7%⁶, which is greater than the prevalence of Crohn’s disease and ulcerative colitis combined. The existence of a dietary approach to management of CD complicates selection of outcome measures. Particularly for pharmacologic adjuncts and alternatives to a GFD, gluten exposure must be assessed and considered in the analysis, yet it may not be a meaningful measure of the efficacy of the intervention being studied.

Small intestinal histology and serology are essential outcomes to include in assessment of novel treatment interventions for CD, but in many cases may not be appropriate primary endpoints. Practical limitations of histology as a primary outcome in CD include invasiveness, interobserver variability in interpretation, sampling error due to the patchy nature of the celiac mucosal lesion and lack of ability to assess extent of intestinal disease⁷. These limitations likely contribute to the inconsistent association between histologic findings and some long-term outcomes^8–10, and observed variability in kinetics of mucosal damage. Although more practical to obtain, serum levels of antibodies directed to tissue transglutaminase or (deamidated) gliadin do not correlate closely with histologic damage in patients with CD on a GFD¹¹ nor are they responsive to gluten challenge¹². It is possible that a drug that ameliorates symptoms may actually lead to worsening in histology or serology by allowing for greater gluten exposure. The currently available treatment for CD (i.e., GFD) has few serious direct adverse effects, so the safety profile for any new therapy intended as an adjunct to or replacement of a GFD must be extremely favorable. Thus, the appropriate role of serology and histology in clinical trials may be as important safety outcomes rather than a primary outcome.

Consistent with this paradigm, the third Gastroenterology Regulatory Endpoints and Advancement of Therapeutics workshop, which was sponsored by the FDA, recommended that patient reported outcomes (PROs) appear to be most suitable as primary outcomes in Phase 2 and 3 clinical trials for CD either as sole end-point or as co-primary end-point with other objective measures¹³. While health-related quality of life (HRQoL) might be a more relevant concern for patients with CD compared to individual symptoms, it might be less amenable to change with therapeutic interventions. Given the multi-dimensional nature of HRQoL, certain aspects of HRQoL might be non-modifiable or might be confounded by factors unrelated to CD.

In this article, we will review non-invasive clinical outcome measures and provide case scenarios to illustrate key considerations in selecting appropriate clinical outcome measures for treatment interventions in CD.

Measures of intestinal function and nutrition

CD causes impaired intestinal absorptive function, particularly in the duodenum. Classical measures of intestinal permeability, such as lactulose-mannitol fractional excretion, are difficult to administer and have had mixed results, with significant inter-individual variability that may be of a greater magnitude than intra-individual changes observed during gluten challenge¹². In contrast, serum levels of intestinal fatty acid binding protein (I-FABP) - a proposed marker of small intestinal epithelial damage - are more sensitive to gluten challenge¹⁴ and to a gluten-free diet¹⁵. Despite responsiveness within individuals, absolute levels of I-FABP do not correlate well with histologic damage in the population, even at diagnosis¹⁵. More recently, oral bioavailability of simvastatin, which is primarily metabolized by cytochrome P450 3A4 (CYP3A4) in villous type enterocytes, has been proposed as a functional measure of villus integrity, but has not been thoroughly tested¹⁶.

Nutritional measures are an even cruder measure of intestinal function, yet they may play a role, particularly in studies that involve dietary interventions. Clinically, treatment of villus atrophy increases nutrient absorption leading to weight gain and development of metabolic syndrome¹⁷, both of which are important considerations given many patients are already overweight at the time of CD diagnosis^18,19. Iron is absorbed from villous tips in the proximal duodenum, so it may be especially relevant as a marker of malabsorption in CD. At diagnosis, iron deficiency is highly prevalent and anemia may be a marker of disease severity²⁰. Malabsorption of other nutrients, such as zinc, vitamin B12 or folate, is less common at diagnosis, but may reflect more extensive enteropathy or may develop on a GFD⁴.

Patient reported outcomes (PROs) for measurement of CD symptoms

The FDA recommends that PROs should meet the following criteria: content validity (extent to which instrument measures the concept/domain it is used to measure), construct validity (evidence that instrument correlates with another accepted measure of disease activity), criterion validity (degree to which instrument is a reflection of an accepted gold standard), test-retest reliability and responsiveness to change²¹. However, there are several challenges when using PROs in CD. This includes determining an appropriate gold standard to determine criterion validity. There is no clear relationship between histology and CD symptoms, which range from classic gastrointestinal symptoms, such as abdominal pain and diarrhea, to non-gastrointestinal symptoms, such as ataxia, headache and infertility². In addition, a small yet important proportion of patients who are diagnosed by screening of high risk groups, such as first-degree relatives, Type 1 diabetes, may be completely asymptomatic². This is very different from other chronic gastrointestinal diseases, such as IBD, in which the vast majority of patients present with gastrointestinal symptoms alone and there is a closer correlation between intestinal damage and symptoms .²²

Significant symptom heterogeneity associated with CD makes it difficult to have a single PRO which would be appropriate for all celiac patients. It also makes studying responsiveness to change in CD a challenging task. In addition, studies have shown that symptom severity in CD does not correlate well with objective measures of disease activity (both off and on treatment), an observation that is not unique to CD. Despite these challenges with PRO development in CD, there has been progress in the last few years and a brief summary of available PROs in CD is given below.

Gastrointestinal symptoms are a main concern and reason for clinical evaluation for patients with CD, thus the majority of PROs focus predominantly on gastrointestinal symptoms (Table 1). Although generic PRO such as the Gastrointestinal Symptom rating scale (GSRS) are not specific for CD, they have been shown to correlate with established markers of CD activity, such as histology, at diagnosis²³. In addition, they have also been shown to be fairly responsive to therapeutic intervention in placebo controlled trials²⁴. However, the GSRS was developed primarily for functional gastrointestinal disorders and peptic ulcer disease, is not optimized for CD and does not measure any extra-intestinal symptoms²⁵. Recently, several CD specific PROs that incorporate intestinal as well as extra-intestinal symptoms have been developed, including the Celiac Symptom Index (CSI)²⁶, the Celiac Disease PRO (CD-PRO)²⁴, the Celiac Disease Assessment Questionnaire (CDAQ)^27,28 and the Celiac Disease Symptom Diary (CDSD)²⁹ (Table 1)

Table 1.

Commonly used patient reported outcome measures for celiac disease interventional studies

	Format	Domains	Recall Period	Regulatory endpoint?1	Comments
Symptom Measure
Generic
Gastrointestinal Symptoms Rating Scale (GSRS)25	15 items, each rated on intensity, frequency, duration and impact of daily living with (7 point scale)	• Abdominal pain • Diarrhea • Constipation • Reflux • Indigestion	Preceding month	No	• correlates with histology at diagnosis23 • population norms available • interview or self-report • modified CD-GSRS
CD specific
Celiac symptom index (CSI)26	16 items, 5 point likert scale	• Intestinal symptoms • Extraintestinal symptoms • General health	Past 4 weeks	No	• Most extensively studied CD specific PRO
Celiac Disease PRO (CeD PRO)24	11 items, Visual analogue scale from no discomfort (0) to worst possible discomfort (10)	• Intestinal symptoms • Extraintestinal symptoms	One day	Yes	• Construct validity and responsiveness to change not reported
Celiac Disease Symptom Diary (CD-SD)29	10 symptoms, scaled 0-10	• Intestinal symptoms • Extraintestinal symptoms Value in health paper	Each symptom assessed daily on 0-10 scale over 7 days to give a overall score of 0-70	Yes	• Construct validity and responsiveness to change not reported
Quality of Life Measures
Generic
Short form 36 (SF-36)30	36 items, various scales	• Physical health  ∘ Functioning  ∘ Role  ∘ Bodily pain  ∘ General health • Mental health  ∘ Vitality  ∘ Social  ∘ Emotional  ∘ Mental health	Past 4 weeks	No	• Population norms available for healthy as well as other diseases • Gold standard for QoL scale devleopment
Psychological General Well-being Index (PGWB)31	22 items, 6-point likert scale	• Vitality • General Health • Self-control • Anxiety • Positive well being • Depressed mood	Past month	No	• Does not address physiological function or vitality
CD specific
Celiac Disease Quality of Life Survey (CDQOL)32	20 items, 5 point likert scale	• Dysphoria • Inadequate treatment • Limitations • Health concerns	Past month (30 days)	No	• Language negatively loaded
Celiac Disease DUX (CDDUX)33	12 items, 5 faces scale	• Having CD • Communication • Diet	Unspecified – “how feeling these days” and “how feeling lately”	No	• Modified from DUX • Pediatric measure with parent and child versions • Available in many languages
Celiac Disease Questionnaire (CDQ)34	28 items, 7 point likert scale	• Emotional problems • Social problems • Disease-related worries • GI symptoms	Past 2 weeks	No
Celiac Disease Assessment Questionnaire (CDAQ)27,28	32 items, 5 point likert scale	• Stigma • Dietary burden • Symptoms • Social isolation • Worries and concerns	Past 4 weeks	Yes	• Scores scalable with dimension scores and an overall index (analogous to SF-36)

¹Developed in accordance with specifications outlined in United States Food and Drug Administration 2009 Guidance for Industry “Patient-Reported Outcome Measures: Use in Medical Product Development to Support Labeling Claims”.

Although symptom based PROs have been suggested as a primary outcome in therapeutic trials for CD, there are several concerns. Fundamentally, the PROs described above have been shown to be responsive to treatment and can detect statistical differences in symptom severity with a therapeutic intervention; however, the minimum clinically important difference (MCID) has not been well-defined. As the smallest change in an outcome that a patient would identify as important, MCID offers a threshold above which an outcome is perceived as relevant by the patient. There is a need to establish consensus on change in CD specific PRO scores that is perceived as MCID by patients. Another concern with symptom based PROs in CD is the lack of correlation between symptoms and objective measures of CD activity. Thus, objective measures of disease activity, such as histology, should be included as co-primary or secondary outcomes in clinical trials for CD as described above. In addition, most PROs were developed in adults and their applicability in children is not clear and no pediatric observer reported outcomes have been validated.

PROs for measurement of health related quality of life

Although symptom focused PROs are likely to be acceptable as a primary outcome in CD by regulatory agencies, HRQoL might arguably be a more important measure for patients. HRQoL in CD is determined not only by ongoing gastrointestinal and extra-intestinal symptoms but also by high treatment burden due to psychological burden, social isolation and financial challenges associated with a GFD. Any intervention - ether an adjunct or an alternative to a GFD - should aim to improve the psychological burden associated with accidental gluten exposure and social isolation related to a GFD without significantly increasing the economic burden of celiac patients, which is already high. An ideal HRQoL measure for CD should assess these aspects of quality of life. Given there are several aspects of HRQoL which are unique to CD, using a CD specific HRQOL measure might provide insight into overall value of a therapeutic intervention in patients with CD. Several CD specific HRQoL instruments have been validated for adult and pediatric patients with CD (Table 1). The Celiac Disease Quality of Life Survey (CDQOL) is the most extensively studied and assesses QoL along 4 domains: dysphoria, inadequate treatment, limitations, and health-concerns³². However, for each HRQoL measure, attention should be paid to individual domains in addition to overall score. For example, while a new therapy might improve the inadequate treatment domain through availability of more treatment options, it is still important to ensure that there is improvement (or at-least no worsening) in other domains of CDQOL with newer therapies.

Recently, a new instrument the Celiac Disease Assessment Questionnaire (CDAQ) was developed^27,28. It appears to be a hybrid of symptom related PRO and HRQoL which assesses a variety of domains: symptoms, dietary burden, social isolation, stigma, and worry and concerns. Thus, CDAQ might be a simple to use and comprehensive measure of both disease and treatment burden in patients with CD. However, there is a need to assess CDAQ’s responsiveness to change and develop more such instruments.

Although HRQoL is likely the most important outcome measure for patients, measurement of HRQoL in a randomized controlled setting might not be a true reflection of that in a clinical setting. This is because some of the factors inherent to a randomized controlled trial such as more frequent interaction with patients, changes in dietary behavior of patients while in trial, anxiety about side-effects from a new therapy, etc. bias patient population and confound HRQoL. Thus, although it is important to measure HRQoL in therapeutic trials, measuring HRQoL in more real world settings is generally more relevant.

Measures of Sickness Burden

In contrast to quality of life measures, burden of disease instruments are less well developed and less frequently utilized, but should be considered in CD. Two general categorizations of burden of disease are reported in the literature: the Disability Adjusted Life Year (DALY), which was initially developed for the World Health Organization’s (WHO) Global Burden of Disease (GBD) survey³⁵, and the Quality Adjusted Life Year (QALY), which was developed by researchers at the University of York³⁶.

The DALY measurement unit is used to quantify the burden of disease in specific populations, and is based on relevant health care costs³⁷. The DALY includes both years of “healthy” life lost due to death and years of “healthy” life lost due to poor health. DALY can be considered estimate of the difference between optimal health and the health of a specific population, such as individuals with CD. While DALY measures the health loss, QALY measures the health gained. The QALY is able to combine the survival of an individual with their HRQoL into a single index, thereby providing a ‘common currency’ to enable comparisons across different disease areas³⁸. The main use of QALYs is within the framework of cost-effectiveness analysis, to assess the improvement in quality-adjusted life expectancy obtained through a specific health intervention relative to a situation in which either no intervention or a standard alternative intervention is provided³⁹. In the current cost sensitive environment, assessment of healthcare utilization is increasingly important. The need for understanding of healthcare utilization in CD is most commonly cited in discussions regarding population screening or payer considerations for emerging therapies. Money spent per QALY may be a key factor in determination of the acceptability of population screening initiatives, such as are considered necessary to significantly reduce the rate of undiagnosed CD⁴⁰. Similarly, cost to the healthcare system to achieve meaningful improvements in either hrQOL or QALYs may be critical in payer acceptance of adjunctive therapies for CD.

Discussion:

Case study: Dietary interventions for celiac disease

Classically, diet related interventions for patients with CD have focused upon broadening the spectrum of grains considered suitable for a gluten-free diet, such as oats^41,42 or ancient wheat cultivars⁴³. Educational (e.g., CD school⁴⁴), behavioral (e.g., psychological support⁴⁵) and technological (e.g., testing for gluten in food, testing for gluten immunogenic peptides (GIPs) in urine or feces) interventions to improve GFD adherence have also been evaluated. As gluten-free processed foods have become more widely available, concerns regarding the nutritional quality of a GFD have increased⁴⁶. Thus, we consider the case of a hypothetical trial comparing the Mediterranean diet to nutritional counselling to improve the nutritional profile of a GFD.

The primary challenge in assessing dietary interventions is the absence of an agreed-upon measure of GFD adherence⁴⁷. Classically, feeding studies have been conducted in inpatient settings where all food and beverages are provided and strictly controlled. Such an approach is less than ideal for dietary intervention studies in CD. Living in a controlled setting makes it very difficult to measure the important dimension of how the intervention affects quality of life for an illness in which meal preparation and planning may be a considerable component of the treatment burden^48–50. An alternative is to provide gluten-free meals through a home meal delivery service. Either pre-prepared meals or gluten-free ingredients to prepare meals could be provided. This approach has been used successfully in nutritional interventions for oncology patients, the elderly and overweight teenagers⁵¹. Of course, provision of gluten-free food does not prevent gluten exposure entirely, either through inadvertent cross-contact during preparation and serving or through intentional ingestion and is not a sustainable practice outside of clinical trials.

For individuals sourcing their own meals, dietary assessment by a dietitian with expertise in gluten-free diets is commonly accepted as the best available proxy for gluten exposure. Unfortunately, this method has not been standardized and is further limited by recall bias, patient knowledge of whether foods contain gluten⁵², and absence of symptoms to signal unintentional gluten exposure. More recently, tests for excretion of GIPs in stool⁵³ and urine⁵⁴ have been developed. These tests rely upon the fact that human endoproteases are unable to cleave the prolyl peptide bonds of gluten⁵⁵. Consequently, polypeptide fragments of 33 amino acids or longer may pass through the digestive system intact. Conveniently, these fragments are also immunogenic for many patients with CD⁵⁵. The availability of assays to measure GIP excretion in urine and stool directly as a proxy for gluten ingestion has demonstrated gluten exposure that was not suspected following dietitian assessment⁵⁶ and an expert panel recently recommended that GIP testing be considered for use as a tool to select patients and/or document gluten exposure.⁵⁷

Appropriate primary outcomes for dietary intervention studies must consider not only how the intervention affects dietary composition or adherence, but also the impact of this intervention on quality of life. Hypervigilance has been associated with decreased quality of life and emphasis on dietary adherence may also generate or exacerbate anxiety⁵⁸. Thus, PROs may have a role either as a secondary or a co-primary endpoint.

Case study: pharmacological adjunct to a GFD in CD

Leffler, et al reported a multicenter, randomized, double-blind, placebo-controlled study of Larazotide acetate in adults with CD who had persistent symptoms despite at-least 12 months of a GFD²⁴. A range of daily oral doses of Larazotide as adjunct to a GFD was evaluated over a 12 week period. Inclusion criteria were based on symptoms alone, thus it is unclear if the drug-effect varies with degree of histological damage at the time of therapy. The primary end-point was the difference in average weekly on-treatment CD-GSRS score for each dose vs placebo, over the 12-week active treatment period. This trial rightly chose a change in PRO score as the primary outcome. However, CD-GSRS was not developed in accordance with FDA regulatory guidelines and lacks assessment of extra-intestinal symptoms related to CD. While this trial included a change from baseline in celiac specific PRO (CeD PRO) as a secondary outcome (this study also served as a validation exercise for the CeD PRO), future trials should use one of the validated, celiac specific PROs specifically developed in accordance with FDA regulatory guidelines as a primary or co-primary outcome.

The trial also assessed serology at baseline and at several time-points during the study as safety measure. However, histologic assessment was not performed for this trial. Given the lack of correlation between serology and histology on a GFD, consideration should be given to include histology as co-primary or secondary endpoint. This trial also assessed GFD compliance using GFD compliance questionnaire and showed individuals on study drug and placebo reported similar rates of voluntary and accidental gluten consumption during the study period. As diet questionnaires are poor predictors of compliance, objective measures such as GIP excretion should be considered in future trials.

Case study - Assessing outcomes in real world settings

Outcomes for real world studies may include the type of outcomes used in clinical trials, including gastrointestinal symptoms and histology, but are uniquely suited for assessment of critical variables including quality of life, burden of disease and healthcare utilization.

Two classic real world studies in CD by Johnston et al⁵⁹ and Mustalahti et al⁶⁰ assessed the effect of CD screening on quality of life in European populations. While both studies assessed the effect of screening in similar populations (mixed symptomatic and screen detected adults detected by serologic tests) during a similar time period, the Johnston paper concluded “Quality of life in screen detected celiac patients did not differ significantly compared to controls”. In contrast, the Mustalahti paper concluded “Gluten-free diet was associated with improved quality of life for patients with symptom-detected CD and patients with screen-detected CD.”

While these studies are superficially similar, the primary outcomes differed. Johnston et al used the SF-36 while Mustalahti et al used the Psychological General Well Being Index (PGWBI). It is probable that a major reason for the disparate conclusions is the choice of endpoint. Specifically, the SF-36 is heavily weighted to physical disability, whereas the PGWBI encompasses psychological, social and emotional status. Given that the studies were both focused predominantly on the effect of CD diagnosis in screen detected, and thus pauci-symptomatic, individuals, the PGWBI was arguably the more appropriate and responsive choice. At the same time, the PGWBI lacks any questions which allow assessment of the relationships among disease, treatment and symptoms on overall quality of life and thus would not currently be considered to be adequately representative of the experience of living with CD. While better measures currently exist, as described above, these two real world studies of CD diagnosis are helpful reminders of the importance of choosing an appropriate primary outcome tailored for the specific population and study design in question.

Conclusions:

CD is a common condition that is underdiagnosed and for which the current treatments are inadequate and difficult. An unprecedented number of pharmacologic alternatives to a GFD are currently under investigation as well as studies of interventions to improve diagnosis and dietary treatment. A major challenge in any of these areas is designing appropriate trials with adequate outcome measures. Traditionally, investigators have relied upon tissue transglutaminase antibodies and/or histology; however, these are poor proxies for either symptoms or health related quality of life. The absence of a definitive criterion standard threatens to hinder progress in this area. Further work is necessary to establish outcome clinical measures for CD that are robust, relevant, responsive to treatment, non-invasive and clinically meaningful. Such measures would include not only biomarkers of CD activity, but also patient reported outcome measures that can apply and be compared in the context of both a gluten-free and a gluten-containing diet.

Key Points:

• There is an unmet need for dietary and behavioral interventions for celiac disease.

• The existence of a dietary approach to celiac disease management complicates selection of outcome measures.

• Serology and histology are more likely to be appropriate as safety outcomes rather than primary outcomes.

• The available patient-reported outcomes for celiac disease have significant limitations.

• The burden of disease is more appropriate for real world studies than for clinical trials.

Synopsis:

There is an unmet need for diagnostic and treatment interventions for celiac disease. Both clinical trials and real world studies require careful selection of clinical outcome measures. Often, neither serology nor histology is an appropriate primary outcome. In this article, we will review various measures of intestinal function and nutrition, patient-reported outcome measures for symptoms and for health related quality of life and measures of sickness burden as they apply to intervention studies for celiac disease. A series of case studies is presented to illustrate key considerations in selecting outcome measures for dietary interventions, pharmacologic interventions and real world studies.