Skip to main content
Journal of Neurogastroenterology and Motility logoLink to Journal of Neurogastroenterology and Motility
. 2022 Jul 30;28(3):376–389. doi: 10.5056/jnm22004

A Systematic Review of the Development and Psychometric Properties of Constipation-related Patient-reported Outcome Measures: Opportunities for Digital Health

V Vien Lee 1,*, Ni Yin Lau 1, David J Y Xi 1, Anh T L Truong 1,2, Agata Blasiak 1,2,3,*, Kewin T H Siah 4,5,*, Dean Ho 1,2,3,*
PMCID: PMC9274467  PMID: 35719047

Abstract

Background/Aims

Constipation can be a chronic condition that impacts daily functioning and quality of life (QoL). To aid healthcare providers in accurately assessing patient symptoms and treatment outcomes, patient-related outcome measures (PROMs) have been increasingly adopted in clinical settings. This review aims to (1) evaluate the methodological quality and measurement properties of constipation-related PROMs, using the COnsensus-based Standards for the selection of health Measurement INtruments (COSMIN) criteria; and (2) assess the modes of digital dissemination of constipation-related PROMs.

Methods

PubMed, Embase, and PsycINFO databases were searched and 11 011 records ranging from 1989 to 2020 were screened by 2 independent reviewers. A total of 26 studies (23 PROMs; 18 measuring symptom-related items and 5 measuring constipation-related QoL items) were identified for the review and assessed.

Results

There were multiple variations between PROMs, including subtypes of constipation, methods of administration, length of PROM and recall period. While no PROM met all the COSMIN quality standards for development and measurement properties, 5 constipation-related PROMs received at least 4 (out of 7) sufficient ratings. Only 2 PROMs were developed in Asia. Five PROMs were administered through digital methods during the validation process but methods of adapting the PROMs into digital formats were not reported.

Conclusions

The constipation-related PROMs identified in this review present varying quality of development and validation, with an overall need for improvement. Further considerations should be given towards more consistent methodology and reporting of PROM development, increase in culturally-specific PROMs, and better reporting of protocol for the digitization of PROMs.

Keywords: Constipation, Digital health, Patient-reported outcome measures, Quality of life

Introduction

Chronic constipation is a prevalent worldwide problem that affects up to 10-15% of the adult population.1 Symptoms of primary and secondary constipation include hard stools, excessive straining, infrequent bowel movements, bloating, and abdominal pain.2 While constipation can often be managed by medication and lifestyle modification, prolonged constipation can significantly decreases quality of life (QoL).2,3 To better assess patients’ health status and QoL, it is important to have an accessible tool that can accurately assess patients’ symptoms and treatment outcomes, which may enable personalized intervention strategies. The usage of reliable and validated patient-related outcome measure (PROM) can help provide a consistent method of measuring clinical symptoms and QoL outcomes in patients.4 PROMs are standardized, validated questionnaires that measure patients’ perception of their own health status and well-being.5 While PROMs were initially developed for research use, they have been increasingly adopted in clinical practice to aid clinicians provide better and more patient-centered care.6

To date, 2 reviews have examined existing assessment scales measuring constipation symptoms.7,8 A combination of 9 self-reported measures, developed between 1989 to 2010, were assessed by both reviews. While the reviews provided an insight on the reliability and validity of existing constipation PROMs, the reviews were not conducted systematically and constipation-related QoL PROMs were not included. Given the impact of constipation on QoL, including mental, social, and physical functioning,9,10 it is important to consider QoL in treatment outcomes.

As the capabilities and adoption of digital technology expand in healthcare, it is also important for us to explore the potential of digitizing PROMs. This could sustain longitudinal patient assessment, which can further support the individualization of patient care. In patients with inflammatory bowel disease, collecting consistent electronic patient reported outcomes (ePRO) on a cloud-based digital therapeutics and monitoring application has been shown to significantly reduce yearly hospitalizations and emergency room visit rates most likely due to immediate interventions prompted by concerning questionnaire scores. Patients also reported having a better understanding of the nature and causes of their health condition after a year.11 Given the importance of incorporating QoL into treatment outcomes and the potential of incorporating digital health technologies that are patient-centric into constipation management, the current review aims to (1) systematically review constipation-related PROMs, including QoL reporting, using the COnsensus-based Standards for the selection of health Measurement INstruments (COSMIN)12 guideline to evaluate the methodological quality of included studies and the quality of the measurement properties themselves, and (2) assess the current modes of digitization of constipation-related PROMs.

Methods

A systematic review protocol was developed in accordance with the Preferred Reporting for Items for Systematic Reviews and Meta-analyses (PRISMA) and the COSMIN guidelines. The study protocol was registered with the International Prospective Register of Systematic Reviews (PROSPERO) database (No. CRD42021236257).

Search Strategy

The PRISMA guidelines were used to identify studies for this review. A comprehensive literature search was performed using PubMed, Embase, and PsycINFO to identify all articles on the development or validation of constipation-related PROMs. The search was conducted up to February 2021. Searches in all 3 databases were performed using the following keywords: (constipation OR gastrointestinal) AND (question* OR [patient AND outcome AND measure]) AND (validation OR development).

Study Selection

The initial search yielded 11 011 articles after duplicates were removed. Four articles were identified via hand-checking of reference lists of published reviews and were included retrospectively. Two authors (V.V.L. and N.Y.L.) independently reviewed titles and abstracts of the identified records for preliminary inclusion. Articles were included for further screening based on inclusion criteria listed in Table 1. Five hundred and seventy-nine articles satisfied the preliminary inclusion criteria and were accepted for a full review. Interrater agreement was assessed with Cohen’s κ indicator, where κ of 0.60-0.79 was classified as “moderate,” 0.80-0.90 as “strong,” and above 0.90 as “almost perfect” interrater agreement.13 There was a moderate interrater agreement for study selection (Cohen’s κ = 0.72; 95% CI, 0.68-0.75) and discrepancies were resolved by discussion.

Table 1.

Inclusion/Exclusion Criteria for Preliminary Screening and Full-text Screening

Screening criteria
Inclusion criteria for preliminary (abstract and title) screening
1) Gastrointestinal-related
2) Developed or validated PROM or questionnaire or survey or scale
Exclusion criteria for full-text screening
1) PROM did not measure constipation symptoms or constipation-related QoL
2) Revalidated an existing questionnaire for a different language or patient population
3) Review papers or conference abstracts
4) PROM examined constipation as subset or question
5) Paediatric-related PROM
6) Not in English

PROM, patient-reported outcome measures; QoL, quality of life.

Full texts of the eligible articles were retrieved and reviewed. Articles were excluded based the exclusion criteria listed in Table 1. Five hundred and fifty-three articles did not meet the eligibility criteria and were excluded. There was strong interrater agreement for the second screening (Cohen’s κ = 0.80; 95% CI, 0.67-0.93) and discrepancies were resolved by discussion. An independent third reviewer (A.T.) was brought in when discrepancies were not resolved. Figure depicts the flow diagram of the study selection.

Figure.

Figure

Flow diagram of study selection.

Quality Assessment

The methodological quality of the studies and the quality of the PROM itself was assessed using the COSMIN guidelines. Firstly, the COSMIN Risk of Bias checklist14 consisting of 117 questions was used to assess the methodological quality of the studies. The following measurement properties were assessed: PROM development, content validity, structural validity, internal consistency, reliability, measurement error, criterion validity, hypothesis testing for constructive validity, and responsiveness. A 4-point rating system of “very good,” “adequate,” “doubtful,” and “inadequate” was used to rate each property. The final rating was determined by taking the lowest score of an assessment area (ie, “worst score counts” principle). No rating was given if measurement property was not assessed or described.

Following that, the quality of the PROM itself was assessed using the COSMIN updated criteria for good measurement properties.12 The following psychometric properties were assessed: internal consistency, reliability, measurement error, content validity, structural validity, hypotheses testing, criterion validity, and responsiveness. Using the criteria provided, a rating of “+” for sufficient, “–” for insufficient, or “?” for indeterminate was given to each measurement property.

Two authors (V.V.L. and D.J.Y.X.) independently reviewed the included studies using the COSMIN risk of bias checklist and updated criteria for good measurement properties. There was moderate interrater agreement for risk of bias (Cohen’s κ = 0.71; 95% CI, 0.69-0.74) and almost perfect interrater agreement for criteria for good measurement properties (Cohen’s κ = 0.93; 95% CI, 0.88-0.98). Any discrepancies were resolved by discussion.

Results

Summary of Included Studies

A total of 23 PROMs measuring constipation symptoms15-35 or constipation-related QoL36-40 were identified. The PROMs were reported in 26 different studies with publication years ranging from 1989 to 2020. The Bowel Function Index and Patient Assessment of Constipation–Symptom (PAC-SYM) had more than 1 validation study with additional information on measurement properties. A summary of included studies is presented in Table 2.

Table 2.

Characteristics of Included Studies

PROM Authors Type of constipation Participants
(validation study)
Method of administration Recall period Items and subsets Location of study Original language
Constipation symptoms
BF-Diary Camilleri et al,15 2011 OIC 238 patients with chronic pain Handheld electronic PDA device Past 24 hours and immediately after each defecation event for some items 10 items (3 modules per bowel movement, daily assessment, and treatments used) USA English
BFI Rentz et al,16 2009 OIC 985 patients with chronic pain (202 Phase II; 460 Phase III; 323 Phase III) Clinician administered Last 7 days 3 items Germany English
Ducrotté and Caussé,17 2012 OIC 987 patients with chronic pain (202 Phase II, 463 Phase III, 322 Phase III) Clinician administered Last 7 days 3 items France English
Abramowitz et al,18 2013 OIC 520 patients with pain induced by cancer Clinician administered Last 7 days 3 items France English
Chinese Constipation Questionnaire Chan et al,19 2005 Functional constipation 221 (111 patients with chronic constipation, 110 healthy controls) Pen and paper Past 2 weeks 6 items Hong Kong Chinese
CC Symptom Severity Measures Nelson et al,20 2014 Chronic constipation 1579 with chronic constipation (307 Phase IIb; 1272 Phase III) Telephone-based IVRS Administered daily during trial 7 items (2 subscales: bowel and abdominal symptoms) USA English
CAS McMillan and Williams,21 1989 Iatrogenic constipation 64 (32 at risk for constipation, 32 healthy controls) Pen and paper Past week 8 items USA English
Constipation during pregnancy questionnaire Ponce et al,22 2008 Constipation during pregnancy 207 pregnant women Pen and paper, telephone interview (follow ups) None reported 42 items (5 subscales: demographic, obstetric history, bowel habits and laxative consumption, lifestyle and eating habits, and urinary or fecal incontinence) Spain Not reported
CSI Varma et al,23 2008 Multiple subtypes 294 (191 patients with constipation, 103 healthy controls) Pen and paper Last month (2 items), no recall period (14 items) 16 items (3 subscales: obstructive defecation, colonic inertia, and pain) USA English
CSS Agachan et al,24 1996 Idiopathic constipation 232 patients with constipation Pen and paper None reported 8 items USA English
DIBSS-C Coon et al,25 2020 IBS-C 532 IBS-C patients from multicentre phase IIb study Handheld electronic diary Every BM event (3 items), past 24 hours (3 items) 7 items (2 subcales: bowel movement-related symptoms and abdominal symptoms severity) USA English
FICA Bharucha et al,26 2004 Functional constipation & faecal incontinence 83 women Pen and paper Last 12 months (symptoms-related items) 98 items (8 subscales: general bowel habits, abdominal pain, treatment of constipation, faecal incontinence, urinary symptoms, anorectal disease, surgical history and other, health care utilization, and quality of life) USA English
Fecal Incontinence and Constipation Questionnaire Österberg et al,27 1996 Constipation & faecal incontinence 90 (36 with faecal incontinence, 38 with constipation, 16 healthy controls) Pen and paper None reported 38 items Sweden Not reported
IBS-C Symptom Severity Measures Williams et al,28 2014 IBS-C 1608 patients with IBS-C Telephone-based IVRS Administered daily during trial 9 items (2 subscales: bowel and abdominal symptoms subsets) USA English
KESS Knowles et al,29 2000 Chronic constipation 91 (71 patients with chronic constipation, 20 healthy controls) Researcher administered None reported 11 items UK English
ODS-S Renzi et al,30 2013 ODS 200 (100 patients with ODS, 100 healthy controls) Pen and paper None reported 5 items Italy Not reported
ODS Score Altomare et al,31 2008 ODS 106 (76 patients with ODS, 30 healthy controls) Researcher administered None reported 8 items Italy Not reported
PAC-SYM Frank et al,32 1999 Chronic constipation 216 patients with chronic constipation Pen and paper Past 2 weeks 12 items (3 subscales: stool, rectal, and abdominal symptoms) USA English
Modified PAC-SYM Neri et al,33 2015 Chronic constipation 2203 patients with chronic constipation Pen and paper Past 2 weeks 11 items (2 subscales: stool and abdominal symptoms) Italy English
Rome III Criteria Questionnaire Digesu et al,34 2010 Functional constipation (Rome III) 201 women Pen and paper Last 3 months (9 items), last 6 months (2 items), no recall period (6 items) 17 items UK English
VSAQ Pamuk et al,35 2003 Constipation (unspecified) 369 hospital personnel Pen and paper Last 12 months (symptoms-related items) 5 items Turkey Not reported
Constipation-related quality of life
CTSAT-Q Szeinbach et al,36 2009 Chronic constipation and IBS-C 311 patients with chronic constipation and IBS-C Online questionnaire None reported 12 items (5 subscales: expectations, value, treatment satisfaction, activities, and effectiveness) USA English
Constipation-related Disability Scale Hart et al,37 2012 Chronic constipation 343 (240 patients with constipation, 103 healthy controls) Pen and paper Past week 13 items (2 subscales: leisure/work activities and activities of daily living) USA English
CRQOL Wang et al,38 2009 Chronic constipation 343 (240 patients with constipation, 103 healthy controls) Pen and paper Past 12 months 18 items (4 subscales: social impairment, distress, eating habits, and bathroom attitudes) USA English
E-CIS Abdul Wahab et al,39 2020 Chronic constipation 470 elderly people with chronic constipation Researcher administered None reported 22 items (7 subscales: daily activities, treatment satisfaction, lack of control of bodily function, diet restriction, symptom intensity, anxiety, and preventive actions) Malaysia Malay
PAC-QOL Marquis et al,40 2005 Chronic constipation 223 patients with chronic constipation Pen and paper Past 2 weeks 28 items (4 subscales: worries/concerns, physical discomfort, psychosocial discomfort, and satisfaction) USA, UK, France, Sweden English

PROM, patient-reported outcome measure; BF-Diary, Bowel Function Diary; BFI, Bowel Function Index; CC, chronic constipation; CAS, Constipation Assessment Scale; CSI, Constipation Severity Instrument; CSS, Constipation Scoring System; DIBSS-C, Diary for Irritable Bowel Syndrome Symptoms–Constipation; FICA, Fecal Incontinence and Constipation Assessment; IBS-C, constipation-predominant irritable bowel syndrome; KESS, Knowles-Eccersley-Scott Symptom Questionnaire; ODS-S, Obstructive Defecation Syndrome Score; PAC-SYM, Patient Assessment of Constipation–Symptom; VSAQ, Visual Scale Analog Questionnaire; CTSAT-Q, Chronic Constipation Treatment Satisfaction Questionnaire; CRQOL, Constipation-related Quality of Life; E-CIS, Elderly-constipation Impact Scale; PAC-QOL, Patient Assessment of Constipation–Quality of Life; OIC, opioid-induced constipation; PDA, personal digital assistant; IVRS, interactive voice response system; USA, United States of America; UK, United Kingdom.

Method of administration was assumed to be pen and paper unless reported otherwise.

Patient-related Outcome Measures Measuring Constipation Symptoms

Eighteen out of the 23 identified PROMs evaluated constipation symptoms. The majority of the PROMs (n = 10) were developed with intentions to assess severity of constipation in patients.16,19,21,23,24,26,27,30-32 Seven PROMs were developed as a potential diagnosis tool to detect clinically significant constipation, with some having dual functionality for diagnosis and measurement of severity.19,21,27,29,30,34,35 A subset of PROMs (n = 4) were created and/or validated for research purposes, specifically to assess treatment benefits in patients during varying stages of clinical trials.15,20,25,28

The items included in the PROMs can be categorised into 5 categories: abdominal symptoms, bowel movement-related symptoms, stool-related symptoms, anal or rectal symptoms, and others. The most common questionnaire items were incomplete evacuation during bowel movement15,16,19,22-24,26,27,29-32,34,35 and stool consistency,15,19-22,25-29,31,32,34,35 with both items included in 77.8% of PROMs assessed. To measure incomplete bowel movement, PROMs have included a combination of frequency and/or severity related questions. To measure stool consistency, questions include rating of consistency based on the 7-point Bristol stool form scale15,20,28 or a self-constructed scale,27,29,31,35 frequency or severity of hard or lumpy stools,19,32,34 and presence of hard and loose/water stools.21,22,26

More than half of the studies included a measure of abdominal pain,15,20,24-28,30,32,34 abdominal bloating,15,19-21,25-29,32 frequency of bowel movements,19-29,34,35 and straining during bowel movement.15,20,22,23,25-28,30-32,34,35 Abdominal pain, abdominal bloating, and straining during bowel movement have been measured through both severity and/or frequency while questions on frequency of bowel movements have generally been measured using a self-constructed time scale. Two PROMs differentiated complete spontaneous bowel movement from spontaneous bowel movement.20,28 A summary of questionnaire items in PROMs measuring constipation symptoms is presented in Table 3.

Table 3.

Summary of Questionnaire Items in Patient-reported Outcome Measures Measuring Constipation Symptoms

Questionnaire items n (out of 18) %
Overall rating for constipation 4 22.2
Abdominal symptoms
Pain 10 55.6
Bloating 10 55.6
Discomfort 6 33.3
Gas 3 16.7
Cramping 2 11.1
Distention 1 5.6
Fullness 1 5.6
Pressure during defecation 1 5.6
Bowel movement-related symptoms
Incomplete evacuation 14 77.8
Frequency 13 72.2
Straining 13 72.2
Inability to pass 7 38.9
Ease/pain during bowel movement 5 27.8
Urgency 2 11.1
Attempts a day 1 5.6
Lack of urge 1 5.6
Stool-related symptoms
Consistency 14 77.8
Amount 3 16.7
Anal/rectal symptoms
Pain 4 22.1
Bleeding 2 11.1
Anus blockage 2 11.1
Burning 1 5.6
Fullness/pressure 1 5.6
Pruritus ani 1 5.6
Others
Use of laxatives/enemas 9 50.0
Use of digital manoeuvres 8 44.4
Time spent in toilet 5 27.8
History (duration of constipation) 4 22.2
Lack of appetite 1 5.6
Changes to diet 1 5.6

Patient-related Outcome Measures Measuring Constipation-related Quality of Life

Five out of the 23 PROMs assessed constipation-related QoL. Three PROMs were developed to measure the impact of constipation on multiple aspects of QoL, including social relationships, treatment satisfaction, physical symptoms, diet, daily activities, and psychological state.38-40 While all 3 PROMs are suitable for patients with chronic constipation, the Elderly-constipation Impact Scale (E-CIS) was developed for elderly Malay speaking individuals aged 60 years and above.39

Two PROMs evaluated specific aspects of QoL in patients with constipation. The Chronic Constipation Treatment Satisfaction Questionnaire (CTSAT-Q) specifically focused on treatment satisfaction in patients with chronic constipation and constipation-predominant irritable bowel syndrome (IBS-C).36 Items include patient’s expectations on and attitude towards medication, value of medication, interference due to treatment, and effectiveness of treatment. On the other hand, the Constipation-related Disability Scale37 focused on the impact of constipation symptoms on day-to-day activities. The PROM includes a rating of difficulty in performing various leisure, work, and daily activities.

COnsensus-based Standards for the Selection of Health Measurement INstruments Risk of Bias

The COSMIN risk of bias assessment demonstrated very few studies with consistent “very good” and/or “adequate” ratings across all domains. A summary of risk of bias scores for each study are presented in Table 4. Cross-cultural validity was not assessed as the current review only included studies that assessed the original version of the PROM.

Table 4.

Individual PAC-SYM Category Scores for Each Study as Assessed by the COnsensus-based Standards for the Selection of Health Measurement INstruments Risk of Bias Checklist14

PROM Authors PROM development Content validity Structural validity Internal consistency Reliability Measurement error Criterion validity Construct validity Responsive-ness
Constipation symptoms
BF-Diary Camilleri et al,15 2011 Doubtful Doubtful Adequate Adequate Doubtful Doubtful
BFI Rentz et al,16 2009 Inadequate Very good Doubtful Inadequate Doubtful Doubtful
Ducrotté and Caussé,17 2012 Very good Doubtful Doubtful Doubtful
Abramowitz et al,18 2013 Very good Doubtful
Chinese Constipation Questionnaire Chan et al,19 2005 Doubtful Doubtful Doubtful Very good Doubtful Very good Inadequate
CC Symptom Severity Measures Nelson et al,20 2014 Inadequate Very good Doubtful Doubtful Doubtful
CAS McMillan & Williams,21 1989 Inadequate Very good Inadequate Adequate
Constipation during pregnancy questionnaire Ponce et al,22 2008 Inadequate Very good
CSI Varma et al,23 2008 Doubtful Doubtful Very good Very good Adequate Very good
CSS Agachan et al,24 1996 Inadequate Doubtful
DIBSS-C Coon et al,25 2020 Doubtful Doubtful Adequate Very good Adequate Doubtful Doubtful Doubtful
FICA Bharucha et al,26 2004 Inadequate Doubtful Doubtful
Fecal Incontinence and Constipation Questionnaire Österberg et al,27 1996 Inadequate Inadequate Inadequate
IBS-C Symptoms Severity Measures Williams et al,28 2014 Inadequate Very good Adequate Doubtful Doubtful
KESS Knowles et al,29 2000 Inadequate Doubtful
ODS-S Renzi et al,30 2013 Doubtful Doubtful Doubtful Doubtful Very good Doubtful
ODS Score Altomare et al,31 2008 Inadequate Doubtful Doubtful
PAC-SYM Frank et al,32 1999 Doubtful Doubtful Adequate Very good Adequate Adequate Inadequate
Modified PAC-SYM Neri et al,33 2015 Very good Very good Doubtful Doubtful
Rome III Criteria Questionnaire Digesu et al,34 2010 Inadequate Doubtful Very good Adequate Inadequate
VSAQ Pamuk et al,35 2003 Inadequate Inadequate
Constipation-related quality of life
CTSAT-Q Szeinbach et al,36 2009 Doubtful Doubtful Very good Very good
Constipation-related Disability Scale Hart et al,37 2012 Inadequate Very good Very good Doubtful Very good
CRQOL Wang et al,38 2009 Doubtful Doubtful Very good Very good Doubtful Very good
E-CIS Abdul Wahab et al,39 2020 Doubtful Doubtful Very good Very good
PAC-QOL Marquis et al,40 2005 Inadequate Adequate Very good Doubtful Inadequate Doubtful

PROM, patient-reported outcome measure; BF-Diary, Bowel Function Diary; BFI, Bowel Function Index; CC, chronic constipation; CAS, Constipation Assessment Scale; CRQOL, Constipation-Related Quality of Life; CSI, Constipation Severity Instrument; CSS, Constipation Scoring System; CTSAT-Q, Chronic Constipation Treatment Satisfaction Questionnaire; DIBSS-C, Diary for Irritable Bowel Syndrome Symptoms–Constipation; E-CIS, Elderly-Constipation Impact Scale; FICA, Fecal Incontinence and Constipation Assessment; IBS-C, constipation-predominant irritable bowel syndrome; KESS, Knowles-Eccersley-Scott Symptom Questionnaire; PAC-SYM, Patient Assessment of Constipation–Symptom; ODS-S, Obstructive Defecation Syndrome Score; PAC-QOL, Patient Assessment of Constipation–Quality of Life; VSAQ, Visual Scale Analog Questionnaire.

Twenty-three studies were rated on PROM development and the majority of the studies (n = 14) scored “inadequate” due to the lack of a PROM development study involving the target population or a cognitive interview study to assess the comprehensibility or comprehensiveness of the PROM. The remaining 9 studies scored “doubtful” due to poor reporting of study methods including the use of skilled group moderators or interviewers, interview guides, recording and transcription process of interviews and independent coding of data. Poor reporting of methods similarly resulted in “doubtful” ratings for content validity. Only 2 studies19,38 comprehensively examined content validity (ie, asking patients and professionals about relevance, comprehensibility, and comprehensiveness).

Construct validity was the most common measurement properties analysed (n = 23), nevertheless, not all studies examined both convergent and discriminative validity. Four studies15,21,24,31 only examined discriminative validity while 2 studies26,35 only examined convergent validity. Half of the studies that examined construct validity scored “doubtful” due to the lack of detailed description of comparator instruments and/or important characteristics of subgroups.

Following construct validity, reliability (n = 17), and internal consistency (n = 16) were the second and third most analyzed measurement properties. The majority of studies that scored “doubtful” and “inadequate” for reliability did not fulfill appropriate design requirements (eg, patients’ stability in the interim period, similarity of test conditions, and appropriate time interval). Most of the studies that analyzed internal consistency fulfilled the COSMIN criteria for “very good.” “Doubtful” ratings for internal consistency were given due to lack of clarity if scale or subscale was unidimensional.

Less than half of the studies analyzed structural validity, measurement error, criterion validity, and responsiveness. Studies that examined structural validity mostly scored “very good” and “adequate.” Only 1 study scored “doubtful” for structural validity due to the lack of description of rotation method. Two studies scored “doubtful” and “inadequate” for measurement error due to unclear description on stability of patients in the interim period and inadequate calculations of standard error of measurement. All studies that examined responsiveness only examined comparison between subgroups accordingly, ratings were based on that aspect. Scores of “doubtful” and “inadequate” for responsiveness were due to poor description of important characteristics of subgroups and inadequate statistical methods applied.

COnsensus-based Standards for the Selection of Health Measurement INstruments Rating of Measurement Properties

Due to the limited amount of validation studies per PROM, the studies were assessed individually and the total ratings were not provided. The individual ratings for each measurement property of all the studies are presented in Table 5.

Table 5.

Individual Rating for Each Measurement Properties Based on the Updated COnsensus-based Standards for the Selection of Health Measurement INstruments Criteria12

PROM Authors Structural validity Internal consistency Reliability Measurement error Criterion validity Construct validity Responsive-ness
Constipation symptoms
BF-Diary Camilleri et al,15 2011 ? + +
BFI Rentz et al,16 2009 + ? + ?
Ducrotté and Caussé,17 2012 + ? ? ?
Abramowitz et al,18 2013 +
Chinese Constipation Questionnaire Chan et al,19 2005 ? + + + +
CC Symptom Severity Measures Nelson et al,20 2014 + + ?
CAS McMillan and Williams,21 1989 + ? +
Constipation during pregnancy questionnaire Ponce et al,22 2008 ?
CSI Varma et al,23 2008 + + + +
CSS Agachan et al,24 1996 ?
DIBSS-C Coon et al,25 2020 ? + ? + ?
FICA Bharucha et al,26 2004 ?
Fecal Incontinence and Constipation Questionnaire Österberg et al,27 1996 ? +
IBS-C Symptom Severity Measures Williams et al,28 2014 ? + + ?
KESS Knowles et al,29 2000 +
ODS-S Renzi et al,30 2013 + ? + +
ODS Score Altomare et al,31 2008 +
PAC-SYM Frank et al,32 1999 ? + + + +
Modified PAC-SYM Neri et al,33 2015 + + +
Rome III Criteria Questionnaire Digesu et al,34 2010 + +
VSAQ Pamuk et al,35 2003 ?
Constipation-related quality of life
CTSAT-Q Szeinbach et al,36 2009 + +
Constipation-related Disability Scale Hart et al,37 2012 + + + +
PAC-QOL Wang et al,38 2009 + + + +
E-CIS Abdul Wahab et al,39 2020 +
CRQOL Marquis et al,40 2005 ? + ? ?

PROM, patient-reported outcome measure; BF-Diary, Bowel Function Diary; BFI, Bowel Function Index; CC, chronic constipation; CAS, Constipation Assessment Scale; CRQOL, Constipation-Related Quality of Life; CSI, Constipation Severity Instrument; CSS, Constipation Scoring System; CTSAT-Q, Chronic Constipation Treatment Satisfaction Questionnaire; DIBSS-C, Diary for Irritable Bowel Syndrome Symptoms–Constipation; E-CIS, Elderly-Constipation Impact Scale; FICA, Fecal Incontinence and Constipation Assessment; IBS-C, constipation-predominant irritable bowel syndrome; KESS, Knowles-Eccersley-Scott Symptom Questionnaire; PAC-SYM, Patient Assessment of Constipation–Symptom; ODS-S, Obstructive Defecation Syndrome Score; PAC-QOL, Patient Assessment of Constipation–Quality of Life; VSAQ, Visual Scale Analog Questionnaire.

Ratings for measurement properties: +, sufficient; ?, indeterminate; −, insufficient.

Twenty-one studies had at least 1 insufficient (–) or indeterminate (?) rating, and no PROM was fully assessed in all measurement properties, with measurement error and criterion validity most commonly missing. The PROMs with the most sufficient (+) ratings include the Chinese Constipation Questionnaire, Constipation Severity Instrument (CSI), PAC-SYM, Constipation-related Disability Questionnaire, and Patient Assessment of Constipation–Quality of Life (PAC-QOL). To improve ratings of measurements properties, focus should be given to obtaining sufficient rating for measurement error (smallest detectable change/limits of agreement < minimal important change), criterion validity (correlation with gold standard or area under the curve ≥ 0.70), and responsiveness (results in accordance with hypothesis or area under the curve ≥ 0.70).

Digitization of Patient-related Outcome Measures

Five studies reported using digital formats to administer PROMs during the validation process.15,20,25,28,36 Both the Bowel Function Diary and Diary for Irritable Bowel Syndrome Symptoms-Constipation (DIBSS-C) were completed as part of an electronic diary and were administered using a handheld device. For the Bowel Function Diary, participants were given a handheld electronic personal digital assistant (PDA) device while the type of device was not specified for the DIBSS-C. The CTSAT-Q was described to be disseminated online however, no further information was provided.

The Chronic Constipation Symptom Severity Measures and IBS-C Symptom Severity Measures were administered using interactive voice response system technology, a computer-automated telephone system that collects data through spoken answers or keypad responses.41 For all 5 studies, methods of digitizing and validating the digital formats of the PROMs were not reported.

Discussion

Digitizing constipation-related PROMs represents a promising step towards individualizing patient intervention in a longitudinal and scalable manner. Therefore, the current systematic review provides an overview of constipation-related PROMs that have been developed and validated over the past 32 years. The review identified 23 different constipation-related PROMs, with 18 measuring symptom-related measures and 5 measuring constipation-related QoL measures.

The review revealed a large amount of variation between PROMs used to measure symptom-related constipation outcomes. Variations include outcome measures targeting different subtypes of constipation (eg, opioid-induced constipation, obstructive defecation syndrome, and IBS-C), functions of PROM (eg, clinical use and research purposes), methods of administration (eg, pen and paper, clinician administered, and electronic diary), length of PROM (range = 3-98 items), and recall period (eg, last 2 weeks and past 24 hours). Given the multiple possible etiologies of constipation, subtype-specific PROMs can be a useful tool to further facilitate customization of outcome measures monitoring, and to provide more accurate feedback to clinicians about treatment progress. Nevertheless, the heterogeneity of variables between PROMs can indicate the lack of standardization during the PROM development process. For instance, multiple studies scored “inadequate” for PROM development based on the COSMIN checklist due to the lack of patients’ involvement and input when developing PROM items. While physical symptoms and functioning are vital aspects of disease monitoring, patients may be more focused on regaining or preserving QoL, including emotional wellbeing and social functioning.42

Considering the known impact of constipation on QoL,9 the review also examined constipation-specific QoL-related PROMs. Similar to studies that developed symptom-related outcomes measures, none of the QoL-related PROMs reviewed met all the COSMIN quality standards for development and measurement properties. While all 5 of QoL PROMs involved patients in the development process through individual interviews or focus groups, issues include incomplete reporting of interview methods and lack of a follow-up cognitive testing session. Based on the ratings of measurement properties, out of the reviewed PROMs, the current review recommends the Chinese Constipation Questionnaire, CSI, PAC-SYM, Constipation-related Disability Questionnaire, and PAC-QOL to measure constipation symptoms or constipation-related quality of life. Nevertheless, none of the reviewed PROMs report all measurement properties indicated in the COSMIN checklist thus, there is a need for better standardization of PROM creation, from the development stages to the final reporting of validation studies. In their efforts to better regulate PROM creation and usage, the Food and Drug Administration (FDA) released a guidance for the industry on recommendations for PROM development and validation,43 and more recently, a draft on selecting, developing or adapting PROMs for medical device evaluation.44 To ensure more consistent methodological quality and reporting, it would be ideal for future PROM development studies to familiarize themselves with the FDA recommendations and COSMIN guidelines.

Despite the variety of PROMs identified in this review, only 2 PROMs were developed and validated within the Asian context. Chan et al19 developed and validated the Chinese Constipation Questionnaire in the Chinese language with an ethnic Chinese population. Similarly, Abdul Wahab et al39 developed and validated the E-CIS using the Malay language spoken in the local dialects of Terengganu and Kelantan in Malaysia. While there are translated versions of questionnaires including the PAC-SYM32 and PAC-QOL,40 culture and language are intertwined, and language should be examined in conjunction with culturally specific health beliefs and understanding.45 Regardless of English fluency, patients of different ethnic groups may differ in terms of pronunciation, speech delivery, grammar/vocabulary and culturally specific presentation styles when describing their issues to medical practitioners.45 Hence, a relatable and culturally-specific PROM can be beneficial in increasing the efficacy of patient-clinician communication, and further facilitate a more personalised, patient-centric symptoms monitoring and treatment.

The current review also assessed the modes of digital dissemination of currently available PROMs and identified 5 PROMs that used digital formats to administer the questionnaire during the validation process. Methods of dissemination were varied, ranging from electronic diary formats on PDA devices to computer-automated telephone systems. Given the widespread adoption of smart devices, such as smartphones and tablets, the use of ePROs presents as a viable option for remote monitoring led by patients themselves. Consistent symptom reporting through digital means can improve patient-clinician communication, detection of unrecognised problems, and patients’ health behaviors, including patient self-management and patient empowerment.46 Nevertheless, to ensure reliable reporting of ePROs, evidence is needed to support measurement equivalence between the electronic and paper-based PROMs.47 The 5 studies in the current review that utilized digital PROMs did not report methods undertaken to digitize the PROMs. Accordingly, there is a need for better standardization for digitization of PROMs to maximize the potential of ePRO tools. Recommendations from the International Society of Pharmacoeconomics and Outcomes Research (ISPOR) ePRO Task Force include cognitive debriefing, usability testing or full psychometric testing of the electronic versions, depending on the level of modification. Future ePRO development studies can benefit by reporting level of modification and relevant methods undertaken to ensure measurement equivalence.

With growing interests to integrate technology into healthcare, PROM development and implementation should keep pace with the fast and evolving field of digital health. Digital health has the potential to offer new modalities of probing patient state in real time through minimally invasive methods (eg, experience sampling method, day reconstruction method). Furthermore, increasing accessibility to ePROs can open doors to personalization of PROMs. A recent real-world longitudinal study of patients with rheumatic and musculoskeletal diseases allowed patients to customize their tracking on the Arthritis Power mobile application between 3 and 10 PRO symptom measures over a period of 3 months.48 While some PROM items were prioritized more, there were variations between patients in the items chosen for tracking and in ranking of importance. It should also be noted that minimal changes to items tracked were observed, suggesting that patients continue to only track symptoms that are important to them. The ePROs can further benefit from concepts commonly employed in digital health, such as gamification and behavioral nudges, to sustain users’ engagement.49 As the concept of personalized medicine continues to grow, digital technologies can aid in the continual evolution and optimization of PROMs.

A limitation of the current systematic review is the subjective nature of the COSMIN evaluation methods. Multiple items of the COSMIN checklist require subjective judgement of the reviewer based on experience and knowledge, hence, there is possibility of subjectivity within the review.50 Furthermore, we acknowledge that there may be other methods to assess psychometric measures beyond the COSMIN guidelines. Nevertheless, the current review endeavored to reduce subjectivity by utilizing 2 independent reviewers with good interrater reliability and a third for any discrepancies. Secondly, the current review did not include PROMs assessing the pediatrics population. Constipation-related pediatric PROMs rely on patient-reported measures, parent/caregiver-reported measures or a combination of both, and assessing the differences between the method of reporting is beyond the scope of the current review. Poor concordance between parent- and child-reporting have been observed when assessing gastrointestinal symptoms. For instance, children tended to rate their pain/discomfort intensity more severely than parents did, and up to 60% of parents of 10- to 19-year-olds could not answer items relating to defecation habits.51 Therefore, it would be beneficial for future studies to focus on the differences in reporting methods and the involvement of both parent and child in the PROM development process.

In conclusion, this review assessed constipation symptoms and constipation-related QoL PROMs using the COSMIN guidelines and identified a lack of consistent methodology and reporting of development and validation studies. Furthermore, more culturally-specific PROMs, especially in the Asian context, will be beneficial. There are varying modes of digital dissemination of constipation-related PROMs however, greater standardization of the process is required to ensure transparency and consistency. As PROM is a useful tool that can provide clinicians and researchers insights into patients’ health status and health-related QoL, further developments of constipation-related PROMs can be made by more consistent methodology and reporting of PROM development, increase in culturally-specific PROMs, and better reporting of protocol for digitization of PROMs.

Footnotes

Financial support: This study was supported funding from the Institute for Digital Medicine (WisDM) Translational Research Programme (Grant No. R-719-000-037-733) at the Yong Loo Lin School of Medicine, National University of Singapore (Dean Ho).

Conflicts of interest: Agata Blasiak and Dean Ho are co-inventors or previously filed pending patents on artificial intelligence-based therapy development. Dean Ho is a shareholder of KYAN Therapeutics, which has licensed intellectual property pertaining to artificial intelligence -based drug development. Others have nothing to disclose.

Author contributions: V Vien Lee: development of the parameters of the systematic review, review of articles for eligibility and quality, data analysis, and preparation of the manuscript; Ni Yin Lau: review of articles for eligibility and editing of the manuscript; David J Y Xi: review of articles for quality and editing of the manuscript; Anh T L Truong: review of articles for eligibility and editing of the manuscript; Agata Blasiak: development of the parameters of the systematic review and editing of the manuscript; Kewin T H Siah: evaluation of quality and editing of the manuscript; and Dean Ho: evaluation of quality and editing of the manuscript.

References

  • 1.Barberio B, Judge C, Savarino EV, Ford AC. Global prevalence of functional constipation according to the Rome criteria: a systematic review and meta-analysis. Lancet Gastroenterol Hepatol. 2021 doi: 10.2139/ssrn.3800856. [DOI] [PubMed] [Google Scholar]
  • 2.Camilleri M, Ford AC, Mawe GM, et al. Chronic constipation. Nat Rev Dis Primers. 2017;3:17095. doi: 10.1038/nrdp.2017.95. [DOI] [PubMed] [Google Scholar]
  • 3.Belsey J, Greenfield S, Candy D, Geraint M. Systematic review: impact of constipation on quality of life in adults and children. Aliment Pharmacol Ther. 2010;31:938–949. doi: 10.1111/j.1365-2036.2010.04273.x. [DOI] [PubMed] [Google Scholar]
  • 4.Almario CV, Spiegel BMR. Employing Irritable Bowel Syndrome Patient-Reported Outcomes in the Clinical Trenches. Clin Gastroenterol Hepatol. 2018;16:462–466.e462. doi: 10.1016/j.cgh.2017.12.026. [DOI] [PubMed] [Google Scholar]
  • 5.Dawson J, Doll H, Fitzpatrick R, Jenkinson C, Carr AJ. The routine use of patient reported outcome measures in healthcare settings. Bmj. 2010;340:c186. doi: 10.1136/bmj.c186. [DOI] [PubMed] [Google Scholar]
  • 6.Black N. Patient reported outcome measures could help transform healthcare. Bmj. 2013;346:f167. doi: 10.1136/bmj.f167. [DOI] [PubMed] [Google Scholar]
  • 7.Izumi K. The measures to evaluate constipation: a review article. Gastroenterol Nurs. 2014;37:137–146. doi: 10.1097/SGA.0000000000000034. [DOI] [PubMed] [Google Scholar]
  • 8.McCrea GL, Miaskowski C, Stotts NA, Macera L, Hart SA, Varma MG. Review article: self-report measures to evaluate constipation. Aliment Pharmacol Ther. 2008;27:638–648. doi: 10.1111/j.1365-2036.2008.03626.x. [DOI] [PubMed] [Google Scholar]
  • 9.Dennison C, Prasad M, Lloyd A, Bhattacharyya SK, Dhawan R, Coyne K. The health-related quality of life and economic burden of constipation. Pharmacoeconomics. 2005;23:461–476. doi: 10.2165/00019053-200523050-00006. [DOI] [PubMed] [Google Scholar]
  • 10.Wald A, Scarpignato C, Kamm MA, et al. The burden of constipation on quality of life: results of a multinational survey. Aliment Pharmacol Ther. 2007;26:227–236. doi: 10.1111/j.1365-2036.2007.03376.x. [DOI] [PubMed] [Google Scholar]
  • 11.Zhen J, Marshall JK, Nguyen GC, Atreja A, Narula N. Impact of Digital Health Monitoring in the Management of Inflammatory Bowel Disease. J Med Syst. 2021;45:23. doi: 10.1007/s10916-021-01706-x. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 12.Prinsen CAC, Mokkink LB, Bouter LM, et al. COSMIN guideline for systematic reviews of patient-reported outcome measures. Qual Life Res. 2018;27:1147–1157. doi: 10.1007/s11136-018-1798-3. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 13.McHugh ML. Interrater reliability: the kappa statistic. Biochem Med (Zagreb) 2012;22:276–282. doi: 10.11613/BM.2012.031. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 14.Mokkink LB, de Vet HCW, Prinsen CAC, et al. COSMIN Risk of Bias checklist for systematic reviews of Patient-Reported Outcome Measures. Qual Life Res. 2018;27:1171–1179. doi: 10.1007/s11136-017-1765-4. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 15.Camilleri M, Rothman M, Ho KF, Etropolski M. Validation of a bowel function diary for assessing opioid-induced constipation. Am J Gastroenterol. 2011;106:497–506. doi: 10.1038/ajg.2010.431. [DOI] [PubMed] [Google Scholar]
  • 16.Rentz AM, Yu R, Müller-Lissner S, Leyendecker P. Validation of the Bowel Function Index to detect clinically meaningful changes in opioid-induced constipation. J Med Econ. 2009;12:371–383. doi: 10.3111/13696990903430481. [DOI] [PubMed] [Google Scholar]
  • 17.Ducrotté P, Caussé C. The Bowel Function Index: a new validated scale for assessing opioid-induced constipation. Curr Med Res Opin. 2012;28:457–466. doi: 10.1185/03007995.2012.657301. [DOI] [PubMed] [Google Scholar]
  • 18.Abramowitz L, Béziaud N, Caussé C, Chuberre B, Allaert FA, Perrot S. Further validation of the psychometric properties of the Bowel Function Index for evaluating opioid-induced constipation (OIC) J Med Econ. 2013;16:1434–1441. doi: 10.3111/13696998.2013.851083. [DOI] [PubMed] [Google Scholar]
  • 19.Chan AO, Lam KF, Hui WM, et al. Validated questionnaire on diagnosis and symptom severity for functional constipation in the Chinese population. Aliment Pharmacol Ther. 2005;22:483–488. doi: 10.1111/j.1365-2036.2005.02621.x. [DOI] [PubMed] [Google Scholar]
  • 20.Nelson LM, Williams VS, Fehnel SE, et al. Psychometric validation of patient-reported outcome measures assessing chronic constipation. Clin Exp Gastroenterol. 2014;7:385–394. doi: 10.2147/CEG.S64713. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 21.McMillan SC, Williams FA. Validity and reliability of the Constipation Assessment Scale. Cancer Nurs. 1989;12:183–188. doi: 10.1097/00002820-198906000-00012. [DOI] [PubMed] [Google Scholar]
  • 22.Ponce J, Martínez B, Fernández A, et al. Constipation during pregnancy: a longitudinal survey based on self-reported symptoms and the Rome II criteria. Eur J Gastroenterol Hepatol. 2008;20:56–61. doi: 10.1097/MEG.0b013e3281108058. [DOI] [PubMed] [Google Scholar]
  • 23.Varma MG, Wang JY, Berian JR, Patterson TR, McCrea GL, Hart SL. The constipation severity instrument: a validated measure. Dis Colon Rectum. 2008;51:162–172. doi: 10.1007/s10350-007-9140-0. [DOI] [PubMed] [Google Scholar]
  • 24.Agachan F, Chen T, Pfeifer J, Reissman P, Wexner SD. A constipation scoring system to simplify evaluation and management of constipated patients. Dis Colon Rectum. 1996;39:681–685. doi: 10.1007/BF02056950. [DOI] [PubMed] [Google Scholar]
  • 25.Coon CD, Hanlon J, Abel JL, Lundy JJ, Carson RT, Reasner DS. Psychometric Analysis of the Abdominal Score From the Diary for Irritable Bowel Syndrome Symptoms-Constipation Using Phase IIb Clinical Trial Data. Value Health. 2020;23:362–369. doi: 10.1016/j.jval.2020.01.002. [DOI] [PubMed] [Google Scholar]
  • 26.Bharucha AE, Locke GR, Seide BM, Zinsmeister AR 3rd, author. A new questionnaire for constipation and faecal incontinence. Aliment Pharmacol Ther. 2004;20:355–364. doi: 10.1111/j.1365-2036.2004.02028.x. [DOI] [PubMed] [Google Scholar]
  • 27.Österberg A, Graf W, Karlbom U, Påhlman L. Evaluation of a questionnaire in the assessment of patients with faecal incontinence and constipation. Scand J Gastroenterol. 1996;31:575–580. doi: 10.3109/00365529609009130. [DOI] [PubMed] [Google Scholar]
  • 28.Williams VS, Nelson LM, Fehnel SE, et al. Psychometric validation of symptom severity measures in irritable bowel syndrome with constipation. Aliment Pharmacol Ther. 2014;40:298–308. doi: 10.1111/apt.12830. [DOI] [PubMed] [Google Scholar]
  • 29.Knowles CH, Eccersley AJ, Scott SM, Walker SM, Reeves B, Lunniss PJ. Linear discriminant analysis of symptoms in patients with chronic constipation: validation of a new scoring system (KESS) Dis Colon Rectum. 2000;43:1419–1426. doi: 10.1007/BF02236639. [DOI] [PubMed] [Google Scholar]
  • 30.Renzi A, Brillantino A, Di Sarno G d'Aniello F, author. Five-item score for obstructed defecation syndrome: study of validation. Surg Innov. 2013;20:119–125. doi: 10.1177/1553350612446354. [DOI] [PubMed] [Google Scholar]
  • 31.Altomare DF, Spazzafumo L, Rinaldi M, Dodi G, Ghiselli R, Piloni V. Set-up and statistical validation of a new scoring system for obstructed defaecation syndrome. Colorectal Dis. 2008;10:84–88. doi: 10.1111/j.1463-1318.2007.01262.x. [DOI] [PubMed] [Google Scholar]
  • 32.Frank L, Kleinman L, Farup C, Taylor L, Miner P Jr, author. Psychometric validation of a constipation symptom assessment questionnaire. Scand J Gastroenterol. 1999;34:870–877. doi: 10.1080/003655299750025327. [DOI] [PubMed] [Google Scholar]
  • 33.Neri L, Conway PM, Basilisco G. Confirmatory factor analysis of the Patient Assessment of Constipation-Symptoms (PAC-SYM) among patients with chronic constipation. Qual Life Res. 2015;24:1597–1605. doi: 10.1007/s11136-014-0886-2. [DOI] [PubMed] [Google Scholar]
  • 34.Digesu GA, Panayi D, Kundi N, Tekkis P, Fernando R, Khullar V. Validity of the Rome III Criteria in assessing constipation in women. Int Urogynecol J. 2010;21:1185–1193. doi: 10.1007/s00192-010-1179-0. [DOI] [PubMed] [Google Scholar]
  • 35.Pamuk ON, Pamuk GE, Celik AF. Revalidation of description of constipation in terms of recall bias and visual scale analog questionnaire. J Gastroenterol Hepatol. 2003;18:1417–1422. doi: 10.1046/j.1440-1746.2003.03155.x. [DOI] [PubMed] [Google Scholar]
  • 36.Szeinbach SL, Baran RW, Bratten J, Jones MP. Psychometric development and validation of the chronic constipation treatment satisfaction questionnaire (CTSAT-Q) Value Health. 2009;12:1004–1010. doi: 10.1111/j.1524-4733.2009.00533.x. [DOI] [PubMed] [Google Scholar]
  • 37.Hart SL, Albiani JJ, Crangle CJ, Torbit LA, Varma MG. Development and assessment of the constipation-related disability scale. Aliment Pharmacol Ther. 2012;35:183–192. doi: 10.1111/j.1365-2036.2011.04910.x. [DOI] [PubMed] [Google Scholar]
  • 38.Wang JY, Hart SL, Lee J, Berian JR, McCrea GL, Varma MG. A valid and reliable measure of constipation-related quality of life. Dis Colon Rectum. 2009;52:1434–1442. doi: 10.1007/DCR.0b013e3181a51196. [DOI] [PubMed] [Google Scholar]
  • 39.Abdul Wahab P, Mohd Yusoff D, Abdul Kadir A, Ali SH, Lee YY, Kueh YC. Psychometric evaluation of a newly developed Elderly-Constipation Impact Scale. PeerJ. 2020;8:e8581. doi: 10.7717/peerj.8581.79858831f07d4797ab81e5ef7664cfd8 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 40.Marquis P, De La Loge C, Dubois D, McDermott A, Chassany O. Development and validation of the Patient Assessment of Constipation Quality of Life questionnaire. Scand J Gastroenterol. 2005;40:540–551. doi: 10.1080/00365520510012208. [DOI] [PubMed] [Google Scholar]
  • 41.Lee H, Friedman ME, Cukor P, Ahern D. Interactive voice response system (IVRS) in health care services. Nurs Outlook. 2003;51:277–283. doi: 10.1016/S0029-6554(03)00161-1. [DOI] [PubMed] [Google Scholar]
  • 42.Kluetz PG, O'Connor DJ, Soltys K. Incorporating the patient experience into regulatory decision making in the USA, Europe, and Canada. Lancet Oncol. 2018;19:e267–e274. doi: 10.1016/S1470-2045(18)30097-4. [DOI] [PubMed] [Google Scholar]
  • 43.The United States Food and Drug Administration, author. Guidance for industry: patient-reported outcome measures: use in medical product development to support labeling claims: draft guidance. Health Qual Life Outcomes. 2006;4:79. doi: 10.1186/1477-7525-4-79. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 44.The United States Food and Drug Administration, author. Principles for Selecting, Developing, Modifying, and Adapting Patient-Reported Outcome Instruments for Use in Medical Device Evaluation, 2020. [accessed 4 Jul 2022]. Available from URL: https://www.fda.gov/regulatory-information/search-fda-guidance-documents/principles-selecting-developing-modifying-and-adapting-patient-reported-outcome-instruments-use .
  • 45.Roberts C, Moss B, Wass V, Sarangi S, Jones R. Misunderstandings: a qualitative study of primary care consultations in multilingual settings, and educational implications. Med Educ. 2005;39:465–475. doi: 10.1111/j.1365-2929.2005.02121.x. [DOI] [PubMed] [Google Scholar]
  • 46.Gandrup J, Ali SM, McBeth J, van der Veer SN, Dixon WG. Remote symptom monitoring integrated into electronic health records: A systematic review. J Am Med Inform Assoc. 2020;27:1752–1763. doi: 10.1093/jamia/ocaa177. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 47.Coons SJ, Gwaltney CJ, Hays RD, et al. Recommendations on evidence needed to support measurement equivalence between electronic and paper-based patient-reported outcome (PRO) measures: ISPOR ePRO Good Research Practices Task Force report. Value Health. 2009;12:419–429. doi: 10.1111/j.1524-4733.2008.00470.x. [DOI] [PubMed] [Google Scholar]
  • 48.Nowell WB, Gavigan K, Kannowski CL, et al. Which patient-reported outcomes do rheumatology patients find important to track digitally? A real-world longitudinal study in ArthritisPower. Arthritis Res Ther. 2021;23:53. doi: 10.1186/s13075-021-02430-0.a86df5e27f6d492a9d47ad6e0ad2db1b [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 49.Sardi L, Idri A, Fernández-Alemán JL. A systematic review of gamification in e-Health. J Biomed Inform. 2017;71:31–48. doi: 10.1016/j.jbi.2017.05.011. [DOI] [PubMed] [Google Scholar]
  • 50.Mokkink LB, Terwee CB, Gibbons E, et al. Inter-rater agreement and reliability of the COSMIN (COnsensus-based Standards for the selection of health status Measurement Instruments) checklist. BMC Med Res Methodol. 2010;10:82. doi: 10.1186/1471-2288-10-82. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 51.Caplan A, Walker L, Rasquin A. Development and preliminary validation of the questionnaire on pediatric gastrointestinal symptoms to assess functional gastrointestinal disorders in children and adolescents. J Pediatr Gastroenterol Nutr. 2005;41:296–304. doi: 10.1097/01.mpg.0000172748.64103.33. [DOI] [PubMed] [Google Scholar]

Articles from Journal of Neurogastroenterology and Motility are provided here courtesy of The Korean Society of Neurogastroenterology and Motility

RESOURCES