Validity and Reliability of the Japanese Version of the Rome III Diagnostic Questionnaire for Irritable Bowel Syndrome and Functional Dyspepsia

Abstract

Background/Aims

Reliable diagnostic instruments for measuring the presence of functional gastrointestinal disorders based on the Rome III criteria have been lacking in Japan. The aims of the present study were to translate and validate the Rome III diagnostic questionnaire which was widely used in Western countries.

Methods

The original version of Rome III diagnostic questionnaire was translated from English into Japanese through 3 independent forward translations, resolution, back translation and reconciliation of the differences. Forty-nine patients with irritable bowel syndrome (IBS), 32 patients with functional dyspepsia (FD) and 56 subjects without any current GI symptoms as controls were recruited from three hospitals located in different regions of Japan and completed the IBS and FD diagnostic modules twice within 14 days. Kappa statistic was used to assess test-retest reliability. The sensitivity and specificity of each diagnostic module for distinguishing IBS or FD patients from controls was tested.

Results

Median kappa statistics were 0.63 for the translated IBS diagnostic module and 0.68 for the FD module. The sensitivity, specificity, and positive predict value of the IBS module against physician diagnosis was 61.2%, 100%, and 100% and those of the FD module was 53.2%, 98.2%, and 94.4%, respectively. Meanwhile, IBS patients were significantly more likely to report blood in stools compared to controls (18.4% vs 1.8%, P < 0.01).

Conclusions

The IBS and FD diagnostic modules on the Japanese version of the Rome III diagnostic questionnaire are valid and reliable. Further studies are warranted to elucidate the diagnostic utility of the red flag questionnaire.

Introduction

Rome III diagnostic criteria based on subjective gastrointestinal (GI) complaints are the most widely used criteria to diagnose functional gastrointestinal disorders (FGIDs).¹ Irritable bowel syndrome (IBS) and functional dyspepsia (FD) are the most common FGIDs both in the Western and the Eastern countries.^2,3 The Rome committee critically considered the available evidence, and multinational expert opinion, when they revised the former Rome II diagnostic criteria⁴ and updated diagnosis and treatment recommendations.

The Rome III diagnostic criteria for FGIDs provide a framework for symptom-based diagnosis. The Rome committee proposed that subtyping of IBS should be based on frequency and consistency of stools,⁵ which can be assessed with the aid of the Bristol Stool Form Scale.⁶ The committee also proposed that the newly defined entities of (1) meal-induced dyspeptic symptoms (postprandial distress syndrome, PDS), and (2) epigastric pain (epigastric pain syndrome, EPS) should be treated as subgroups of FD for pathophysiological and therapeutic research purposes.³

The Rome Foundation developed a questionnaire for the Rome III diagnostic criteria for FGIDs. They also validated the questionnaire to assess the test-retest reliability of the questionnaire for all diagnoses combined and for individual questions, and to test the sensitivity and specificity of the questionnaire for identifying medically diagnosed patients with FGIDs. The Rome III diagnostic questionnaire for IBS contains 10 items and the one for FD contains 18 items; answers to questions are on an ordinal scale with individual frequency thresholds for each question.⁷ While “red flag” questions are not part of the diagnostic questionnaire, these features, if present, should prompt the clinician to consider further investigations to exclude other serious medical conditions. Briefly, the “red flag” questions include history for the past 3 months of fever, weight loss, cancer in family members, blood mixed with stool, anemia and change in bowel habit after age 50.⁸ Red flags may be useful for identifying patients who require additional diagnostic evaluation in the clinical settings. However, it has been investigated that incorporating them into the Rome II criteria did not improve sensitivity and resulted in too many missed diagnoses of IBS.⁸

Previously, we have developed and validated the Japanese translated version of the Rome II modular questionnaire based on the former diagnostic criteria for IBS.⁹ There are a few studies to investigate prevalence of FGIDs using the Rome III criteria in Japan.^10,11 However, reliable diagnostic instruments for measuring the presence of FGIDs based on the Rome III criteria, comparable to those used in Western countries, have been lacking in Japan. To make cross-cultural comparisons possible, it is indispensable to develop common diagnostic measures for FGIDs.¹² The aims of the present study were to translate the Rome III diagnostic questionnaire into Japanese and to validate them in Japanese patients with IBS and FD. In addition, we investigated whether the utility of red flag symptoms that can be incorporated into the Japanese instruments.

Materials and Methods

Translation

The Rome III diagnostic questionnaire was developed as a diagnostic instrument for functional GI disorders according to the Rome III diagnostic criteria.⁷ All answers in the Rome III diagnostic questionnaire were in ordinal scale with individual frequency thresholds except for 22 of 81 items which had a “yes” or “no” response. A score was given to each of the responses for ordinal scaled items (0, never; 1, sometimes; 2, often; 3, most of the time; and 4, always). There were 10 items for IBS diagnosis, 18 items for FD diagnosis, and 6 items for the “red flag” symptoms/conditions (blood in stools, unintended weight loss of over 10 pounds (4.5 kg), fever, symptom onset after age 50, and family histories of any GI cancer and inflammatory bowel disease) were used.

A group of the Japanese co-investigators (S.F. and M.K.) independently translated the English version of the Rome III diagnostic questionnaire into the Japanese language. Meanwhile, an investigator (S.N.) independently translated it in the same way to resolve the potential incongruity of the linguistic expressions between the Eastern and Western areas in Japan. After the descriptions of each item had been discussed with specialists, the Japanese version of the instruments was then counter-translated into English by a native speaker of English. This back-translated version was sent to the Rome Committee, compared with the original versions, and discussed to confirm that the Japanese-translated versions were comparable to the English versions of the Rome III diagnostic questionnaire. Then, the Japanese version of the instruments was approved by the authors of the original English versions.

Subjects

Forty-nine consecutive patients (21 females, mean age 40.9 years) clinically diagnosed by physicians as having IBS according to Rome III diagnostic criteria, and 32 patients with FD (26 females, mean age 53.8 years) diagnosed according to Rome III were enrolled from GI clinics in Tohoku University Hospital, Hyogo College of Medicine Hospital, and Japan Community Health care Organization Shiga Hospital, Japan. Clinical examinations including upper and/or lower GI endoscopy had been carefully performed and no abnormal findings to explain GI symptoms had been detected when diagnosed as FD or IBS. Fifty-six subjects (32 females, mean age 51.4 years) who visited at the same hospitals for an annual health check (n = 31) or treatment of mild hypertension/hyperlipidemia (n = 25) but did not have any current GI complaints and any past history of abdominal surgeries except for appendectomy were also recruited as controls. Control subjects were carefully diagnosed as not having any FGID by their physicians.

All the participants were Japanese and were 18 years of age or older. They did not have any organic GI diseases or any other severe physical or psychiatric complications. Informed consent was obtained from all the subjects. However, a study ID was assigned to insure that the investigators could not identify any subjects. This study was approved by the Ethics Committees of Tohoku University Graduate School of Medicine, Hyogo College of Medicine and Japan Community Health care Organization Shiga Hospital.

Severity of Irritable Bowel Syndrome Symptoms

The IBS symptom severity scale (IBS-SSS) was developed and is widely used in Western countries to assess the severity of lower GI symptoms and the degree to which the quality of life is impaired by IBS.¹² This instrument has five items, and the total score can range from 0 to 500. The IBS-SSS scores severity and frequency of abdominal pain, severity of abdominal distension, dissatisfaction of bowel movements, and the interference with life, with a 100-point scale (0, none and 100, worst) for each question. With the possible exception of the bowel dissatisfaction item, these symptom questions are appropriate for characterizing the severity of symptoms in FD as well as the severity of IBS. In the original English version, IBS is graded as mild (75–174), moderate (175–299), or severe (300–500) on the basis of clinical observations of IBS patients.¹³ Our group previously developed the Japanese version of IBS-SSS and has confirmed the reliability and validity of this questionnaire.⁹

Procedure

Physician diagnosis for IBS, FD, and control was coded into numbers. The diagnostic code and a study ID were written on a cover page of the first questionnaire for each subject in advance. The first questionnaire containing all 81 questions on the original complete version plus the IBS-SSS was completed by each participant during a clinic visit. Fourteen days later, all subjects were asked to complete a second questionnaire and return it by post to test for reproducibility of answers. The follow-up questionnaire included only the modules for diagnoses of IBS (10 items, the IBS diagnostic module) and FD (18 items, the FD diagnostic module).

Statistical Methods

The test-retest reliability of the questionnaire was judged with the use of the Kappa statistic to assess concordance between questionnaire responses on two separate occasions. The Pearson’s chi-square tests were used to assess the agreement between a positive diagnosis by the Japanese version of Rome III diagnostic questionnaire and a clinical diagnosis assessed by their physicians. The sensitivity, specificity, and positive predictive value of the diagnostic questionnaire for discriminating IBS patients from controls, and for discriminating FD patients from controls were also analyzed. One-way analysis of variance (one-way ANOVA) was used to assess difference in the symptom severity scores between the clinical diagnostic groups. The discriminant validity was measured by comparing the grade of severity based on the IBS-SSS between subjects with Rome-positive and Rome-negative IBS using the nonparametric correlation coefficient, Kendall’s tau-b. The proportion of each red flag symptom/condition was compared using the Fisher’s exact test between the clinical diagnostic groups.

Results

No apparent modifications were required to resolve inconsistencies between the forward translations into the Japanese language. The translation prepared by S.F. and M.K. was primarily selected as the final version at this stage. When comparing the back translation to the original English version minor adjustments in the choice of words for “bothersome” (questions 26 and 39), “retching” (question 36), and “several” (question 79) were made in the Japanese translation. The Rome foundation board approved the completed translation in January 2010 (Supplementary Figure).

Characteristics of the sample are shown in Table 1. There was no difference in gender ratio among the groups. Patients with IBS were significantly younger in age compared with FD patients or control subjects. Each patient with IBS was subtyped as IBS with diarrhea (n = 28), IBS with constipation (n = 9), mixed IBS (n = 11), or unclassified IBS (n = 1) by their physician according to dominant stool consistencies. Each patient with FD was subtyped as postprandial distress syndrome (PDS, n = 25) or epigastric pain syndrome (EPS, n = 7) by their physician based on the upper GI symptoms. None of FD patients was diagnosed as overlap of PDS and EPS.

Table 1.

Characteristics of Samples

	Controls	FD	IBS
Number (females)	56 (32)	32 (26)	49 (21)
Age (yr)	51.4 ± 2.2	53.8 ± 3.0	40.9 ± 2.3a,c
Number of subtypes of IBS
Diarrhea	-	-	28
Constipation	-	-	9
Mixed	-	-	11
Unclassified	-	-	1
Number of subtypes of FD
PDS	-	25	-
EPS	-	7	-
Number of types of non-GI controls
Annual health check	31	-	-
Mild hypertension/hyperlipidemia	25	-	-
IBS-SSS total score	30 ± 7	192 ± 16a	252 ± 16a,c
Abdominal pain severity	1 ± 1	21 ± 4a	45 ± 4a,c
Abdominal pain frequency	0.1 ± 0.0	2.7 ± 0.6a	4.3 ± 0.5a,b
Abdominal bloating	2 ± 1	42 ± 6a	38 ± 5a
Bowel dissatisfaction	18 ± 4	55 ± 6a	65 ± 4a
Interference with life	8 ± 3	46 ± 6a	62 ± 4a

Characteristics of each group were shown.

FD, functional dyspepsia; IBS, irritable bowel syndrome; PDS, postprandial distress syndrome; EPS, epigastric pain syndrome; GI, gastrointestinal; IBS-SSS, IBS symptom severity scale.

Data were expressed as mean ± SEM,

^aP < 0.01 vs Controls;

^bP < 0.05,

^cP < 0.01 vs FD.

The total score on the IBS-SSS in patients with IBS was significantly higher than FD patients or control subjects (P < 0.01, respectively; Table 1). IBS patients also showed significantly higher scores than FD patients on IBS-SSS items measuring the intensity and frequency of abdominal pain (Table 1). However, there was no difference in the scores for abdominal bloating, bowel dissatisfaction or interference with life on the IBS-SSS between IBS and FD groups. Control subjects had significantly lower symptom scores for all subscales and the overall symptom severity score on the IBS-SSS compared with IBS or FD patients (P < 0.01, respectively; Table 1).

With respect to the test-retest reliability for the Rome III diagnostic questionnaire, median kappa statistic for the IBS diagnostic module was 0.63. Median kappa statistic for the FD module was 0.68.

The instrument for the symptom-based IBS diagnostic questionnaire was in modest agreement with physician diagnosis; sensitivity of 61.2% and specificity of 100% for distinguishing IBS from health (Table 2). The positive predictive value (PPV) on the IBS module was 100%. When applied the functional bowel disorders (FBD) module on the Rome III criteria¹ instead of the IBS module, the sensitivity and specificity were 91.8% and 50.0%. PPV on the FBD module was 61.6%.

Table 2.

Agreement Between Physician Diagnosis and the Irritable Bowel Syndrome Diagnostic Module

Diagnosis	IBS	Controls	Total
Rome IBS (+)	30	0	30
Rome IBS (−)	19	56	75
Total	49	56	105

Sensitivity of the irritable bowel syndrome (IBS) diagnostic module was 61.2%, specificity (100%), and positive predict value (100%).

Data were expressed as number.

The FD module was also in modest agreement with physician diagnosis; sensitivity of 53.2%, specificity of 98.2% and PPV of 94.4% for distinguishing FD from health (Table 3). None of subjects with hypertension or hyperlipidemia (n = 25) was identified as IBS or FD. It is interesting to note that 19 of 49 (39%) patients with IBS were also diagnosed as FD when using the Rome III diagnostic questionnaire. On the other hand, 3 of 32 (9%) patients with FD were overlapping IBS.

Table 3.

Agreement Between Physician Diagnosis and the Functional Dyspepsia Diagnostic Module

Diagnosis	FD	Controls	Total
Rome FD (+)	17	1	18
Rome FD (−)	15	55	70
Total	32	56	88

Sensitivity of the functional dyspepsia (FD) diagnostic module was 53.2%, specificity (98.2%), and positive predict value (94.4%).

Data were expressed as number.

When the score on the IBS-SSS for all the participants was graded as mild graded as mild (75–174), moderate (175–299), or severe (300–500),¹³ subjects with Rome-positive IBS (n = 33) significantly had more severe symptoms compared to those who were not diagnosed as IBS (Rome-negative IBS, n = 100) using the diagnostic questionnaire (Kendall’s tau-b = 0.45, P < 0.001; Table 4).

Table 4.

Discriminative Validity With the Irritable Bowel Syndrome-Symptom Severity Scale in the Rome III Diagnostic Questionnaire (Adapted from Francis et al¹³)

IBS-SSS Score	0–74	75–174	175–299	300–500	Total
Rome IBS (+)	0	7	15	11	33
Rome IBS (−)	49	22	20	9	100
Total	49	29	35	20	133

Score on the irritable bowel syndrome-symptom severity scale (IBS-SSS) was graded as mild (75-174), moderate (175–299), or severe (300–500). Subjects with Rome-positive IBS had more severe symptoms compared to those who were not diagnosed as IBS (P < 0.001).

Data were expressed as number.

With respect to frequencies of the red flags, IBS patients were significantly more likely to report blood in stools (18.4% vs 1.8%, P < 0.01; Fisher’s exact test) (Table 5), unintended weight loss (10.2% vs 0%, P < 0.05) and new bowel symptom onset after age 50 (40.0% vs 10.8%, P < 0.05) compared to controls. FD patients were more likely to report only unintended weight loss (12.5% vs 0%, P < 0.05) compared to controls. There was no significant difference in frequency of fever or family history of any GI cancer or inflammatory bowel disease between the groups.

Table 5.

Frequencies of the “Red Flag” Symptoms

Frequency (%)	Controls (n = 56)	FD (n = 32)	IBS (n = 49)
Blood in stools	1.8	12.5	18.4b
Unintended weight loss	0	12.5a	10.2a
Fever	0	0	6.0
Bowel symptom onset after age 50	10.8	30.0	40.0a
Family history of any GI cancer	28.6	28.1	25.5
Family history of IBD	0	3.4	4.2

Frequencies of the self-reported symptoms or family histories which would suggest further evaluation to exclude organic gastrointestinal (GI) diseases were shown.

FD, functional dyspepsia; IBS, irritable bowel syndrome; IBD, inflammatory bowel disease.

^aP < 0.05,

^bP < 0.01 vs Controls (Fisher’s exact test).

Discussion

The Rome III diagnostic questionnaire was translated into Japanese using the guidelines recommended by the Rome Foundation, and the translated instrument was reviewed and approved by the Rome Foundation. In this study we evaluated the validity of the Japanese-translated Rome III diagnostic questionnaire by examining the concordance between questionnaire based diagnoses and clinical diagnoses in patients with IBS and FD attending gastroenterology out-patient clinics in Japan. We found modest sensitivity for both the IBS questionnaire module (61.2%) and the FD module (53.2%). However, the specificity (100% for IBS, 98.2% for FD) and the PPV (100% for IBS and 94.4% for FD) were excellent. This suggests that clinician-diagnosed FD may be missed by the Japanese Rome III diagnostic questionnaire, but few subjects without FD will be incorrectly classified as IBS or FD cases.

We compared the sensitivity and specificity of the Japanese Rome III Diagnostic Questionnaire to the sensitivity and specificity of the Rome III Diagnostic Questionnaire in other languages. The original English language version of the questionnaire was tested in 328 patients with IBS diagnosed by gastroenterologists and 554 healthy controls, and sensitivity-defined as the proportion of medically diagnosed IBS patients identified by the questionnaire-was 71%, while specificity-defined as the proportion of healthy controls correctly classified as non-IBS-was 88%.¹⁴ A translation of the Rome III questionnaire into the Malaysian language similarly showed a sensitivity of 81% and specificity of 100% for discriminating 31 clinically diagnosed IBS patients from 31 healthy controls.¹⁵ A translation of the Rome III questionnaire into Portuguese discriminated medically diagnosed patients with FD from health controls with a sensitivity of 91% and specificity of 95%.¹⁶ It has been reported that a sensitivity of the modified Rome III Diagnostic Questionnaire differs among the Asian countries,¹⁷ suggesting that socio-cultural perspective and/or the linguistic nuances might be taken into account to translate the original questionnaire into different languages.

In studies which have tested the ability of the Rome III questionnaire to discriminate medically diagnosed IBS patients from patients with other gastrointestinal diagnoses such as inflammatory bowel disease or FD rather than discriminating them from healthy controls, performance has been more modest: In one study using the Rome II criteria, the questionnaire’s sensitivity-defined as the ability to discriminate IBS patients from those with other gastrointestinal diagnoses-was 60% and specificity was 56%.¹⁸ In a Norwegian study, which tested the discrimination of IBS from patients with other causes of abdominal pain, sensitivity was 39% and specificity was 63%.¹⁹ On the other hand, using with the Korean version of the Rome III questionnaire, the sensitivity was 77.8% for the IBS module and 70.0% for the FD module and specificity was 81.6% and 63.0% when discriminated clinical IBS or FD patients from subjects who visited at gastroenterologists for any other reasons.²⁰ These data show that the sensitivity and specificity of the Japanese Rome III Diagnostic Questionnaire is as good as the English original and other translations of the Rome III questionnaire when similar methodologies are used to evaluate sensitivity and specificity. In this study, test-retest reliability over an interval of two weeks was acceptable with median kappa of 0.63 for IBS and 0.68 for FD. However, this is somewhat below the test-retest coefficient of 82% found for the English language version of the questionnaire.¹⁴

In the present study, frequencies of reporting “red flag” symptoms (eg, blood in stools and unintended weight loss) were concordant with previous reports.^8,21 It is not surprising that the frequency of family history of any GI cancer in our findings was relatively high in each group (25.5 to 28.6%, Table 5) since gastric and colon cancers are very common in the Japanese population.²² Notably, the red flags are not automatically cause for alarm signs. A separate, benign problem (eg, rectal bleeding caused by hemorrhoids) is often found that explains them. In the present study, no organic abnormality had been detected in the colon for all of patients who were diagnosed as IBS with colonoscopy or barium enema despite the self-reported alarm symptoms on the red flag questionnaire. Whitehead et al⁸ reported that patients with IBS or other functional bowel disorders are highly likely to report red flag symptoms, suggesting that these symptoms should not be incorporated into the symptom-based Rome criteria as exclusions. Therefore, the “red flag” questionnaires are not meant to be used for the discrimination of patients with FGIDs from patients who have any serious conditions but are used for identifying patients who may require further diagnostic testing to determine whether organic diseases are present.

A limitation of this and all other studies seeking to validate symptom based diagnostic criteria for FGIDs is that there is no biomarker to serve as a gold standard. The approach taken here of comparing questionnaire based diagnoses to medical diagnoses made by experienced clinicians has been employed in other studies, but it should be acknowledged that clinicians often disagree about the diagnosis of an FGID in a specific patient.¹⁴ A second study limitation was that estimates of sensitivity and specificity were based on the ability of the Rome III Diagnostic Questionnaire to discriminate healthy controls from patients with FGIDs. This over-estimates the ability to discriminate patients with FGIDs from patients with other gastrointestinal symptoms, which is more important to clinicians than the ability to distinguish patients with FGIDs from healthy controls. Moreover, it is also important to evaluate whether the questionnaire could differentiate different types of FGIDs. Thirdly, we did not investigate whether participants may take any medication for GI and/or the other reasons. It is unlikely that participants selected by physicians may be biased since physicians were asked to exclude FGID patients who had improved their GI symptoms and the mean severity score in IBS patients was considered as a moderate level at the index visit on the IBS-SSS,¹³ which was almost concordant with the previous findings of validation studies for IBS.^9,23,24 Lastly, IBS patients were significantly younger in age compared with FD patients or control subjects. Younger population might be more educated than older population in Japan. In the further studies, socioeconomic status such as an education level in each subject should be taken into account.

In conclusion, the present findings demonstrate that both the IBS and FD diagnostic modules on the Japanese version of the Rome III diagnostic questionnaire demonstrate good validity and reliability. In addition, our findings support the idea that patient-reported red flag symptoms should not be incorporated into the Rome criteria as exclusions. Thus, this Japanese Rome III Diagnostic Questionnaire can be used for cross-cultural comparisons on functional GI disorders between Japan and other countries.

Supplementary Materials

Note: To access the supplementary Figure mentioned in this article, visit the online version of Journal of Neurogastroenterology and Motility at http://www.jnmjournal.org/, and at http://dx.doi.org/10.5056/jnm15016.