The role of the Acute Cystitis Symptom Score questionnaire for research and antimicrobial stewardship. Validation of the Hungarian version

Abstract

Introduction

The Acute Cystitis Symptom Score (ACSS) is a new self-reporting tool to evaluate the symptoms of uncomplicated acute cystitis (AC) in women. The linguistic and clinical validation process of the Hungarian version used in this study may serve as a guide for the validation of the ACSS in other languages.

Material and methods

In this prospective cohort study, women with AC (Patients) and those without (Controls) filled in the Hungarian ACSS version, during their visits to physician's office. Statistical analysis included ordinary descriptive values, calculation of reliability, validity, discriminative ability, responsiveness (sensitivity, specificity) and comparative analysis.

Results

Thirty-one patients were recruited for validation along with 37 controls. Statistical analyses resulted in excellent values of internal consistency, discriminative ability and validity for diagnosis of AC. At the cut-off at a score of 6 in the ‘typical’ domain, positive and negative predictive values were 97% and 92%, sensitivity and specificity were 90% and 97%, respectively.

Conclusions

The ACSS has demonstrated benefits for diagnosis and patient-reported outcome assessment. It is objective, fast, and cost-effective, and may help to easily confirm the accurate diagnosis of AC. Therefore, it may be especially important for clinical and epidemiological studies on AC in women.

INTRODUCTION

Women, suffering from acute uncomplicated cystitis (AC) represent the vast majority of the cases of the urinary tract infections (UTIs) – the most widespread infectious diseases worldwide [1].

Non-standardised and subjective evaluation of symptoms of AC, inappropriate and prolonged administration of antibiotics, low adherence to international guidelines have led to excessive antibiotic prescription and inevitably to increasing antimicrobial resistance, and unfortunately, the development of novel antimicrobial agents is not expected in the near future [2]. Antibiotic stewardship programs aim to set coordinated strategies to enhance patient health outcomes, decrease the use of broad-spectrum antibiotics and to slow down the increase of antimicrobial resistance [3, 4, 5]. In order to reach these aims, standardised, high-quality investigations of available and future modalities for treatment and prevention of UTIs have to be conducted [5–9].

Clinical studies on UTIs often rely only on patients' self-diagnosis of episodes of AC. The major limitation of current evidence making the results of the studies often unreliable is the use of non-standardised, non-validated methods of self-diagnosis. Therefore, objective self-diagnosis and self-assessment of outcome are essential, as a basis for comparing the efficacy of different treatment modalities, either of antibiotic and non-antibiotic [10, 11].

The Acute Cystitis Symptom Score (ACSS) was developed under the hypothesis that diagnosis of AC can be made with high probability, based on typical symptomatology, such as frequency, urgency and dysuria, in the absence of vaginal and/or urethral discharge [2]. Uzbek and Russian versions of the ACSS were tested in Uzbek and Russian speaking female populations of the Republic of Uzbekistan. Thereafter, the ACSS was translated into and validated in German and British English languages. The ACSS and its scoring system has demonstrated high values of reliability, validity and discriminative abilities in all studies held in Uzbekistan, Germany and Great Britain [12–16]. The evaluation of the ACSS in other languages, such as Polish, Romanian, Ukrainian, and American English is in preparation.

Our study was designed as a prospective cohort study of the associations between symptomatology and diagnosis of AC, and the assessment of its outcomes in women, with the ACSS used as a standardised tool.

This paper mainly encompasses an internal validation part of the study, which aimed to develop a Hungarian version of the ACSS. The current study may also serve as an example and methodological guide for linguistic and clinical validation of the ACSS in other languages.

MATERIAL AND METHODS

The Acute Cystitis Symptom Score questionnaire

The ACSS contains 18 questions (items), which are divided into 4 domains: 6 items regarding typical acute cystitis symptoms (‘Typical’ domain), 4 items for differential diagnosis (‘Differential’ domain), 3 items on quality of life (‘QoL’ domain) and 5 additional questions regarding other relevant circumstances, such as menses and pregnancy (‘Additional’ domain). The first 3 domains are designed and scored in a Likert-type scale in order to measure severity of symptoms, while the items of the last domain are designed as dichotomous requiring only simple ‘Yes/No’ answers.

Translation process

The translation and linguistic validation of the Hungarian version of the ACSS was performed in accordance with the Linguistic Validation Manual for Patient-Reported Outcomes (PRO) Instruments guidelines [17]. The validated Russian version of the ACSS was taken as a source. First, two independent primary, forward translations into the Hungarian language were produced by two professional translators, followed by a consultative meeting between the two primary translators and the local project manager to obtain a consensus ‘provisional’ version. Then the provisional version was back-translated into the source language by an independent translator, which was compared with the source version of the questionnaire, as a test. This step was followed by pilot testing of the corrected provisional version of the questionnaire. The notes of the patients were then discussed and the final version of the questionnaire was created. The validated British English version was taken into account as a reference before the final Hungarian version was accepted. The process of translation and linguistic validation was published by our workgroup in details earlier [12–16, 18].

Ethical approval

The protocol of the validation process was approved by the Egészségügyi Tudományos Tanács (ETT) TUKEB Medical Research Council, Budapest Hungary on 24th February 2015 (Approval number: 34/2015.–6423/2015/EKU) and the Local Ethical Committee of the tertiary care hospital, where the work was conducted. The research was performed in accordance with the 1964 Helsinki declaration and its later amendments or comparable ethical standards. Before inclusion into the study, all patients and subjects were requested to sign written patient informed consent.

Pilot test

A pilot test of the translated Hungarian version of the ACSS was carried out in 6 female respondents, of different ages, with different levels of education and belonging to different social groups, who had experienced AC in their history at least once.

Recruitment

Female respondents aged 18 years and older, who visited the urological outpatient clinic of a Hungarian tertiary care hospital from September 2015 to February 2016, diagnosed with uncomplicated AC (Patients) were enrolled along with healthy women without AC or any significant urological condition, not visiting the hospital as patients (Controls).

Patient groups

For allocation of the respondents into groups and further analysis of responsiveness, a project supervisor and two members of the research team were chosen. One of them (A.B.) had access to the case histories and the results of respondents' clinical and laboratory investigations, but was blinded to the results of the questionnaire survey while the second member (A.M.) was blinded to all results of respondents' investigations apart from the ACSS test results and the final diagnosis of the urologist. Based on information given to them, they have made independent diagnostic decisions whether the respondent had or did not have AC. Their decisions were documented and compared by the research supervisor (P.T.). In cases when their opinions coincided true negative (both of them decided that the patient did not have AC) or true positive (both of them decided that patient did have AC), diagnoses were marked. All disagreements were discussed with the project supervisor and a final decision was achieved by consensus. Using this algorithm, respondents were divided into two groups: control group (Controls) and acute cystitis group (Patients).

Examination, data collection

The diagnosis of AC was strictly adjusted to the European Association of Urology Guidelines [2]. The criteria and the examinations performed for the diagnosis of AC in this study are summarised in Table 1.

Table 1.

Criteria and examinations performed for the diagnosis of acute cystitis

Examination type	Criteria
Focused medical history exploration and standard urological physical examination (performed under conditions of clinical practice)	history of lower urinary tract symptoms (dysuria, frequency and urgency, suprapubic pain, hematuria, cloudy/foul smelling urine), exclusion of asymptomatic bacteruria absence of vaginal discharge or irritation no risk factors for complicated urinary tract infections no signs of acute pyelonephritis, absence of fever, systemic symptoms or flank pain
Microscopy of centrifuged urine sediment or urine dipstick test (mid-stream clean catch urine samples)	Sediment examination: at least one of the following conditions had to be true: 6 or more leucocytes per unit/high power field (HPF) 100 or more bacteria per unit/HPF 3 or more red blood cells per unit/HPF in urine sediment OR Urine dipstick test suggesting urinary tract infection (nitrite, leukocyturia, haematuria)
Urine culture (mid-stream clean catch urine samples)	103 or more CFU of uropathogens in 1 ml of unspun urine
Additional examinations, if indicated, to exclude other conditions than acute uncomplicated cystitis	In the case of atypical symptoms the physician in charge (expert urologist specialist) decided if additional diagnostic studies were necessary: kidney and bladder ultrasound kidney-ureter-bladder x-ray other clinical and/or instrumental diagnostic procedures and tests

In addition to the urological examinations, the patients were asked to fill out the ACSS questionnaire (Part A).

Appropriate therapy, for those women who had AC, was prescribed according to European Association of Urology Guidelines [2]. Patients were suggested to come for test-of-cure (TOC) visit after finishing of prescribed therapy. During TOC, they were requested to fill out the ‘follow-up’ form (Part B) of the ACSS.

All data were recorded into an electronic database via the latest version of a specific client software (e-USQOLAT) [19].

Statistical analysis

Data obtained from ACSS survey were analysed using Statistical Package for the Social Sciences (IBM SPSS Statistics for Windows, Version 21.0. IBM, GmbH, Ehningen, Germany). Statistical analysis included ordinary descriptive statistical values (average values such as means and medians, etc.).

Calculations of Cronbach's alpha, split-half reliability and Spearman-Brown prophecy were used for the assessment of internal consistency (reliability analysis) of the Hungarian ACSS [20, 21]. Splitting into halves was performed in dependence of odd end even ordinary numbers of items.

Validity and discriminative ability were evaluated via calculation of responsiveness (sensitivity and specificity), using area under receiver operating characteristic curve (AUC) analysis along with positive and negative likelihood ratios as well as diagnostic odds ratio. Two by two (2x2) tables were used with taking the acute cystitis as an ‘exposure’, and scores of the ACSS – as an ‘outcome’.

Normality of distributions was assessed visually and numerically, using Q-Q plots and Shapiro-Wilk test [22].

Comparative analysis was performed using Mann-Whitney's U (non-parametric) and Student's t (parametric) tests [23, 24]. Differences between variables were measured using standard deviations and 95% confidence intervals (95% CI). Statistical significance of differences was evaluated using P-value; substantive significance was estimated via effect size calculation by correlation coefficient (rho).

Success and non-success rates were defined using individual criteria. The definitions along with the results are presented in Table 2.

Table 2.

Success and non-success rates using individual criteria

Mode	Domain(s)	Definition of Success (Scores)	Success N (%)	Non-Success N (%)
1	Dynamics	≤1	21 (91.3%)	2 (8.7%)
2	Main symptomsa	≤3, but no item >1 (mild)	20 (87%)	3 (13%)
3	Typicals	≤4, but no item >1 (mild)	20 (87%)	3 (13%)
4	QoL	≤3, but no item >1 (mild)	22 (95.7%)	1 (4.3%)
5	Typicals+QoL	≤7, but no item >1 (mild)	20 (87%)	3 (13%)
6	Typicals/ QoL	≤4/≤3, but no item >1 (mild)	20 (87%)	3 (13%)

^aMAIN SYMPTOMS include TYPICALS 1–3 only: frequency, urgency, painful urination

QoL – Quality of Life; N – number

Substantive significance was estimated via effect size calculation by correlation coefficient (rho) and Cohen's d. The statistical power of the test between cases (Patients vs Controls) and within cases (Visit 1 vs. Visit 2) for the ACSS domains was assessed using Wilks' lambda.

RESULTS

Translation and linguistic validation

The process of translation and linguistic validation of the ACSS resulted in the final Hungarian ACSS. After approval by hospital authorities, the final version was used for the pilot test. Both the Hungarian and the British English versions of the ACSS are available at the ACSS website [25].

Pilot test

All six respondents of the pilot test have found the questionnaire to be understandable, and the scale to be adequate and clear in that they could not have answered it more than one way, what may judge for this Hungarian version of the ACSS to be used as final version for the clinical validation study.

Demography

Sixty-eight Hungarian women were recruited for validation. Thirty-seven of them were recognised as having no acute cystitis (Controls), whereas diagnosis of AC was approved in 31. The median (range) age of Controls and Patients was 48 (19–85) and 42 (18–78) years, respectively.

‘Typical’ domain

Cronbach's α for ‘Typical domain was 0.89 (95% CI; 0.84 to 0.93), the correlation between forms was 0.86, the Guttman split half and the Spearman-Brown coefficients were 0.91 and 0.93 respectively.

The mean score achieved by the Controls was 0.84 ±1.79, while Patients achieved 9.42 ±3.33 (p <0.0001, Table 3). Results of comparative analysis of typical symptom scores between Patients and Controls are demonstrated in Figures 1 and 2.

Table 3.

Differences in scores of the different domains of the Acute Cystitis Symptom Score (ACSS) between groups of Patients and Controls and comparison between two visits

Typical scores	Controls	Patients	P value	Cohen’s d/effect-size r (power)	Patients’ Visit 1	Patients’ Visit 2	P value	Cohen’s d/effect-size r (power)
Number	37	31	23	23
Range	0 to 9	3 to 16	<0.0001	3.20/0.85 (1.00)	6 to 16	0 to 9	<0.0001	3.52/0.87 (1.00)
Mean ±SD	0.84 ±1.79	9.42 ±3.33			9.86 ±2.89	1.05 ±2.04
95% CI for Mean	0.24 to 1.43	8.20 to 10.64			8.54 to 11.17	0.12 to 1.97
Median	0	10.00			10.00	0.00
0.25 percentile	0	6.00			7.00	0.00
0.75 percentile	1	11.00			11.00	1.25
Differential scores	Controls	Patients	P value	Cohen’s d/effect-size r (power)	Patients’ Visit 1	Patients’ Visit 2	P value	Cohen’s d/effect-size r (power)
Range	0 to 2	0 to 5	<0.0001	0.94/0.43 (0.98)	0 to 4	0 to 2	0.021	0.65/0.31 (0.74)
Mean ±SD	0.11 ±0.46	1.03 ±1.30			0.86 ±1.11	0.29 ±0.56
95% CI for Mean	-0.04 to 0.26	0.55 to 1.51			0.35 to 1.36	0.03 to 0.54
Median	0	1.00			1.00	0.00
0.25 percentile	0	0.00			0.00	0.00
0.75 percentile	0	2.00			2.00	0.25
Quality of Life (QoL) scores	Controls	Patients	P value	Cohen’s d/effect-size r (power)	Patients’ Visit 1	Patients’ Visit 2	P value	Cohen’s d/effect-size r (power)
Range	0 to 8	0 to 9	<0.0001	2.14/0.73 (1.00)	0 to 8	0 to 6	<0.0001	2.06/0.77 (1.00)
Mean ±SD	0.84 ±1.74	4.94 ±2.08			4.95 ±2.20	1.05 ±1.53
95% CI for Mean	0.26 to 1.42	4.17 to 5.70			3.95 to 5.95	0.35 to 1.75
Median	0	5.00			6.00	0.00
0.25 percentile	0	6.00			3.00	0.00
0.75 percentile	0.50	6.00			6.00	2.00
Typical+QoL scores	Controls	Patients	P value	Cohen’s d/effect-size r (power)	Patients’ Visit 1	Patients’ Visit 2	P value	Cohen’s d/effect-size r (power)
Range	0 to 13	6 to 25	<0.0001	3.08/0.84 (1.00)	6 to 22	0 to 12	<0.0001	3.38/0.86 (1.00)
Mean ±SD	1.68 ±3.18	14.35 ±4.86			14.81 ±4.40	2.10 ±2.98
95% CI for Mean	0.62 to 2.74	12.57 to 16.14			12.81 to 16.81	0.74 to 3.45
Median	0	15.00			15.00	1.00
0.25 percentile	0	10.00			11.00	0.00
0.75 percentile	2.50	17.00			17.00	3.00

Figure 1.

Mean score comparison of typical symptoms between patients and controls.

Figure 2.

Sum scores (box-and-whisker plots) of ‘typical’ symptoms among patients and controls with cut-off line.

ROC analysis resulted in AUC = 0.99 (95% CI; 0.96 to 1.0; p <0.001). The most representative sign for AC according to our data was painful urination and it was observed in 78% of Patients. It also showed the highest AUC, while ’haematuria’ had the lowest.

‘Differential’ domain

The Cronbach's α value was 0.45 (95% CI; 0.17-0.65). The mean total score, achieved by Patients (1.03 ±1.3) was significantly higher than that achieved by Controls (0.11 ±0.46, p <0.0001) (Table 3). The AUC for this domain was 0.74 (95% CI, 0.6 to 0.87, p = 0.001).

’Quality of Life’ domain

The reliability in the category ‘quality of life’ was high: Cronbach's α was 0.95 (CI 95%; 0.93 to 0.97). Mean total score was 0.84 ±1.74 vs. 4.94 ±2.08 in Controls vs. Patients respectively (p <0.0001, Table 3). The AUC was 0.94 (95% CI: 0.88 to 1.00).

‘Typical’ and ‘Quality of Life’ domains

Since the ‘QoL’ domain consists of three items, it is not reasonable to apply the analysis of split-half reliability for this domain. Therefore, we performed this analysis for combined ‘Typical’ and ‘QoL’ domains. The Cronbach's α value for this ‘combined domain’ (9 items) was 0.93 (CI 95%: 0.91 to 0.96), with 0.92 for the first part and 0.80 – for the second part. Correlation between parts was 0.93, Spearman-Brown coefficient was 0.96, and the Guttman's split-half coefficient was 0.93.

Analysis of validity

For prediction of acute cystitis, at cut-off score 6 of Typical domain, positive and negative predictive values were 96.55% and 92.31%, sensitivity and specificity were 90% and 97%, respectively.

Follow-up visit, comparison between the two visits

Twenty-three (74%) members of the Patients group came back for TOC visit, with 61% patients who felt back to normal and 30% felt much better. The average interval between visits was 15 days. Results of comparative analysis of the ACSS scores between the two visits are presented on Table 3.

Table 2 represents various possibilities to differentiate between success and non-success, using part B of the ACSS (23 Patients treated for AC). Application of four different modalities (main symptoms, Typicals, Typicals + QoL, Typicals/QoL) revealed the same numbers of patients showing success and non-success in 20 (87%) and 3 (13%) of patients, respectively. These results were proven by clinical investigation of mentioned Patients. A very similar pattern was found, when patient-reported outcome was assessed earlier using the Russian and Uzbek versions of the ACSS [13].

Since ‘Typical’ domain have shown excellent results concerning sensitivity and specificity for diagnosing AC (the cut-off score of 6), it may be reasonable to use the same domain at a score of ≤4, but no item >1 for patient-reported outcome of success of treatment.

Statistical power and effect size analysis

The results of the analysis are presented in Table 3.

DISCUSSION

Emerging antibiotic resistance of uropathogens is a serious and well-known problem. The excessive use of broad-spectrum antibiotics leads to increased bacterial resistance. Development of multi-drug resistant bacteria results in higher rate of therapeutic failure and leads to administration of broader spectrum antibiotics for empirical treatment. Broader spectrum antibiotics, such as carbapenems, fluoroquinolones or cephalosporins, however, should be saved for patients with more severe infections, whereas patients with benign infections like AC should be treated initially with narrow spectrum antibiotics in accordance with the actual guidelines, availability and local patterns of susceptibility [2]. Therefore, fast and unequivocal diagnosis and outcome assessment of uncomplicated AC is extremely important both for clinical and research purposes. It is especially reasonable, since the first Phase 3 studies comparing antibiotic versus non-antibiotic treatment have shown that symptomatic treatment by itself may become an accepted treatment modality in the future [11, 26].

Validated patient questionnaires are becoming increasingly popular in all fields of modern medical practice. Specific questionnaires, such as International Prostate Symptom Score (I-PSS) [27] and the International Index of Erectile Dysfunction (IIEF) [28] – translated and validated in several languages, have become widely used instruments of urological examination, and are inevitable in the course of comparing different treatment strategies. However, up to most recently there has been no widespread validated questionnaire suitable for diagnosis and outcome of AC.

The Acute Cystitis Symptom Score questionnaire was initially reported in 2013. It evaluates patient symptoms, estimates the effect of the disease on quality of life and contributes to differential diagnostics. Originally developed in Uzbekistan in Uzbek language, the ACSS is now already translated and clinically tested in Russian, British English and German languages [12–16, 18]. The ACSS nowadays is filled out by hundreds of female patients suffering from AC and has proved to be valuable in clinical practice and was also included in the updated German guidelines on uncomplicated UTI [29].

The aim of the current study was to perform the linguistic and clinical validation of the Hungarian version of the ACCS. The study revealed that the Hungarian version of ACSS is well-designed and the questions are clear and understandable. The statistical power and effect size analysis revealed, that the number of the respondents and their allocation was appropriate for the validation study. The validation process of the translated Hungarian ACSS version has demonstrated excellent values of internal consistency, discriminative and predictive abilities, and validity for diagnosis of AC in women. Values of interclass correlation were also very good. The analysis of responses and the symptoms showed significant differences between control and AC group in each category. Strong correlation between different categories was observed.

The ‘Typical’ symptoms were highly specific and almost exhaustive predictors of AC. The leading symptom for AC according to Hungarian data is painful urination, observed in 78% of patients with AC. Cut-off score of 6 of ‘Typical’ domain can be excellently used to differentiate between cases positive and negative for AC, thanks to high predictive values (96.55% and 92.31%, respectively). As well, ‘Typical’ domain, at a cut-off score of 4 or lower, is reliable to assess effectiveness of the therapy, at the test-of-cure visits, either in combination with ‘Quality of Life’ domain or not.

The current study was performed at a single centre, which may be considered a limitation. This also explains the relatively low number of participants. In addition, 8 of 31 patients did not return for any control visit and therefore did not fill in the second part (control visit) of the ACSS. Nevertheless, the similarity of results of the current study to those of previous studies, may judge that the effect of possible selection bias if any, is however non-significant and the results are representative. Moreover, high sensitivity, specificity, association between laboratory tests and questionnaire results, the clear difference in scores between control group and patient group, all these together suggest, that the questionnaire is able to describe the dynamics of the clinical condition very well.

In our study, we have validated the Hungarian version of the ACSS not only linguistically but also clinically. Thus, the Hungarian ACSS can now be used as an effective tool in the diagnosis and outcome assessment of AC in Hungarian speaking women for clinical and research purposes not only by urologists, but also by gynaecologists and general practitioners in their clinical practice. Methodology of validation, described in this paper, may be used as an example for translation and validation of the ACSS questionnaire into other languages.

CONCLUSIONS

The ACCS is an easy, fast, cost-effective tool and might be used as standardised tool in UTI research and clinical practice. The ACSS provides an objective evaluation of diagnosis and patient-reported outcome assessment, and is therefore, especially important for both analytical and descriptive studies, such as clinical trials and epidemiological studies. For this reason, we recommend to translate and validate the ACSS questionnaire into other languages as well.

Conflicts of interest

Funding

Alidjanov JF has received a 1-year Clinical/Lab Scholarship grant from European Urological Scholarship Programme for his Research Project on urinary tract infections. Wagenlehner FME has received consultancy fees and study support (third-party funding) from Achaogen, Astellas, AstraZeneca, Bionorica, Calixa Pharmaceuticals, Cerexa Pharmaceuticals, Cubist Pharmaceuticals, LEO, MSD, Pfizer, Rempex Pharmaceuticals, Rosen Pharma, Shionogi, and Vifor Pharma. Naber KG has received personal fees from Basilea, Bionorica, Boehringer Ingelheim, Cubist/MSD, DaiichiSankyo, Enteris, Biopharm, Galenus, Helperby, Leo Pharma, Melinta, MerLion, OM Pharma/Vifor, Paratek, Pierre Fabre, Roche, Rosen Pharma, Shionogi, and Zambon.

The other authors did not have any conflicts of interest.