Validation of the Turkish Version of the Caffeine Use Disorder Questionnaire in an Adult Population

Abstract

The aim of this study was to examine the reliability and validity of the Turkish version of the Caffeine Use Disorder Questionnaire (CUDQ) in an adult population. In this instrument validation study, a total of 310 individuals (253 female, 57 males), with a mean age of 25.96 ± 6.29 years were included. The questionnaire consisted of 4 parts, comprising the sociodemographic characteristics, CUDQ, caffeine withdrawal symptoms, and caffeine consumption. For the evaluation of the data, confirmatory factor analysis (CFA), descriptive statistics, and the t test were used. In the reliability analysis, the Cronbach alpha internal consistency coefficient was 0.86, and the intraclass correlation coefficient was 0.83 for CUDQ. The CMIN/df was 0.54, and the model generally fits well to the structure (RMSEA = 0.08, CFI = 1, NFI = 1, GFI = 0.99, AGFI = 0.99, TLI = 1, NNFI = 1, RFI = 0.98). The findings suggested that the CUDQ has validity of structure, internal consistency, and construct validity for assessing Caffeine Use Disorder the tendency in the Turkish adult population.

Graphical abstract

Supplementary Information

The online version contains supplementary material available at 10.1007/s11469-021-00688-8.

Caffeine, a natural psychostimulant alkaloid that is found in more than 60 plants, including coffee beans, tea leaves, kola nuts, and cocoa pods, is a widely used bioactive substance worldwide (Heckman et al., 2010; National Coffee Association of USA (NCA), 2012). Important dietary sources of caffeine include coffee, tea, yerba mate, caffeinated soda (cola type), and energy drinks (Zucconi et al., 2013). The caffeine concentration in products varies greatly, but coffee has the highest caffeine content when compared to other beverage categories (Mitchell et al., 2014). Among addictive substances, coffee is the most common legally used beverage in the world. The absence of any restrictions on its sale and use further increases its consumption (Uzbay, 2015). More than 8 million tons of coffee was consumed worldwide in 2014 (NCA, 2018). In Turkey, since 2013, it is known that coffee consumption has increased rapidly (NCA, 2018).

The potential health consequences of the intake of caffeine are the subject of scientific research. There are several statements from health authorities (e.g., European Food Safety Authority (EFSA), Health Canada, US Food and Drug Administration (FDA)) that the general population of healthy adults are not at risk for the potential side effects of caffeine (Verster & Koenig, 2018). According to comprehensive research on caffeine safety, it has been shown that a caffeine intake of up to 400 mg per day for healthy adult individuals is not associated with adverse side effects (Nawrot et al., 2003; Rotstein et al., 2013). The EFSA has confirmed that an average daily intake of 400 mg of caffeine for an adult does not cause side effects (Agostoni et al., 2015). Epidemiological studies support the fact that moderate caffeine intake plays a beneficial role in reducing the risk of several chronic diseases (Parkinson’s disease, type 2 diabetes, chronic kidney disease, and cognitive disorders), but it should be noted that excessive intake could possibly be harmful to at-risk groups (children, adolescents, pregnant, and/or lactating women) (Grosso et al., 2017; Poole et al., 2017; Reis et al., 2019; Seifert et al., 2011; Srithongkul & Ungprasert, 2020). Although the short-term effects of caffeine on health (acute elevation of blood pressure, stimulation of the central nervous system, increased metabolism, etc.) are well known, the long-term effects, depending on the amount, are not yet clear (Daneschvar et al., 2020; Grosso et al., 2017; Higdon & Frei, 2006). Caffeine is being studied specifically for its effects on alertness and cognitive performance. However, in some people, if consumed late in the day, it can have a dose-dependent negative effect on sleep (Lieberman et al., 2019). The positive effects of caffeine consumption are mainly related to psychomotor speed, but it is suggested the withdrawal reversal mechanism in individuals who habitually consume caffeine. Since caffeine acts as a stimulant, its psychoactive effects and withdrawal symptoms when it is not used by habitual users make them mandatory (Luciano et al., 2005).

It was noted in the literature that caffeine, like other psychoactive substances, has an addictive potential. Striley et al. (2011) argued that including caffeine addiction in the Diagnostic and Statistical Manual of Mental Disorders, Fourth Edition (DSM-IV) could be clinically beneficial (Striley et al., 2011). The DSM-IV does not specifically recognize substance abuse applied to caffeine, although several studies have applied the substance abuse criteria defined as meeting at least 3 of the following criteria: tolerance, withdrawal, consuming larger amounts than intended, an inability to quit, too much time spent obtaining or getting rid of caffeine, significant activities are reduced/discontinued due to caffeine, and continued use despite a problem (Meredith et al., 2013). In the Diagnostic and Statistical Manual of Mental Disorders, Fifth Edition (DSM-5), caffeine use disorder (CUD) was added due to preclinical and clinical evidence in Section 3  (Further Working Conditions section) (American Psychiatric Association, 2013).

In the world and in Turkey, caffeine addiction occurs due to increased caffeine consumption and has become a serious issue. Therefore, it is important to help people who are seeking treatment or who wish to quit or reduce their caffeine consumption and have an appropriate scale to measure the full range of CUD. The aim of this study was to examine the reliability and validity of the Turkish version of the Caffeine Use Disorder Questionnaire (CUDQ), which was developed in line with the symptoms of CUD recommended in the DSM-5 in the adult Turkish population.

Material and Method

This study was an instrument validation study. First, permission for use of the scale was obtained via e-mail from Ágoston et al. (2018), who developed it. Then, the adaptation process was carried out in 2 stages: cross-cultural adaptation process and psychometric testing. The adaptation and translation process was conducted according to the methodology of Beaton et al. (2000).

Stage 1: Cross-Cultural Adaptation Process

The cross-cultural adaptation process was done in 5 steps:

• Step I: Two independent translations were made by a bilingual translator and a bilingual academician whose mother tongue is Turkish. These 2 translators provided additional comments to highlight difficult statements or ambiguities.

• Step II: Based on the versions of the first translator (T1) and the second translator (T2), a synthesis of these translations was made (a joint translation, T-12, was produced) by the 3 translators, with a written report that carefully documented the synthesis process.

• Step III: Beginning with the T-12 version of the questionnaire and completely blind to the original version, a translator then conducted back translation into the original language.

• Step IV: A committee of 3 experts (a health professional, a translator, and an academic) evaluated all versions of the questionnaire to combine and determine what would be considered the final version of the questionnaire for field testing. This expert committee evaluated it in terms of 4 equivalences (semantic equivalence, idiomatic equivalence, experiential equivalence, conceptual equivalence).

• Step V: All of the reports were evaluated by the researchers conducting the study. There were no major changes made during this cultural adaptation.

In the pre-test, a pilot study was performed with 10 individuals selected from the target population to obtain initial assessment of the scale and determine the time required and the difficulty involved in responding in an online survey. Feedback was requested for the participants to suggest any changes that might be necessary.

Stage 2: Psychometric Testing

This study was conducted between August and November 2020 (hence, during the COVID-19 pandemic). Therefore, all of the data were collected online. The survey was constructed using Google Forms. An online survey was circulated on different online lists and social media platforms (Instagram, Twitter, and Facebook). Moreover, the online survey was distributed and posted in online community platforms such as university forums. Informed consent was obtained electronically before the data were collected from the participants. Those who did not give informed consent were not included in this study. It was also stated that the participants could fill out the questionnaire only once. The inclusion criteria included being over 18 years of age, having consumed caffeine in the last year at least 1 time, not being pregnant/lactating, being a Turkish native, and currently living in Turkey. The procedures used in this study adhered to the tenets of the Declaration of Helsinki. The research protocol of the study was approved by the Ethics Committee of the Ankara University (number:56786525-050.04.04/85330).

In the factor analysis by Nunnally (1994), the sample size was 10 times the number of items, while in that by Gorsuch (1983), the sample size was 15 times the number of items. Tavsancıl (2002) stated that the sample size should be between 5 and 10 times the number of items. Therefore, it was aimed to reach at least 150 individuals. A convenience sample of the general Turkish population was recruited via social media platforms to participate in the present study. Data were collected from 310 individuals (aged 18 to 57 years), and 10 individuals were excluded from the study because of they did not meet the inclusion criteria.

Measures

An anonymous online questionnaire was developed by the researchers for this study. First, the individuals were asked about their age, gender, and marital and smoking status. Second, to determine their caffeine consumption, the visual frequency of food consumption was revised, evaluating a comprehensive beverage containing caffeine developed by McIlvain et al. (2011) of the different sizes of the widely sold beverage. The questions asked how many of each size were consumed daily by those who completed the questionnaire. The question was: “What is the average number of each drinks that you consumed last month?” For example, if you drank cola, how many 250 cubic centimeter (cc) drinks, 160 cc drinks, 250 cc drinks, 330 cc bottles, or 500 cc drinks did you consume every day on average? Then, the total was calculated according to the amount consumed in cc and the amount of caffeine in each beverage. Following this, the consumption amounts of the foods that contain caffeine were asked about in terms of the number of grams. Caffeine amounts were calculated using the United States Department of Agriculture (USDA) International Nutritional Composition Database, and the products measurements were based on studies on the labels and caffeine contents of foods (Fredholm et al., 1999; USDA accessed, 2021).

Third, for the caffeine withdrawal symptoms (headache, fatigue or drowsiness, depressed mood or irritability, difficulty concentrating, flu-like symptoms, and inconvenience or suffering in daily life), 7 dichotomous items were used to assess whether the participants experienced this after not drinking caffeine for 24 h or more during the past year (Heinz et al., 2009).

Finally, the CUDQ was applied (Ágoston et al., 2018). A maximum likelihood confirmatory factor analysis (CFA) was then conducted to evaluate the unidimensional construct validity. The CUDQ is a 10-item scale and has a stable unidimensional structure with robust psychometric properties. Factor loads (0.46 and 0.78) and corrected item-total correlation (0.47–0.56) of the CUDQ were found to be acceptable. The internal consistency and test-retest reliability (α = 0.87 and intra-class correlation coefficient (ICC) = 0.83) of the scale were at acceptable levels. The participants were asked to indicate their level of agreement with the statements using a 4-item Likert-type scale. The answers included “never,” “sometimes,” “often,” and “very often.” The Likert-type scale ranged from 1 (never) to 4 (very often). The total raw scores could range from 10 to 40. The higher the score, the greater the CUD was.

Data Analysis

The validation analysis of the CUDQ, which was adapted to Turkish, was conducted. Since the developed scale was defined as a 1-factor structure, validations were also performed on one dimension. In the study, first, the frequency analysis results of demographic variables and descriptive statistics for the scale items were given. A check was performed to determine whether there were any missing data, and no missing data were found. Internal consistency, which is “the degree of interrelatedness among the items” (Mokkink et al., 2010), was calculated with Cronbach alpha. In addition, it was calculated the ICC a concordance index for continuous data. CFA was used to test construct validity of the adapted scale. Since the scale data were categorical, the diagonal weighted least squares (DWLS) technique was used in the estimation stage. In addition, the independent samples t test was used to examine the relationship of the adapted scale scores with the symptoms. The t test, which is a parametric test, was chosen because the number of observations of the groups with regard to the symptoms was higher than 30. All of the statistical analyses were obtained using the R-Project program (R Core Team, 2020) and Lavaan (Rosseel, 2012) package. The test results of the study were evaluated at a 95% confidence level.

Results

The target population consisted of a total of 310 Turkish individuals who were between 18 and 65 years of age. The frequency distributions and descriptive statistics of the demographic information of the individuals participating in the research are given in Table 1. The majority of the sample was female (81.6%), and 79.0% of the individuals were single. The mean age of the individuals was 25.96 ± 6.29 years, and the mean body mass index (BMI) was 23.23 ± 9.95 kg/m².

Table 1.

Frequency distribution of demographic information of individuals

Variables	n (%) or $\overline{\mathrm{X}}\pm \mathrm{Sd}$
Age (years)	25.96 ± 6.29
BMI (kg/m2)	23.23 ± 9.95
Gender
Female	253 (81.6)
Male	57 (18.4)
Marital status
Single	245 (79.0)
Married	65 (21.0)
Chronic disease
Yes	46 (14.8)
No	264 (85.2)
Smoking status
Yes	57 (18.4)
No	253 (81.6)
Using nutritional supplements
Yes	68 (21.9)
No	242 (78.1)

Values are expressed as n (%) or mean ± SD

The fit indices of the CUDQ, which was adapted to Turkish, were calculated as a result of CFA. In CFA, the chi square goodness of fit index was calculated as 18.829 (df = 35, P = 0.988). The other goodness of fit indices comprised the following: CFI = 1, GFI = 0.99, AGFI = 0.99, TLI = 1, NNFI = 1, RFI = 0.98, RMSEA = 0.000, and SRMR = 0.05. According to the CFA findings, the Turkish adapted CUDQ was highly consistent in terms of validity (Mulaik et al., 1989).

Table 2 shows the results of the descriptive analysis for the sub-items of the CUDQ and the Cronbach alpha reliability analysis of the CUDQ. According to these results, the item with the highest average was CUDQ3, and the item with the lowest average was CUDQ1. As a result of the reliability analysis, the adjusted correlation lower and upper limits were given for the items of the caffeine consumption scale, and all of the corrected correlations of these items were positive. The intraclass correlation coefficient of the scale was 0.83, and the Cronbach alpha coefficient was 0.86. In addition, since no increase in the reliability coefficient was observed when an item was removed, no item was excluded from the caffeine consumption scale. According to the ICC and reliability coefficients, it was concluded that the caffeine consumption scale was highly reliable in terms of internal consistency.

Table 2.

The reliability analysis of CUDQ

Variable	$\overline{\mathrm{X}}\pm \mathrm{Sd}$	Cronbach alpha if item deleted	Corrected item-total correlation	ICC	Cronbach alpha
CUDQ1	3.65 ± 1.08	0.86	[0.39, 0.70]	0.83	0.86
CUDQ2	4.56 ± 0.8	0.85
CUDQ3	4.66 ± 0.76	0.84
CUDQ4	4.47 ± 0.8	0.84
CUDQ5	4.19 ± 0.9	0.86
CUDQ6	4.41 ± 0.87	0.86
CUDQ7	4.24 ± 0.93	0.84
CUDQ8	4.43 ± 0.83	0.85
CUDQ9	4.51 ± 0.81	0.84
CUDQ10	4.51 ± 0.85	0.84

Data are presented as mean ± standard deviation. $\overline{X}$ Mean, Sd standard deviation, ICC intraclass coefficient

Table 3 shows the item-based CFA for the CUDQ. According to the CFA findings, all of the items were grouped under 1 dimension in a statistically significant way (p < 0.05).

Table 3.

The confirmatory factor analyzes of the CUDQ

Item	$\beta$	STD ($\beta$)	z P
CUDQ1	1	0.52	- -
CUDQ2	1.619	0.56	9.23 < 0.001
CUDQ3	1.996	0.70	9.7 < 0.001
CUDQ4	2.054	0.74	9.79 < 0.001
CUDQ5	0.554	0.39	7.46 < 0.001
CUDQ6	0.846	0.41	7.1 < 0.001
CUDQ7	1.814	0.70	9.60 < 0.001
CUDQ8	1.244	0.59	9.03 < 0.001
CUDQ9	2.199	0.77	9.82 < 0.001
CUDQ10	1.792	0.74	9.64 < 0.001

Table 4 shows the CUDQ score results for the caffeine withdrawal symptoms. According to these findings, 31.3% of the individuals had headache, 31.9% had fatigue and constant sleepiness, 21.9% had depression or restlessness, 28.7% had a lack of concentration, 11.6% had flu-like symptoms, and 11.6% experienced inconvenience or suffering in daily life. In addition, the CUDQ scores of the individuals who experienced caffeine withdrawal symptoms were significantly higher than those without symptoms (p < 0.05). Figure 1 shows the average caffeine intake of the participants from caffeinated foods and drinks.

Table 4.

The comparison of CUDQ score scores based on caffeine withdrawal symptoms

Variable	Group	n (%)	$\overline{\mathrm{X}}\pm \mathrm{Sd}$	t- Value P
Headache	Yes	97 (31.3)	20.33 ± 5.70	8.86 < 0.001
	No	213 (68.7)	14.58 ± 4.30
Fatigue or drowsiness	Yes	99 (31.9)	20.11 ± 5.86	8.33 < 0.001
	No	211 (68.1)	14.63 ± 4.28
Depresses mood or irritability	Yes	68 (21.9)	21.59 ± 5.63	9.02 < 0.001
	No	242 (78.1)	14.91 ± 4.44
Difficulty concentrating	Yes	89 (28.7)	20.88 ± 5.92	9.23 < 0.001
	No	221 (71.3)	14.57 ± 4.06
Flu-like symptoms (e.g., nausea, vomiting, muscle pain)	Yes	36 (11.6)	21.42 ± 6.67	4.95 < 0.001
	No	274 (88.4)	15.72 ± 4.94
Inconvenience or suffering in daily life	Yes	36 (11.6)	22.81 ± 6.94	6.11 < 0.001
	No	274 (88.4)	15.53 ± 4.64

Fig. 1.

Total caffeine intake from caffeinated food and drinks

Table 5 shows the relationship between the total CUDQ score and the caffeine intake of individuals from caffeinated foods and drinks. There was a statistically significant relationship between the total scores of the CUDQ and the caffeine intake of individuals from caffeinated foods and drinks (p < 0.05). A positive and low relationship was found between the total score of the CUDQ and caffeinated beverages (r = 0.35) and caffeinated foods (r = 0.14). In addition, the correlation coefficient between the total score of the CUDQ and total caffeine intake was statistically significant (p < 0.05). A positive and low level (r = 0.35) correlation was found between the total score of the CUDQ and total caffeine intake. In other words, as the CUDQ scores increased, the individuals’ daily caffeine intake increased, and vice versa.

Table 5.

Correlation of between CUDQ total scores and daily caffeine intakes from foods and drinks

	Caffeinated drinks	Caffeinated foods	Total caffeine intake
CUDQ	0.35**	0.14*	0.35**

^**Correlation was significant at 0.01, *correlation was significant at 0.05

Discussion

The present study aimed to adapt the CUDQ to an adult Turkish population. This study was the first attempt to validate the original version translated into the Turkish language. The Turkish form of the scale was sent online to adult individuals, and the data obtained from the 310 individuals were analyzed.

In this study, the findings suggested that the model for CUD was psychometrically robust. The level of internal consistency among the items of the CUDQ, item total correlation numbers, and Cronbach alpha internal consistency coefficients were examined, and the internal consistency coefficient of the scale was accordingly found to be 0.874. The Cronbach alpha coefficient comprises a value between 0 and 1. Approaching a value of 1 indicated that the scale was perfectly good. In addition, the correlation of all of the items with the total score was positive and above 0.50. This indicated that the scale was able to measure CUD fear as a whole. As a result, the findings allowed for the conclusion that the CUDQ presented sufficient empirical evidence of reliability and validity to support its use in the context presented in this study. The Cronbach alpha coefficient for the original scale was 0.87 (Ágoston et al., 2018). In the present study, the Cronbach alpha coefficient was 0.86. The findings also indicated that the scale had a reliability coefficient that was over 0.70, which met the criterion of acceptable reliability recommended that was determined by Nunnally (Nunnally & Bernstein, 1978)

Caffeine consumption is a socially acceptable activity, and its increasing consumption has caused it to become a part of our daily lives. Today, the introduction of new caffeine-containing food products and changes in consumption habits of more traditional caffeine sources have increased overall caffeine consumption and potential cumulative effects (Temple et al., 2017). High caffeine consumption can have significant public health impacts. Therefore, it is not surprising that caffeine has aroused the interest of researchers and clinicians (Nieber, 2017). There is no agreement about the real health danger upon caffeine regular consumption (Addicott, 2014). However, for some individuals, the use of caffeine can interfere with the fulfillment of obligations or cause interpersonal problems. As these 2 criteria are the most severe, it is highly recommended to examine possible manifestations in clinical studies. Therefore, the Turkish version of the CUDQ as a whole may be the subject of further studies to distinguish latent continuity from CUD with low persistence and mild symptoms.

For the caffeine withdrawal symptoms, some psychometric properties were used to assess whether the participants experienced the following symptoms after not drinking caffeine for 24 h or more during the previous year: headache, drowsiness, fatigue, depressed mood, difficulty concentrating, flu-like symptoms, and inconvenience or suffering in daily life (Heinz et al., 2009). The concurrent validity of the CUDQ showed significant correlation with caffeine withdrawal symptoms. In addition, the CUDQ scores of the individuals who experienced caffeine withdrawal symptoms were significantly higher than those without symptoms. Therefore, it was thought that CUD may increase psychiatric disorders and associated symptoms. Similar to this study, in other studies, CUD was found to be associated with symptoms such as headache, drowsiness, fatigue, depressed mood, sleep, and wakefulness (Budney et al., 2015; James & Keane, 2007; James & Rogers, 2005; Meredith et al., 2013).

In this study, a statistically significant relationship was found between the total scores of the CUDQ and the individuals’ caffeine intake from both caffeinated food and beverages. Positive correlations were found between the caffeine intake and CUDQ scores, similar to the study of Ágoston et al. (2018). In this study, the relationship between general caffeine use and the CUDQ was investigated. In future studies, the consumption of different caffeinated products can be examined separately. In addition, the groups of caffeine consumers should be investigated, while considering both the amount and type of caffeine consumed in the development of CUD, as well as the role of various demographic variables (e.g., socioeconomic status, educational status).

Although it was a relatively heterogeneous and representative sample of the general population in Turkey, several limitations should also be noted. First, in the study population, the females were quite overrepresented. Second, the findings of this study were based on self-report data which has the risk of source bias. In further research, it could be used the multiple methods of assessment (e.g., experimental paradigms, clinical administration). Third, the sample power analysis result evaluated in this study was sufficient, but the data collection occurred online. The data were collected during the initial COVID-19 pandemic; hence, to minimize the risk of infection, the data collection occurred online. Proper sampling may be limited in terms of the ability to access all layers of the population (for example, individuals without the internet). Its suitability for use in clinical and research settings (it via other modes of administrations, such as clinical administration and/or face-to-face verbal interviews) and more diverse populations, including clinical samples, should be examined further in future studies. However, online data collection methods tend to give more honest and truthful results than offline methods (Griffiths, 2010). Finally, this study calculates the criterion validity and test-retest. In conclusion, the findings of this study provided evidence that the CUDQ has validity of structure, internal consistency, and construct validity for assessing CUD. Further studies could determine the tendency of individuals in the adult Turkish population towards CUD, and it could be applied to different groups.

Supplementary Information

Below is the link to the electronic supplementary material.

Author Contribution

Seda Kaya: Conceptualization; Methodology; Investigation; Writing, original draft

Mahmut Bodur: Conceptualization; Methodology; Investigation; Writing, original draft

Merve İlhan-Esgin: Conceptualization; Methodology; Investigation; Writing, original draft

Funda Pınar Çakiroğlu: Conceptualization, Project administration, Validation, Supervision

Ayşe Özfer Özçelik: Conceptualization, Project administration, Validation, Supervision

Declarations

Ethics Approval

Informed consent was obtained electronically before data were collected from the participants. The procedures used in this study adhere to the tenets of the Declaration of Helsinki. The research protocol of the study was approved by the Ethics Committee of the Ankara University (number: 56786525-050.04.04/85330).

Conflict of Interest

The authors declare no competing interests.