ABSTRACT
Aim
To adapt and validate the Problem Areas in Diabetes scale in type 2 diabetes patients in Indonesia.
Design
This was a cross‐sectional study.
Methods
The study with 1044 participants was conducted at public health centers. The construct validity used an exploratory factor analysis to measure the scale structure, and a confirmatory factor analysis was used to assess model suitability. Reliability was calculated using Cronbach's alpha, composite reliability, average variance extracted and test–retest results. The sample was divided into equal numbers for the exploratory factor analysis and confirmatory factor analysis.
Results
The exploratory factor analysis showed that the Kaiser‐Meyer‐Olkin test was 0.899, and Bartlett's test of sphericity was significant. Three factors were obtained with a factor loading of > 0.34. Cronbach's alpha for all items was 0.858, and individual factors ranged from 0.636 to 0.864. An interclass correlation coefficient and test–retest reliability analysis of all question items exhibited good results. Pearson correlations for all items or each factor showed significant results. The confirmatory factor analysis showed that the instrument had an acceptable model fit.
Patient or Public Contribution
Type 2 diabetes patients and nurses in charge at public health centers were involved in this study. The findings show that the instrument is valid and reliable; it can be an option for measuring the stress diabetes mellitus. The subscales in this instrument can be used simultaneously or separately, according to each individual's needs. Indonesian PAID can be used for stress screening in diabetes mellitus patients starting from the primary health service level so that initial treatment can be provided more quickly.
1. Introduction
The global prevalence of diabetes mellitus has consistently increased, with as many as 537 million adults living with diabetes mellitus reported in 2021, which is projected to be 643 million by 2030 and 783 million in 2045. Type 2 diabetes mellitus accounts for more than 90% of the total cases. Geographically, the United States is ranked third in terms of diabetes mellitus total population, with 31 million. The European continent accounts for 8%–14% of total cases, with the prevalence in Western Europe greater than in Eastern Europe. The Asian continent accounts for around 60% of total diabetes cases, with several countries having the highest levels of diabetes mellitus population. These countries are China ranked first (116.4 million), India ranked second (77 million) and Indonesia ranked fourth (19.5 million) (Ong et al. 2023).
Diabetes mellitus not only affects individual health but also causes a greater social and economic burden. Countries, including low‐middle‐income countries, feel more significant burdens and problems than countries with high income, one of which is due to limited resources in access to preventive and curative health services (Tesfaye Yunka et al. 2020). The persistent increase in blood glucose levels in people with diabetes mellitus is a significant problem and will worsen if not well managed. Uncontrolled blood glucose conditions can trigger serious complications, including cardiovascular problems, nephropathy, and neuropathy (Guo et al. 2023). Diabetes mellitus is estimated to be the cause of 6.7 million deaths in 2021, while more than two‐thirds of them are related to cardiovascular complications (Tesfaye Yunka et al. 2020).
Apart from physical problems, individuals with diabetes mellitus also face psychological issues. Changing physical conditions, complex management and lifelong effects cause psychological problems in diabetes mellitus. The psychological problem that individuals often experience is diabetes distress, which is a condition of emotional burden with characteristic feelings of frustration, tiredness and exhaustion related to diabetes mellitus. Diabetes distress can impact emotional well‐being, feelings of helplessness, poor mood and reduced quality of life. Prolonged stress can trigger worse conditions, such as anxiety and depression, and is associated with poor glycemic control and non‐compliance with therapy, which increases and worsens complications (Kalra et al. 2018). Early detection of stress in people with diabetes mellitus gives individuals the opportunity to get treatment more quickly and appropriately so that worse conditions can be prevented (Guo et al. 2023).
2. Background
A systematic review study provided information on several instruments that can be used to measure stress in people with diabetes mellitus, such as the Problem Areas in Diabetes (PAID) scale. Compared to other instruments, the PAID was more often used in previous studies (Lee et al. 2015). The PAID scale was first developed with a single factor and 20 items (Polonsky et al. 1995). During its development, there were several variations in the number of factors and items, such as PAID‐5, PAID‐1, and PAID SF (Lee et al. 2015). The PAID scale is specifically designed to assess emotional distress experienced by individuals with diabetes and can play a critical role in comprehensive diabetes management. Since its development, the PAID scale has been adapted globally, including in Europe (Beléndez et al. 2014; Cichoń et al. 2021; Papathanasiou et al. 2014; Schmitt et al. 2016), Africa (Lygidakis et al. 2021; Sayed Ahmed et al. 2022), the USA (Gross et al. 2007; Miller and Elasy 2008; Snoek et al. 2000), and Asia (Altıkardeş et al. 2024; Arzaghi et al. 2011; Huis In'T Veld et al. 2011; Lee et al. 2014). Indonesia is included in the Southeast Asian region; in this region, the PAID‐20 scale has only recently undergone an adaptation process in Singapore (Venkataraman et al. 2015) and Malaysia (Jannoo et al. 2019).
Cultural differences can significantly influence health perceptions and behaviours, so it is essential to adapt and validate psychological tools such as the PAID scale to ensure that they are effective and culturally sensitive. A valid and reliable measuring tool is expected to improve diabetes management by providing healthcare professionals in Indonesia with a robust tool to diagnose and address the emotional distress associated with diabetes. Previous studies carried out adaptation and psychometric testing of the PAID scale but used the five‐item version and only assessed the content validity, construct validity using Pearson's product–moment correlation test, and test reliability using Cronbach's alpha coefficient to show internal consistency. Their study was also only conducted in one province in Indonesia with a relatively small sample size, thereby reducing the generalizability of the study results (Budi et al. 2023).
The PAID scale with complete items is expected to be able to measure all aspects of stress experienced by people with diabetes mellitus. However, adaptation and psychometric studies of the total item PAID have never been conducted in Indonesia, and the absence of a comprehensive psychometric evaluation in previous studies could hamper understanding of the scale's construct validity within the Indonesian context. Therefore, this study satisfies the need for a detailed examination of the scale's reliability, validity, and cultural appropriateness to enhance its clinical and research utility in Indonesia. We aimed to adapt and test the validity and reliability of the PAID‐20 scale in Indonesians with type 2 diabetes in a comprehensive manner.
3. Methods
3.1. Design
The study had a cross‐sectional design and was conducted from July to October 2023.
3.2. Study Setting and Sample
Data collection was carried out at public health centers in 15 provinces in Indonesia. The research sample was people with type 2 diabetes who visited a public health center. The type of diabetes mellitus was confirmed through a doctor's diagnosis in their medical records. There are several opinions regarding calculating the minimum sample size for psychometric tests; we applied a minimum sample of 20 people per item (Kyriazos 2018). The PAID scale comprises 20 items, so the minimum sample size required was 400 participants. The total sample in this study was 1044 participants; we randomly divided them into 522 people for the exploratory factor analysis (EFA) and 522 participants for the confirmatory factor analysis (CFA). Test–retest reliability was conducted on 80 participants who filled out the same questionnaire again 14 days after first filling out the questionnaire. Reliability studies generally consider a sample size of 50–100 participants adequate (Terwee et al. 2007). Previous studies show that ICC values remain stable and interpretable with sample sizes above 50 (Koo and Li 2016). Samples were taken by accidental sampling with determining criteria: a minimum age of 20 years, an ability to communicate well, and not taking medication for mental disorders. Patients with a history of or are currently experiencing mental problems and patients with diabetic wounds were the study exclusion criteria.
3.3. Procedures
The translation process was carried out after we obtained permission from the original author (Polonsky et al. 1995) via personal communication on May 24, 2023. The translation process is shown in Figure 1. Our adaptation process and psychometric tests as follows:
FIGURE 1.
Translation, cultural adaptation, and psychometric validation process of the PAID scale into Bahasa Indonesia. Forward and backward translation, synthesis, pilot testing, revision, content validity assessment (CVI), and psychometric evaluation were conducted to develop the final PAID Bahasa Indonesia instrument.
3.3.1. Translation Process
The forward translation process, translating the English version of the PAID questionnaire into Bahasa Indonesian, was carried out by two independent translators fluent in Bahasa Indonesian and English, who are nurses living in Taiwan and the UK. An internal research team of three people compared the Bahasa Indonesia version of the translated results, which were called draft version 1. Draft version 1 was then translated back into English by two translators, one of whom was an English teacher and the other from an English‐Indonesian translation institute. The internal team, together with the translators, carried out comparisons through online meetings. The internal team, together with the translators and the diabetes specialist, carried out comparisons through online meetings. We reviewed semantic, idiomatic, experiential, and conceptual equivalence to ensure that the translated items were culturally appropriate in the Indonesian context. There were no significant differences between the two translation results; consensus was reached during the discussion, and the result was referred to as draft version 2.
3.3.2. Pilot Testing of the Pre‐Final Version
The pilot testing process involved 10 type 2 diabetes mellitus and 10 non‐diabetes mellitus subjects who were asked to rate each item regarding ease of understanding. In the pilot study, we involved a non‐diabetes mellitus population to provide essential data for analysing various response ranges, creating a normative data set, and increasing the ability to generalise. Apart from that, it can also increase study efficiency and reduce research costs (Teresi et al. 2022). Revisions were carried out based on the results of pilot testing and continued with content validity testing by a panel of 12 experts consisting of a doctor, one public health specialist, two psychiatric nurses working as academics, two nurses at a mental hospital, two community nurses, two medical‐surgical nurses working as academics, and two medical‐surgical nurses at a hospital. The scale‐content validity index average (S‐CVI/ave) results were 0.95 (Table S1). We did not conduct preliminary psychometric testing with a bilingual sample because limited participants could communicate using Bahasa Indonesian and English.
3.3.3. Psychometric Testing in the Target Population
The final PAID form was used for psychometric testing of people with type 2 diabetes mellitus at public health centers. We called this instrument the PAID‐Bahasa Indonesia. The psychometric testing process was assisted by 25 research assistants who were nursing staff at community health centers at the research locations. The assistants distributed questionnaires to participants who face‐to‐face had signed an informed consent form.
3.4. Measures
3.4.1. Participant Characteristics
Participant characteristics were measured using a self‐reported questionnaire consisting of age, age at diagnosis of diabetes mellitus, duration of diabetes mellitus, gender, educational level, occupation, ethnicity, place of diagnosis of diabetes mellitus, family history of diabetes mellitus, marital status, health insurance participation and income. Blood pressure was measured using a digital blood pressure monitor with the patient sitting after resting for around 5 min. Body weight and height were respectively measured using a digital scale and height measurement device in an upright position without shoes. The measurement results were used to calculate the body‐mass index (BMI) using the formula body weight (kg) divided by height2 (m2). All blood pressure, height and weight measurements used standard operational procedures.
3.4.2. Original Version of the PAID
The PAID is an instrument for measuring diabetes‐related psychosocial distress. The original version of PAID consists of one factor, 20 items and six answer choices using a Likert scale (0–5), and during its development, the scale became five answer choices (0–4). The answer choice scores used were ‘not a problem’ (0), ‘minor problem’ (1), ‘moderate problem’ (2), ‘somewhat serious problem’ (3), and ‘serious problem’ (4). The summed raw score ranges from 0 to 80, which is typically multiplied by 1.25 to yield a standardized score ranging from 0 to 100, with a higher score indicating greater stress. Initial testing of the PAID showed high internal reliability, with good item‐to‐total correlations (Polonsky et al. 1995). Factor loading in the second test showed that the results for each item ranged 0.38–0.89, and Cronbach's alpha was high (0.95) (Welch et al. 1997). The PAID‐20 scale has been translated into various languages, with results showing that it is valid and reliable even with variations in the number of factors.
3.5. Data Analysis
There were no missing data in this study. Data analyses were carried out using SPSS vers. 27. Categorical data are presented using numbers and percentages, while continuous data are the mean and standard deviation (SD) with the addition of minimum, maximum, skewness, and kurtosis for each item. Data are normally distributed if the skewness range is ±2 and the kurtosis coefficient is ±4 (Mishra et al. 2019).
In EFA, data extraction utilised the Maximum Likelihood method with Direct Oblimin rotation, based on eigenvalue > 1. Maximum Likelihood is appropriate for normally distributed data and allows us to calculate model fit indices, statistical significance of factor loading, factor intercorrelations, and confidence intervals (Zitzmann et al. 2022). Direct Oblimin, an oblique (non‐orthogonal) rotation, was also selected since the hypothesized factors were theoretically expected to be correlated, as with most psychological and behavioural constructs. Our results also indicated small‐to‐moderate intercorrelations among the extracted factors, thus substantiating the application of an oblique rotation method (Akhtar‐Danesh 2023). To verify the sample size, we used the Kaiser‐Meyer‐Olkin (KMO) test and Bartlett's test of sphericity. A factor loading of > 0.30 was used to identify each item. Some methodological sources indicate it is acceptable in exploration conditions, particularly if the sample size is large enough, to have lower cut‐offs (≥ 0.30) (Hair 2009; Tabachnick and Fidell 2019). In the current study, with a sample size of 1044 participants, factor stability was enhanced, and it was considered reasonable to use a cut‐off value of > 0.30. As in earlier psychometric literature, the cut‐off value still indicates a meaningful association between items and their corresponding factors (Tavakol and Wetzel 2020). Furthermore, the retained items were conceptually meaningful and provided support for the construct validity of the PAID‐Bahasa Indonesia scale. We determined the number of factors using Kaiser's criteria with an eigenvalue of > 1 (Shrestha 2021). The convergent validity was analysed using the Heterotrait‐Monotrait (HTMT) ratio with respective acceptable values of < 0.85 (Rönkkö and Cho 2022). Pearson correlation statistics were used to measure the relationship between constructs.
To confirm data model fit and establish the quality of the resultant model, we conducted supplementary analysis using CFA with AMOS version. 26 (IBM, Armonk, NY, USA). We confirmed acceptability of model fit by a chi‐squared test value (χ 2)/df of ≤ 3, a goodness‐of‐fit index (GFI), adjusted GFI (AGFI), a comparative fit index (CFI), a Tucker‐Lewis index (TLI), and an incremental fit index (IFI) of > 0.9 (Hu and Bentler 1999; Schermelleh‐Engel et al. 2003), a root mean squared error of approximation (RMSEA) and standardised root mean squared residual (SRMR) of ≤ 0.08 (Goretzko et al. 2024; Hu and Bentler 1999).
We used Cronbach's alpha to test the internal consistency with a good consistency criterion with a value of ≥ 0.70 and a total item correlation of ≥ 0.30 (Barbera et al. 2021). The test–retest reliability of the PAID‐Bahasa Indonesia was analysed using the interclass correlation coefficient (ICC) with a value of 0.75 ~ 0.90 indicating good reliability results, and > 0.90 indicating perfect reliability (Ten Hove et al. 2022). We used the two‐way mixed‐effects model with absolute agreement and single measures ICC, which is appropriate for assessing test–retest reliability of scale scores (Koo and Li 2016).
3.6. Ethical Considerations
All participants signed an informed consent form as a legal requirement before participating in the study. Ethical clearance for the study was obtained from the Ethic Research Committee of Universitas Respati Yogyakarta (no. 090.3/FIKES/PL/V/2023) and by the Taipei Medical University‐Joint Institutional Board (no. N202308068). Data were collected after obtaining permission from the Health Service of Indonesia and public health centers at the research locations.
4. Results
4.1. Sample Characteristics of Participants
In total, 1044 people with type 2 diabetes mellitus participated in the study with continuous data characteristics displayed as the mean (SD) as follows: age 56.95 (8.787) years, weight 61.83 (10.481) kg, height 158.57 (9.947) cm, BMI 25.16 (12.829) kg/m2, systolic blood pressure 142.04 (16.985) mmHg, diastolic blood pressure 87.46 (24.339) mmHg, diagnosed with diabetes mellitus since the age of 50.70 (8.179) years, being diagnosed by diabetes mellitus for 6.25 (4.816) years and with a total PAID score of 19.63 (10.894). Similar results were also obtained in the test–retest reliability results of 80 people with type 2 diabetes mellitus as follows: age 60.25 (8.624) years, weight 58.96 (10.440) kg, height 156.90 (7.645) cm, calculated BMI 23.94 (3.878) kg/m2, systolic blood pressure 139.24 (21.506) mmHg, diastolic blood pressure 87.21 (12.783) mmHg, diagnosed with diabetes mellitus since the age of 53.84 (7.975) years, being diagnosed by diabetes mellitus for 6.41 (5.976) years, and with a total PAID score of 13.46 (8.291). Categorical data on total participants and test–retest participants showed that the majority of participants were female (52.9% and 70.0%), had finished senior high school (32.2% and 32.5%), were Javanese (83.7% and 86.3%), had been diagnosed with diabetes mellitus the first time in a public health center (77.7% and 96.3%), had no family history of diabetes mellitus (59.0% and 67.5%), had health insurance (89.1% and 100.0%) and had a monthly income of <Indonesian Rupiah (IDR) 2,500,000 (57.9% and 86.2%). Detailed data on participant characteristics are presented in Table 1.
TABLE 1.
Demographic characteristic of respondents.
| Characteristic | All participants (N = 1044) | Test–retest reliability (N = 80) |
|---|---|---|
| Mean (SD) or n (%) | Mean (SD) or n (%) | |
| Age (years) | 56.95 (8.787) | 60.25 (8.624) |
| Weight (kg) | 61.83 (10.481) | 58.96 (10.440) |
| Height (cm) | 158.57 (9.947) | 156.90 (7.645) |
| BMI (kg/m2) | 25.16 (12.829) | 23.94 (3.878) |
| SBP (mmHg) | 142.04 (16.985) | 139.24 (21.506) |
| DBP (mmHg) | 87.46 (24.339) | 87.21 (12.783) |
| Age at DM diagnosis (years) | 50.70 (8.179) | 53.84 (7.975) |
| Length of DM (years) | 6.25 (4.816) | 6.41 (5.976) |
| PAID total score | 19.63 (10.894) | 13.46 (8.291) |
| Gender | ||
| Male | 492 (47.1) | 24 (30.0) |
| Female | 552 (52.9) | 56 (70.0) |
| Educational level | ||
| None | 34 (3.3) | 1 (1.3) |
| Elementary School | 280 (26.8) | 25 (31.3) |
| Junior High School | 151 (14.5) | 13 (16.3) |
| Senior High School | 336 (32.2) | 26 (32.5) |
| Bachelor | 220 (21.1) | 12 (15.0) |
| Master | 16 (1.5) | 2 (2.5) |
| Doctoral | 7 (0.7) | 1 (1.3) |
| Occupation | ||
| None | 168 (16.1) | 4 (5.0) |
| Farmer | 135 (12.9) | 4 (5.0) |
| Housewife | 140 (13.4) | 38 (47.5) |
| Civil servant | 77 (7.4) | 0 (0.0) |
| Retired | 28 (2.7) | 8 (10.0) |
| Private staff | 311 (29.8) | 14 (17.5) |
| Others | 185 (17.7) | 12 (15.0) |
| Ethnicity | ||
| Javanese | 874 (83.7) | 69 (86.3) |
| Non‐Javanese | 170 (16.3) | 11 (13.8) |
| Place first diagnosed | ||
| Public health center | 811 (77.7) | 77 (96.3) |
| Hospital | 191 (18.3) | 3 (3.8) |
| Others | 42 (4.0) | |
| Family history of DM | ||
| No | 616 (59.0) | 54 (67.5) |
| Yes | 428 (41.0) | 26 (32.5) |
| Marriage | ||
| No | 20 (1.9) | 3 (3.8) |
| Yes | 877 (84.0) | 56 (70.0) |
| Widow/Widower | 147 (14.1) | 21 (26.3) |
| Health insurance | ||
| No | 28 (2.7) | 0 (0.0) |
| Yes | 1016 (97.3) | 80 (100.0) |
| Income | ||
| < IDR 2,500,000 | 604 (57.9) | 69 (86.2) |
| IDR 2,500,000–5,000,000 | 349 (33.4) | 11 (13.8) |
| > IDR 5,000,000 | 91 (8.7) | 0 (0.0) |
Abbreviations: BMI, body‐mass index; DBP, diastolic blood pressure; DM, diabetes mellitus; IDR, Indonesian Rupiah; n, number; PAID, problem areas in diabetes; SBP, sytolic blood pressure; SD, standard deviation. [In 2023, the average exchange rate was US$1.00 ≈ IDR15,200].
The S‐CVI/ave. result for the overall PAID‐Bahasa Indonesia items was 0.95 (Table S1). The average score, skewness, and kurtosis of the PAID‐Bahasa Indonesia version for each item are shown in Table 2. The lowest and highest means of all items were obtained for items 18 and 12, with mean (SD) scores of 0.65 (0.932) and 1.49 (1.247), respectively. All items showed skewness scores ranging 0.483 to 1.512 and kurtosis scores ranging −0.788 to 1.795.
TABLE 2.
Average Scores of the PAID‐Bahasa Indonesia Version According to Items and Domains (N = 1044).
| Item | Min | Max | Mean | SD | Skewness | Kurtosis |
|---|---|---|---|---|---|---|
| PAID_1 | 0 | 4 | 0.91 | 1.044 | 1.130 | 0.677 |
| PAID _2 | 0 | 4 | 0.84 | 1.022 | 1.146 | 0.745 |
| PAID _3 | 0 | 4 | 1.00 | 1.042 | 0.946 | 0.386 |
| PAID _4 | 0 | 4 | 0.73 | 1.006 | 1.464 | 1.565 |
| PAID _5 | 0 | 4 | 0.93 | 1.088 | 1.071 | 0.379 |
| PAID _6 | 0 | 4 | 1.00 | 1.091 | 1.046 | 0.413 |
| PAID _7 | 0 | 4 | 0.88 | 0.983 | 1.168 | 1.016 |
| PAID _8 | 0 | 4 | 1.06 | 1.101 | 0.853 | −0.059 |
| PAID _9 | 0 | 4 | 1.11 | 1.110 | 0.789 | −0.219 |
| PAID _10 | 0 | 4 | 0.94 | 1.140 | 1.192 | 0.592 |
| PAID _11 | 0 | 4 | 1.00 | 1.063 | 0.974 | 0.280 |
| PAID _12 | 0 | 4 | 1.49 | 1.247 | 0.483 | −0.788 |
| PAID _13 | 0 | 4 | 1.18 | 1.106 | 0.774 | −0.095 |
| PAID _14 | 0 | 4 | 0.78 | 1.057 | 1.354 | 1.035 |
| PAID _15 | 0 | 4 | 0.82 | 1.104 | 1.290 | 0.694 |
| PAID _16 | 0 | 4 | 0.98 | 1.135 | 1.137 | 0.422 |
| PAID _17 | 0 | 4 | 0.76 | 1.046 | 1.433 | 1.351 |
| PAID _18 | 0 | 4 | 0.65 | 0.932 | 1.512 | 1.795 |
| PAID _19 | 0 | 4 | 1.44 | 1.258 | 0.510 | −0.747 |
| PAID _20 | 0 | 4 | 1.13 | 1.102 | 0.661 | −0.482 |
Abbreviations: Max, maximum; Min, minimum; PAID, problem areas in diabetes; SD, standard deviation.
4.2. Validity of the PAID‐Bahasa Indonesia
4.2.1. Construct Validity With the EFA of the PAID‐Bahasa Indonesia
The results of EFA are shown in Table 3. The data were suitable for factor analysis since the value of KMO was 0.899 and Bartlett's test of sphericity was significant (p < 0.001). All items showed acceptable factor loading above 0.30. Based on the Kaiser criterion (eigenvalue > 1), three factors were retained, and they explained 45.05% and 36.23% total variance before and after rotation, respectively (Table 3 footnote and Table S2). The scree plot in Figure S1 confirmed this solution since there was an apparent decline after the third factor. The first factor, emotional and diabetes self‐management distress, was made up of 11 items with loadings from 0.386 to 0.728. The second factor, treatment‐related and social support distress, comprised six items with loadings from 0.349 to 0.726. The third factor, complication distress, comprised three items loading from 0.437 to 0.779.
TABLE 3.
Exploratory factor analysis for the PAID‐Bahasa Indonesia (N = 522).
| Item | Factor loading | |||
|---|---|---|---|---|
| Emotional and diabetes self‐management distress (Distres emosional dan manajemen diri diabetes) | Treatment‐related and social support distress (Distres yang berhubungan dengan terapi dan dukungan sosial) | Complications distress (Distres komplikasi) | ||
| PAID_6 |
Feeling depressed when you think about living with diabetes. (Merasa tertekan ketika Anda berpikir tentang hidup dengan diabetes.) |
0.728 | 0.298 | 0.259 |
| PAID_11 | Feeling constantly concerned about food and eating. (Merasa terus‐menerus khawatir tentang makanan dan makan.) | 0.676 | 0.203 | 0.382 |
| PAID_8 | Feeling overwhelmed by your diabetes. (Merasa kewalahan dengan diabetes Anda.) | 0.667 | 0.213 | 0.238 |
| PAID_5 | Feelings of deprivation regarding food and meals. (Merasa makanan dan minuman Anda dirampas oleh diabetes.) | 0.657 | 0.251 | 0.207 |
| PAID_10 | Feeling angry when you think about living with diabetes. (Merasa marah ketika Anda berpikir tentang hidup dengan diabetes.) | 0.649 | 0.277 | 0.241 |
| PAID_3 | Feeling scared when you think about living with diabetes. (Merasa takut ketika Anda berpikir tentang hidup dengan diabetes.) | 0.627 | 0.350 | 0.253 |
| PAID_16 | Feeling that diabetes is taking up too much of your mental and physical energy every day. (Merasa bahwa diabetes menghabiskan terlalu banyak energi baik mental dan fisik Anda setiap hari.) | 0.618 | 0.270 | 0.261 |
| PAID_7 | Not knowing if your mood or feelings are related to your diabetes. (Tidak tahu apakah suasana hati atau perasaan Anda berhubungan dengan diabetes Anda.) | 0.593 | 0.354 | 0.186 |
| PAID_20 | Feeling ‘burned out’ by the constant effort needed to manage diabetes. (Merasa ‘kelelahan’ karena upaya terus‐menerus yang diperlukan untuk mengelola diabetes) | 0.540 | 0.285 | 0.333 |
| PAID_9 | Worrying about low blood sugar reactions. (Khawatir tentang reaksi gula darah rendah.) | 0.501 | 0.236 | 0.270 |
| PAID_14 | Not ‘accepting’ your diabetes. (Tidak ‘menerima’ diabetes Anda.) | 0.386 | 0.369 | 0.002 |
| PAID_18 | Feeling that your friends and family are not supportive of your diabetes management efforts. (Merasa bahwa teman dan keluarga Anda tidak mendukung upaya manajemen diabetes Anda.) | 0.181 | 0.726 | 0.067 |
| PAID_4 | Uncomfortable in social situations related to your diabetes care (e.g., people telling you what to eat)? (Situasi sosial yang tidak nyaman terkait dengan perawatan diabetes Anda) (misalnya, orang‐orang memberi tahu Anda apa yang harus dimakan) | 0.332 | 0.550 | −0.052 |
| PAID_2 | Feeling discouraged with your diabetes treatment plan? (Merasa kecil hati dengan rencana perawatan diabetes Anda.) | 0.428 | 0.547 | 0.014 |
| PAID_17 | Feeling alone with your diabetes. (Merasa sendirian denga diabetes Anda.) | 0.255 | 0.468 | 0.075 |
| PAID_15 | Feeling unsatisfied with your diabetes physician. (Merasa tidak puas dengan dokter diabetes Anda.) | 0.198 | 0.379 | −0.104 |
| PAID_1 | Not having clear and concrete goals for your diabetes care. (Tidak memiliki tujuan yang jelas dan konkret untuk perawatan diabetes Anda.) | 0.152 | 0.349 | 0.013 |
| PAID_12 | Worrying about the future and the possibility of serious complications. (Mengkhawatirkan masa depan dan kemungkinan komplikasi serius.) | 0.412 | 0.009 | 0.779 |
| PAID_19 | Coping with complications of diabetes. (Koping saat mengatasi komplikasi diabetes.) | 0.263 | 0.075 | 0.556 |
| PAID_13 | Feelings of guilt or anxiety when you get off track with your diabetes management. (Merasa bersalah atau cemas ketika Anda keluar jalur manajemen diabetes Anda.) | 0.416 | 0.191 | 0.437 |
Note: Extraction method: maximum likelihood. Rotation method: oblimin with Kaiser normalisation. The three‐factor solution was supported by initial eigenvalues > 1 (5.807, 1.982 and 1.221), explaining 45.05% of the total variance before rotation and 36.23% after rotation. Bold values indicate the highest factor loading for each item, representing the factor to which the item was assigned. Kaiser‐Mayer‐Olkin = 0.899; Bartlett's test of sphericity: degrees of freedom = 190, p < 0.001.
4.2.2. Construct Validity With a CFA of the PAID‐Bahasa Indonesia
The structure of the PAID‐Bahasa Indonesia model is presented in Figure 2. Results of the CFA with three factors showed results of χ 2 = 332.995, df = 158, χ 2/df = 2.11, p < 0.001, GFI = 0.940, AGFI = 0.920, IFI = 0.917, TLI = 0.900, CFI = 0.915, RMSEA = 0.046, and SRMR = 0.066. Study findings showed the goodness of fit of the model structure.
FIGURE 2.
Confirmatory factor analysis of the PAID–Bahasa Indonesia. The model comprises three latent factors: Emotional and diabetes self‐management distress, treatment‐related and social support distress, and complication distress. Standardised factor loadings are shown on the arrows, and error terms (e1–e20) are specified for each observed variable. Model fit indices indicated an acceptable fit (χ 2 = 332.995, df = 158, χ 2/df = 2.11, p < 0.001; GFI = 0.940; AGFI = 0.920; IFI = 0.917; TLI = 0.900; CFI = 0.915; RMSEA = 0.046, 90% CI [0.039, 0.053]; SRMR = 0.066).
4.2.3. Pearson's Correlation of the PAID‐Bahasa Indonesia
Table 4 shows results of the Pearson correlation test for variables. Each factor showed results significantly related to the total PAID score (p < 0.01, r = 0.615 ~ 0.938). Findings of the Pearson analysis showed a significant relationship between each factor (p < 0.01 and p < 0.05) with r values of 0.108 ~ 0.476. Gender, BMI, systolic and diastolic were not significantly related to the total PAID score or each factor (p > 0.05). Only age showed a significant relationship with treatment‐related and social support distress factors (p < 0.05, r = −0.111) and complication distress factors (p < 0.05, r = 0.120). Meanwhile, the duration of suffering from diabetes mellitus was related to the total PAID score (p < 0.05, r = 0.091) and emotional and diabetes self‐management distress factors (p < 0.05, r = 0.096).
TABLE 4.
Pearson's correlation for the PAID.
| Variable | PAID‐all items | Emotional and diabetes self‐management distress factors | Treatment‐related and social support distress factors | Complications distress factors |
|---|---|---|---|---|
| Age | −0.033 | −0.040 | −0.111 a | 0.120 a |
| Gender | −0.022 | −0.033 | −0.019 | 0.020 |
| Length of DM | 0.091 a | 0.096 a | 0.063 | 0.027 |
| BMI | −0.050 | −0.066 | 0.021 | −0.050 |
| SBP | 0.002 | −0.015 | 0.054 | −0.023 |
| DBP | −0.016 | −0.014 | 0.007 | −0.036 |
| Emotional and diabetes self‐management distress factors | 0.938 b | 1 | 0.426 b | 0.476 b |
| Treatment‐related and social support distress factors | 0.639 b | 0.426 b | 1 | 0.108 a |
| Complications distress factors | 0.615 b | 0.476 b | 0.108 a | 1 |
Abbreviations: BMI, body‐mass index; DBP, diastolic blood pressure; DM, diabetes mellitus; PAID, problem areas in diabetes; SBP, systolic blood pressure.
Correlation was significant at the 0.05 level (2‐tailed).
Correlation was significant at the 0.01 level (2‐tailed).
4.3. Reliability of the PAID‐Bahasa Indonesia
Results of the PAID‐Bahasa Indonesia reliability test are presented in Table 5; we determined Cronbach's alpha value for all items to be 0.858, while for factors 1 to 3, respective values were 0.864, 0.651 and 0.636. Item total correlations (ITCs) were determined for factor 1 (0.352 ~ 0.662), factor 2 (0.267 ~ 0.537) and factor 3 (0.390 ~ 0.517). Test–retest reliability showed excellent results for all factors, with the ICC (95% CI) value for all items of ICC = 0.938 (95% CI: 0.903–0.960), factor 1 of ICC = 0.936 (95% CI: 0.900–0.959), factor 2 of ICC = 0.835 (95% CI: 0.7245–0.895) and factor 3 of ICC = 0.949 (95% CI: 0.921–0.967). HTMT ratios of factor 1 to factor 2, factor 1 to factor 3 and factor 2 to factor 3 are 0.564, 0.508 and 0.137, respectively.
TABLE 5.
Reliability analysis and convergent validity.
| Domain and scale item | Construct validity (N = 522) | Test–retest reliability (N = 80) | HTMT ratio | ||||
|---|---|---|---|---|---|---|---|
| ITC | Cronbach's alpha | ICC | 95% CI | F1 vs. F2 | F1 vs. F2 | F2 vs. F3 | |
| All Items | 0.858 | 0.938 | 0.903–0.960 | 0.564 | 0.508 | 0.137 | |
| Factor 1 | 0.864 | 0.936 | 0.900–0.959 | ||||
| PAID_6 | 0.662 | ||||||
| PAID_8 | 0.622 | ||||||
| PAID_5 | 0.595 | ||||||
| PAID_11 | 0.635 | ||||||
| PAID_10 | 0.598 | ||||||
| PAID_16 | 0.579 | ||||||
| PAID_3 | 0.570 | ||||||
| PAID_7 | 0.538 | ||||||
| PAID_20 | 0.516 | ||||||
| PAID_9 | 0.472 | ||||||
| PAID_14 | 0.352 | ||||||
| Factor 2 | 0.651 | 0.835 | 0.7245–0.895 | ||||
| PAID_18 | 0.537 | ||||||
| PAID_4 | 0.417 | ||||||
| PAID_2 | 0.453 | ||||||
| PAID_17 | 0.340 | ||||||
| PAID_15 | 0.307 | ||||||
| PAID_1 | 0.302 | ||||||
| Factor 3 | 0.636 | 0.949 | 0.921–0.967 | ||||
| PAID_12 | 0.517 | ||||||
| PAID_19 | 0.437 | ||||||
| PAID_13 | 0.390 | ||||||
Abbreviations: CI, confidence interval; F, factor; HTMT, the heterotrait‐monotrait; ICC, interclass correlation coefficient; ITC, item total correlation; N, number; PAID, problem areas in diabetes.
A summary of the study methods and results is given in Table 6. In general, the study consisted of phases 1 and 2. Phase 1 was the translation process by two bilingual translators each and an internal team of three people, then a content validity analysis by 12 experts. Phase 2 was a psychometric analysis activity consisting of validity and reliability tests. The validity test assessed the construct validity using two methods, EFA and CFA, while the reliability test used internal consistency and test–retest results.
TABLE 6.
Summary of the psychometric process.
| No | Description | Method | Result |
|---|---|---|---|
| Phase 1: Translation process and content validity of the PAID | |||
| 1 | Translation process, back translation, and committee approach |
Indonesian‐to‐English translator (n = 2) English‐to‐Indonesian translator (n = 2) Internal team (n = 3) |
Final version of the PAID‐Bahasa Indonesia |
| 2 | Content validity | Expert review (n = 12) | Mean S‐CVI/ave. for the total score was 0.95 |
| Phase 2: Psychometric analysis of the PAID | |||
| 1 | Construct validity | ||
| EFA | Factor loading > 0.30 | Factor loading range 0.349 to 0.779 | |
| CFA | χ 2/df ≤ 3.0, GFI, AGFI, CFI TLI, and IFI > 0.9 (Hu and Bentler 1999; Schermelleh‐Engel et al. 2003), and RMSEA and SRMR ≤ 0.08 (Goretzko et al. 2024; Hu and Bentler 1999). | χ 2 = 332.995, df = 158, χ 2/df = 2.11, p < 0.001, GFI = 0.940, AGFI = 0.920, IFI = 0.917, TLI = 0.900, CFI = 0.915, RMSEA = 0.046, 90% CI [0.039, 0.053], and SRMR = 0.066 | |
| 2 | Reliability | ||
| Internal consistency Test–retest |
Cronbach's α ≥ 0.7 ICC Good reliability (0.75–0.90) Excellent (> 0.90) |
Cronbach's alpha value for all items was 0.858. ICC all item 0.938 ICC each factor in sequence were 0.936, 0.835, and 0.949 |
|
Abbreviations: AGFI, adjusted goodness‐of‐fit index; CFA, confirmatory factor analysis; CFI, comparative fit index; CVI, content validity index; EFA, exploratory factor analysis; GFI, goodness‐of‐fit index; ICC, interclass correlation coefficient; IFI, incremental fit index; PAID, problem areas in diabetes; RMSEA, root mean squared error of approximation; S‐CVI/ave., scale‐content validity index average; SRMR, standardised root mean squared residual; TLI, Tucker‐Lewis index.
5. Discussion
Based on our best knowledge, the 20‐item PAID has never been translated into Bahasa Indonesia. This study is the first to test the validity and reliability of a Bahasa Indonesia version of the PAID instrument. The findings in this study indicated that the PAID‐Bahasa Indonesia scale is valid and reliable for measuring stress in people with type 2 diabetes. To our knowledge, only two Southeast Asian countries have conducted psychometric studies of the PAID instrument: Malaysia (Jannoo et al. 2019) and Singapore (Venkataraman et al. 2015). These findings follow the Malay version of the PAID psychometric study, which used CFA and EFA analysis with three factors and showed valid and reliable results. Meanwhile, a study in Singapore with CFA and Rasch analysis showed valid and reliable single‐factor results with a reduction to 16 items.
Indonesia, Singapore, and Malaysia are multi‐ethnic countries; Malaysia and Singapore consist of Malay, Chinese, and Indian ethnicities, while in Indonesia, Chinese and Indians are ethnic minorities. With different regional distributions, Indonesia is dominated by indigenous ethnicities such as Javanese, Sundanese and Malay (Sealy et al. 2022). Compared to Singapore, Malaysia is more similar to Indonesia in terms of cultural similarities, history, religion and language similarities, which influence diet patterns, health beliefs and lifestyle. Besides that, Indonesia and Malaysia are upper‐middle‐income countries, unlike Singapore, a high‐income country (Ke 2024). Besides, the study in Singapore used the same English as the original version with participants fluent in English communication, so language adaptation was not carried out. Cultural differences affect health perceptions, behaviours and stress interpretation; this PAID‐20 Bahasa Indonesia version can provide an effective and culturally sensitive scale to early detection of stress and its sources accurately and comprehensively for Indonesians with type 2 diabetes.
Previous PAID psychometric studies also used CFA and EFA analysis for the construct validity analysis, including those conducted in the US (Snoek et al. 2000), Norway (Graue et al. 2012), Germany (Schmitt et al. 2016) and Rwanda (Lygidakis et al. 2021). Meanwhile, other cross‐cultural studies show valid and reliable results using various variations in data analysis methods, including Brazil (Gross et al. 2007), African‐Americans (Miller and Elasy 2008), Iceland (Sigurdardottir and Benediktsson 2008), Sweden (Amsberg et al. 2008), Taiwan (Huang et al. 2010), Iran (Arzaghi et al. 2011), Greece (Papathanasiou et al. 2014), Spain (Beléndez et al. 2014), Republic of Korea (Lee et al. 2014), Poland (Cichoń et al. 2021), Egypt (Sayed Ahmed et al. 2022) and Turkey (Altıkardeş et al. 2024; Huis In'T Veld et al. 2011).
The original PAID instrument contained 20 items with a single factor (Polonsky et al. 1995; Welch et al. 1997), and several years later, the developer conducted a follow‐up study and determined that there were four factors (Snoek et al. 2000). In the adaptation of the scale to languages in various countries, there were variations in the number of factors, including one factor (Cichoń et al. 2021; Huang et al. 2010; Venkataraman et al. 2015), two factors (Miller and Elasy 2008; Sigurdardottir and Benediktsson 2008), three factors (Amsberg et al. 2008; Jannoo et al. 2019; Papathanasiou et al. 2014) and four factors (Gross et al. 2007; Schmitt et al. 2016). Several studies even analysed variations in the number of factors, such as studies in Turkey (Huis In'T Veld et al. 2011), Spain (Beléndez et al. 2014), Republic of Korea (Lee et al. 2014), Rwanda (Lygidakis et al. 2021) and Egypt (Sayed Ahmed et al. 2022) which displayed one, two, three and four factors. This differed from other studies in Turkey, which showed numbers of factors of two, three and four (Altıkardeş et al. 2024). Meanwhile, studies in Norway showed two and four factors (Graue et al. 2012). These variations also resulted in variations in the naming of factors in each previous study.
This study used an eigenvalue of > 1 and found three factors with the best model fit results. There were slight variations and differences in item structure compared to other previous studies. The first factor comprised items that support emotional and diabetes self‐management problems, so we named it ‘emotional and diabetes self‐management distress’. Meanwhile, the second factor is a combination of items related to treatment‐related emotional problems and social support, so it was named ‘treatment‐related and social support distress’. The third factor contained items about diabetes mellitus complications and was therefore named ‘complication distress’. Based on the number of items, the first factor (emotional and diabetes self‐management distress) had the greatest number (11 items). This is reasonable considering that diabetes mellitus is a chronic disease which poses a severe threat to sufferers because they will live with this disease for the rest of their lives, causing a very significant burden for patients. Apart from that, just like in other developing countries, people with diabetes mellitus still experience stigma from society, which ultimately develops into self‐stigma by patients themselves. Stigmatisation, whether from social or self‐stigma, ultimately causes psychosocial problems (such as reduced self‐esteem, hopelessness/depression, and low self‐efficacy) and behaviours (Kalra et al. 2018).
The study used the HTMT ratio to analyse discriminant validity, with findings all < 0.85 indicating adequate discriminant validity (Rönkkö and Cho 2022). We found only two PAID adaptation studies that carried out a convergent validity analysis but used different analytical methods (Jannoo et al. 2019; Lygidakis et al. 2021). Lygidakis et al. (2021) found the composite reliability (CR) for each factor of 0.602 ~ 0.875; however, average variance extracted (AVE) values were not analysed, and Jannoo et al. (2019) showed CR and AVE for each factor of 0.500 ~ 0.590 and 0.832 ~ 0.911, respectively.
Results of the CFA strengthened the findings that the PAID‐Bahasa Indonesia with three factors is valid for measuring stress in people with type 2 diabetes in Indonesia. This study displayed only one CFA, using the structure obtained by the EFA, in line with previous studies, which also only displayed results of one model analysis (Huang et al. 2010; Jannoo et al. 2019; Schmitt et al. 2016; Venkataraman et al. 2015). In contrast to several other studies, the CFA was carried out with several models and different variations in the number of factors to obtain good model fit results (Beléndez et al. 2014; Lee et al. 2014; Lygidakis et al. 2021; Sayed Ahmed et al. 2022).
All PAID‐Bahasa Indonesia items showed good internal consistency as evidenced by Cronbach's α coefficient of 0.858, in line with previous studies with values of 0.85 (Huis In'T Veld et al. 2011) and 0.88 (Lygidakis et al. 2021). However, this differed from previous ones which had excellent internal consistency with Cronbach's α coefficients of > 0.9 (Cichoń et al. 2021; Sayed Ahmed et al. 2022; Schmitt et al. 2016; Venkataraman et al. 2015). In the internal consistency analysis of each factor, only emotional and diabetes self‐management distress factors were found to have Cronbach's α coefficient of > 0.86, while those of treatment‐related and social support distress and complication distress factors were 0.651 and 0.636. However, Cronbach's α value of > 0.6 still shows that the internal consistency can be accepted (Van Griethuijsen et al. 2015). Previous methodological literature reveals that alpha values between 0.60–0.70 are still acceptable in exploratory research or when factors include a small number of items (Taber 2018). Moreover, both factors demonstrated good test–retest reliability. Therefore, we retained these factors as they showed consistent reliability and conceptual coherence. Other previous study findings also showed the existence of subdimensions with Cronbach's α coefficients of < 0.7 (Altıkardeş et al. 2024; Cichoń et al. 2021; Lee et al. 2014; Lygidakis et al. 2021).
The ITC analysis revealed that several items had low values, namely item 1 with a value of < 0.3 and four items (13, 14, 15 and 17) with values of < 0.4; for multidimensional scales, the acceptable ITC value is > 0.2 (Kristiansen et al. 2016). We assumed the low Cronbach's α coefficient of the second and third factors and the ITC value due to participant heterogeneity. A diverse or heterogeneous sample can produce inconsistent responses across items, thus causing low internal consistency (Gharaibeh et al. 2017). Our research locations in 15 provinces involving rural and urban areas in Indonesia allowed for wide variations in respondent characteristics, such as age, ethnicity, educational level, and occupation. Variations in these characteristics may affect how participants interpret the items. Several previous studies also analysed ITC with values 0.32 ~ 0.84 (Polonsky et al. 1995), > 0.3 (Welch et al. 1997), 0.34 ~ 0.79 (Amsberg et al. 2008) and 0.386 ~ 0.828 (Papathanasiou et al. 2014). In contrast to this study, which carried out an ITC analysis based on each factor, two of those studies directly analysed each item without grouping them into factors.
Based on the ICC value, the PAID‐Bahasa Indonesia test–retest of all items and of factors 1 and 3 showed excellent reliability, while factor 2 showed good reliability. ICC values of 0.75 ~ 0.9 indicate good reliability, while > 0.9 indicates excellent (Liljequist et al. 2019). Several studies also conducted test–retest reliability with variations in the analysis presented and the distance between data collection from the baseline, including ICC of 0.89 (10 days) (Lee et al. 2014), r = 0.83 and p < 0.01 (1 month) (Huang et al. 2010), ICC of 0.79 (1 month) (Graue et al. 2012), total item ICC of 0.97 and ICC per factor ranging 0.92 ~ 0.97 (1 month) (Sayed Ahmed et al. 2022), Pearson's correlation of 0.83 for all PAID items and each factor ranging 0.64 ~ 0.80 (2 months) (Snoek et al. 2000), and an ICC of 0.942 (60 ± 15 days) (Papathanasiou et al. 2014). Based on previous studies, this study applied 14 days as a time interval for the test–retest reliability assessment (Park et al. 2018).
Diabetes mellitus is a chronic disease and cannot be cured, so it requires lifelong management, which impacts all aspects of the individual, both physically and psychologically (Kalra et al. 2018). Early detection of stress and its sources accurately and comprehensively can prevent more severe psychosocial problems, such as more severe mental disorders or a worsening quality of life. People with diabetes mellitus are expected to have regular checks on their condition every month under recommendations in clinical guidelines for diabetes mellitus management (Magliano and Boyko 2021). Nurses and other health teams can incorporate stress evaluation for people with diabetes mellitus into a routine program at primary or other healthcare facilities.
The PAID scale can be an option and can be relied upon to examine more comprehensive problems. The PAID‐Bahasa Indonesia is a good choice for studying problems that can cause distress to people with type 2 diabetes. The PAID‐Bahasa Indonesia consists of three factors, which can be used as a whole or separately, and each factor can be adjusted for measurement purposes. The 20 items are sufficient, being neither too many nor too few, so it does not take a long time to fill out the scale, but problems can be thoroughly studied. The short time needed to complete the scale should benefit nurses by allowing them to carry out other tasks more freely, and patients feel free when filling it out. In this study, subjects completed all of the instruments in an average of 5 ~ 10 min, so it is very suitable for clinical settings where everything needs to be fast and agile; the time in this study was shorter than previous studies that required an estimated time to complete the instruments of 20 ~ 40 min (Miller and Elasy 2008). This longer time may have been due to previous studies using several comparison instruments other than the PAID scale.
The results of stress examinations can increase nurses' attention to the psychological problems of people with diabetes mellitus and become a basis for determining the need for appropriate interventions such as psychoeducation, psychosocial, or collaborative programs involving other health workers. Nurses and other health workers can also use Indonesian PAID for further research related to stress in people with diabetes mellitus.
Although the PAID‐Bahasa Indonesia shows benefits for measuring people with type 2 diabetes stress, the study has some limitations. First, we did not assess the convergent validity but used other assessments to show correspondence or convergence between similar constructs. Second, the participants only included people with type 2 diabetes mellitus who checked themselves in primary health care facilities, so it limits the generalisation to all diabetes mellitus or diabetes patients in secondary or tertiary health care facilities. Third, the Cronbach alpha factor two and three values < 0.7 indicate suboptimal internal consistency. However, the Cronbach alpha values for all items show good internal consistency. Fourth, the characteristics of respondents obtained using a self‐reported questionnaire can trigger bias and inaccuracies; to minimise the increasing number of biased clinical statuses, height, weight, and blood pressure are measured directly. However, this research used a sufficient number of participants derived from 15 provinces in Indonesia to represent the diversity of the population, and the validity and reliability results obtained were very stable and accurate. Various analytical approaches can better be used to explain results when conducting cross‐cultural adaptation studies.
6. Conclusions
The study showed that the PAID‐Bahasa Indonesia has good psychometric results for measuring stress, as indicated by adequate validity and reliability test results. The PAID‐Bahasa Indonesia consists of three factors, namely emotional and diabetes self‐management distress, treatment‐related and social support distress, and complications distress. Based on these findings, it can be concluded that the PAID‐Bahasa Indonesia is suitable for measuring stress in people with type 2 diabetes in Indonesia.
Author Contributions
Siti Fadlilah: conceptualization, data curation, formal analysis, investigation, methodology, project administration and writing – original draft. Wahyu Rochdiat Murdhiono: formal analysis, investigation, project administration and writing – original draft. Meidiana Dwidiyanti: methodology, supervision, validation and writing‐review and editing. Hiroshi Sugimoto: methodology, supervision, validation and writing – review and editing. Andi Muhammad Fiqri Muslih Djaya: formal analysis, investigation, project administration and writing – original draft. Chia Ling Lin: methodology, resources, and writing‐original draft. Hsiu Ting Tsai: conceptualization, data curation, funding acquisition, methodology, supervision, validation and writing – review and editing.
Funding
This study was funded by The National Science and Technology Council, Taiwan through grant 10.13039/100020595.
Conflicts of Interest
The authors declare no conflicts of interest.
Supporting information
Table S1: presents the content validity results, including item‐level content validity index (I‐CVI), content validity ratio (CVR) and scale‐level indices based on expert evaluation.
Table S2: reports the exploratory factor analysis results, including eigenvalues and the proportion of variance explained for each factor.
Figure S1: displays the scree plot of the PAID scale, illustrating the distribution of eigenvalues and supporting the retention of a three‐factor solution.
Acknowledgements
We gratefully acknowledge the physicians and nurses at public health centers in the setting of this study for their assistance with recruiting study participants.
Data Availability Statement
All data from the findings of this study are available upon request to the corresponding researcher.
References
- Akhtar‐Danesh, N. 2023. “Impact of Factor Rotation on Q‐Methodology Analysis.” PLoS One 18, no. 9: e0290728. 10.1371/journal.pone.0290728. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Altıkardeş, D. K. , Nefs G., Hacışahinoğulları H., Smeets T., and Pouwer F.. 2024. “Reliability and Validity of the Turkish Version of the Problem Areas in Diabetes (PAID) Survey: Results From Diabetes MILES–Turkey.” Primary Care Diabetes 18, no. 2: 218–223. 10.1016/j.pcd.2024.01.001. [DOI] [PubMed] [Google Scholar]
- Amsberg, S. , Wredling R., Lins P.‐E., Adamson U., and Johansson U.‐B.. 2008. “The Psychometric Properties of the Swedish Version of the Problem Areas in Diabetes Scale (Swe‐PAID‐20): Scale Development.” International Journal of Nursing Studies 45, no. 9: 1319–1328. 10.1016/j.ijnurstu.2007.09.010. [DOI] [PubMed] [Google Scholar]
- Arzaghi, S. M. , Mahjouri M. Y., Heshmat R., Khashayar P., and Larijani B.. 2011. “Psychometric Properties of the Iranian Version of the Problem Areas in Diabetes Scale (IR‐PAID‐20).” Journal of Diabetes and Metabolic Disorders 10: 16. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Barbera, J. , Naibert N., Komperda R., and Pentecost T. C.. 2021. “Clarity on Cronbach's Alpha Use.” Journal of Chemical Education 98, no. 2: 257–258. 10.1021/acs.jchemed.0c00183. [DOI] [Google Scholar]
- Beléndez, M. , Hernández‐Mijares A., Marco J., Domínguez J. R., and Pomares F. J.. 2014. “Validation of the Spanish Version of the Problem Areas in Diabetes (PAID‐SP) Scale.” Diabetes Research and Clinical Practice 106, no. 3: e93–e95. 10.1016/j.diabres.2014.09.012. [DOI] [PubMed] [Google Scholar]
- Budi, T. A. , Pramono R. B., and Wicaksana A. L.. 2023. “Validity and Reliability Test for Problem Areas in Diabetes‐Five Item Short Form (PAID‐5) Indonesia Version.” Pakistan Journal of Medical Sciences 39, no. 3: 737–741. 10.12669/pjms.39.3.7088. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Cichoń, E. , Kiejna A., Gondek T. M., et al. 2021. “PAID‐PL‐The Polish Version of the Problem Areas in Diabetes Scale: Perfect Reliability and a One‐Factor Structure.” Diabetes, Metabolic Syndrome and Obesity: Targets and Therapy 14: 4433–4441. 10.2147/dmso.S322850. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Gharaibeh, B. , Al‐Smadi A. M., and Boyle D.. 2017. “Psychometric Properties and Characteristics of the Diabetes Self Management Scale.” International Journal of Nursing Sciences 4, no. 3: 252–259. 10.1016/j.ijnss.2017.04.001. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Goretzko, D. , Siemund K., and Sterner P.. 2024. “Evaluating Model Fit of Measurement Models in Confirmatory Factor Analysis.” Educational and Psychological Measurement 84, no. 1: 123–144. 10.1177/00131644231163813. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Graue, M. , Haugstvedt A., Wentzel‐Larsen T., Iversen M. M., Karlsen B., and Rokne B.. 2012. “Diabetes‐Related Emotional Distress in Adults: Reliability and Validity of the Norwegian Versions of the Problem Areas in Diabetes Scale (PAID) and the Diabetes Distress Scale (DDS).” International Journal of Nursing Studies 49, no. 2: 174–182. 10.1016/j.ijnurstu.2011.08.007. [DOI] [PubMed] [Google Scholar]
- Gross, C. C. , Scain S. F., Scheffel R., Gross J. L., and Hutz C. S.. 2007. “Brazilian Version of the Problem Areas in Diabetes Scale (B‐PAID): Validation and Identification of Individuals at High Risk for Emotional Distress.” Diabetes Research and Clinical Practice 76, no. 3: 455–459. 10.1016/j.diabres.2006.09.022. [DOI] [PubMed] [Google Scholar]
- Guo, H. , Wu H., and Li Z.. 2023. “The Pathogenesis of Diabetes.” International Journal of Molecular Sciences 24, no. 8: 6978. 10.3390/ijms24086978. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Hair, J. F. 2009. “Multivariate Data Analysis.”
- Hu, L. t. , and Bentler P. M.. 1999. “Cutoff Criteria for Fit Indexes in Covariance Structure Analysis: Conventional Criteria Versus New Alternatives.” Structural Equation Modeling: A Multidisciplinary Journal 6, no. 1: 1–55. [Google Scholar]
- Huang, M.‐F. , Courtney M., Edwards H., and McDowell J.. 2010. “Validation of the Chinese Version of the Problem Areas in Diabetes (PAID‐C) Scale.” Diabetes Care 33, no. 1: 38–40. 10.2337/dc09-0768. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Huis In'T Veld, E. M. , Makine C., Nouwen A., et al. 2011. “Validation of the Turkish Version of the Problem Areas in Diabetes Scale.” Cardiovascular Psychiatry and Neurology 2011: 315068. 10.1155/2011/315068. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Jannoo, Z. , Yap B. W., Khan N. M., and Farcomeni A.. 2019. “Assessing Diabetes Distress Among Type 2 Diabetes Mellitus in Malaysia Using the Problem Areas in Diabetes Scale.” Value in Health Regional Issues 18: 159–164. 10.1016/j.vhri.2019.03.004. [DOI] [PubMed] [Google Scholar]
- Kalra, S. , Jena B. N., and Yeravdekar R.. 2018. “Emotional and Psychological Needs of People With Diabetes.” Indian Journal of Endocrinology and Metabolism 22, no. 5: 696–704. 10.4103/ijem.IJEM_579_17. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Ke, Y. 2024. “ASEAN Four's Middle Income Trap Dilemma: Evidence of the Middle Technology Trap.” Asian Review of Political Economy 3, no. 1: 14. 10.1007/s44216-024-00033-5. [DOI] [Google Scholar]
- Koo, T. K. , and Li M. Y.. 2016. “A Guideline of Selecting and Reporting Intraclass Correlation Coefficients for Reliability Research.” Journal of Chiropractic Medicine 15, no. 2: 155–163. 10.1016/j.jcm.2016.02.012. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Kristiansen, L. , Holmstrom M. R., and Olofsson N.. 2016. “Assessing the Construct Validity and Reliability of School Health Records Using the ‘Health Dialogue Questionnaire’ in the Eleventh Grade.” AIMS Public Health 3, no. 3: 470–486. 10.3934/publichealth.2016.3.470. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Kyriazos, T. A. 2018. “Applied Psychometrics: Sample Size and Sample Power Considerations in Factor Analysis (EFA, CFA) and SEM in General.” Psychology 9, no. 8: 2207. 10.4236/psych.2018.98126. [DOI] [Google Scholar]
- Lee, E.‐H. , Lee Y. W., Lee K.‐W., Kim Y. S., and Nam M.‐S.. 2014. “Measurement of Diabetes‐Related Emotional Distress Using the Problem Areas in Diabetes Scale: Psychometric Evaluations Show That the Short Form Is Better Than the Full Form.” Health and Quality of Life Outcomes 12: 1–9. 10.1186/s12955-014-0142-z. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Lee, J. , Lee E.‐H., Kim C.‐J., and Moon S. H.. 2015. “Diabetes‐Related Emotional Distress Instruments: A Systematic Review of Measurement Properties.” International Journal of Nursing Studies 52, no. 12: 1868–1878. 10.1016/j.ijnurstu.2015.07.004. [DOI] [PubMed] [Google Scholar]
- Liljequist, D. , Elfving B., and Skavberg Roaldsen K.. 2019. “Intraclass Correlation–A Discussion and Demonstration of Basic Features.” PLoS One 14, no. 7: e0219854. 10.1371/journal.pone.0219854. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Lygidakis, C. , Uwizihiwe J. P., Bia M., et al. 2021. “Cultural Adaptation and Psychometric Evaluation of the Kinyarwanda Version of the Problem Areas in Diabetes (PAID) Questionnaire.” Health and Quality of Life Outcomes 19, no. 1: 183. 10.1186/s12955-021-01821-w. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Magliano, D. J. , and Boyko E. J.. 2021. IDF Diabetes Atlas 10th Edition Scientific Committee. International Diabetes Federation. https://www.ncbi.nlm.nih.gov/books/NBK581934/. [Google Scholar]
- Miller, S. T. , and Elasy T. A.. 2008. “Psychometric Evaluation of the Problem Areas in Diabetes (PAID) Survey in Southern, Rural African American Women With Type 2 Diabetes.” BMC Public Health 8: 1–7. 10.1186/1471-2458-8-70. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Mishra, P. , Pandey C. M., Singh U., Gupta A., Sahu C., and Keshri A.. 2019. “Descriptive Statistics and Normality Tests for Statistical Data.” Annals of Cardiac Anaesthesia 22, no. 1: 67–72. 10.4103/aca.ACA_157_18. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Ong, K. L. , Stafford L. K., McLaughlin S. A., et al. 2023. “Global, Regional, and National Burden of Diabetes From 1990 to 2021, With Projections of Prevalence to 2050: A Systematic Analysis for the Global Burden of Disease Study 2021.” Lancet 402, no. 10397: 203–234. 10.1016/S0140-6736(23)01301-6. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Papathanasiou, A. , Koutsovasilis A., Shea S., et al. 2014. “The Problem Areas in Diabetes (PAID) Scale: Psychometric Evaluation Survey in a Greek Sample With Type 2 Diabetes.” Journal of Psychiatric and Mental Health Nursing 21, no. 4: 345–353. 10.1111/j.1365-2850.2012.01875.x. [DOI] [PubMed] [Google Scholar]
- Park, M. S. , Kang K. J., Jang S. J., Lee J. Y., and Chang S. J.. 2018. “Evaluating Test‐Retest Reliability in Patient‐Reported Outcome Measures for Older People: A Systematic Review.” International Journal of Nursing Studies 79: 58–69. 10.1016/j.ijnurstu.2017.11.003. [DOI] [PubMed] [Google Scholar]
- Polonsky, W. H. , Anderson B. J., Lohrer P. A., et al. 1995. “Assessment of Diabetes‐Related Distress.” Diabetes Care 18, no. 6: 754–760. 10.2337/diacare.18.6.754. [DOI] [PubMed] [Google Scholar]
- Rönkkö, M. , and Cho E.. 2022. “An Updated Guideline for Assessing Discriminant Validity.” Organizational Research Methods 25, no. 1: 6–14. 10.1177/1094428120968614. [DOI] [Google Scholar]
- Sayed Ahmed, H. A. , Mohamed S. F., Elotla S. F., Mostafa M., Shah J., and Fouad A. M.. 2022. “Psychometric Properties of the Arabic Version of the Problem Areas in Diabetes Scale in Primary Care.” Frontiers in Public Health 10: 843164. 10.3389/fpubh.2022.843164. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Schermelleh‐Engel, K. , Moosbrugger H., and Müller H.. 2003. “Evaluating the Fit of Structural Equation Models: Tests of Significance and Descriptive Goodness‐Of‐Fit Measures.” Methods of Psychological Research Online 8, no. 2: 23–74. [Google Scholar]
- Schmitt, A. , Reimer A., Kulzer B., Haak T., Ehrmann D., and Hermanns N.. 2016. “How to Assess Diabetes Distress: Comparison of the Problem Areas in Diabetes Scale (PAID) and the Diabetes Distress Scale (DDS).” Diabetic Medicine 33, no. 6: 835–843. 10.1111/dme.12887. [DOI] [PubMed] [Google Scholar]
- Sealy, T. , Ibrahim Z., Zulian P. B., and Rasid I. M.. 2022. “South and Southeast Asia: Deep Diversity Under Strain.” Religion, State and Society 50, no. 4: 452–468. 10.1080/09637494.2022.2126258. [DOI] [Google Scholar]
- Shrestha, N. 2021. “Factor Analysis as a Tool for Survey Analysis.” American Journal of Applied Mathematics and Statistics 9, no. 1: 4–11. 10.12691/ajams-9-1-2. [DOI] [Google Scholar]
- Sigurdardottir, A. , and Benediktsson R.. 2008. “Reliability and Validity of the Icelandic Version of the Problem Area in Diabetes (PAID) Scale.” International Journal of Nursing Studies 45, no. 4: 526–533. 10.1016/j.ijnurstu.2006.10.008. [DOI] [PubMed] [Google Scholar]
- Snoek, F. J. , Pouwer F., Welch G. W., and Polonsky W. H.. 2000. “Diabetes‐Related Emotional Distress in Dutch and US Diabetic Patients: Cross‐Cultural Validity of the Problem Areas in Diabetes Scale.” Diabetes Care 23, no. 9: 1305–1309. 10.2337/diacare.23.9.1305. [DOI] [PubMed] [Google Scholar]
- Tabachnick, B. , and Fidell L.. 2019. Using Multivariate Statistics. 7th Еd ed. Allyn & Bacon/Pearson Education. [Google Scholar]
- Taber, K. S. 2018. “The Use of Cronbach's Alpha When Developing and Reporting Research Instruments in Science Education.” Research in Science Education 48, no. 6: 1273–1296. 10.1007/s11165-016-9602-2. [DOI] [Google Scholar]
- Tavakol, M. , and Wetzel A.. 2020. “Factor Analysis: A Means for Theory and Instrument Development in Support of Construct Validity.” International Journal of Medical Education 11: 245–247. 10.5116/ijme.5f96.0f4a. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Ten Hove, D. , Jorgensen T. D., and van der Ark L. A.. 2022. “Updated Guidelines on Selecting an Intraclass Correlation Coefficient for Interrater Reliability, With Applications to Incomplete Observational Designs.” Psychological Methods 29: 967–979. 10.1037/met0000516. [DOI] [PubMed] [Google Scholar]
- Teresi, J. A. , Yu X., Stewart A. L., and Hays R. D.. 2022. “Guidelines for Designing and Evaluating Feasibility Pilot Studies.” Medical Care 60, no. 1: 95–103. 10.1097/mlr.0000000000001664. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Terwee, C. B. , Bot S. D., de Boer M. R., et al. 2007. “Quality Criteria Were Proposed for Measurement Properties of Health Status Questionnaires.” Journal of Clinical Epidemiology 60, no. 1: 34–42. 10.1016/j.jclinepi.2006.03.012. [DOI] [PubMed] [Google Scholar]
- Tesfaye Yunka, T. , Berhanu Mogas S., Zawdie B., et al. 2020. “The Hidden Burden of Diabetes Mellitus in an Urban Community of Southwest Ethiopia.” Diabetes, Metabolic Syndrome and Obesity: Targets and Therapy 13: 2925–2933. 10.2147/dmso.S269386. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Van Griethuijsen, R. A. , van Eijck M. W., Haste H., et al. 2015. “Global Patterns in Students' Views of Science and Interest in Science.” Research in Science Education 45: 581–603. 10.1007/s11165-014-9438-6. [DOI] [Google Scholar]
- Venkataraman, K. , Tan L. S. M., Bautista D. C. T., et al. 2015. “Psychometric Properties of the Problem Areas in Diabetes (PAID) Instrument in Singapore.” PLoS One 10, no. 9: e0136759. 10.1371/journal.pone.0136759. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Welch, G. W. , Jacobson A. M., and Polonsky W. H.. 1997. “The Problem Areas in Diabetes Scale: An Evaluation of Its Clinical Utility.” Diabetes Care 20, no. 5: 760–766. 10.2337/diacare.20.5.760. [DOI] [PubMed] [Google Scholar]
- Zitzmann, S. , Walther J.‐K., Hecht M., and Nagengast B.. 2022. “What Is the Maximum Likelihood Estimate When the Initial Solution to the Optimization Problem Is Inadmissible? The Case of Negatively Estimated Variances.” Psychiatrist 4, no. 3: 343–356. 10.3390/psych4030029. [DOI] [Google Scholar]
Associated Data
This section collects any data citations, data availability statements, or supplementary materials included in this article.
Supplementary Materials
Table S1: presents the content validity results, including item‐level content validity index (I‐CVI), content validity ratio (CVR) and scale‐level indices based on expert evaluation.
Table S2: reports the exploratory factor analysis results, including eigenvalues and the proportion of variance explained for each factor.
Figure S1: displays the scree plot of the PAID scale, illustrating the distribution of eigenvalues and supporting the retention of a three‐factor solution.
Data Availability Statement
All data from the findings of this study are available upon request to the corresponding researcher.