Confirmatory Factor Analysis of the Malay Version of the Confusion, Hubbub and Order Scale (CHAOS-6) among Myocardial Infarction Survivors in a Malaysian Cardiac Healthcare Facility

Abstract

Background

The six item Confusion, Hubbub and Order Scale (CHAOS-6) has been validated as a reliable tool to measure levels of household disorder. We aimed to investigate the goodness of fit and reliability of a new Malay version of the CHAOS-6.

Methods

The original English version of the CHAOS-6 underwent forward-backward translation into the Malay language. The finalised Malay version was administered to 105 myocardial infarction survivors in a Malaysian cardiac health facility. We performed confirmatory factor analyses (CFAs) using structural equation modelling. A path diagram and fit statistics were yielded to determine the Malay version’s validity. Composite reliability was tested to determine the scale’s reliability.

Results

All 105 myocardial infarction survivors participated in the study. The CFA yielded a six-item, one-factor model with excellent fit statistics. Composite reliability for the single factor CHAOS-6 was 0.65, confirming that the scale is reliable for Malay speakers.

Conclusion

The Malay version of the CHAOS-6 was reliable and showed the best fit statistics for our study sample. We thus offer a simple, brief, validated, reliable and novel instrument to measure chaos, the Skala Kecelaruan, Keriuhan & Tertib Terubahsuai (CHAOS-6), for the Malaysian population.

Introduction

People who have survived chronic lifethreatening illnesses are often confronted with multiple physical and psychological stressors: unemployment, homelessness, poverty, noise, overcrowding, reduced social support and deteriorating mental status, among others (1–3). These factors can disrupt people’s ability to access and implement medical care, predisposing patients to engage in risky health behaviours, miss appointments and adhere poorly to treatment plans (3–5).

In this context, the concept of chaos was initially employed to quantify the disorder in children’s homes that could impact their development and health. In 1994, Bronfenbrener and Cecil (6) studied chaos among caregivers and parents of children in middle childhood using a bio-ecological model. They conceptualised chaotic systems to be characterised by frenetic activity, lack of structure and unpredictable daily activities or routines (7). However, these claims were contested, as a validated global instrument to measure chaos was not available. In 1995, Matheny and colleagues (8) initially developed the Confusion, Hubbub and Order Scale (CHAOS) to assess a child’s home in terms of the amount of commotion, ability to find things, feeling rushed, ability to relax, having regular routines or a calm atmosphere. The 15item measure was constructed to be answered by parents about their children’s home environment (8). Although much literature has used this original scale as a basis for investigating life chaos in a child’s home, preliminary efforts were also initiated to measure life chaos in adults with chronic diseases (3, 4, 9).

In 2011, Wong and colleagues (3) adapted several concepts measured by the original CHAOS scale to the adult population. They restructured the text to reflect items to measure a person’s life in general, rather than focusing solely on the home environment. Items related to finance, employment, housing and appointments were added, yielding a set of 30 items that was subsequently reduced to 12 after iterative oneon-one cognitive interviews found some items to be confusing or irrelevant. A multi-trait scaling analysis was performed to eliminate items with lower correlations, resulting in a final brief scale known as CHAOS-6. This six-item scale consisted of self-referential statements using a five-point Likert scale ranging from ‘definitely true’ to ‘definitely false’. The six-item scale yielded a Cronbach’s alpha value of 0.67, suggesting an acceptable internal consistency (3).

Only three studies have thus far used CHAOS-6 in adult populations: the first was among a sample of HIV-infected persons (3), while the other two investigated American cohorts (4, 9). They all found emerging associations between life chaos and perceived barriers to medical care. The importance of research on life chaos suggests the need for a valid and reliable tool to measure and assess universal perceptions of chaos. The present study aims to validate the first non-English version of the CHAOS-6, specifically for Malay speakers.

Methods

Subjects and Study Setting

This cross-sectional study was conducted between May and September 2016. We recruited 105 myocardial infarction patients from the Cardiology Clinic of Hospital Serdang, Selangor, Malaysia. The study’s objectives and benefits were explained to the participants, whose confidentiality and anonymity were assured. Written informed consent was obtained from each participant before data collection began. Ethical approval was obtained from the Malaysian Ministry of Health’s Medical Research Ethics Committee (approval number: NMRR-152210-28696-IIR).

Instrument

The Brief Confusion, Hubbub and Order Scale (CHAOS-6)

CHAOS-6 addresses the consistencies of people’s daily routines, their ability to plan or anticipate future activities and their punctuality (4). This single-factor, six-item brief scale is measured on a five-point Likert scale (1 = ‘definitely true’ to 5 = ‘definitely false’). Total scores were obtained by reversing the responses (1 = 5, 2 = 4, 3 = 3, 4 = 2, 5 = 1) to the four negatively worded items (items 2, 4, 5, 6) and summing across all items in the scale. Sum scores range from 6 to 30, with higher scores representing a more chaotic lifestyle characterised by greater household disorder, inconsistencies in daily routines, lack of punctuality and failure to carry out planned activities (4, 9).

Translation

Adaptation of the Malay version of CHAOS-6 from the original English version was conducted according to commonly accepted guidelines of translating and adapting health status questionnaires (10, 11). During this process, a forward translation was produced from the original English version into Malay. The Malay version was then translated back into English and compared with the original version. Errors in the targeted language version that altered item meaning in the backward translation were identified and rectified. This process was repeated until a satisfactory translation was obtained. The final Malay version was reviewed by an expert and pilot-tested on ten patients who did not participate in the study. Additional grammatical errors or misspellings were subsequently corrected to produce the final Malay version of the CHAOS-6 (Supplementary file 1).

Statistical Analysis

The analysis was performed using IBM SPSS AMOS (v.23) statistical software, with total scores obtained by summing across all six items after reversing the four negatively worded items. The multivariate normality of the items was assessed statistically by Mardia’s normalised estimate of multivariate kurtosis in the form of critical ratio of kurtosis in the output. A critical ratio of kurtosis below 5.0 indicates multivariate normality (12). Descriptive analysis was conducted to describe the characteristics of the sample and the distribution of CHAOS-6 items. The means, standard deviations (SDs), frequencies and percentages were reported.

Confirmatory Factor Analysis

We conducted a CFA to examine the universal fit of the scale and determine its suitability for Malay speakers. Path analysis was yielded. Several recommended fit indices— relative chi-square (χ²/df), goodness of fit index (GFI), comparative fit index (CFI), incremental fit index (IFI), normed fit index (NFI), Tucker-Lewis Index (TLI), standardised root mean square residual (SRMR) and root mean square error of approximation (RMSEA)—were reported (13). The relative chi-square is the ratio of the chi-square to degrees of freedom, with a value below 3 indicating acceptable fitness of the model (14) and a value of 2 or less indicating superior fitness (15). GFI values range between 0 and 1, with 1 indicating a perfect fit (16). The values for CFI, IFI, NFI and TLI range between 0 and 1, with values closer to 1 indicating better data fitness (17). The acceptable value for SRMR is less than 0.10 (18), while an RMSEA between 0.08 and 0.10 exhibits a mediocre fit and an RMSEA below 0.08 indicates a good fit (19).

Reliability

Composite reliability (CR) was used to test the reliability and internal consistency of a latent construct. A value of CR ≥ 0.6 was required to achieve composite reliability for a given construct (20).

Results

Socio-demographic Characteristics of the Respondents

Our sample was comprised of 95 (90.5%) men and 10 (9.5%) women. The mean age of the respondents was 47.5 (SD = 5.5) years, with 72 (68.6%) respondents over 45 years old. 81 respondents (77.1%) had completed secondary education (Table 1).

Table 1.

Socio-demographics of the respondents (n = 105)

Characteristics	n(%)
Gender
Men	95 (90.5)
Women	10 (9.5)
Age (years)
≤ 45	33 (31.4)
> 45	72 (68.6)
Education level
Primary	13 (12.4)
Secondary	81 (77.1)
Tertiary	11 (10.5)

Distribution of the CHAOS-6 Item Score among Myocardial Infarction Survivors

Response scores to the 6-item CHAOS scale ranged from 1 (‘definitely true’) to 5 (‘definitely false’). On a question-by-question basis, 8.6% (9/105) to 40.9% (43/105) of the 105 respondents stated a score of 1, while 4.8% (5/105) to 34.3% (36/105) stated a score of 5 (Table 2).

Table 2.

Distribution of respondent scores of the CHAOS-6 (n = 105)

Items	Score
	1, n(%)	2, n(%)	3, n(%)	4, n(%)	5, n(%)
1. My life is organized.	43(40.9)	31(29.5)	11(10.5)	15(14.3)	5(4.8)
2. My life is unstable.	9(8.6)	16(15.2)	13(12.4)	31(29.5)	36(34.3)
3. My routine is the same from week to week.	24(22.9)	46(43.8)	14(13.3)	16(15.2)	5(4.8)
4. My daily activities from week to week are unpredictable.	21(20.0)	31(29.5)	21(20.0)	21(20.0)	11(10.5)
5. Keeping a schedule is difficult for me.	20(19.0)	23(21.9)	13(12.4)	30(28.6)	19(18.1)
6. I do not like to make appointments too far in advance because I do not know what might come up.	22(21.0)	18(17.1)	15(14.3)	31(29.5)	19(18.1)

Floor and Ceiling Effects of the CHAOS-6

The mean (SD) of single-factor total CHAOS score among myocardial infarction survivors was 16.0 (SD = 4.6), with scores ranging from 6 to 26; one respondent (1.0%) had a score of 6 and three respondents (2.9%) scored 26 (Table 3).

Table 3.

Floor and ceiling of the single factor of the CHAOS-6 (n = 105)

Factor	Item Number	Mean (SD)	Minimum	Maximum	Range	Reaching Floor, n(%)	Reaching Ceiling, n(%)
1	1,2,3,4,5,6	16.0 (4.6)	6.0	26.0	20.0	1 (1.0)	3 (2.9)

Confirmatory Factor Analyses

All items in the multivariate normality level were normally distributed. The critical ratio of kurtosis was 3.47. Following identification of the single-factor solution, we assessed the universal fit, focusing on the quality of the model using the SPSS AMOS package. CHAOS-6 was subjected to CFAs using structural equation modelling (SEM). Figure 1 illustrates the observed items and unobserved factors synthesised through CFA, with standardised regression weights ranging between 0.11 and 0.86.

Figure 1.

A single-factor model of the CHAOS-6 (N = 105)

The relative chi-square was 1.12, indicative of a superior fitness for the model (P = 0.347). All comparative indices in the model—CFI, IFI, NFI, TLI—were greater than 0.85 (0.99, 0.99, 0.89, 0.98 respectively), showing goodness of fit for the data. The GFI of 0.97 suggested a nearly perfect fit of the sample data. The SRMR value was below 0.10 (0.09) and the model’s RMSEA value was 0.033 (90% confidence interval (CI) 0.000–0.118), suggesting an acceptable and good fit model for Malay speakers.

Reliability of the CHAOS-6

The composite reliability was 0.65, confirming that the scale is a reliable tool for Malay speakers.

Discussion

We found a 6-question CHAOS questionnaire to be reliable and valid in an adult Malay-speaking population. The singlefactor model was logically sound, informative and meaningful, as the comparative indices GFI, SRMR and RMSEA demonstrated strong support for model fitness.

To date, only one original investigation examined the factor structure of CHAOS-6 by exploratory factor analysis (EFA) (3). With the intention of developing and validating the first non-English CHAOS-6, the scale underwent evaluation by CFA. We believe that CFA is nothing less than an indispensable strategy for questionnaire validation and is superior to EFA with regards to model measurement, in line with the justifications listed by Arifin and Yusoff (21). CFA verifies the relationships between items and their respective factors, allowing for fixing relationships to provide ways to assess fitness of the proposed theoretical model (22). In Wong’s original analysis, factor loadings ranged from 0.50 to 0.71 (3). The minimum acceptable value for factor loading in EFA is 0.40 (23). In the present study, the standardised factor loadings yielded from CFA were reported, and ranged between 0.11 and 0.86. Factor loadings in CFA have been recommended to be greater than 0.50 (20). However, it has also been postulated that this assumption could be overridden, such that item deletion is not warranted if CFA of the total model showed excellent fit statistics (20). As the fit parameters in our study supported this notion, we decided to maintain all six items in the model.

The composite reliability was 0.65, confirming that the Malay CHAOS-6 is adequately reliable to be administered among speakers of that language (20). This value was almost similar to that reported in the original version (3), but lower than that reported in a recent investigation that tested the internal consistency of the scale among an American myocardial infarction cohort (4). The fit statistics from CFAs revealed that the final singlefactor model of CHAOS-6 concurred with the sample data and displayed the best fit for our study sample. We are unable to compare the consistencies of the study findings, as there were no identified studies that employed a CFA approach to CHAOS-6 among myocardial infarction survivors or other populations. Further studies that are suitable for comparing with the findings of the current study are warranted.

Because Malaysia has high rates of cardiovascular diseases (24), this instrument may be used to understand the association of chaos with non-adherence with clinical practice (4), prompting caregivers and policymakers to identify individuals with chaotic lives as potentially being non-adherent to agreed treatment plans. Potential life-coping skills and benefits from referral to social work or other community-based services may be implemented through early screening via this brief, easily administered questionnaire.

Worldwide, the approach to health is undergoing a paradigm shift from disease to risk (25). Human health is strongly influenced by the environment in which people live—the healthy face greater risks of becoming sick, while the sick become more ill due to poverty, poor housing, noise, rapid urbanisation, pollution, rare neighbourhood integration, elevated frustrations regarding socio-financial decline, perceived insecurity and unemployment following illness, any or all of which may impede rapid recuperation from sickness by hindering people’s ability to manage or organise their lives (3). Given that Malaysians today are certainly exposed to these fundamental socioenvironmental factors due to the country’s raid development and with the projection that mental illness is expected to be the second biggest health problem within the country by 2020 (26), we felt it was timely to validate a tool for measuring chaos from the perspective of Malay speakers. We regard chaos as a hypothesised novel modifiable attribute that can be refined by further research to contribute to order in the home, in the immediate community and in the greater environment.

Limitations

Sample homogeneity of myocardial infarction survivors from a single health facility affects generalisability of the study findings. Future cross-validation studies should include more diverse and heterogeneous samples to exclude any homogeneity effects in our results. Certain low standardised factor loadings in the current study could be improved with the application of item response theory (IRT), a behaviour change assessment that uses a model-based approach to measure whether each item is fitted properly to the study population and culture (27). Our analyses indicate that CHAOS-6 reliably measure something, but we cannot yet state with certainty that what it measures is ‘chaos’. To refine what it does measure, subsequent efforts could involve sending observers to affected households to see whether observed chaos and survey-measured chaos are significantly correlated. In addition, exploration using robust methodological techniques is warranted in subsequent studies, as we hypothesise that ‘chaos’ is a potential novel modifiable trait affecting human lives.

Conclusion

The results of the current study showed that the Malay version of the brief Confusion, Hubbub and Order Scale (CHAOS-6) reveals a psychometrically sound and reliable tool for measuring chaotic lifestyles among myocardial infarction survivors. We hereby offer Skala Kecelaruan, Keriuhan & Tertib Terubahsuai (CHAOS-6), a simple, brief, validated, reliable and novel instrument to measure chaos in the lives of Malay speakers.

Appendix. SKALA KECELARUAN, KERIUHAN & TERTIB TERUBAHSUAI (CHAOS-6)

Kami berminat untuk mengetahui tentang gaya hidup anda yang teratur sejak bulan lalu. Bagi pernyataan yang berikut, sila tandakan “X” pada lajur respons yang bersesuaian.

Item	Pernyataan	Memang Benar	Agak Benar	Tidak Pasti	Agak Tidak Benar	Memang Tidak Benar
1.	Hidup saya teratur.
2.	Hidup saya tidak stabil.
3.	Rutin saya sama dari seminggu ke seminggu.
4.	Aktiviti harian saya dari seminggu ke seminggu tidak dapat dijangka.
5.	Saya sukar mengikut jadual.
6.	Saya tidak suka membuat janji temu terlalu awal kerana saya tidak tahu perkara yang akan berlaku.

(Note: The Malay CHAOS-6 scale is freely available and its use is permitted to anyone for non-profit purposes, either for research, academics or clinical purposes with appropriate citation of the authors, Ganasegeran K, Selvaraj K, Rashid A, 2017. Alteration of the scale, the use for profit purposes or distribution of the scale requires further permissions from the authors. To request for permissions, please email medkuru@yahoo.com).