Measuring reliability and validity of “Stressometer^®”: A computer-based mass screening and assessment tool for evaluation of stress level and sources of stressors

Abstract

Introduction:

It is essential to develop tools that can identify stress manifestation, source of stressors, and suffering in an effort to bridge the treatment gap and enhance behavioral health in the developing world. To that aim, the Stressometer^® (SOM) was developed as a comprehensive scale of stress and behavioral health for use around the world.

Materials and Methods:

A validation study of the Stressometer^® (SOM) was undertaken with a sample in India that included a nonclinical group and a group of patients at a clinic in New Delhi. For validation purposes, participants were also administered three currently validated scales, including Perceived Stress Scale, Stress Overload Scale (SOS), and Depression Anxiety Stress Scale (DASS).

Results:

The Stressometer^® (SOM) was found to be reliable and had high correlations with established scales.

Conclusion:

Stressometer^® (SOM) is a valid and reliable, computer based mass screening tool for evaluation of stress level and sources of stress. Overall, Stressometer^® (SOM) creates a robust measurement of stress and behavioral health that is likely culturally neutral and thus has universal applicability. A scale such as this one is ideal for use in the developing world to help bridge the treatment gap created and enhance behavioral health, especially in those suffering.

INTRODUCTION

Many suffering from behavioral health (or emotional /mental health) issues do not receive adequate treatment and may experience stigma or discrimination.[1] In the developing world, there are additional challenges to maintain adequate behavioral health, including an overtaxed health-care system, poverty, illiteracy, stigma, and a lack of knowledge on behavioral health.[2] For instance, the reality of mental health care in India is bleak,[3] with inadequate mental health professionals. Therefore, it is essential to develop tools that can bridge the treatment gap.

Stress, which can threaten behavioral health, is the result of an imbalance between the demands placed and their ability to handle those demands, with large individual differences in reactions.[4] Stress and inability to cope has been related to physical health issues, such as hypertension,[5] and also to behavioral health issues. Theories on behavioral health have been developed and studied mainly from the Western point of view,[6] which focuses on stress and its results from a more biomedical angle.[7] Studies examining behavioral health in other cultures have discovered differences in stress, coping, and behavioral health,[8] thus suggesting that standard Western instruments cannot be simply translated and used across cultures.[9] For instance, many non-Western patients present with somatic complaints, such as aches, pains, and gastrointestinal distress, and not offer cognitive and mood symptoms until asked,[10] possibly because of a cultural difference that physical symptoms are expected from ill people or possibly because of stigma toward the mentally ill.[11]

Many scales related to stress and behavioral health have been developed and validated in diverse cultures. Three scales of interest are the Depression, Anxiety, and Stress Scale (DASS),[12] Stress Overload Scale (SOS),[13] and Perceived Stress Scale (PSS).[14] All three have been used with samples from developing countries (DASS,[15] SOS,[16] and PSS[17]) with focus on specific aspects of stress and overlook aspects of stress manifestation in Indian populations. Likewise, the previous version of Stressometer^® (SOM Version 1)[18,19,20] assessed only components of stressed and missed on diagnostic capacity, thus indicating toward a new and revised version with Likert scale. Therefore, the current study attempted to validate a new and more comprehensive scale of stress and behavioral health for use in India.

MATERIALS AND METHODS

Participants

There were 100 participants: 50 patients from a private clinic in New Delhi and 50 were individuals who had not sought treatment. Patients were recruited directly during the clinic visits. The nonpatients were recruited through E-mails, posts on the private clinic Facebook page and website and were not close friends or relatives of the patients.

Inclusion criteria

• All participants must be above 18 years of age

• Participants for the patient group must suffer from a diagnosable mental health condition as per the Diagnostic and Statistical Manual of Mental Disorders, 5^th Edition/International Classification of Diseases, 10^th Revision, and nonpatient group participants should not be experiencing severe physical or mental health conditions as assessed on a brief clinical interview.

Questionnaire

The Stressometer^® (SOM) is a self-administered assessment tool developed by the first author for mass screening to evaluate stress level of individuals and identify the sources of stressors (See attached Appendix 1). The scale has been reviewed in the past by experts in the field of psychiatry and psychology. Previous versions (SOM version 1) have been used to measure stress among nurses,[18] paramilitary forces,[19] and medical students and professionals.[20] The previous version consisted of 50 questions divided over five subscales with each comprising ten questions. Theses subscales related to nature (e.g., irritability), circumstances (e.g., recent job change or marriage), body and mind (symptoms of stress, like, anxiety, or disturbed sleep), home life (e.g., lack of family support), and work life (e.g., unsupportive colleagues). The scale allows participants to choose between “Yes,” “No,” “Can’t Say,” or “Not Applicable” for the nature and circumstances subscales and between “Never,” “Sometimes,” “Often,” “Always,” “Can’t Say,” or “Not Applicable” for body and mind, home life, and worklife subscales.

Inspired by questions in Thorson's Principles of Stress Management by Peiffer,[21] the Stressometer^® (SOM) can be administered online or through software installed on computers. It consists of basic demographic questions as well as 55 questions addressing recent possible stressors (Circumstances, Domestic Life, and Professional Life), the participant's nature (Human Nature), and the presence of potential stress symptoms (Clinical Symptoms). All questions use a five-point Likert scale with the additional options of “Can’t Say” and “Not applicable.” Four questions identify major symptoms of common disorders and will be used as determinants of need for immediate referral for intervention.

For convergent validation purposes, three currently validated scales were used. The first is the PSS, which consists of 14 questions focused on stress experienced in the last month.[14] The second is the DASS, which consists of 42 questions measuring the negative emotional states of depression, anxiety, and stress in the past week.[12] The third is the SOS, which consists of 30 items examining to what degree a person is experiencing stress overload that may affect their well-being.[13] The SOS is split into two subscales: personal vulnerability and event load.

Procedure

Data were collected from March to April and July 2017. Participants were informed of the details of the study and written informed consent was obtained from them. Participants were asked to fill out all four scales in a fixed order of demographics, SOM, PSS, SPS, and DASS. All scales were self-administered on provided computers or paper as per the convenience of the participants. After scale administration, to test criterion validity, each participant underwent a brief clinical interview with psychiatrist and/or psychologist, who were blinded as to the results of the tests administered. The clinical interview provided each participant with a rating addressing their need for intervention. Researchers were available for support if needed.

Ethical considerations

The Institutional Review Board approved the study protocol (reference number: A20/2017). Participants gave written informed consent to take part in the research.

Data analysis

For the four questions that addressed major symptoms of disorders (wanting to leave everything and go away, persistently sad and low, uncontrollable thoughts or compulsive actions, and hearing voices or feeling paranoia), patterns of responses were compared for patient and control groups and a Chi-squared was calculated. Subscale scores for Human Nature, Life Circumstances, Human Body and Mind, Home Life, and Work Life were computed. In addition, ratings for all questions were added to get a total score. Spearman correlations were used to analyze the relationships among the five subscales, total score, and validation scales. Reliability of the scale was established using the Cronbach's alpha test. In addition, a logistic regression analysis was performed to use for prediction of future individuals who need referral for possible diagnosis and treatment.

RESULTS

Of the 100 participants, 48 (20 patients and 28 nonpatients) were female and 52 (30 patients and 22 nonpatients) were male. The participants ranged in age from 18 to 74, with a mean age of 35.0.

The Stressometer^® (SOM) scale was found to be highly reliable (55 items; α = 0.935). Patients were more likely to agree that they wanted to leave everything and go away (χ² (4) = 10.93, P = 0.027), were persistently sad and low (χ² (4) = 18.84, P = 0.001), had uncontrollable thoughts or compulsive actions (χ² (4) = 24.53, P < 0.001), or that they heard voices or were paranoid (χ² (4) = 15.04, P = 0.005). Patients more frequently did not report jobs in the demographics and 25 patients selected “Can’t Say” on the Professional Life Subscale. Therefore, that subscale was excluded from the correlation analysis. There were significant correlations between clinical symptoms and patient status (rs = 0.375, P < 0.001), Dass-total (rs = 0.721, P < 0.001), PSS total (rs = 0.503, P < 0.001), SOS personal vulnerability (rs = 0.584, P < 0.001), and SOS event load (rs = 0.385, P < 0.001). There are also significant correlations between the validation scales used [Table 1].

Table 1.

Correlations between the Stressometer^® subscales, patient status, and validation scales

Subscales and validation scales	Correlation coefficient	Subscales and validation scales
		Nature	Circumstances	Clinical symptoms	Domestic life	SOM total	PSS total	SOS personal vulnerability	SOS event load scale	DASS total
Nature	0.247 (0.013)	-
Circumstances	−0.252 (0.011)	0.181 (0.071)	-
Clinical symptoms	0.375 (<0.001)	0.617 (<0.001)	0.102 (0.314)	-
Domestic life	0.347 (<0.001)	0.337 (0.001)	0.057 (0.571)	0.467 (<0.001)	-
SOM total	0.067 (0.509)	0.641 (<0.001)	0.665 (<0.001)	0.656 (<0.001)	0.473 (<0.001)	-
PSS total	0.020 (0.843)	0.459 (<0.001)	0.093 (0.359)	0.503 (<0.001)	0.348 (<0.001)	0.426 (<0.001)	-
SOS personal vulnerability	0.024 (0.813)	0.593 (<0.001)	0.126 (0.213)	0.584 (<0.001)	0.350 (<0.001)	0.535 (<0.001)	0.690 (<0.001)	-
SOS event load scale	−0.192 (0.055)	0.445 (<0.001)	0.234 (0.019)	0.385 (<0.001)	0.264 (0.008)	0.484 (<0.001)	0.501 (<0.001)	0.645 (<0.001)	-
DASS total	0.213 (0.033)	0.579 (<0.001)	0.060 (0.554)	0.721 (<0.001)	0.440 (<0.001)	0.571 (<0.001)	0.571 (<0.001)	0.731 (<0.001)	0.577 (<0.001)	-

Significant correlations are bolded. SOM – Stressometer^®; PSS – Perceived Stress Scale; SOS – Stress Overload Scale; DASS – Depression Anxiety Stress Scale

There was a significant logistic regression using the subscales as administered. A logistic regression was calculated to predict if an individual needed intervention based on status as patient or nonpatient [Table 2]. The significant predictors were clinical symptoms subscale, SOS event load subscale, and the reduced domestic life subscale. That combination was also highly reliable (33 items; α = 0.899). A test of the full model against a constant only model was statistically significant, indicating that the predictors as a set reliably distinguished participant group membership (χ²(3) = 41.02, P < 0.001). Nagelkerke's R² of 0.449 indicates a moderate relationship between prediction and group. Prediction success overall was 76% (72% nonpatients and 80% patients). The Wald criterion indicates that all predictors were significant.

Table 2.

Logistic regression equation to predict group membership as patient or nonpatient

Variable	B	SE	Wald	Significant
SOS event load scale	−0.137	0.036	14.773	<0.001
Circumstances	−0.108	0.04	7.26	0.007
Clinical symptoms	0.146	0.038	14.762	<0.001
Domestic life	0.17	0.058	8.699	0.003
Constant	−2.351	1.239	3.598	0.058

SOS – Stress Overload Scale; SE – Standard error

DISCUSSION

The Stressometer^® (SOM) was identified to be a valid and highly reliable measure of stress. The scale was correlated with other measures of stress (PSS, SOS, and DASS) used in the study, thus suggesting concurrent validity. In other words, stressometer is identified to be a tool with the capacity to gauge stress. The unique subscales of stressometer correlated significantly well with the other scales of stress (PSS, SOS, and DASS), thus indicating the origin of stress from various dimensions of an individual's life. In other words, the tool is comparable with existing global stress scales, besides additionally analyzing the source of stress suggesting that as a whole it can function as a self-assessment of stress and its causes. Based on it being self-administered, if used across the world, can bridge the treatment gap and efficaciously identify those requiring treatment.

In addition, the scale is likely to be culturally neutral with universal applicability, thus making it ideal for use in the developing world to bridge the treatment gap created by an overtaxed health care system and enhance behavioral health. Simultaneously, it helps in cutting down the stigma of mental health by being a self-rated computer-based questionnaire with the potential to be used by the user at his/her comfort zone.

Limitations

This study has limitation of being mostly based on self-report data, thus can be subject to response bias. The study also had a small sample size with 25 participants selecting “Can’t Say” on the professional life subscale resulting in exclusion of the same from the analysis. Future studies can include a full detailed clinical or physical analysis of the participants. Furthermore, the future study can also take into account the professional aspect of the participants so as to understand the level of stress or causation from the same.

Financial support and sponsorship

Nil.

Conflicts of interest

There are no conflicts of interest.

Appendix

Appendix 1.

Sample of questions in Stressometer^® (SOM)

Subscale	Sample Questions
Nature	It is in my nature that I feel people take advantage of me It is in my nature that I give too much importance to what others think of me It is in my nature that I put others need first and then think about myself It is in my nature to be very particular about timings, cleanliness, and orderliness It is in my nature that I can’t express my feelings or get work done according to my needs
Circumstances	My type of job or work or assignments changed which is leading to increased stress levels My responsibility at work has increased which is leading to increasing stress levels I have left job, or retired from job, or have been without a job due to some reason which is leading to increased stress levels An increase in my responsibilities due to my marriage has increased my stress levels An increase in my responsibilities due to of my child has increased my stress levels
Clinical symptoms	I feel that I have become more emotional or irritable I have started getting worrying thoughts or feel anxious My heart beats very fast even though I am resting I have had frequent uncontrollable thoughts or felt forced to perform certain repetitive actions which despite my best efforts cannot be stopped (e.g., repetitive handwashing or checking locks) I have been experiencing few phenomena which have never happened before like listening to some voices which no one else around me listens to or feeling people are plotting against me or experiencing some more weird things
Domestic life	I feel my family and/or partner do not give me sufficient emotional/moral support I find lack of appreciation from my family and/or partner One of my close family member or friend or partner is stressed I have a face regular nuisance of some kind in my neighborhood I feel insecure about my life at home or outside
Professional life	I find atmosphere around my area of work stressful I find my colleagues/peers unsupportive I normally work much more than my colleagues/peers, boss, or customer/client I have faced sexual harassment I feel insecure about my current job