Reliability and Validity of the Adolescence Stress Scale (ADOSS) for Indian Adolescents

Abstract

Background:

Stress levels increase tremendously in adolescence. Indian adolescents derive much of their stress from school and family pressures. Developing a standardized tool to assess adolescent stress can help early diagnosis. This article aimed to assess the reliability and validity of the ADOlescence Stress Scale (ADOSS).

Methods:

ADOSS was administrated by trained interviewers to 100 adolescents (10–17 years) of both sexes in Chennai, South India. Salivary cortisol was used as an objective measure of stress in 40 adolescents. Reliability was assessed within a two weeks interval. Exploratory factor analysis was done for the baseline ADOSS scores. The internal consistency (Cronbach’s alpha and spilt-half correlation), test-retest agreement (Kappa value), and inter-rater reliability (ICC) were assessed. Pearson correlation between salivary cortisol and ADOSS score was done.

Results:

The inter-factor correlation between the inherent four ADOSS factors (personal, academic, family, and social) was between 0.73 and 0.87, and split-half correlation was between 0.70 and 0.83, showing that the factors are reliable and coherent. Test-retest agreement was Kappa: 0.63, P = 0.001, and ICC was 0.84. Pearson correlation showed a statistically significant positive correlation between cortisol levels and ADOSS scores (r = 0.87, P = 0.001).

Conclusion:

ADOSS is a reliable and valid tool to assess the overall stress of Indian adolescents in the age group of 10–17 years.

Key messages:

ADOSS can be used as a clinical and research tool to capture all major domains (personal, academic, family, and social) of stress in adolescence. This tool based on sound methodology provides a short, replicable model to be tested among other ethnic groups.

Stress can be defined as a person’s lack of self-efficacy or personal competence to control or adapt to a particular situation. ¹ In general, stress is not caused by a particular factor or stressor but is a compilation of many stressors leading to issues in adaptation. ² India is home to more than 250 million youths aged 10–19 years, constituting 17% of the worldwide adolescent population. ³ Adolescents go through several common stressors, including physical and sexual changes linked with puberty, meeting demands of school/college life, issues with beginning and maintaining friendships and romantic relationships, making career choices, dealing with certain responsibilities, and obtaining gradual independence from their families. ⁴

National Mental Health Survey of 2016 estimated that 13.5% of adolescents residing in metropolitan areas suffer from mental health disorders, which is double the prevalence compared to rural areas in India. ⁵ Studies from India, using standardized and validated scales for Indian adolescents such as the Adolescent Stress Assessment scale (25 items) and Depression, Anxiety and Stress Scale—21 (DASS-21), have reported that 10.1% ⁶ and 51.1% ⁷ of Indian adolescents were under moderate to severe stress. A few studies have used adult stress scales or scales not validated on Indian adolescents and reported moderate educational stress of 39.2% ⁸ and only <12% of adolescents as having low stress. ⁹ The Adolescent Stress Questionnaire (ASQ) ¹⁰ is used globally to understand a broad span of adolescent stressors. The Academic Stress Scale (ASS) ¹¹ is a widely used 40-item self-reported questionnaire to capture academic stress in adolescents. The only study that slightly modified the original ASQ to suit the Indian adolescents was by D’souza et al, ¹² wherein their tool collapsed the original 58-item ASQ to 38 items, leaving out a few adolescent stressors pertinent in current times, such as questions on relationship issues.

While other scales are available to capture adolescent stress, gaps exist in length, recall ability, reliability, validity, and final adaptation of these tools to our cultural setting from an Indian adolescent perspective. The ADOlescent Stress Scale (ADOSS) is novel and subjective. It is short (with 20 questions) and includes stressors that Indian adolescents encounter daily and major issues that are “bothersome” to them. Additionally, the assessment of salivary cortisol is a valid and reliable reflection of the unbound hormone level in blood.^{13, 14} Studies examining the association between salivary cortisol and stress scale scores among adolescents in India are lacking. This article aimed to evaluate the reliability and validity of ADOSS and also validate it against a physiological and objective stress indicator (salivary cortisol), which has not been carried out by other studies.

Methods

Participants

This pilot study to validate ADOSS was part of a larger study that aimed to assess the impact of yoga on stress, metabolic parameters, and cognition (attention and concentration) of Indian adolescents. The study was approved by Madras Diabetes Research Foundation Institutional Ethics Committee (REF/2017/07/014804) and registered with Clinical Trials Registry, India (CTRI/2017/08/009203). Ten schools were randomly approached in Chennai, South India, for this validation study, and two private schools agreed to participate. Once the schools agreed to the study, written informed consent and assent were obtained from parents and children. Convenience sampling was used to recruit an equal percentage of both sexes in the age group of 10 to 17 years between 8th and 10th grades in these schools. A total of 108 adolescents’ data at two points were collected for the validation study. Participants with missing ADOSS data (n = 8) were excluded. For assessing validity, the baseline data were collected during January 2020 by trained research staff, and repeated data was collected after 2–3 weeks.

Instrument

ADOSS was developed by the psychologist in our team (LS), and the items used in the questionnaire were pooled from the original versions of the ASQ ¹⁰ and ASS, ¹¹ with a few new items (which were specific to Indian adolescents) added (Table 1). As the first step of scale development, a three-member expert team consisting of a psychologist, physician, and public health scientist came together for a group discussion with the objective of developing a short inventory to assess major concerns of Indian adolescents, especially the academic burden that the current generation of adolescents faces. Items from ASQ and ASS that were chosen for inclusion were done keeping in light the personal experience of researchers working with Indian adolescents who often report specific areas of concern. The item pooling was further carried out by the expert team keeping in mind the criteria to identify “bothersome” or stressful issues from an adolescent’s perspective. The concept of stressful experience was identified in terms of a recurring feeling of disturbance due to particular thoughts or situations. Accordingly, culturally relevant new items were also developed. Item pooling was then carried out from the three sources—ASQ, ASS, and personal experiences of the expert team working with adolescents. Culturally appropriate items from the ASQ and ASS were first selected, and then the new items were added. Items were written in simple English to phrase various stressors or problems an adolescent may face across various settings. Pooled items were rated by the expert team for their clarity, representativeness, and relevance. Next, the finalized scale was given to 10 teachers and 10 parents from a local school for review. Based on their comments, sentences were reframed and edited to make them short, clear, and crisp. Lastly, the scale was administrated to 20 adolescents, and after obtaining 100% concurrence, 20 items were retained with four response anchors for administration. Concurrence in ranking and the response from adolescents indicated good agreement; thus, all items were retained for further psychometric validity without rejecting any items.

Table 1.

ADOlescence Stress Scale (ADOSS)

S.No.	Over the Last 2 Weeks, How Often Have You Been Bothered by Any of the Following Problems?	Source
A	I experience pressure from my parents over rules and expectation	ASQ
B	I have problems in meeting academic and extracurricular demands	ASS
C	I am able to achieve what I need	ASS
D	I feel my friends are always teasing/bullying me	ASQ
E	I am unable to concentrate on studies	New
F	I am least scared about my performance in exams	New
G	I have problems of speaking to persons in authority	ASS
H	I experience lots of conflicts with my parents	ASQ
I	I tend to overeat or eat very less	New
J	I worry about my physical /body image	ASQ
K	I feel inadequate compared to my own age group boys and girls	ASQ
L	I aspire to be someone and focus to achieve it	New
M	I have difficulties with peer group/influenced by the peers	ASQ
N	I am often bothered by negative/sexual thoughts	New
O	I have difficulty in communication with the opposite (boys/girls)	New
P	I am involved in a love relationship which is causing emotional strain	New
Q	I feel loss of self confidence	ASS
R	I am disturbed by separation/quarrelling of my parents	ASQ
S	I am worried about my prolonged illness	New
T	I have pressure due to financial problem at home	ASQ

ASQ: Adolescent Stress Questionnaire, ASS: Academic Stress Scale, New: Newly Developed by a team of psychologist, doctor and public health scientist.

The ADOSS asks respondents about their thoughts and feelings over the last two weeks. Each item is rated on a 4-point scale ranging from 0 for “never” to 3 for “nearly every day.” ADOSS scores are calculated by summing up the scores obtained on all scale items except two positively worded items (items 3 and12) that need reverse scoring (e.g.: 3 = 0, 2 = 1, 1 = 2, and 0 = 3). Thus, the total score is 60, with higher scores indicating high stress (Supplementary table). ADOSS captures adolescents’ stress across a wide age range (10–17 years). The way the questions are comprehended by the lower (10–13 years) and higher (14-17 years) age groups could vary. Hence, in this study, it was administered by trained research staff to ensure uniformity in administration and scale completion.

Salivary Cortisol

Salivary cortisol is a measure of active free cortisol. This was assessed in 40 adolescents from a subset of 100, during baseline data collection. A single saliva sample was collected before the short mid-morning break at 10 am. All instructions regarding the sample collection were given to the participants a day prior. Participants were made to gargle with water 10–15 minutes before the sample collection. They were instructed to gently chew a cotton roll, to stimulate the salivary flow rate. It normally takes less than a minute to collect a saliva sample of 0.5–1 ml using this method. The test was performed after ruling out factors that can affect the accuracy of the test, like the use of tobacco, licorice, steroid-containing lotions, or oral gels; gingivitis; oral sores/injury, or stress immediately before sample collection. Cortisol levels were estimated by the radioimmunoassay (RIA) method. ¹⁵ Samples were processed at our in-house central laboratory, certified nationally by the National Accreditation Board for Testing and Calibration of Laboratories and internationally by the College of American Pathologists.

Statistical Analyses

Statistical analysis was carried out using the IBM Statistical Package for the Social Sciences (SPSS version 24).

Exploratory factor analysis was carried out to understand the significant underlying factors. This was done to establish the fundamental factorial validity of the questionnaire and thereby identify the main factors under which the ADOSS items were being grouped. Each extracted factor showed the number of items differently loaded on that factor. The amount of variance for each factor was measured by the eigen value. To construct the most reliable factors, varimax rotation factor analysis was done using the baseline data. The item loading cut-off of 0.4 was taken to arrive at a significant loading of the item on a specific factor. Cronbach alpha coefficient of 0.6 indicates the reliability and internal consistency of the items of a scale. ¹⁶

Internal consistency was evaluated using Cronbach’s alpha coefficients and Spearman’s split-half correlation, where 20 items were split into two sets such as odd- and even-numbered items, a score was calculated for each set of items, and the relationship between the scores obtained for the two sets was analyzed for the general agreement between the multiple items of ADOSS. Cronbach’s alpha values of <0.5 were considered as unacceptable, 0.5 as poor, 0.6 as questionable, 0.7 as acceptable, 0.8 as good, and ≥0.9 as excellent consistency. ¹⁷ The values of split-half correlation were classified as: < 0.39 as weak, 0.40–0.69 as moderate, and 0.70–1.00 as strong correlation. ¹⁸

The percentage of agreement between data collected at the two time points was examined by calculating Cohen’s Kappa coefficient. The values of Kappa were classified as follows: <0.4 as poor, 0.41–0.75 as good, and >0.76 as excellent agreement. ¹⁹

To assess inter-rater reliability, data collected by the two raters was examined by calculating Intraclass Correlation Coefficient (ICC). Values of >0.9 were considered as excellent, 0.75–0.9 as good, 0.40–0.50 as fair, and <0.50 as poor inter-rater agreement. ²⁰

Results

Participants’ Baseline Data

We recruited 50 boys and 50 girls for this reliability and validity study. The mean (±SD) age of boys and girls was 13.5 (±0.5) and 13.3 (±0.8) years, respectively.

Reliability and Validity

The scree plot showed that a four-factor solution emerged with an eigenvalue of 1, indicating the purity of the factors. (Figure 1). The initial eigen values of the four factors were 4.97, 1.54, 1.25, and 1.05, respectively. From factor 5 onwards, the eigen value gradually decreased to <1.

Figure 1. Scree Plot from the Exploratory Factor Analysis on ADOSS.

ADOSS: ADOlescence Stress Scale.

Exploratory factor analysis of ADOSS showed that data and correlation matrix were appropriate for factor analysis (KMO Index = 0.61; Bartlett’s test of sphericity was significant [X2 (780, N =100) = 1,407.08, P < 0.001]). Most of the ADOSS variables, as shown in Table 2, had reasonable factor loadings between factors 1–4 and cumulatively accounted for 63.4% of the total variance. Items C, I, J, L, N, O, P, Q, and S are related to the stress due to personal life, so factor 1 was labeled as “Personal stress.” Items B, E, F, and G are related to stress from education. Hence, factor 2 was labeled as “Academic stress.” Four items (A, H, R, and T) are related to stress from family, so factor 3 was labeled as “Family stress.” Only three items (D, K, and M) fall on factor 3 related to stress from society; hence, factor 4 was labeled as “Social stress.” The major and unique contribution to the variance was seen in factors 1 (9 items), 2 (4 items), 3 (4 items), and 4 (3 items).

Table 2.

Varimax Rotated Factor Analysis of ADOSS

Q.No	Description	Loadings	Factor
C	Ability to achieve	0.54	Factor 1 (Personal stress)
I	Overeat or eat less	0.60
J	Worry about body image	0.58
L	Focus to achieve goal	0.60
N	Negative thoughts	0.51
O	Problem in communication with opposite gender	0.58
P	Love and relationship	0.51
Q	Loss of self confidence	0.55
S	Prolonged illness	0.56
B	Problem in academic	0.59	Factor 2 (Academic stress)
E	Unable to conc. in studies	0.64
F	Least scared about exams	0.42
G	Problems to speak to authority	0.47
A	Over rules & expectation	0.52	Factor 3 (Family stress)
H	Conflicts with parents	0.53
R	Disturbed by parents’ separation	0.67
T	Financial problem in home	0.61
D	Teasing & bullying	0.49	Factor 4 Social stress
K	Compared to own age group	0.42
M	Peer influence	0.44

ADOSS: ADOlescence Stress Scale.

Table 3 shows the item analysis of the four main ADOSS factors. The mean value of all line items for personal, academic, family, and social factors of ADOSS was 6.3±0.8, 3.8±0.3, 2.9±0.2, and 2.5±0.2, respectively. The Cronbach’s alpha coefficients for the four main factors were between 0.73 and 0.87, while for the entire questionnaire, it was 0.84. Similarly, the Spearman–Brown split-half test correlation was between 0.70 and 0.83. The values of Cronbach’s alpha coefficients and split-half test correlations were found to be highly sufficient to prove the reliability of ADOSS. All items appeared worthy enough to retain, and deleting items from the scale lowered the Cronbach’s alpha. Test-retest agreement was measured for the total scores of ADOSS by Cohen’s Kappa, which showed a fair agreement (0.63) between data collected at the two time points. This indicated a moderate level of stability of the scores across time. Inter-rater reliability was evaluated using ICC, and a value of 0.84 was obtained. This score indicates a good level of concurrence in the scale administration across raters.

Table 3.

Item Analysis of the Four Main ADOSS Factors to Check Internal Consistency

Factor	Domain	Cronbach’s Alpha Coefficients	Spearman Brown Split-Half Test Correlations	Mean Value (SD) of Line Items Under Each Factor
1	Personal stress	0.87	0.83	6.3 (0.8)
2	Academic stress	0.84	0.80	3.8 (0.3)
3	Family stress	0.73	0.75	2.9 (0.2)
4	Social stress	0.79	0.70	2.5 (0.2)

ADOSS: ADOlescence Stress Scale.

Table 4 describes the mean and rank orders of the 20 items of ADOSS. The mean ranged from 1.71 (Question “L”) to 0.40 (Question “M”). It is evident from the table that the first two ranks of ADOSS fall on two positive questions related to the future and that the next three ranks related to education.

Table 4.

Stress Rank Based on ADOSS Items Mean Score

Q. No	Stressors	%	Mean ± SD	Stress Rank
L	Focus to achieve goal	81.0	1.71 ± 0.1	1
C	Able to achieve	80.0	1.55 ± 0.1	2
Q	Loss of self -confidence	78.2	1.40 ± 0.1	3
E	Unable to conc. in studies	74.5	1.24 ± 0.3	4
B	Problem in academic	74.0	1.11 ± 0.1	5
F	Least scared about exams	70.0	1.07 ± 0.1	6
A	Over rules & expectation	69.8	0.98 ± 0.1	7
K	Compared to own age group	69.8	0.94 ± 0.1	8
T	Financial problem in home	68.9	0.90 ± 0.1	9
D	Teasing & bullying	67.1	0.88 ± 0.1	10
H	Conflicts with parents	67.0	0.84 ± 0.09	11
I	Overeat or eat less	67.0	0.8 3 ± 0.09	12
J	Worry about body image	66.4	0.81 ± 0.09	13
G	Problems to speak to authority	64.9	0.75 ± 0.09	14
S	Prolonged illness	64.9	0.74 ± 0.09	15
O	Problem in communication with opposite gender	45.2	0.5 9 ± 0.09	16
P	Love and relationship	44.8	0.50 ± 0.09	17
N	Negative thoughts	44.7	0.47 ± 0.07	18
R	Disturbed by parents’ separation	44.0	0.42 ± 0.07	19
M	Peer influence	43.8	0.40 ± 0.07	20

ADOSS: ADOlescence Stress Scale.

A scatter plot was used to show the association between the ADOSS score and salivary cortisol (Figure 2). ADOSS score was positively correlated with salivary cortisol (r ² = 0.75, P = 0.002). The Pearson correlation further showed a statistically significant linear relationship between the ADOSS score and cortisol level (r = 0.86, P = 0.001).

Figure 2. Scatter Plot Showing the Correlation Between ADOSS Score and Salivary Cortisol.

ADOSS: ADOlescence Stress Scale.

Discussion

This study investigated the reliability and validity of the ADOSS for assessing stress in Indian adolescents. Both the test–retest and the reliability analyses indicated fair agreement, and the validity of ADOSS showed a relatively high internal consistency. To our knowledge, this is the first stress scale developed in India for Indian adolescents that has been validated against an objective measure (salivary cortisol).

Factor analysis of ADOSS showed that the 20 items were highly loaded on four factors: personal, academic, family, and social. The Cronbach’s alpha coefficients for each of these domains were between 0.73 and 0.87. While comparing ADOSS to other stress scales, the exploratory factor analysis of the Adolescent Boys Stress Questionnaire (ABSQ-2015) showed that the 16 items of the scale were also grouped into three factors, namely academic, family, and personal and social stress ²¹ ; however, it was not cross-validated in any other population nor did the authors compare the scale with other clinical parameters. Similarly, the factor analysis of the Questionnaire on Academic Stress in Secondary Education showed that its 30 items fall under four main factors: academic overload, interaction with classmates, family pressure, and future-oriented perspective. ²² Stress is known to be high just before and during examination, compared to the initial point of the academic year. Hence, the time of data collection becomes very critical while capturing not just academic stress but other factors also, as it can have a cascading effect.

Personal stress had a majority of events grouped in factor 1. Some items related to personal stress included focus to achieve, worry about body image, prolonged illness, and problems in communication with the opposite gender. The academic stressors factor evaluates school events such as academic performance and pressure from studies in a manner that relates to the Educational Stress Scale in Adolescents. ²³ The third factor on family stress includes stressors related to family dynamics (e.g., fights between parents) and family worries in general. This result is consistent with the family domain of the Perceived Stress Scale ²⁴ and identifies the daily parent–child relationship ²⁵ and the problems with overall family functioning. ²⁶ The family financial situation was considered as affecting adolescents in this study. These findings indicate that the economic burden in the family is also crucial to adolescents and would potentially have a negative impact on psychological adjustment, such as depression or antisocial behavior. ²⁷ The fourth factor on social stress identifies stress from interpersonal relationships, ²⁵ which evaluates issues similar to changes in the social environment that make teenagers fearfully perceive “common” situations which weren’t a stress in the early stages of life, such as “teasing and bullying” or “peer influence.” ²⁵ The scree-plot suggests a four-factor configuration which was based on several statistical criteria and theoretical considerations. ²⁸

The internal consistency measured by Cronbach’s α for the four factors in ADOSS ranged from 0.73 to 0.87. The factor scale correlations were also generally higher across all the factors of the scale, with the highest reported in personal stress. The top five highest-ranking stressful life events in this study were: focus to achieve goal, able to achieve, loss of self-confidence, unable to concentrate in studies, and problems in academics. These stressful life events are basically academic and interpersonal in nature.

The four-factor structure found in the current sample was in line with a previous study on the use of the Depression, Anxiety, and Stress Scale-21 (DASS-21) among adolescents in Vietnam. ²⁹ In another study, there were six factors analyzed in the use of the Perceived Stressors Global Scale for Adolescents: critical events, daily hassles, social exhibition, family concerns, academic stressors, and social pressure. ³⁰ A similar finding was observed in a study conducted at Dhaka where the analysis revealed six factors. ³¹

According to the baseline cortisol levels analyzed against the ADOSS scores, higher salivary cortisol levels were found in those experiencing high rates of daily stressors, indicating a significant relationship between the two measures. In contrast, findings from the Healthy Lifestyle in Europe by Nutrition in Adolescence (HELENA) study did not report any consistent relationships, in adolescents, between wake-up salivary cortisol and perceived stress measured by a stress scale. ³²

Our study had some limitations. We used convenience sampling in a moderately sized sample. The ADOSS was tested only in adolescents of a major metropolis in southern India. Hence, cross-country appropriateness is yet to be evaluated. Details were collected by solely relying on students’ self-reports, and hence, some recall bias cannot be ignored. However, this tool based on sound methodology provides a short, replicable model for further expansion among other ethnic groups.

In conclusion, the ADOSS was found to be a simple, reliable, and valid instrument for assessing adolescent stress, which can be used for clinical, research, and training purposes as the questionnaire covers all the four main domains (personal, academic, family, and social) of stress in an adolescent’s life. Replication on a larger sample can help further enhance the tool’s psychometric property.