Test-retest reliability of the Seasonal Pattern Assessment Questionnaire in Old Order Amish

Abstract

Background

Research on test-retest reliability of the Season Pattern Assessment Questionnaire (SPAQ) is sparse, and to date, has not been done with subgroups such as the Old Order Amish.

Methods

We examined the test-retest reliability of the SPAQ in a sample of Old Order Amish. A total of 68 Old Order Amish participants completed the SPAQ twice, with 4 months between administrations. Quantitative data analyses were carried out to determine respective strengths of test-restest reliability for two variables [i.e., Global Seasonality Score (GSS), and Problem Rating Score (PRS)].

Results and conclusions

Results revealed the test-retest reliability of the SPAQ in this population to be strong within the respective variables (GSS, α= 0.87; and PRS, α= 0.79) using Cronbach’s alpha.

Introduction

Mood variations with changing seasons have been well established across cultures and ethnicities [1]. Often, these mood differences reach clinical significance (i.e., seasonal affective disorder) for which accurate assessment and treatment is necessary [2]. Seasonal affective disorder (SAD) is a syndrome that is characterized by annual and recurrent episodes of depression during a particular season, with a spontaneous resolution of symptoms when the season is over. When symptomatic months are within fall and winter, it is referred to as Winter SAD, and when those months are between spring and summer, it is referred to as Summer SAD. Subsyndromal SAD refers to the occurrence of similar depressive symptoms and seasonal trends, but to a milder degree, allowing for more normal functioning. The epidemiological method to estimate seasonal mood changes is the Seasonal Pattern Assessment Questionnaire (SPAQ; [3] ) which has shown adequate reliability and validity in estimating participants’ seasonal mood fluctuations [4–10]. As research grows in this specialty area, ethnically and culturally diverse populations are being studied in order to validate and fine tune our awareness of the prevalence, etiology, and course of SAD to better understand the interplay between its risk and resilience factors as well as endogenous and environmental factors. Germane to SAD assessment in sub-populations is the impact of socio-cultural factors. Research is presently exploring variables such as vulnerability and environmental triggers for SAD. For example, day length and higher latitude have been linked to higher winter SAD prevalence [11]. Agumadu et al. [12] and Guzman et al. [13] have used the SPAQ to examine seasonal mood changes in various ethnic groups (e.g., African-Americans and African students in Washington DC) and found a greater prevalence of winter SAD than summer SAD. In contrast, several populations (e.g., Lapps, Icelanders, Asiatic groups, and Chinese) have shown resiliency with regards to winter SAD [14–18].

Extensive scientific investigation has shown bright light therapy to be a safe and effective modality of treatment of SAD, viz.bright light therapy [19–27]. Given that all studies on the epidemiology of SAD have been performed in populations using electric light at home and work, it is possible that exposure to artificial light may affect prevalence of this condition. Thus, investigating SAD in the OOA is important because they represent a population that generally does not use bright electric lighting. To our knowledge, there are no studies of SAD in the OOA. Prior to analyzing responses of OOA on SPAQ, it is important to have a sense of the reliability of the instrument in this population taking into consideration their communication style. Unique to the OOA culture are norms related to humility and subsequent communicative content. These cultural variables are highly likely to have a significant impact on questionnaire responses. The OOA’s overarching, long-standing zeitgeist entails humility (e.g., Gelassenheit). As per Kraybill [28], this cultural tenet refers to a “yieldedness” and “deep unconditional spiritual surrender to God (p. 93)”. Gelassenheit pervades all aspects of their lives and is fleshed out as, “ A meek and mild personality that is willing to suffer rather than defend itself (p. 93)”. Further, Kraybill [28] asserts that the OOA’s belief in nonresistance (e.g., command to love one’s enemies) may predispose members to inhibit all anger and aggression. In addition to the effects explained above, the cultural practices of humility and nonresistance create quietness and brevity of communication. Therefore, when given questionnaires regarding mental health, in which the participant is asked to disclose personal struggles, there is a good possibility that OOA can consciously or unconsciously minimize or completely avoid report of emotional pain and not make complaints based on all of the above cultural dynamics.

The current study examines the adequacy of test-retest reliability of SPAQ in the Old Order Amish sub-population. In line with Rohan and Sigmon [5], we hypothesize that there is a strong test-retest reliability (e.g., α>0.7) between a participant’s initial and follow-up (e.g., 4 months later) Global Seasonality Score (GSS). Similarly, we hypothesize that there is a strong test-reliability (e.g., α>0.7) between an Old Order Amish participant’s initial and follow-up Problem Rating Scores (PRS).

Methods

Our survey was conducted on a group of OOA adults residing in Lan-caster County, Pennsylvania. The study was approved by the Institutional Review Board of University of Maryland-School of Medicine. The participants were those who were enrolled in several other metabolic and cardiovascular genetic studies, signed the informed consent form, and specifically agreed to be recontacted for other studies. We mailed out the SPAQ along with an explanation that we were collecting data on the relationship between season and mood. Standard instructions involved completion of the questionnaire and mailing it back to us. The participants were sent a 1 dollar bill each as a token of appreciation for participating in the study.

Non-Amish individuals, Amish individuals not enrolled in the above mentioned studies, and Amish individuals under the age of 18 years were excluded from the study. Due to logistic reasons certain participants received the SPAQ twice with a 4-month gap in between mailings and return of completed measures. This resulted in some OOA respondents completing two SPAQs. The second questionnaire was excluded from the parent study but provided the opportunity for the test-retest analysis of the current report. GSS is one of two composite scores derived from the SPAQ and measures the degree to which sleep, weight, mood, social activity, appetite, and energy are affected by season. The GSS was calculated according to Kasper et al. [4]. The problem rating score (PRS) was calculated based on the extent to which participants indicated that seasonal changes were a problem for them on a scale of zero (no problem) to five (disabling problem).

We compared the GSS of each of the 68 Old Order Amish adults who completed the SPAQ in May 2011 with their GSS from the SPAQ completed in September 2011. In similar fashion, we compared their PRS between the two SPAQ administrations. Calculations for SPAQ test-retest reliability were conducted using Cronbach’s alpha for the respective variables of GSS and PRS.

Results

The characteristics of the sample included 38 women and 30 men. The average age was 55.97 years with a standard deviation of 15.04. As hypothesized, test-retest reliability was strong (α=0.87, p< 0.001) between Old Order Amish participants’ initial and follow-up (e.g., 4 months later) GSS on the SPAQ using Cronbach’s alpha. Likewise, test-retest reliability was strong (α=0.79, p<0.001) between Old Order Amish participants’ initial and follow-up (e.g., 4 months later) PRS on the SPAQ as measured by Cronbach’s alpha.

Discussion

Test-retest reliability of SPAQ in the OOA is analogous to the work of Rohan and Sigmon [5], revealing high degrees of test-retest reliability with the SPAQ as regards GSS and PRS. Reliability strength was found despite the fact that initial and follow-up SPAQ completion were done in different seasons (e.g., winter and spring).

As explained above, the OOA culture has a certain response style to questionnaires and psychological testing based on beliefs and core values. A lack of disclosure regarding personal struggles is often seen. For example, OOA could rank order months of the year according to months in which they felt “best” yet frequently leave blank a similar question in which participants were asked to rank order months according to when they felt “worst ”. However, this pattern was not likely to affect inter-rater reliability, but rather validity, since this cultural influence was likely to occur in a uniform manner (e.g., all participants were Amish) not affecting SPAQ test-retest reliability.

In our study on Seasonality of Mood in the OOA [29], we found the prevalence of winter SAD in the OOA using the SPAQ was 0.84%. Taking into account subsyndromal winter SAD with winter SAD (total winter SAD), the prevalence was found to be 2.59%. The possible impact of Amish-specific cultural factors was considered one of the possible factors which might account for the low prevalence of winter SAD in the OOA, which was found to be the lowest in all SPAQ-based studies which have been conducted in predominantly Caucasian populations. We considered participants who met criteria for winter SAD or winter s-SAD who had indicated when they felt “best” but had left blank the months in which they felt “worst,” thereby excluding them from a SPAQ-based diagnosis. Taking the mirror-image of the months in which they felt best to indicate the months in which they possibly felt worst (for example, if a participant felt best in August, we assumed he/she felt worst in February), we identified another three possible cases of winter SAD and another thirteen possible cases of total winter SAD. Although this resulted in an increase in prevalence of winter SAD to 1.1% and an increase in prevalence of total winter SAD to 3.6%, the prevalence rates still remained lower than in most other studies.

The study had the following limitations: 1) it employed a relatively small sample (e.g., 68 participants), 2) it only measured test-retest reliability and not validity, and 3) it used a sample of convenience. A more definitive work is required to relate the SPAQ to a structured clinical interview, such as the Structured Clinical Interview for DSM Disorders-Research Version (SCID-RV). In conclusion, the good test-retest reliability of the SPAQ-derived GSS supports its use in the Amish.