Abstract
Objectives
The Mental Health Index (MHI) is widely used as a measure of mental health status, but has not been evaluated in the geriatric oncology population. This study evaluated the MHI-17 in a geriatric oncology population, to establish validity and scoring rules.
Materials and Methods
The Carolina Senior Registry (NCT01137825) was used to obtain data for 686 patients with cancer 65 and older who completed the MHI-17. The 17-item patient-reported measure produces one total score summing across four domains: anxiety, depression, positive affect, and sense of belonging. Cronbach’s alpha (α), confirmatory factor analyses (CFA), item-response theory (IRT) analyses, and differential item functioning (DIF) analyses were used to evaluate internal consistency and validity.
Results and Discussion
The revised MHI retained the 13 best-fitting items from the MHI-17 and resulted in a final model that included two subscales: anxiety (four items, RMSEA 0.11; CFI 0.99; TLI 0.98) and depression (9 items, RMSEA 0.10; CFI 0.96; TL 0.95). IRT analyses of the four anxiety items indicated good fit (RMSEA 0.08) and precise measurement of adults with poor mental health, and the nine depression items also fit well (RMSEA 0.05). No meaningful differences were found by sex, education, or treatment stage. Scores were developed to provide meaningful norms. The new MHI-13 is a shorter, more accurate way to assess mental health in older adults with cancer and most importantly allows clinicians to separately identify anxiety and/or depression - a clinically important distinction as treatment differs among these two types of mental health impairment.
Keywords: Mental health, geriatric oncology, reliability, validity, psychometric validation, geriatric assessment
Introduction
The number of older adults with significant mental health needs is expected to rise to over 15 million by 2030.1 Older adults with poor mental health are significantly more likely to have functional status decline, are at increased risk for sustaining a fall, and report poorer overall quality of life.2 Further, adults with cancer are at higher risk for depressive symptoms than those without cancer,3 and survivors who report higher levels of depressive symptoms also have higher levels of somatic symptom burden, poorer overall quality of life, and higher perceived disability.4 Older women with breast cancer who report depressive symptoms are less likely to receive curative treatment and have lower survival rates than those without mental health needs.5 Depressive and anxiety symptoms are treatable, yet older adults with cancer are significantly less likely to receive a referral to mental health professionals.6
Screening for mental health issues in oncology has mostly been centered on distress screening, but even with increased distress screening occurring, referral to supportive care services remains poor.6 There are few mental health screens validated for older adults with cancer in particular, and those that are, lack specific scoring rules. The Mental Health Index (MHI) has been used in the geriatric assessment for almost a decade, 7 yet its psychometric properties for older adults with cancer have not been evaluated. In fact, the MHI-17 is incorporated in the most commonly utilized geriatric assessment that has been adopted by the Cancer and Aging Research Group (CARG) and has been incorporated in multiple studies/sites, including several cooperative group trials within the Alliance. Although this instrument is commonly employed, the scoring has never been elucidated and remains unclear.8,9 Moreover, the scoring of the MHI-17 does not separate anxiety from depression, a major distinction affecting treatment. Although anxiety and depression can co-occur and are often related, for initial screening purposes, for treatment, and future research, it is important to distinctly identify them.10 This study aimed to examine the validity and reliability of the MHI-17, explore the dimensionality, and provide meaningful scoring rules in older adults with cancer.
Materials and methods
Participants
This study is a retrospective secondary data analysis derived from the Carolina Seniors Registry (CSR) (NCT01137825). The CSR was developed in 2009 as a large observational cancer registry to collect geriatric assessment (GA) data on older adults (65 and older) with cancer. For this specific study, we included all patients that completed the GA and MHI-17 between January 2009 and October 2014. For a more comprehensive description of the CSR including the recruiting procedures, sampling methods, and assessments performed, please see Williams et al.9 This study was approved by the Institutional Review Board of the University of North Carolina (IRB #15-1524).
Measure
MHI-17 was developed as a measure of psychological distress and wellbeing.11 It has also been referred to as the MHI-II (MHI-I has 32 items). The seventeen items were developed by RAND corporation in 1992 as part of the Medical Outcomes Study (MOS).11 The initial Cronbach’s alpha of MHI -17 was 0.97.11 Initial psychometric performance of MHI-17 was reported by Leidy et al. in euthymic and depressed patients with bipolar disorder.12 The MHI-5 was developed later as a shortened version of the MHI-17, and includes the 5 mental health questions with a different time frame of four weeks as compared to two weeks.
For the study reported here, the MHI-17 was administered as part of a brief Geriatric Assessment used in a research registry of older patients with cancer.13 The MHI-17 was scored, as developed, with 4 domains: depression/behavioral-emotional control was measured by 8 items (see Appendix 1; 2, 5, 8, 9, 11, 13, 15, 16); anxiety by 4 items (4, 6, 10, 17); sense of belonging by 1 item (3); and positive affect by 4 items (1, 7, 12, 14). Item response options are on a 6-point frequency scale ranging from “All of the time” to “None of the time”. Some items were reverse scored so that when summed, higher scores represented better mental health. Scores ranged from 0-100.
Analysis
Descriptive Statistics
Item-level descriptive statistics, including frequencies, means, and standard deviations (SD), were computed in SAS. Floor and ceiling effects were calculated as the proportion of older adults who had the minimum and maximum summed score, respectively. Item-to-total correlations were computed between each item and the total of the remaining items. All correlations were hypothesized to be positive and statistically significant, with patterns of negative or low item-to-total correlations indicative of items that could be revised or removed from the MHI-17.
Reliability
Internal consistency was evaluated by Cronbach’s alpha. For group-level assessment, alpha values of at least 0.70 were considered ideal.14,15
Structural Validity
Factor analyses
To explore the dimensionality of the MHI-17 items and provide a check on assumptions of the unidimensional item response theory (IRT) model to be used, we used confirmatory factor analysis (CFA) of the inter-item tetrachoric correlation matrix using Mplus to explore the relationships between the items and the hypothesized underlying constructs of depression and anxiety. Fit of this model was evaluated based on three indices: the root mean square error approximation (RMSEA; (RMSEA; acceptable if <0.05), an absolute fit measure; and incremental fit indices, the Tucker-Lewis index (TLI; acceptable if >0.95) and the comparative fix index (CFI; acceptable if >0.95). Fitting residual correlations significantly greater than zero served as indicators of local dependence (LD), which indicates content redundancy between two or more items and may be suggestive of additional factors.
Item response theory analyses
IRT analysis offer more in-depth information than classical test theory methods, including evaluation of variation in item performance (differential item functioning) across demographic subgroups. The IRT-based graded response model (GRM) characterizes each item by two parameters: the discrimination parameter, a, which reflects the degree to which item responses are associated with the latent construct being measured (e.g., how effectively an item discriminates between individuals with higher versus lower depression) and the threshold parameter, b, which reflects where along the continuum of the latent construct the item response categories are providing the most discrimination (e.g., if the item is tapping the high end or low end of depression). For each dimension identified in the CFA, the GRM was conducted using IRTPRO (Scientific Software International, Lincolnwood, IL). Overall model fit was evaluated based on the root mean square error of approximation (RMSEA), with values less than 0.08 considered acceptable. The S-X2 statistic compares expected and observed item response frequencies and was used to assess item-level fit, for which significant results (p < 0.05, adjusted for multiple comparisons) was an indicator of poor fit. The standardized local dependence (LD) X2 statistic compares observed and model-implied cell counts and was used to identify items that were excessively related after controlling for the underlying domain; values larger than 10 indicated substantial LD.16-21
Next, we examined differential item functioning (DIF), which enables evaluation of whether items behave differently across subgroups after holding the underlying construct (depression or anxiety) constant. It detects a form of measurement bias that occurs when people in different groups with the same level of the underlying construct have a different probability of getting a particular score on a scale. DIF may indicate that attributes other than those the scale is intended to measure are affecting responses. For this analysis, potential DIF was examined to detect whether items behaved differently across sex, education (high school or less vs. more than high school), and treatment stage (before cancer treatment, during treatment, and after treatment) groups. The Wald X2 test was used to evaluate DIF at the 0.05 level with the Benjamini-Hochberg procedure used to correct for multiple comparisons. The magnitude of DIF was evaluated graphically by examining the characteristic curves.
Items that did not fit well, substantially violated local dependence, or functioned differently for key groups were examined to see if they were in line with the theory of mental health and treatment with use of the DSM-V,22 and to improve the scoring and usability of the MHI. A final IRT model was calibrated with the remaining items. Final IRT parameters were used to examine how well each item measured mental health.
Item response theory scoring and reliability
IRT scores were computed based on the final IRT parameters. These scores are relative to the population of this sample, assuming a normal distribution with a mean of 0 and standard deviation of 1. To be more easily interpretable, the IRT scores were then scaled to the T-score metric with a mean of 50 and a standard deviation of 10. Although IRT scale scores may be computed for either response patterns or summed scores, summed scores tend to be more widely used since they do not require special software.23 Thus, we computed summed scores to ease use of the revised scale. IRT test information was used to examine score precision in distinguishing among individuals; more information indicates greater precision and reliability.
Results and discussion
Descriptive Statistics
Our sample included 686 older adults diagnosed with cancer, with a mean age of 73 (range: 65 – 93). Sixty-eight percent were women, 86 % self-reported as White, and 49 % were college graduates. A majority of the sample had at least one comorbid condition, and 45 % were diagnosed with breast cancer, although multiple different cancer types were represented (see Table 1).
Table 1:
Characteristics of the study sample.
| Sample Characteristics | n = 686 |
|---|---|
| Mean Age | 73 (range: 65-93, SD=6.87) |
| Sex | |
| Women | 68% |
| Self-reported ethnicity | |
| White | 86% |
| Education | |
| High school or less | 31% |
| Some College | 20% |
| College graduates | 49% |
| Number of comorbid conditions | |
| 0 | 9% |
| 1–3 | 62% |
| 4+ | 29% |
| Type of cancer | |
| Breast | 45% |
| Lung | 11% |
| Other | 44% |
| Treatment Period | |
| Pre-treatment | 37% |
| During treatment | 39% |
| Post-treatment | 24% |
Note: Other cancers = colorectal, head and neck, leukemia, lymphoma, and gynecologic.
Validity and Internal Consistency Reliability
The MHI-17 had high internal consistency reliability (α=0.90) and substantial item-total correlations (0.33-0.70). However, a one-factor (mental health domain) confirmatory factor analysis model failed to support the data (RMSEA=0.12; CFI=0.92; TLI=0.91). A one-factor model fit to the five MHI-5 items provided poor results (RMSEA=0.16; CFI=0.96; TLI=0.91). We then removed items for a few reasons: items that were considered double-barreled questions (asking more than one question in one item) and/or did not address directly treatable mental health issues. This resulted in dropping four items (3,8,10 and 14): “have you ever felt loved and wanted?”, “have you felt emotionally stable”, “have you felt restless, fidgety, or impatient?”, and “were you a happy person?”. Given the poor fit of the model to the overall 17 item scale, we reconceptualized what constructs the MHI-17 is measuring. We hypothesized that the MHI measures two domains, depression and anxiety, and reviewed the instrument based on the DSM-V to assign the remaining items to these two domains.22 The resulting 13 item scale consisted of two subscales, four anxiety items (RMSEA=0.11; CFI=0.99; TLI=0.98, factor loadings: 0.71-0.77) and nine depression items (RMSEA=0.10; CFI=0.96; TLI=0.95, factor loadings: 0.55-0.80), which both fit well and provided evidence of its structure. See table 2 for items on MHI-13.
Table 2:
MHI-13 items by domain
| Anxiety Items | Depression Items |
|---|---|
| • Have you been a very nervous person? |
• Has your daily life been full of things that were interesting to you? |
| • Have you felt tense or high-strung? | • Did you feel depressed? |
| • Have you felt calm or peaceful? | • Have you been in firm control of your behavior, thoughts, emotions, feelings? |
| • Have you been anxious or worried? |
• How you felt downhearted and blue? |
| • Have you been moody, or brooded about things? |
|
| • Heave you felt cheerful, light-hearted? | |
| • Have you been in low or very low spirits? |
|
| • Did you feel you have nothing to look forward to? | |
| • Have you felt so down in the dumps that nothing could cheer you up? |
IRT analyses of the four anxiety items indicated good overall fit (RMSEA= 0.08), good item-level fit, no local dependence, and precise measurement of adults with poor mental health, from 3 SDs below to 1 SD above the mean for anxiety. IRT analyses of the nine depression items indicated good overall fit (RMSEA= 0.05), good item-level fit, no local dependence, and precise measurement of adults with poor mental health, from 4 SDs below to 1 SD above the mean of depression. No differential item functioning by sex, education, or treatment stage was found for either subscale. Items with the highest parameter is considered the most discriminating. IRT scores were developed to provide meaningful norms. See Table 3 for item parameters.
Table 3.
IRT GRM Item Parameters
| Anxiety Item |
a | b1 | b2 | b3 | b4 | b5 |
|---|---|---|---|---|---|---|
| 10 | 2.22 | 0.08 | 1.06 | 1.94 | 2.35 | 3.13 |
| 11 | 2.28 | −0.05 | 0.93 | 1.94 | 2.37 | 3.09 |
| 12 | 1.82 | −1.02 | 0.71 | 1.20 | 2.15 | 3.13 |
| 13 | 2.17 | −0.65 | 0.63 | 1.62 | 2.29 | 2.82 |
| Depression Item |
a | b1 | b2 | b3 | b4 | b5 |
| 1 | 1.38 | −1.11 | 0.54 | 1.10 | 2.20 | 3.13 |
| 2 | 2.09 | −0.04 | 1.19 | 2.11 | 2.69 | 3.17 |
| 3 | 1.11 | 0.19 | 2.19 | 2.80 | 3.40 | 3.99 |
| 4 | 2.42 | −0.09 | 1.10 | 2.15 | 2.48 | 3.08 |
| 5 | 1.68 | 0.23 | 1.42 | 2.41 | 2.76 | 3.32 |
| 6 | 2.04 | −1.25 | 0.41 | 0.88 | 1.78 | 2.51 |
| 7 | 2.34 | 0.01 | 1.19 | 2.05 | 2.31 | 3.03 |
| 8 | 1.99 | 1.05 | 1.83 | 2.56 | 2.96 | |
| 9 | 2.43 | 1.37 | 2.02 | 2.65 | 3.02 |
Scoring
The MHI-13 has thirteen individual items that are scored from 0-5 with higher scores reflecting higher levels of anxiety or depression. For each of the two domains, item scores are summed and translated into T-scores. Recursive partitioning in JMP was used to determine cut-scores for each subscale by maximizing the difference between poor and good health groups calculated with current MHI-5 cut-points. The cutting value for the split was determined by maximizing the LogWorth, which is related to the p-value associated with the sum of squares due to the difference between means. T-scores greater than 57.8 for depression and 55.1 for anxiety can used as indicators of potential poor mental health, see appendix 2 for MHI-13, detailed instructions and scoring sheets. See appendix 1 for details.
Practice Implications
This study examined the validity and reliability of the MHI-17 for a population of older adults with cancer. The data from our sample population indicated that the items on the MHI-17 did not fit into one mental health domain. Four items were removed from the original MHI-17 that did not directly address mental health issues or were considered double-barreled (asking more than one question in one item), leaving thirteen items that represent two distinct domains: anxiety and depression. The resulting MHI-13 provides a shorter, and potentially more accurate way to assess anxiety and depressive symptoms for practical referral in older adults with cancer. Incorporating the MHI-13 into the evaluation process of older adults with cancer may allow for earlier detection of mental health issues, and perhaps most importantly, allows clinicians and researchers to separately identify anxiety and depression. This is an important clinical distinction, as treatment options differ among these two types of mental dysfunction and have different relationships with aging.10 We believe that MHI-13 represents a measure of anxiety and depressive symptoms in the geriatric oncology population, and our results will improve our understanding of mental health disorders in older adults with cancer in future studies.
Acknowledgements
This work was supported by the National Cancer Institute (R25CA116339), the Lineberger Cancer Center University Cancer Research Fund (UL1RR025747), and the Clinical and Translational Science Award program of the National Center for Advancing Translational Sciences (1UL1TR001111). Portions of this research were presented at the American Society of Clinical Oncology Annual Meeting in Chicago, Illinois in June 2016. We would also like to acknowledge Emily Fawaz for her final editing and support.
Appendix 1
MHI-13 Administration and Scoring Directions
Administration Instructions
All 13 items of the MHI-13 can be administered together (MHI-13 Anxiety + Depression for Administration.docx). Alternatively, these two scales can be administered separately (MHI-13 Anxiety for Administration.docx; MHI-13 Depression for Administration.docx). Whether these scales are administered together or separately, separate scores are calculated for Anxiety and Depression.
Scoring Instructions
Score each of the MHI-13 items as indicated below. The sum of the first 9 items produces a Depression summed score, and the sum of the last 4 items produces an Anxiety summed score. Following the scoring rubric on the next page is a translation table to convert the summed scores to T-scores.
| How much of | A | A | Item | ||||
|---|---|---|---|---|---|---|---|
| the time | All | Most | Good | Some | Little | None | Score |
| during the | of | of | Bit | of | of | of | given |
| past two | the | the | of | the | the | the | Response |
| weeks: | Time | Time | the | Time | Time | Time | Selected |
| Time | |||||||
| 1. has your daily life been full of things that were interesting to you? | 0 | 1 | 2 | 3 | 4 | 5 | |
| 2. did you feel depressed? | 5 | 4 | 3 | 2 | 1 | 0 | |
| 3. have you been in firm control of your behavior, thoughts emotions, feelings? | 0 | 1 | 2 | 3 | 4 | 5 | |
| 4. have you felt downhear ted and blue? | 5 | 4 | 3 | 2 | 1 | 0 | |
| 5. have you been moody, or brooded about things? | 5 | 4 | 3 | 2 | 1 | 0 | |
| 6. have you felt cheerful, light-hearted? | 0 | 1 | 2 | 3 | 4 | 5 | |
| 7. have you been in low or very low spirits? | 5 | 4 | 3 | 2 | 1 | 0 | |
| 8. did you feel you had nothing to look forward to? | 4 | 4 | 3 | 2 | 1 | 0 | |
| 9. have you felt so down in the dumps that nothing could cheer you up? | 4 | 4 | 3 | 2 | 1 | 0 | |
| Column Sum= Depression Summed Score | |||||||
| 10. have you been a very nervous person? | 5 | 4 | 3 | 2 | 1 | 0 | |
| 11. have you felt tense or high-strung? | 5 | 4 | 3 | 2 | 1 | 0 | |
| 12. have you felt calm or peaceful? | 0 | 1 | 2 | 3 | 4 | 5 | |
| 13. have you been anxious or worried? | 5 | 4 | 3 | 2 | 1 | 0 | |
| Column Sum= Anxiety Summed Score | |||||||
Scoring Table
For the Depression scale. T-scores ≥ 57.8, summed scores of at least 12, indicate poor mental health.
For the Anxiety scale. T-scores ≥ 55.1, summed scores of at least 6, indicate poor mental health.
| Summed Score | Depression T-Score | Anxiety T-Score |
|---|---|---|
| 0 | 33.1 | 35.7 |
| 1 | 38.0 | 41.5 |
| 2 | 41.5 | 45.7 |
| 3 | 44.2 | 48.9 |
| 4 | 46.4 | 51.5 |
| 5 | 48.3 | 53.8 |
| 6 | 50.0 | 55.9 |
| 7 | 51.6 | 57.9 |
| 8 | 53.0 | 59.8 |
| 9 | 54.4 | 61.6 |
| 10 | 55.6 | 63.3 |
| 11 | 56.8 | 65.0 |
| 12 | 58.0 | 66.6 |
| 13 | 59.1 | 68.3 |
| 14 | 60.2 | 69.9 |
| 15 | 61.2 | 71.6 |
| 16 | 62.2 | 73.3 |
| 17 | 63.1 | 75.1 |
| 18 | 64.1 | 77.1 |
| 19 | 65.0 | 79.5 |
| 20 | 65.8 | 83.0 |
| 21 | 66.7 | |
| 22 | 67.5 | |
| 23 | 68.4 | |
| 24 | 69.2 | |
| 25 | 70.0 | |
| 26 | 70.8 | |
| 27 | 71.6 | |
| 28 | 72.4 | |
| 29 | 73.2 | |
| 30 | 74.0 | |
| 31 | 74.8 | |
| 32 | 75.6 | |
| 33 | 76.5 | |
| 34 | 77.3 | |
| 35 | 78.2 | |
| 36 | 79.1 | |
| 37 | 80.0 | |
| 38 | 81.1 | |
| 39 | 82.1 | |
| 40 | 83.1 | |
| 41 | 84.3 | |
| 42 | 85.8 | |
| 43 | 88.2 |
Footnotes
Mackenzi Pergolotti is now with ReVital Cancer Rehabilitation, Select Medical.
Grant R. Williams is now at the Division of Hematology and Oncology at University of Alabama at Birmingham.
Publisher's Disclaimer: This is a PDF file of an unedited manuscript that has been accepted for publication. As a service to our customers we are providing this early version of the manuscript. The manuscript will undergo copyediting, typesetting, and review of the resulting proof before it is published in its final citable form. Please note that during the production process errors may be discovered which could affect the content, and all legal disclaimers that apply to the journal pertain.
Disclosure of potential conflicts of interest
The authors declare that they have no conflicts of interest related to this research.
REFERENCES
- 1.Jeste DV, Alexopoulos GS, Bartels SJ, et al. Consensus statement on the upcoming crisis in geriatric mental health: Research agenda for the next 2 decades. Archives of General Psychiatry. 1999;56(9):848–853. [DOI] [PubMed] [Google Scholar]
- 2.Whooley MA, Kip KE, Cauley JA, et al. Depression, falls, and risk of fracture in older women. Archives of Internal Medicine. 1999;159(5):484–490. [DOI] [PubMed] [Google Scholar]
- 3.Reyes-Gibby CC, Aday LA, Anderson KO, Mendoza TR, Cleeland CS. Pain, depression, and fatigue in community-dwelling adults with and without a history of cancer. Journal of pain and symptom management. 2006;32(2):118–128. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 4.Brown LF, Kroenke K, Theobald DE, Wu J, Tu W. The association of depression and anxiety with health-related quality of life in cancer patients with depression and/or pain. Psycho-Oncology. 2010;19(7):734–741. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 5.Goodwin JS, Zhang DD, Ostir GV. Effect of Depression on Diagnosis, Treatment, and Survival of Older Women with Breast Cancer. Journal of the American Geriatrics Society. 2004;52(1):106–111. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 6.Ellis J, Lin J, Walsh A, et al. Predictors of referral for specialized psychosocial oncology care in patients with metastatic cancer: the contributions of age, distress, and marital status. Journal of clinical oncology : official journal of the American Society of Clinical Oncology. 2009;27(5):699–705. [DOI] [PubMed] [Google Scholar]
- 7.Hsu T, Loscalzo M, Ramani R, et al. Factors associated with high burden in caregivers of older adults with cancer. Cancer. 2014;120(18):2927–2935. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 8.Jolly TA, Deal AM, Nyrop KA, et al. Geriatric assessment-identified deficits in older cancer patients with normal performance status. The oncologist. 2015;20(4):379–385. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 9.Williams GR, Deal AM, Jolly TA, et al. Feasibility of geriatric assessment in community oncology clinics. Journal of geriatric oncology. 2014;5(3):245–251. [DOI] [PubMed] [Google Scholar]
- 10.Weiss Wiesel TR, Nelson CJ, Tew WP, et al. The relationship between age, anxiety, and depression in older adults with cancer. Psychooncology. 2015;24(6):712–717. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 11.Stewart AL. Measuring functioning and well-being: the medical outcomes study approach. duke university Press; 1992. [Google Scholar]
- 12.Leidy NK, Palmer C, Murray M, Robb J, Revicki DA. Health-related quality of life assessment in euthymic and depressed patients with bipolar disorder: psychometric performance of four self-report measures. Journal of Affective Disorders. 1998;48(2):207–214. [DOI] [PubMed] [Google Scholar]
- 13.Hurria A, Gupta S, Zauderer M, et al. Developing a cancer-specific geriatric assessment. Cancer. 2005;104(9):1998–2005. [DOI] [PubMed] [Google Scholar]
- 14.Cronbach LJ. Coefficient alpha and the internal structure of tests. Psychometrika. 1951;16(3):297–334. [Google Scholar]
- 15.Streiner DL, Norman GR. Health Measurement Scales: A practical guide to their development and use. Oxford University Press; 2008. [Google Scholar]
- 16.Tucker LR, Lewis C. A reliability coefficient for maximum likelihood factor analysis. Psychometrika. 1973;38(1):1–10. [Google Scholar]
- 17.Bentler PM. Comparative fit indexes in structural models. Psychol Bull. 1990;107(2):238–246. [DOI] [PubMed] [Google Scholar]
- 18.Edelen MO, Reeve BB. Applying item response theory (IRT) modeling to questionnaire development, evaluation, and refinement. Quality of life research : an international journal of quality of life aspects of treatment, care and rehabilitation. 2007;16 Suppl 1:5–18. [DOI] [PubMed] [Google Scholar]
- 19.Kang T, Chen TT. Performance of the generalized S-X2 item fit index for the graded response model. Asia Pacific Education Review. 2010;12(1):89–96. [Google Scholar]
- 20.Chen WH, Thissen D. Local Dependence Indexes for Item Pairs Using Item Response Theory Journal of Educational and Behavioral Statistics. 1997;22(3). [Google Scholar]
- 21.Steiger JH. A note on multiple sample extensions of the RMSEA fit index. Structural Equation Modeling: A Multidisciplinary Journal. 1998;5(4):411–419. [Google Scholar]
- 22.Association AP. Diagnostic and Statistical Manual of Mental Disorders. 5th ed Washington, DC: 2013. [Google Scholar]
- 23.Thissen D, Wainer H. Test scoring. Journal of Educational Measurement. 2002;3(39):98–137. [Google Scholar]