Construct validity of the Herth Hope Index: A systematic review

Abstract

Objective:

The psychological construct of hope is an important determinant for mental health and well-being. The availability of valid and reliable instruments to measure hope is, therefore, critical. Despite a large number of psychometric studies on the Herth Hope Index (HHI), its construct validity has not yet been determined. Therefore, this paper aimed to conduct a systematic review of the psychometric properties of the HHI.

Methods:

Databases such as PubMed, Science Direct, Google Scholar, Magiran, SID, IranDoc, and IranMedex were evaluated systematically using the terms “HHI,” “psychometric,” “validity,” “reliability,” and related terms (with the use of OR and AND operators) and no restrictions on the year of publication. A total of 13 eligible studies were found published between 1992 and 2018 in the USA, Portugal, Switzerland, Iran, Germany, Petersburg, Japan, the Netherlands, Lima, Peru, and Norway. The methodology used in the available studies included principal component analysis (n = 6), maximum likelihood estimation (n = 5), and principal axis factoring (n = 1). One study did not point the methodology.

Results:

Four studies reported the total extracted variances to be less than 50%, six studies reported variance between 50% and 60%, and three papers reported variance that exceeded 60%. Of the papers that examined the factor structure of the HHI, two studies reported a one-factor solution, seven reported two factors, and four reported a three-factor solution. Although the HHI is the most widely translated and psychometrically tested tool in languages other than English, psychometric variations in factor solutions remain inconsistent.

Conclusion:

Findings highlight the need for future research that appraises the validity of the HHI in different countries, and how the measure relates to other scales that evaluate hope.

Introduction

Hope is a universal multidimensional psychological construct that relates to a person’s expectation that they will obtain desirable results based on realistic, future goals. Hope is a motivating factor for initiating or maintaining actions to achieve a goal and can be facilitated or sustained through an individual’s support networks and interpersonal relationships. Hope is positively associated with happiness, perseverance, success, health, compatibility, and social support[1-3] and can neutralize the effects of stress on mental, behavioral, and physical health.[1] Furthermore, studies indicate that hope is considered an inner source[4] that is crucial to finding the meaning of ones’ life, solidarity, positive self-assessment, and self-worth,[5] which can ultimately lead to self-actualization.[6] Schrank et al. (2008) completed a systematic review investigating hope in psychiatry and found that the concept is vital as both a process and outcome variable in the recovery from severe mental illness.[7] Hope has gradually received attention from both policymakers and practitioners concerning its potential impact on well-being and quality of life for individuals.[8]

The plethora of studies focusing on hope as a core construct for well-being is pervasive across different fields. Schrank et al. (2008) indicated that 32 instruments have been designed to measure hope. Some examples of these instruments include the Snyder Hope Scale (1991), State Hope Scale (1997), Miller Hope Scale (1988), and the Herth Hope Scale (HHS) (1989).[7] The creation of the Herth Hope Index (HHI) is based on Dufault and Martocchio’s (1985) multidimensional hope theory which takes into account philosophical, religious, sociological, and psychological factors.[9,10] The purpose of this index was to access multiple dimensions of hope, provide a clear expression of the unique dimensions of hope in clinical populations, and reduce both the complexity and number of items of the other available measures. The HHI was designed in 1991 to evaluate hope in young people with chronic illnesses. It consists of 12 items responded to on a four-point Likert-type scale, from one (completely disagree) to four (completely agree).[11]

The original HHS and the abbreviated version, the HHI, are the most widely translated and psychometrically tested tools in a language other than English.[9,12-14] Despite a large number of psychometric studies on this index and its wide use in non-English speaking cultures, its validity as an appropriate tool has not yet been fully confirmed. Such research points to a need to better understand hope and further investigate the HHI in varied cultural contexts.[14] It has been argued that some items of the tool may be problematic for use in certain cultures leading to invalid results.[9] As many studies have only focused on conceptualizing the HHI rather than evaluating the measure, it is necessary to conduct a comprehensive review of the index to evaluate its potential usefulness. A search of previous literature indicates that no systematic review has been conducted on the evaluation of the index psychometrically. Therefore, the present study aims to present a systematic review of the psychometric properties of the HHI.

Methods

Study design and search strategy

A systematic review of the literature was conducted using PubMed, Scopus, Web of Science, Embase, Science Direct, and Google Scholar databases to assess articles with English-language abstracts. Persian-language articles from databases, including Magiran, SID, IranDoc, and IranMedex, were evaluated as well. Date of publication was not restricted and all past studies up until December 2018 were considered. The search terms “HHI,” “psychometric,” “validity,” “reliability,” and related terms (with the use of OR and AND operators) were used. Two researchers assessed the titles and abstracts of related articles independently. Articles were included into the study if they used only the HHI in the study, contained the search term keywords in the title or abstract, were in English or Persian with an English abstract, and conducted a cross-cultural psychometric assessment, including measures of reliability, validity, or diagnostic accuracy of the measure. Only studies that reported the construct validity using factor analysis were included in the study. Figure 1 displays the stages of article selection based on preferred reporting items for systematic reviews and meta-analyses guidelines.

Figure 1.

PRISMA diagram

Checking the quality of studies

The quality of the procedure was assessed by two independent investigators using two bias risk assessment tools, including the Quality Assessment of Diagnostic Accuracy Studies (QUADAS-2)[15] and the checklist of Standards for Reporting of Diagnostic Accuracy (STARD).[16] QUADAS-2 was designed to carefully review the method of the studies reviewed. This tool evaluates the internal consistency of the study and contains 14 items with a three-option answer (i.e., yes, no, and unknown).[15] The 25-item checklist of Standards for Reporting Diagnostic accuracy studies (STARD) was used to check the study design quality (including the sample collection and data). In the current study, the revised version of these tools was used to assess the studies that analyzed the validity and reliability of the HHI. Inconsistencies between the scales were resolved by discussion among the two independent investigators. If an agreement was not reached about the validity and reliability of the assessed articles, a third researcher was invited to the study to assess the articles.

Data abstraction

After evaluating the quality and accuracy of the articles, the following data were extracted: Authors’ name(s), publication year, study-type, number of participants (and their mean age), number of extracted variables, and explained total variance and validity. The appraisal checklists were approved by ten faculty members (five assistant professors, three associated professors, and two full professors in nursing) before determining the quality of each study to ensure the validity aligned with the aim of the present study.

Ethical considerations

Ethical issues (including plagiarism, informed consent, misconduct, data fabrication and/or falsification, double publication and/or submission, and redundancy) were considered by authors.

Results

A total of 38 studies using the HHI were initially extracted. From these, 21 duplicated papers were excluded resulting in 16 papers with full-text access. Based on exclusion criteria, 13 articles were deemed eligible for a systematic review. Figure 1 indicates the process of selecting studies and their inclusion in the present systematic review.

Quality assessments

A total of 13 studies that focused on the validation of the HHI were evaluated using QUADAS-2 and STARD, described previously. In general, the quality of all studies was acceptable based on the risk assessment tools. The risk for bias that was noted included possible loss of important results, lack of descriptions of the study inclusion and exclusion criteria, and issues related to the acceptability and management of selected samples.

Studies description

Studies meeting criteria for final evaluation were published between 1992 and 2018 in the USA, Portugal, Switzerland, Iran, Germany, Russia (Petersburg), Japan, the Netherlands, Lima, Peru, and Norway. Most of the included studies utilized simple and convenience sampling, and sample sizes were reported to be between 45 and 500 participants. Four studies did not indicate the sampling method.

Six studies used a principal component analysis (PCA),[17-19] one utilized principal axis factoring,[25] five used maximum likelihood (ML) estimation,[20-23] and one study did not point the methodology. Three studies used confirmatory factor analysis.[5,18,22] Two studies reported a one-factor solution of the HHI,[22-24] seven reported two factors,[5,18,20,23-25] and four reported a three-factor solution.[17,21,23,26] Four studies reported the total extracted variance to be less than 50%,[17-18,25] six reported the variance to be between 50% and 60%,[19,20,22,24] and the total variance reported for three studies was reported to be greater than 60%.[21,23,26]

To examine the construct validity of the studies, weighted averages of the percentage of extracted variances were assessed. While there are variances in the techniques used to extract factors between PCA and exploratory factor analysis (EFA), many scientists use them interchangeably and PCA is frequently used when EFA would be more appropriate. EFA emphasizes the shared variance of the variables, whereas PCA focuses on the total variation among the variables; therefore, the percentage of the extracted variance of PCA will be greater than EFA.[33]

Reliability of the HHI

Two papers performed test-retest using correlation coefficients and reported this value as >0.7.[18,23] The other 11 studies used Cronbach’s alpha to assess internal consistency, of which two studies had an alpha of <0.7 for one of the extracted factors[21] and the rest were above 0.7.[24] Moreover, one study reported composite reliability for the two extracted factors, with values of 0.88 and 0.69.[25]

Discussion

The present study provides a systematic evaluation of 13 studies that evaluated the HHI. It is crucial to use a valid and reliable instrument to measure hope, considering the importance of its role in both morbidity and well-being. In addition, psychometric assessment is very important for nursing researchers as a whole. Over the past two decades, researchers focusing on nurses have used the principles of psychometric assessment to design and test instruments for measuring important phenomena in the field of nursing. Therefore, it is vital to use instruments with established validity and reliability to conduct this research or design new instruments when needed.[28]

The studies in this review were conducted across different disciplines, contexts, countries, and languages. The increasing interest in the HHI is evident by the growing number of publications over the years, with nine papers published after 2010. As the HHI takes into account philosophical, religious, sociological, and psychological contexts,[14,29] the index can be highly contextual based on where the measure is administered. Therefore, measuring the HHI using a one-fits-all approach does not seem appropriate and might limit the ability to generalize the findings. Thirteen studies incorporating the HHI by utilizing a standardized methodology.

Of the 13 studies that utilized the HHI, 30% of the selected studies did not report the sampling method, while the remainder utilized simple and convenience sampling methods. Although the convenience sampling method may be the only available option under some research circumstances, additional methods should be considered to generalize the study findings.

Various analytical techniques (e.g., PCA, ML, and PAF) were applied in the selected studies, which resulted in different construct structures. However, none of the selected papers discussed the rationale for choosing their selected method. Average variance extracted (AVE) of 0.5 and above is one of the criteria used to establish construct validity.[30] In eight of the studies, the AVE was greater than 0.5, whereas the remainder showed less validity.

More than 80% of the articles (11 total) incorporated Cronbach’s alpha to assess internal consistency and reliability. Irrespective of the reported value of Cronbach’s alpha, to the best of the authors’ knowledge, no single reliability index can be considered a perfect assessment tool. Although alpha is always a lower bound of reliability, other formulae have been developed for estimating reliability; some are more accurate than alpha, but none have gained as much popularity.[31] It is encouraged that researchers use at least two indices to ensure valid reliability. Reliability is the equality of the results over time and under the same condition which is measured by the repeatability and reproducibility of the results.[32] Reliability consists of three different parts, including stability, equivalence, and homogeneity. Researchers use different methods to measure each of the three parts (e.g., test-retest, Cronbach’s alpha, and split-half).[33] Cronbach’s alpha determines the average internal consistency of questions. It is the most common measure of internal consistency when the questions have multiple Likert or Likert-type measures. However, test-retest measures test the consistency by giving the same test twice to the same people at different times to see if the scores are the same. It is supposed that the measured concept and the samples’ features have not been changed during the two times.[34] As the concept of hope is a relatively stable concept over time, some studies have used the test-retest method to assess the reliability of the HHI.[18,23] However, memory effect, fatigue effect, and genuine change effect can have an impact on test-retest reliability. For reliability, as discussed in the theory of reliability, it is not possible to calculate reliability exactly; instead, researchers intending to estimate reliability will find it always remains an imperfect endeavor. Two reliability estimates used in the selected 13 studies are test-retest and Cronbach alpha test. Cronbach alpha, which is the most familiar flavor under internal consistency strategy, has been applied in all the selected studies, while only two studies (15% of the sample) used test-retest as a complementary test. It is worth mentioning the amount of time that elapses between two tests among a sample is critical in the test-retest estimate. The shorter the gap, the higher the correlation. While comparing the correlation coefficients of different research outputs, one should bear in mind the test-retest method can present considerably different estimates depending on intervals. Although the time intervals were not the same in both studies (1 week and 2 weeks), the correlation coefficients remained relatively similar. Besides these methods, intraclass correlation coefficient is the more desirable measure of reliability that reflects both degrees of correlation and agreement between measurements. It has been widely used to evaluate inter-rater, test-retest, and intra-rater reliability (Table 1).[35] The results of the present study indicate that one study[25] assessed consistency using the average inter-item correlation.[3]

Table 1.

Extracted studies used HHI

All selected studies tested the psychometric properties of the HHI scale. Three factors of temporality and future, positive readiness and expectancy, and interconnectedness were first introduced by Herth (1999) as underlying factors to develop the HHI. Since then, the above-mentioned factorial structures of the HHI were examined in different countries to investigate if cross-cultural, demographic, and societal factors can influence the validity and reliability of Herth’s (1999) hypothesized factors. However, exploratory factor analyzes in the selected studies did not confirm the original three-factor structure by Herth (1999). Although in 30% of selected studies, the data could fit all three underlying factors introduced by Herth (1999), 54% of studies confirmed a two-factor structure. Moreover, in 15% of selected studies, the validity and reliability test approved a one-factor structure. These results demonstrated that using a scale in a culture other than the one it was originally designed for can generate a potential threat to the validity and reliability of an index. Furthermore, the variety of outcomes from using the same index in different cultures sheds light on the fact that underlying factors of hope are not unambiguous.[20] This may open new avenues for further research to develop a new instrument to measure hope in each specific culture. Moreover, all tools need to be culturally relevant and any translation of the HHI or any other tool needs to specifically and systematically address the cultural implications in the translation.

Conclusions

In summary, the HHI has been widely translated and psychometrically tested in many languages across numerous countries. Studies utilizing an EFA revealed that one to three factors were extracted from the HHI. The reliability of the HHI was reported as being acceptable in all of the studies. The review findings highlight the need for additional studies that appraise how the HHI is translated and interpreted in other countries and how it compares to other tools.

Authors’ Declaration Statements

Ethics approval and consent to participate

Not applicable.

Availability of data and material

The data used in this study are available and will be provided by the corresponding author on a reasonable request.

Competing Interests

The authors declare that there are no conflicts of interest.

Funding Statement

None.

Authors’ Contributions

All authors equally contributed.