Skip to main content
PMC home page
NIHPA Author Manuscripts logoLink to NIHPA Author Manuscripts
. Author manuscript; available in PMC: 2015 Oct 1.
Published in final edited form as: Health Commun. 2014 Jan 21;29(9):937–946. doi: 10.1080/10410236.2013.833580

Predicting Continuance—Findings from a Longitudinal Study of Older Adults Using an eHealth Newsletter

Heather A Forquer 1, John L Christensen 2, Andy SL Tan 1
PMCID: PMC4029923  NIHMSID: NIHMS548315  PMID: 24446900

Abstract

While eHealth technologies are promisingly efficient and widespread, theoretical frameworks capable of predicting long-term use, termed continuance, are lacking. Attempts to extend prominent information technology (IT) theories to the area of eHealth have been limited by small sample sizes, cross-sectional designs, self-reported as opposed to actual use measures, and a focus on technology adoption rather than continuance. To address these gaps in the literature, the present analysis includes empirical evidence of actual use of an eHealth technology over the course of one year. This large (n=4,570) longitudinal study focuses on older adults, a population with many health needs, and among whom eHealth use may be particularly important. With three measurement points over the course of a year, this study examined the effects of utilitarian and hedonic beliefs on the continued use of an eHealth newsletter using constructs from IT adoption and continuance theories. Additional analyses compared the relative strength of intentions compared to earlier use in predicting later use. Usage intention was strongly predicted by both hedonic beliefs and utilitarian beliefs. In addition, utilitarian beliefs had both direct effects on intention, as well as indirect effects, mediated by hedonic beliefs. While intention predicted subsequent use, earlier use was a significantly stronger predictor of use than intention. These findings make a theoretical contribution to an emerging literature by shedding light on the complex interplay of reasoned action and automaticity in the context of eHealth continuance.

INTRODUCTION

Electronic health technology or eHealth, defined by the World Health Organization as “the use of information and communication technologies for health” (WHO, 2012), is an efficient and convenient method for delivering public health interventions (Cline & Haynes, 2001). Use of the Internet for health information is widespread; nearly 60% of adults in the United States have used the Internet to access health information. However, a primary challenge to the efficacy of eHealth is long-term use. Reports describing the use of Internet-based health interventions show that repeated use is rare (Leslie, Marshall, Owen, & Bauman, 2005; Verheijden, 2007), and little is known about predicting long-term use of eHealth technologies.

Research on the adoption and use of eHealth technologies has been criticized for lacking a theoretical framework (Or & Karsh, 2009). In contrast, identifying the determinants of initial technology adoption decisions has long been a focus of study among information technology (IT) researchers. Several recent studies have united these two streams of research (D. Kim & Chang, 2007; Lemire, Pare, Sicotte, & Harvey, 2008; Or & Karsh, 2009; Silvestre, Sue, & Allen, 2009) but are limited by small sample sizes, cross-sectional designs, and predict adoption rather than long-term use. Consequently, there is a need to apply theoretical models of IT adoption to predict long-term use of eHealth technologies.

This research is further informed by a lifespan approach to communication behaviors and focuses specifically on older adults. Prior research supports the notion that aging adults face unique changes (e.g., neural, cognitive, and emotional abilities) and as a result respond differentially to mediated and interpersonal communications compared to the younger population (Ryan & Butler, 1996; Southwell, 2010; Sparks, 2003). Based on this approach and the reality of increasing health care needs of a rapidly aging population in the United States, our goal in this present study is to examine how to predict continued use of potentially beneficial eHealth technologies by older adults in the context of cancer communication.

Study Overview

This study assesses predictors of long-term use of an eHealth technology by using data from part of a year-long study in which a monthly health e-newsletter was sent to a large nationwide sample (n=4,570) of adults aged 50-70 from July 2010 to May 2011. Subscribers were recruited from an online panel and randomized to receive one of four versions of the e-newsletter focusing on behaviors related to cancer screening and prevention. Subscriber use of the e-newsletter was tracked using an online system. In addition, subscribers completed questionnaires in three waves: at baseline, halfway through the study (6 months following baseline), and at the end of the study (12 months following baseline).

The dataset has unique strengths for understanding predictors of long-term use. Most importantly, unlike the majority of IT adoption and long-term use studies which rely on self-reports of use (Legris, Ingham, & Collerette, 2003; Turner, Kitchenham, Brereton, Charters, & Budgen, 2010), this dataset includes empirical evidence of actual use of an eHealth technology by using an online system to track newsletter use. This may be particularly important for older adults as worsening memory may make recall of exposure less accurate than in younger adults (Southwell, 2010). In addition, IT adoption and long-term use studies are often cross-sectional, limiting the conclusions that can be made (see for example, D. Kim & Chang, 2007; Silvestre et al., 2009). In fact, we are unaware of any IT use studies using three or more time points, a design which is crucial for understanding how relationships between constructs may change over time (S. S. Kim & Malhotra, 2005). Finally, the study focuses on an older population of Internet users, who are likely to have greater health needs than younger adults, but are less likely to use the Internet for health information (Fox, 2011a).

We first outline the theoretical framework that guided the explication of the predictive model, research questions, and hypotheses. This is followed by the study design and analytic procedures. The study aims to inform health communication research and practice by assessing the relative roles of meaningful theoretical concepts in predicting continued use of eHealth technologies.

THEORETICAL FRAMEWORK & RESEARCH MODEL

Predicting Continuance

Theories of technology adoption have been inspired by social psychological models of behavior such as the Reasoned Action Approach (Fishbein & Ajzen, 2010) and its prior incarnation called the Theory of Reasoned Action (TRA). The Reasoned Action Approach views behavior as the result of behavioral intentions which are thought to be derived from various beliefs regarding the expected outcomes of the behavior, normative pressure, and perceived control over the behavior.

Dominant models of IT adoption such as the Technology Acceptance Model (TAM; Davis, 1989) and the Unified Theory of Acceptance and Use of Technology (UTAUT; Venkatesh, Morris, Davis, & Davis, 2003) build upon TRA in an attempt to explain adoption behaviors specific to IT. The expansive IT adoption literature has identified beliefs that are particularly important determinants of behavioral intentions in various IT contexts. For example, perceived usefulness, which is defined as the extent to which individuals expect that the technology will help them achieve an intended goal has been used to predict adoption and use of health-related web sites (D. Kim & Chang, 2007; Lemire et al., 2008; Leslie et al., 2005; Silvestre et al., 2009).

Very few studies predict long-term use of eHealth web sites, a concern that is not unique to the eHealth field. Although frameworks such as TAM have been shown to successfully explain IT adoption behaviors, many IT researchers have argued that they do not adequately explain users’ post-adoption behaviors (Bhattacherjee, 2001). Successful implementation of a technology requires sustained usage over time, long after its initial adoption. This long-term usage is referred to as continuance. It is logical to assume that adoption and continuance are conceptually distinct, and that their determinants may differ.

Noting that existing adoption models “provide a limited explanation of, and may sometimes contradict, observed continuance behaviors” (p. 352), Bhattacherjee (2001) has introduced both an original and updated model of IT continuance (Bhattacherjee & Barfar, 2011). According to this model, continuance intentions are determined by both perceptions of usefulness and by one's satisfaction with post-adoption IT usage. These models attempt to be inclusive of technologies that afford benefits that are not necessarily utilitarian in nature. For instance, the authors drew the field's attention to the fact that, “in this day and age of Internet-enabled systems, many...are intended to enhance user enjoyment” (Bhattacherjee & Barfar, 2011, p.7). This may be particularly important for predicting the use of consumer health technologies (Or & Karsh, 2009), as opposed to more traditional health technologies, whose use may not be related to enjoyment.

Other researchers have also addressed the distinction between utilitarian beliefs (e.g., usefulness) and hedonic beliefs (e.g., enjoyment) in their approaches to IT usage. This divide has been described in multiple ways using different sets of labels. For example, Tojib and Tsarenko (2012) examined intrinsic and extrinsic factors in a post-adoption context. Still others have described intrinsic and extrinsic motivations (Venkatesh & Brown, 2001). Hedonic motivation has also been modeled as a predictor of intention in the UTAUT2 model, an updated version of the original UTAUT model intended to incorporate predictors of use in the consumer context (Venkatesh, Thong, & Xu, 2012). Research suggests that older adults are more motivated to carry out behaviors to reach a particular emotional state (e.g., satisfaction), as opposed to behaviors to increase knowledge (Yoon, Cole, & Lee, 2009). Thus, understanding the contribution of utilitarian and hedonic beliefs is important for predicting use of eHealth technologies by older adults. Overall, there is support for the idea that both types of beliefs are significant determinants of continuance intention. Based on this literature, we hypothesize that utilitarian and hedonic beliefs will be positively associated with behavioral intention at time 1, time 2, and time 3 (Hypotheses 1a). All hypotheses are summarized in Table 1 and illustrated as structural equation paths in Figure 1.

Table 1.

Summary of hypotheses

No. Hypotheses Hypothesis supported
Continuance constructs and relationships
H1a Utilitarian (U) and hedonic beliefs (H) will be positively associated with intention (BI). Yes
H1b The magnitude of the effect of utilitarian beliefs on intention will be stable over time. Yes
H1c The magnitude of the effect of hedonic beliefs on intention will become stronger over time. No
RQ1 Do hedonic beliefs mediate the effect of utilitarian beliefs on behavioral intention? Yes
User evaluation updating
H2a User evaluations will be positively associated over time. Partially
H2b The associations between earlier evaluations will be weaker than the associations between later evaluations. Partially
Influence of past behavior
H3 Past use will be positively associated with subsequent utilitarian beliefs, hedonic beliefs, and behavioral intention over time. Partially
H4 Behavioral intention will predict subsequent use. Yes
H5 Past use will predict subsequent use. Yes
H6 The magnitude of the relationship between intention and use will decrease over time. No
H7 Past use will be a stronger predictor of subsequent use than intention. Yes
Open in a new tab

Figure 1. Results of the final model.

Figure 1

Open in a new tab

Notes. Maximum-likelihood standardized coefficient estimates are presented. Significance levels are set at *p<.05, **p<.01, ***p<.005. The final model included correlated errors between identical measures over waves and controlled for confounders (i.e., age, gender, race (white vs. others), education level, marital status, employment status, income, newsletter version) but these are omitted from this figure for clarity. To account for autocorrelation of measures within individuals over time (individuals who are persistently high or persistently low on one measure over time), correlations between disturbance (error) terms of the corresponding observed measures across waves were also included in the estimation as recommended in Kline (2010) but omitted in Figure 1 for clarity. For example, the error term of H1 was modeled to be correlated with error terms of H2 and H3

In a study of mobile devices, Kim and Oh (2011) examined utilitarian versus hedonic beliefs, comparing their predictive power in both pre-adoption and continuance contexts. Utilitarian beliefs were strong predictors of both adoption and continuance. In contrast, hedonic beliefs predicted continuance, but not adoption. Based on these findings, we hypothesize that the effect of utilitarian beliefs on intentions should remain stable as users gain experience with the newsletter (Hypothesis 1b). However, the effect of hedonic beliefs on intentions should become stronger over time as user experience increases (Hypothesis 1c).

Some IT usage frameworks have modeled utilitarian and hedonic beliefs as being directly related to behavioral intentions, yet others have suggested that hedonic beliefs may mediate the effect of utilitarian beliefs on intentions (B. Kim & Oh, 2011; Luo, Chea, & Chen, 2011; Tojib & Tsarenko, 2012; Venkatesh & Brown, 2001). To gain clarity regarding the relationship between these constructs, we pose the following research questions: Do hedonic beliefs mediate the effect of utilitarian beliefs on continuance intentions at each time point (Research Questions 1a-1c)?

Evaluation Updating and the Influence of Past Use on Later Evaluations

Kim and Malhotra (2005) proposed a longitudinal model of continuance that addresses several underlying mechanisms of post-adoption phenomena. The authors tested their integrative model in the context of an Internet-based higher education web portal and provided compelling evidence that their model is an effective approach to explaining IT continuance. One of the mechanisms included in this model focuses on the dynamic nature of user evaluations and judgments (e.g., beliefs and intentions). The authors noted that previous attempts to explain continuance behaviors view user evaluations as static, largely ignoring the fact that they can evolve over time as users gain experience with a technology. The authors referred to the theory of belief updating (Hogarth & Einhorn, 1992) when arguing that earlier evaluations serve as bases for subsequent evaluations. There is indeed evidence that users engage in anchoring and adjustment processes during which initial evaluative judgments influence those that come later. In other words, later evaluations are formed partially in relation to earlier evaluations (rather than being newly formed at each time point).

This intertemporal updating mechanism has been shown to operate on beliefs as well as intentions (see for example, Bolton & Drew, 1991). We therefore expect that utilitarian beliefs at time 1 will positively predict utilitarian beliefs at time 2 and, furthermore, that these time 2 beliefs will predict beliefs at time 3. Similar evaluation updating processes should occur for hedonic beliefs and behavioral intentions (Hypothesis 2a). It is reasonable to assume that early evaluations might be less informed or precise because they are based on limited interaction with the technology. It is expected that, as users gain more experience with the technology over time, little to no adjustment will be needed. Evaluations should eventually stabilize over time, becoming more highly correlated with each other. More specifically, we hypothesize that the relationship between neighboring evaluations will grow in magnitude over time as users gain experience with the technology (Hypothesis 2b). Said another way, the link between earlier evaluations (between time 1 and time 2) will be weaker than the link between later evaluations (between time 2 and time 3).

In their longitudinal model, Kim and Malhotra (2005) also proposed that earlier use of a technology should influence later evaluations. They argued that based on self-perception theory (Bem, 1972), individuals base their beliefs on their past behaviors. Instead of a unidirectional model in which beliefs predict use, researchers expected a reciprocal relationship whereby the more an individual uses a technology, the more positively she will evaluate it later. In other words, they infer that users have positive beliefs about a technology because they have previously used it. As Kim and Malhotra (2005) explained, this reversed pathway opposes the theory of evaluation updating. Whereas evaluation updating suggests that a cognitive process is taking place in which users reflect on earlier and later beliefs, self-perception theory suggests that there is an automatic process in which users instantly evaluate a technology positively based on the fact that they have used it in the past. We therefore hypothesize that past use will be positively related to later evaluations of the newsletter (Hypothesis 3).

Predicting Use

According to dominant theories of behavior such as TRA (Fishbein & Ajzen, 2010) and technology acceptance theories such as TAM (Davis, 1989), intention predicts later behavior. We therefore hypothesize that behavioral intentions reported at time 1 and time 2 will significantly predict subsequent use of the newsletter (Hypothesis 4).

Previous experience with a technology may influence later intentions and behavior indirectly as described above, but may also have direct effects on later behavior (S. S. Kim, Malhotra, & Narasimhan, 2005). Intentions are formed during a cognitive process based on beliefs about outcomes and attitudes towards the behavior. However, many theories additionally point to a role of habit formation, in which past behavior predicts later behavior directly, without necessarily forming and acting on an intention to carry out the behavior (Jasperson, Carter, & Zmud, 2005; Ouelette & Wood, 1998). Habit is defined as the tendency to repeat past behavior in a stable context in response to a cue (Ajzen, 2002) and is most likely to occur with frequent opportunities to perform the behavior (Ajzen, 2002; Ouelette & Wood, 1998). We therefore hypothesize that the past newsletter usage will be positively related to subsequent use (Hypothesis 5).

The role of habit may be particularly important for older adults, as cognitive decline leads to a greater reliance on automatic processing over deliberative processing. This has important implications; although a shift towards automatic processing could be described as an adaptive behavior, it may also reduce the quality of decision-making as older adults consider fewer pieces of information when making a decision (Cole et al., 2008; Peters, 2010). Therefore, we expect that as habit forms, past use will be a stronger predictor of later behavior compared with behavioral intention. In other words, we hypothesize both that the relationship between intention and subsequent use will weaken over time (Hypothesis 6), and that behavioral intention will be a weaker predictor of subsequent use compared with past use (Hypothesis 7).

METHODS

Study population & procedure

Subscribers were invited from an online panel of US adults maintained by Survey Sampling International and were randomized to receive one of four versions of the [University Name] Health Digest. Each monthly e-newsletter included both “general” articles on a variety of health topics such as aging and heart disease, and “treatment” articles featuring one of four cancer prevention and screening behaviors: fruit and vegetable consumption, exercise, colorectal cancer screening, or mammography. The articles were written by a professional health news editor with input from medical experts affiliated with the [University Name] School of Medicine. Our goal was to keep “busy adults up-to-date on the latest health research” by providing an e-newsletter that was on par with other, widely distributed e-newsletters such as the Harvard Health Newsletter, Consumer Reports on Health, and the AARP Health Newsletter. Further details about the newsletter design are presented in Hornik et al. (2012). Pilot data from 716 members of the target population confirmed that the e-newsletter was visually appealing, easy to read, and covered health topics that appealed to older consumers (e.g., aging, heart disease).

Subscribers received an email each month notifying them of updates to the e-newsletter, which was hosted on an accompanying website. Each email alert included clickable images, headlines, and teasers that led subscribers to full versions of the articles online. Those who did not respond to initial emails were sent weekly reminders to access the newsletter. An electronic system tracked subscribers’ opening of email alerts. Subscribers were invited to participate in surveys after viewing the first newsletter (time 1), six months later (time 2), and an additional six months later (time 3). The survey included questions about health behaviors and about user evaluations of the newsletter. In this analysis, we excluded subscribers who did not view the first newsletter because the analysis focuses on long-term continuance behavior rather than initial adoption of a technology. Of 15,824 participants enrolled in the study, 4,570 met this criterion (28.8%). The study was approved by the university's Institutional Review Board.

Measures

Use

Electronic tracking records of subscribers’ opening of email alerts were used to construct a measure of use between time 1 and time 2 (USE1-2, maximum 5), and between time 2 and time 3 (USE2-3, maximum 6). Utilitarian beliefs (U), hedonic beliefs (H), & behavioral intention (BI). All items were rated on a 5-point Likert scale ranging from “strongly disagree” to “strongly agree”. Respondents could also choose “no opinion”. Respondents were instructed to select “no opinion” if they did not read the newsletter. For respondents who selected “no opinion”, but for whom use was verified with data from the online tracking system, responses were coded as missing. The constructs are defined in Table 2. Control variables. We included several covariates including age, gender, race, education level, marital status, employment status, income, and newsletter version assignment.

Table 2.

Model constructs and corresponding items

Construct Item
Utilitarian beliefs (U) In general, the articles included useful information
(Cronbach's alpha=0.92 at T1, 0.93 at T2, and 0.93 at T3) The articles were useful to my health
I learned new health information about the articles
Hedonic beliefs (H) I enjoyed reading the e-newsletter
Behavioral intention (BI) I am interested in reading future issues of the e-newsletter
Open in a new tab

Statistical Analysis

Preliminary univariate analyses revealed that several of the construct variables were not normally distributed (skewness ranged from −1.37 to 0.50; kurtosis ranged from −1.42 to 1.86; all univariate Shapiro-Wilk tests were significant at p<.0005). We assessed for multivariate outliers by examining model-estimated standardized dfBetas for each of the main model predictors in the linear equations implied by Figure 1. An outlier is defined as an absolute standardized dfBeta value greater than 1.0. No major outliers were detected (standardized dfBetas ranged from −0.96 to 0.65). We visually inspected the standardized residuals versus the predicted values of each of the implied linear equations in Figure 1 and found evidence suggesting heteroscedasticity of residuals variances in several of the equations.

Missing values in the continuance model variables ranged from 0 to 64%. The missingness occurred mainly due to non-response to the surveys at time 1 (30.7%), time 2 (45.5%), and time 3 (51.4%). The remainder of the missingness occurred due to respondents answering “no opinion” about the e-newsletter evaluation questions despite the fact that their use was verified using an online tracking system. In a sensitivity analysis, we repeated the analysis after excluding the respondents who answered “no opinion”. The substantive findings were identical to the original analysis. We therefore reported the model that included these responses coded as missing.

We fitted the hypothesized model with the MPlus 7 statistical software (Muthén & Muthén, 1998-2012) using a full-information maximum likelihood (FIML) algorithm with Huber-White covariance adjustments to address the missingness in the predictors and the above minor violations of linear regression assumptions (non-normality, outliers, and heteroscedasticity) (Kline, 2010). The FIML technique is shown to be superior to ad hoc methods for dealing with missing data (e.g., listwise deletion, pairwise deletion, mean imputation) and reduces bias and sampling variability in multiple regression models (Enders, 2001; Newman, 2003). Goodness of fit was assessed using a combination of indices including the chi-square test of model fit, comparative fit index (CFI), Tucker-Lewis Index (TLI), root mean square error of approximation (RMSEA), and the standardized root mean square residual (SRMR). We tested the equivalence of coefficients (Hypotheses 1b, 1c, 2b, 6) using individual Wald tests for each hypothesis.

RESULTS

Mean age of the study participants was 60 years. The majority were female (71%) and white (84%). Table 2 shows the other demographic characteristics of the participants. Using Current Population Survey (CPS) data for July 2010 for comparison, the analytic sample was slightly older, more female, more White, more educated, had lower household income, were less likely to be employed, and were less likely to be married compared with the general population of adults ages 50-70 (U.S. Census Bureau, 2010).

The distributions and zero-order correlations between the key measures in the continuance model are shown in Table 3. All the correlations were positive and statistically significant at the p=0.05 level. Of five possible newsletters in the first half of the study, and six in the second half, the average number of newsletters opened was 2.5 and 2.3, respectively.

Table 3.

Sample Characteristics (n=4,570)

Characteristic Analytic Sample Current Population Survey
Age, years (M) 59.9 58.6
Female (%) 71.1 51.8
Race-ethnicity (% White) 86.4 83.3
Education (% some college or more) 79.1 56.4
Income (% >40k) 56.4 61.7
Employment (% part or full time) 46.7 57.5
Marital status (% married or domestic partner) 62.5 66.9
Open in a new tab

Note. Current Population Survey estimates are from July 2010, the time at which enrollment began for the study.

Figure 1 presents the standardized parameter estimates for the model paths of interest. The fit indices indicated good model fit based on the hypothesized model with the exception of the overall chi square test of model fit (χ2(165)=344.413, p<.0001). However, the chi square test is known to be sensitive to sample sizes greater than 200 (Hu & Bentler, 1999). The other fit indices pointed to good model fit to the data. The CFI was 0.990, TLI was 0.978, RMSEA was 0.015, and SRMR=0.026. An inspection of the modification indices did not reveal theoretically meaningful points of poor fit in the model. We tested additional alternative models that included direct paths between utilitarian beliefs and hedonic beliefs on use to examine the potential for model misspecification due to the omission of these paths. These paths were non-significant and were subsequently omitted from the final model presented here.

We summarize the results of the final model and hypotheses in Table 1. We found that the hypothesized constructs of utilitarian and hedonic beliefs were related to behavioral intention at all three time points (H1a). Both utilitarian and hedonic beliefs predicted intention to read later e-newsletters. In addition, we found evidence that the influence of utilitarian beliefs on behavioral intention had both direct and indirect paths, mediated by hedonic beliefs (RQ1). The hypothesis that utilitarian beliefs would remain as stable predictors of intention over time was supported (H1b). However, we did not find evidence to support the hypothesis that the relationship between hedonic beliefs and intention would strengthen over time (H1c). In fact, the results suggested that the relationship between hedonic beliefs and intentions weakened over time.

The hypotheses regarding user evaluation updating (H2a, H2b) were partially supported. In particular, the relationship between hedonic beliefs at time 2 and time 3 was significantly stronger than the relationship between hedonic beliefs at time 1 and time 2. While utilitarian beliefs were related across all time points, the strength of the relationship did not change significantly. Finally, behavioral intention at time 1 was not related to behavioral intention at time 2, but intention was related between time 2 and time 3. Prior use showed significant influences on later evaluations at the midpoint and final rounds (H3); however, the strength of these relationships was relatively small and in one case not significant.

The relationship between behavioral intention and subsequent use was significant across all time points, and the strength of the relationship was moderate (H4). However, there was no evidence to support the hypothesis that the relationship between intention and subsequent use weakened over time (H6). As expected, prior use predicted later use (H5), and prior use was a significantly stronger predictor of later use than intention (H7).

DISCUSSION

Based on the Reasoned Action Approach (Fishbein & Ajzen, 2010), dominant models of technology usage such as TAM (Davis, 1989) and UTAUT (Venkatesh, Morris, Davis, & Davis, 2003) focus on the importance of utilitarian beliefs as determinants of usage. The findings of our study, however, challenge this dominant perspective, suggesting that both utilitarian beliefs (e.g., this technology will be useful) and hedonic beliefs (e.g., this technology will be enjoyable) are important predictors of usage. Both types of beliefs had direct effects on intention to use the e-newsletter. Furthermore, our findings demonstrate that hedonic beliefs partially mediate the effect of utilitarian beliefs on intention. In other words, enjoyment is important because it accounts for a portion of the abovementioned effect. This indirect pathway is consistent with the results of other technology usage studies (Bhattacherjee, 2001; Limayem, Hirt, & Cheung, 2007; Tojib & Tsarenko, 2012). Taken together, these findings suggest that eHealth technologies should be not only informative, but also enjoyable.

We examined the predictive ability of utilitarian and hedonic beliefs over time. Specifically, we expected the link between utilitarian beliefs and intentions to remain stable over time. Our data support this hypothesis, suggesting that perceptions of utility are important determinants of usage even after consumers have gained experience with the technology. Surprisingly, our hypothesis regarding the predictive ability of hedonic beliefs over time was not supported. We expected the relationship between hedonic beliefs and intentions to grow stronger over time but we actually observed the opposite pattern. This hypothesis was based on a study conducted by Kim and Oh (2011), who found that hedonic beliefs did not predict adoption intentions but did predict continuance intentions. The theoretical rationale was that decisions to use a given technology are partially driven by prior perceptions of enjoyment. As consumers gain experience with the technology, these affective markers accumulate and become more salient during the decision-making process. Our findings challenge this theoretical perspective; the relationship between hedonic beliefs and intentions actually weakened as consumers gained experience with the e-newsletter. One possible explanation for this unexpected pattern is related to novelty seeking. As Venkatesh and colleagues (2012) argue, consumers initially pay attention to a product's novelty, potentially driving adoption and early use. The novelty decreases as time goes on, however. Since novelty is thought to contribute to perceptions of enjoyment, the relationship between hedonic beliefs and continuance intentions might be expected to weaken over time, as was the case in our dataset. Thus, at least in terms of explaining older adults’ usage of eHealth technology, the affective marker explanation should be reexamined.

Hypotheses related to user evaluation updating were mostly supported, with the exception of the relationship between behavioral intention at time 1 and time 2. The lack of a significant path is consistent with other studies (S. S. Kim & Malhotra, 2005) that have failed to observe a hypothesized association between intentions formed in the early stages of continuance. It appears that the anchoring and adjustment mechanism for intention formation becomes salient at later stages in the continuance process relative to belief formation. Although the relationship between utilitarian beliefs did not differ significantly over time (B=0.671, 0.720), this difference was not significant. This may explained by a ceiling effect, as the correlation between time 1 and 2 was already high. In contrast, the correlations between hedonic beliefs increased over time (B=0.133, 0.338), and were relatively smaller than correlations between utilitarian beliefs. This result suggests that hedonic beliefs may be more unstable and likely to change over time than utilitarian beliefs. Our study used a single measure of hedonic beliefs (enjoyment), but it is possible that other underlying dimensions, such as novelty or mood management, contribute to hedonic beliefs. These dimensions have been explored in communication theory in general (Vorderer, Klimmt, & Ritterfeld, 2004), and models of eHealth continuance could benefit from a similar exploration.

Past use had weak or nonsignificant relationships with subsequent evaluations of the newsletter. While the existence of significant paths supports theories of reciprocal relationships based on self-perception (Bem, 1972; Ouellette & Wood, 1998), the magnitude of the relationships relative to other pathways were less substantial than those found in other studies (Bajaj & Nidumolu, 1998, S.S. Kim & Malhotra, 2005). A potential explanation for this discrepancy is that the reciprocal pathway observed in past studies is an artifact of the way use was measured. In both Kim and Malhotra (2005) and Bajaj and Nidumolu (1998), past use was self-reported. As a result, participants may have reported corresponding beliefs to decrease cognitive dissonance between amount of use and favorability of beliefs. In this study, past use was measured objectively, so the possibility that participants experienced cognitive dissonance is less likely. Research on theories of continuance should measure use empirically in order to reconsider the existence and importance of pathways between past use and later evaluations.

Consistent with TRA (Fishbein & Ajzen, 2010), behavioral intention was a significant predictor of later behavior. Prior studies of technology continuance have found that intention only predicts use immediately post-adoption, but loses its predictive power over time as habit develops (Venkatesh et al., 2000; S.S. Kim & Malhotra, 2005). Accordingly, we expected the association between intention and later use to weaken over time. However, the magnitude of the relationship between intention and use was stable across waves. It is unclear how long the intention-behavior relationship will persist before a habit is formed and intention loses its predictive power. Studies of continuance models should include a range of intervals of use to examine this issue.

As expected, a strong habit formed over the course of the newsletter subscription and this is consistent with the claim that older adults may rely on automatic processing more heavily than deliberative processing (Cole et al., 2008; Peters, 2010). While past use was a significantly stronger predictor of subsequent use (B=0.804) compared with intention (B=0.120), the persistence of the intention-use relationship has two implications. First, intention is not a suitable proxy for measuring behavior. Second, given the relative predictive power of intention and past use, it is tempting to focus on automatic processing (e.g., past use predicting later use) rather than deliberative processing (e.g., beliefs predicting intention) for promoting eHealth continuance. However, beliefs and corresponding intentions were predictive of later use over and above the strong influence of prior use. While the predictive power of intention is comparatively weaker than past use, beliefs and corresponding intentions are modifiable, as opposed to past behavior, which is not. Accordingly, it will be important for eHealth designers to make a good “first impression” to form favorable utilitarian and hedonic beliefs, ensuring sustained use (Venkatesh et al., 2000, p. 51).

This study has several limitations. Due to the design of the original field experiment, the study population of an opt-in panel of subscribers was not representative of all older Americans, and future research should address this. However, this study represents one of few studies focusing on older adults, who likely have greater health needs than younger adults, and who are known to use the Internet for health information less frequently than younger adults (Fox, 2011b). In addition, the level of dropout in the study was substantial, with little more than half of enrolled subscribers completing the final survey at time 3. However, given the one-year length of follow-up, this level of dropout was not unexpected. Similar longitudinal studies with higher response rates had a follow-up period of 2-6 months (S. S. Kim & Malhotra, 2005; Venkatesh & Brown, 2001).

There were also limitations in the measurement of continuance constructs. First, the study relied on the measure of opening the monthly emails as a proxy of actual usage of the eHealth newsletter. However, the original field study showed that the targeted health behaviors were influenced as a function of newsletter opens. That is, the more individuals opened the newsletter, the more likely they were to change their health behavior, suggesting that opens are in fact a suitable proxy for use (Hornik et al., 2012). Furthermore, use of email alerts is recognized as an important behavior that is widespread (Fox, 2011a) and that may improve engagement and long-term use of eHealth technologies (see for example Robroek, Lindeboom, & Burdorf, 2012). Second, the survey items utilized in this study were not operationalized and validated a priori as measures of the continuance predictors of utilitarian beliefs, hedonic beliefs, or behavioral intentions. Despite this, the survey items were closely related to similar measures based on earlier theories of technology adoption reviewed in the introduction. In future research, we advise designing and pre-testing measures that may better capture the key variables of continuance predictors and assessing if the findings from this present study could be replicated.

CONCLUSION

Use of eHealth technologies, such as health information on the Internet, is widespread (Fox, 2011a) but not well understood in terms of its predictors. This study presents a preliminary effort to use technology adoption and continuance models from the IT literature to predict use of an eHealth technology. The results have at least three novel contributions to continuance theories. First, inclusion of both utilitarian and hedonic beliefs as constructs is necessary for predicting continued use of consumer eHealth technologies among older adults. Second, the results challenge the significance of the relationship between past use and later evaluations, suggesting that self-perception may not play a substantial role in eHealth continuance. Finally, both habit formation and intention play a role in predicting eHealth usage over time, suggesting that intention is not a suitable proxy for behavior, and efforts to promote continuance should focus not only on habit, but also on forming positive utilitarian and hedonic beliefs.

Table 4.

Descriptive statistics and correlation matrix of model constructs

Correlation matrix
Constructs Mean SD 1 2 3 4 5 6 7 8 9 10 11
1. BI1 4.37 0.82 1.00
2. BI2 4.20 0.95 0.47 1.00
3. BI3 4.24 0.94 0.42 0.64 1.00
4. U1 4.10 0.76 0.55 0.41 0.43 1.00
5. U2 4.14 0.75 0.45 0.62 0.52 0.54 1.00
6. U3 4.21 0.73 0.43 0.55 0.64 0.51 0.59 1.00
7. H1 4.28 0.80 0.76 0.46 0.45 0.61 0.49 0.51 1.00
8. H2 4.18 0.87 0.50 0.77 0.59 0.49 0.70 0.57 0.54 1.00
9. H3 4.22 0.87 0.48 0.60 0.76 0.47 0.54 0.69 0.52 0.65 1.00
10. USE1-2 2.53 1.83 0.13 0.18 0.07 0.07 0.10 0.10 0.08 0.16 0.12 1.00
11. USE2-3 2.28 2.40 0.17 0.26 0.25 0.11 0.15 0.20 0.13 0.24 0.25 0.65 1.00
Open in a new tab

Note. All correlations are significant at the p=0.05 level.

Acknowledgements

This work was supported by funding from the National Cancer Institute at the National Institutes of Health's Center of Excellence in Cancer Communication (CECCR) located at the Annenberg School for Communication, University of Pennsylvania [P20-CA095856-07S1]. Its contents are solely the responsibility of the authors and do not necessarily represent the official view of the National Cancer Institute or the National Institutes of Health.

Footnotes

Current Addresses of Authors Andy S.L. Tan Penn's Center of Excellence in Cancer Communication Research Annenberg School for Communication University of Pennsylvania 3620 Walnut St. Philadelphia, PA 19104, John L. Christensen Department of Communication College of Liberal Arts and Sciences University of Connecticut 850 Bolton Road, U-1085 Storrs-Mansfield, CT 06269

REFERENCES

  1. Bajaj A, Nidumolu SR. A feedback model to understand information system usage. Information & Management. 1998;33:213–224. [Google Scholar]
  2. Bem DJ. Self-perception theory. Advances in Experimental Social Psychology. 1972;6:1–62. [Google Scholar]
  3. Bhattacherjee A. Understanding information systems continuance: an expectation-confirmation model. MIS Quarterly. 2001;25(3):351–370. [Google Scholar]
  4. Bhattacherjee A, Barfar A. Information technology continuance research: current state and future directions. Asia Pacific Journal of Information Systems. 2011;21(2):1–18. [Google Scholar]
  5. Bolton RN, Drew JH. A Longitudinal Analysis of the Impact of Service Changes on Customer Attitudes. Journal of Marketing. 1991;55(1):1–9. [Google Scholar]
  6. Cline RJW, Haynes KM. Consumer health information seeking on the internet: the state of the art. Health Education Research. 2001;16(6):671–692. doi: 10.1093/her/16.6.671. [DOI] [PubMed] [Google Scholar]
  7. Cole C, Laurent G, Drolet A, Ebert J, Gutchess A, Lambert-Pandraud R, Peters E. Decision making and brand choice by older consumers. Marketing Letters. 2008;19(3-4):355–365. [Google Scholar]
  8. Davis FD. Perceived usefulness, perceived ease of use, and user acceptance of information technology. MIS Quarterly. 1989;13(3):319–340. [Google Scholar]
  9. Enders CK. The performance of the full information maximum likelihood estimator in multiple regression models with missing data. Educational and Psychological Measurement. 2001;61(5):713–740. [Google Scholar]
  10. Fishbein M, Ajzen I. Predicting and changing behavior: the reasoned action approach. Psychology Press; New York: 2010. [Google Scholar]
  11. Fox S. Health Topics. Internet & American Life Project. 2011a Retrieved from http://pewinternet.org/Reports/2011/HealthTopics.aspx.
  12. Fox S. The Social Life of Health Information. Internet & American Life Project. 2011b Retrieved from http://pewinternet.org/Reports/2011/Social-Life-of-Health-Info.aspx.
  13. Hogarth RM, Einhorn HJ. Order effects in belief updating: The belief-adjustment model. Cognitive Psychology. 1992;241(1):1–55. [Google Scholar]
  14. Hornik RC, Mello S, Forquer H, Tan ASL, Johnson M, Rusko J, Schwartz JS. Results from a randomized controlled trial testing the effects of routine health information exposure on cancer prevention and screening behaviors.. Paper presented at the 98th Annual Convention of the National Communication Association; Orlando, FL. 2012. [Google Scholar]
  15. Hu LT, Bentler PM. Cutoff Criteria for Fit Indexes in Covariance Structure Analysis: Conventional Criteria Versus New Alternatives. Structural Equation Modeling-a Multidisciplinary Journal. 1999;6(1):1–55. [Google Scholar]
  16. Jasperson JS, Carter PE, Zmud RW. A comprehensive conceptualization of post-adoptive behaviors associated with information technology enabled work systems. MIS Quarterly. 2005;29(3):525–557. [Google Scholar]
  17. Karahanna E, Straub DW, Chervany NL. Information technology adoption across time: a cross-sectional comparison of pre-adoption and post-adoption beliefs. MIS Quarterly. 1999;23(2):183–213S. [Google Scholar]
  18. Kim B, Oh J. The difference of determinants of acceptance and continuance of mobile data services: A value perspective. Expert Systems with Applications. 2011;38(3):1798–1804. [Google Scholar]
  19. Kim D, Chang H. Key functional characteristics in designing and operating health information websites for user satisfaction: an application of the extended technology acceptance model. International Journal of Medical Informatics. 2007;76(11-12):790–800. doi: 10.1016/j.ijmedinf.2006.09.001. [DOI] [PubMed] [Google Scholar]
  20. Kim SS, Malhotra NK. A longitudinal model of continued IS use: an integrative view of four mechanisms underlying postadoption phenomena. Management Science. 2005;51(5):741–755. [Google Scholar]
  21. Kline RB. Estimation Principles and Practice of Structural Equation Modeling. 3rd edition Guilford press; New York: 2010. pp. 115–177. [Google Scholar]
  22. Legris P, Ingham J, Collerette P. Why do we people use information technology? A critical review of the technology acceptance model. Information & Management. 2003;40:191–204. [Google Scholar]
  23. Lemire M, Pare G, Sicotte C, Harvey C. Determinants of Internet use as a preferred source of information on personal health. International Journal of Medical Informatics. 2008;77(11):723–734. doi: 10.1016/j.ijmedinf.2008.03.002. [DOI] [PubMed] [Google Scholar]
  24. Leslie E, Marshall AL, Owen N, Bauman A. Engagement and retention of participants in a physical activity website. Preventive Medicine. 2005;40(1):54–59. doi: 10.1016/j.ypmed.2004.05.002. [DOI] [PubMed] [Google Scholar]
  25. Limayem M, Hirt SG, Cheung CMK. How habit limits the predictive power of intention: the case of information systems continuance. MIS Quarterly. 2007;31(4):705–737. [Google Scholar]
  26. Luo MM, Chea S, Chen JS. Web-based information service adoption: A comparison of the motivational model and the uses and gratifications theory. Decision Support Systems. 2011;51(1):21–30. [Google Scholar]
  27. Muthén LK, Muthén BO. Mplus User's Guide. Seventh Edition ed. Muthén & Muthén; Los Angeles, CA: 1998-2012. [Google Scholar]
  28. Newman DA. Longitudinal modeling with randomly and systematically missing data: A simulation of ad hoc, maximum likelihood, and multiple imputation techniques. Organizational Research Methods. 2003;6(3):328–362. [Google Scholar]
  29. Or CK, Karsh BT. A systematic review of patient acceptance of consumer health information technology. Journal of the American Medical Informatics Association. 2009;16(4):550–560. doi: 10.1197/jamia.M2888. [DOI] [PMC free article] [PubMed] [Google Scholar]
  30. Ouelette JA, Wood W. Habit and intention in everyday life: the multiple processes by which past behavior predicts future behavior. Psychological Bulletin. 1998;124(1):54–74. [Google Scholar]
  31. Peters E. The aging consumer: Perspectives from psychology and economics. Psychology Press; New York, NY: 2010. Aging-related changes in decision making. pp. 75–101. [Google Scholar]
  32. Robroek SJW, Lindeboom DEM, Burdorf A. Initial and sustained participation in an Internet-delivered long-term worksite health promotion program on physical activity and nutrition. Journal of Medical Internet Research. 2012;14(2):e43. doi: 10.2196/jmir.1788. [DOI] [PMC free article] [PubMed] [Google Scholar]
  33. Ryan EB, Butler RN. Communication, aging, and health: Toward understanding health provider relationships with older clients. Health Communication. 1996;8(3):191–197. [Google Scholar]
  34. Silvestre AL, Sue VM, Allen JY. If you build it, will they come? The Kaiser Permanente model of online health care. Health Affairs (Millwood) 2009;28(2):334–344. doi: 10.1377/hlthaff.28.2.334. [DOI] [PubMed] [Google Scholar]
  35. Southwell BG. On the Need for a Life-Span Approach to Health Campaign Evaluation. Health Communication. 2010;25(6-7):525–528. doi: 10.1080/10410236.2010.496701. [DOI] [PubMed] [Google Scholar]
  36. Sparks L. An introduction to cancer communication and aging: Theoretical and research insights. Health Communication. 2003;15(2):123–131. doi: 10.1207/S15327027HC1502_1. [DOI] [PubMed] [Google Scholar]
  37. Tojib D, Tsarenko Y. Post-adoption modeling of advanced mobile service use. Journal of Business Research. 2012;65(7):922–928. [Google Scholar]
  38. Turner M, Kitchenham B, Brereton P, Charters S, Budgen D. Does the technology acceptance model predict actual use? A systematic literature review. Information and Software Technology. 2010;52(5):463–479. [Google Scholar]
  39. U.S. Census Bureau DataFerrett, Current Population Survey. 2010 Jul; [Google Scholar]
  40. Venkatesh V, Brown SA. A longitudinal investigation of personal computers in homes: adoption determinants and emerging challenges. MIS Quarterly. 2001;25(1):71–102. [Google Scholar]
  41. Venkatesh V, Morris MG, Ackerman PI. A longitudinal field investigation of gender differences in individual technology adoption decision-making processes. Organizational Behavior and Human Decision Processes. 2000;83(1):33–60. doi: 10.1006/obhd.2000.2896. [DOI] [PubMed] [Google Scholar]
  42. Venkatesh V, Morris MG, Davis GB, Davis FD. User acceptance of information technology: Toward a unified view. MIS Quarterly. 2003;27(3):425–478. [Google Scholar]
  43. Venkatesh V, Thong JYL, Xu X. Consumer Acceptance and Use of Information Technology: Extending the Unified Theory of Acceptance and Use of Technology. MIS Quarterly. 2012;36(1):157–178. [Google Scholar]
  44. Verheijden M. Rates and determinants of repeated participation in a web-based behavior change program for healthy body weight and healthy lifestyle. Journal of Medical Internet Research. 2007;9(1):e1. doi: 10.2196/jmir.9.1.e1. [DOI] [PMC free article] [PubMed] [Google Scholar]
  45. Vorderer P, Klimmt C, Ritterfeld U. Enjoyment: At the heart of media entertainment. Communication Theory. 2004;4:388–408. [Google Scholar]
  46. WHO eHealth. 2012 Retrieved from http://www.who.int/topics/ehealth/en/
  47. Yoon C, Cole CA, Lee MP. Consumer decision making and aging: Current knowledge and future directions. Journal of Consumer Psychology. 2009;19(1):2–16. [Google Scholar]

RESOURCES