Susceptibility to Spear-Phishing Emails: Effects of Internet User Demographics and Email Content

Abstract

Phishing is fundamental to cyber attacks. This research determined the effect of Internet user age and email content such as weapons of influence (persuasive techniques that attackers can use to lure individuals to fall for an attack) and life domains (a specific topic or aspect of an individual’s life that attackers can focus an emails on) on spear-phishing (targeted phishing) susceptibility. One-hundred young and 58 older users received, without their knowledge, daily simulated phishing emails over 21 days. A browser plugin recorded their clicking on links in the emails as an indicator of their susceptibility. Forty-three percent of users fell for the simulated phishing emails, with older women showing the highest susceptibility. While susceptibility in young users declined across the study, susceptibility in older users remained stable. The relative effectiveness of the attacks differed by weapons of influence and life domains with age-group variability. In addition, older compared to young users reported lower susceptibility awareness. These findings support effects of Internet user demographics and email content on susceptibility to phishing and emphasize the need for personalization of the next generation of security solutions.

1. Introduction

Phishing emails are a type of targeted email attack where social engineers lure the recipient into performing specific actions such as clicking on a malicious link, opening a malicious attachment, or visiting a web page and entering their personal information (APWG, 2014; Hong, 2012). These attacks are appealing, not only because they are simple and low cost, but also because they make attack attribution difficult (Bradley, 2011; Carr, 2011). As a result, phishing is a fundamental component of many cyber attacks and is often used as a first step in advanced persistent threats (APTs; James, 2005; Singer & Friedman, 2014). Susceptibility to phishing, including spear-phishing (a more targeted variation of phishing, where usually the victim is addressed by name¹), in older adults is a critical and growing public health concern but is currently largely overlooked in the scientific literature. There are several factors that may make older adults a particular target for such attacks: (i) Older adults are the fastest growing segment of the U.S. population, in part due to the ‘baby-boomer’ generation (National Center for Chronic Disease Prevention and Health Promotion, 2009; United States 2010 Census, 2010); (ii) Older adults often have accumulated financial assets over a lifetime, which they increasingly manage online (Perrin & Duggan, 2015); (iii) Many older adults occupy powerful positions in finances and politics and are confronted with a variety of decision-making situations in cyber space (e.g., clicking on a link in an email and visiting web sites); (iv) Finally, general cognitive-processing capacities and sensitivity to deception decline with age, while self-reported trust increases (Johnson, 1990; Mata, Josef, Samanez-Larkin, & Hertwig, 2011; Mather, 2006; Tentori, Osherson, Hasher, & May, 2001; Verhaeghen & Salthouse, 1997).

Some previous survey-based studies showed that older adults were not more likely to fall for fraud than young adults (Lichtenberg, Sugarman, Paulson, Ficker, & Rahman-Filipiak, 2016; Ross, Grossmann, & Schryer, 2014; Sugiura, 2013; Wood, Liu, Hanoch, & Estevez-Cores, 2015). However, these findings referred to more general forms of fraud (e.g., “Have you been the victim of financial fraud in the past five years (Yes, No)?”). As Ross et al. (2014) suggested, lack of age effects in susceptibility across some types of fraud does not mean that there are no age-related differences in any type of fraud (e.g., phishing). Also, compared to survey-based studies that largely depend on self- reflection about past experience with fraud, a field experiment can directly capture behavioral response to attempted fraud. Thus, we propose that a behavior-based methodology is particularly advantageous when determining susceptibility to phishing “in the wild” as well as among older adults, considering self-report response biases among older adults (e.g., positivity bias in autobiographical memory; Fernandes, Ross, Wiegand, & Schryer 2008; Tomaszczyk & Fernandes, 2012).

In addition to Internet user characteristics such as their age as potential risk factors to phishing susceptibility, the specific content of a phishing email may contribute to the effectiveness of the attack, also in interaction with user demographics. The literature suggests a variety of techniques that can be applied to lure users into clicking on malicious email links or procuring personal information. In particular, Cialdini (2007) and Hadnagy (2010) proposed seven psychological principles or weapons of influence that can be applied in phishing emails to lure individuals to fall for the attack. For example, to make the opportunities mentioned in an email seem more valuable, attackers will make those opportunities seem rare (scarcity). In the present study, we used the term ‘weapons of influence’, instead of ‘principles of influence’ from Cialdini (2007), to particularly reflect the malicious intention of attackers who use those strategies in phishing.

In addition to weapons of influence, attackers can focus an email on a specific topic or aspect of an individual’s life (life domain; Baltes, 1987), which may be particularly relevant to the recipient. For example, they can create an email as an advertisement for some medication (health), which potentially appeals to certain demographics (e.g., older adults).

Based on the considerations above and to fill a significant research gap, we conducted a field experiment (user study) over a period of 21 days to investigate susceptibility to phishing (i.e., whether individuals clicked on links in phishing emails) and susceptibility awareness (i.e., individuals’ self-reported likelihood of clicking on links in phishing emails) among young and older Internet users. In addition, we examined the extent to which young and older Internet users’ susceptibility and susceptibility awareness to phishing emails varied by email content (weapons of influence and life domains).

The study protocol was approved by the Institutional Review Board (IRB) at the University of Florida. The study took place at the participants’ homes. During the study interval, 100 young and 58 older Internet users received, without their knowledge, simulated phishing emails from our study team. These emails systematically varied weapons of influence and life domains and contained a web link (URL), which led participants to an accompanying web façade created by our group. A browser plug-in developed for this study recorded all web sites visited by the user during the study. The plug-in also tracked whether users clicked on the embedded links in the simulated phishing emails, which was our central dependent variable and indicated participants’ susceptibility to the phishing emails. On the last study day, users reviewed 21 previously unseen simulated phishing emails and indicated how likely they would click on the link in each email. This measure served as the indicator of participants’ susceptibility awareness to the phishing emails.

We found that older women were particularly susceptible to phishing. In addition, young users’ susceptibility decreased across the course of the intervention, while older users’ susceptibility did not decrease. The effectiveness of phishing emails varied depending on the weapons of influence and life domains. In particular, susceptibility was highest for scarcity and legal emails and lowest for social proof and financial emails. Further, the relative effectiveness of specific weapons of influence differed between young and older users. While young compared to older users showed greater susceptibility to scarcity and authority emails, older compared to young users showed greater susceptibility to reciprocation and liking emails. Older users showed lower susceptibility awareness than young users. In addition, susceptibility awareness varied by the weapons of influence and life domains. In particular, susceptibility awareness was highest for scarcity and legal emails and lowest for social proof and health emails.

Currently, the literature on deception and aging is scarce, especially in the context of cybersecurity. The majority of current research on decision-making in aging focuses on financial or health decisions and is conducted in laboratory settings or via surveys (Peters, 2006; Samanez-Larkin & Knutson, 2015; Westerman & Davies, 2000). Further, current human factors of phishing research has largely focused on general education and anti-phishing training (Caputo, Pfleeger, Freeman, & Johnson, 2014), but not on the identification of particularly vulnerable groups among diverse demographics for designing more tailored intervention approaches (for an exception see Neupane et al., 2018). To our knowledge, this study is the first behavioral measurement of aging effects on deception related to phishing emails in an ecologically valid context.

This paper starts with a review of related work on phishing as well as on age-related changes in trust and susceptibility to deception. We then provide the theoretical background for the key constructs of weapons of influence and life domains. The literature review concludes with a summary of our central research hypotheses. Next, the paper outlines our study methodology and framework, including details about the Phishing Internet Task (PHIT), developed to assess susceptibility to phishing emails. We then report detailed results, before presenting concluding remarks.

2. Related Work

2.1. Phishing

Phishing is a type of cyber social engineering attack, through which adversaries lure the recipient into clicking on a malicious link, opening a malicious attachment, or visiting a web page, procuring the user’s personal information. The security of individual Internet users and organizations, all the way up to nation-states, can be affected by such attacks. These attacks constitute a widespread threat; they are increasingly combined with malware and used as gateways for future attacks, such as APTs (James, 2005; Fisher, 2011; Singer & Friedman, 2014; Wrightson, 2014). Phishing can leverage emails, instant messaging, social media, and even Quick Response (QR) codes; called QRishing (Vidas, Owusu, Wang, Zeng, Cranor, & Christin, 2013). QRishing encourages mobile users to scan unauthenticated QR codes from various places, which directs users to visit malicious websites or install malware. Current protection methods propose the filtering of malicious emails before users see them (usually via machine learning methods) or support decision-making by training users in recognition of suspicious emails (Fette, Sadeh, & Tomasic, 2007; Sheng et al., 2009; Netcraft Toolbar, 2010; Zhang, Egelman, Cranor, & Hong, 2007). For example, Anti-phishing Phil, which teaches an organization’s employees good habits to avoid phishing via a game, is one example of such an effort (Sheng et al., 2007). PhishGuru was developed as an embedded training system for identifying phishing in emails (Kumaraguru, Cranor, & Mather, 2009). Phishing training faces many challenges, not the least of which is that people tend to forget the learned material and fall for the similar malicious emails again a short time after the training (as early as a few weeks). For example, an investigation into the efficacy of anti-phishing exercises in the context of a large organization found that the training had no significant effect on the likelihood that users would click on phishing email links (Caputo et al., 2014). Instead, users typically either clicked on all links or clicked on none, independent of the training.

These previous intervention studies suggest a considerable inter-individual difference in susceptibility to phishing. Some empirical work investigated demographic characteristics of individuals with high susceptibility to phishing and found that women compared to men were more susceptible (Halevi, Memon, & Nov, 2015; Sheng et al., 2010). In addition, individuals with a higher capability of deliberative processing, a decision-making process based on reflective reasoning rather than intuitive thinking (Evans, 2008, 2010), showed better performance in the differentiation of phishing emails from legitimate emails (Jones, Towse, Race, & Harrison, 2019). Survey-based studies examined the susceptibility to phishing as a function of personality traits and found that users with higher conscientiousness showed higher likelihood for falling for phishing (Halevi, Memon, & Nov, 2015). Some studies also examined susceptibility to phishing as a function of age and found that users between the ages of 18 and 25 years were more susceptible to phishing than other age groups (Kumaraguru et al., 2009; Sheng et al., 2010). These studies considered individuals up to middle age, but not older users (60 years and older). There is some indication that a “golden age” of decision-making occurs in adults aged around 55 years old, which is characterized by high levels of fluid intelligence (e.g., processing speed, working memory) and crystallized intelligence (e.g., experience; Agarwal, Driscoll, Gabaix, & Laibson, 2009; Samanez-Larkin, 2013). Thus, the middle-aged group may not be particularly at risk for such attacks. However, as we will discuss in more detail below, older users may be particularly vulnerable to phishing but have been under- investigated so far; thus, older users need to be explicitly considered in empirical investigations (Garg, Lorenzen-Huber. Camp, & Connelly, 2012).

Importantly, previous user studies on cybersecurity did not consider the effect of the content of phishing emails on their effectiveness (Benenson, Gassmann, & Landwirth, 2017; Halevi, Memon, & Nov, 2015; Jagatic, Johnson, Jakobsson, & Menczer, 2007; Mohebzada, El Zarka, BHojani, & Darwish, 2012; Uebelacker & Quiel 2014; but see Butavicius, Parsons, Pattinson, & McCormac, 2016). One exception is a study by Ferreira and Lenzini (2015) that used qualitative content analysis to identify five weapons of influence in phishing emails (i.e., authority, social proof, liking/similarity/deception, commitment/reciprocation/consistency, and distraction). Also, using a semi-structured interview, Whitty (2013) found that similar weapons of influence were commonly used in online dating romance scam. However, these previous qualitative studies did not determine the relative effectiveness of specific principles of persuasion in luring users to fall for attacks. Uebelacker and Quiel (2014) proposed a theoretical model attempting to link weapons of influence used in phishing to personality traits of individuals who had fallen for attacks. However, as of yet, their theoretical framework has not been empirically supported. This lack of a solid body of empirical work is particularly surprising, considering that certain phishing emails are more convincing than others. To systematically address variations in effectiveness, the present study manipulated the phishing email content. In particular, we created a set of simulated phishing emails that varied in weapons of influence and life domains and examined young and older users’ susceptibility as a function of these email content characteristics.

Next, we review the literature on trust and susceptibility in aging. We then introduce the two main concepts, weapons of influence and life domains as two factors that we propose to contribute to the effectiveness of phishing emails.

2.2. Trust and Susceptibility in Aging

Despite a lack of direct investigation into susceptibility, multiple lines of research provide support for the hypothesis that older adults may be more susceptible to phishing than young adults. For example, as individuals age, perceived trust increases (Zebrowitz, Boshyan, Ward, Gutchess, & Hadjikhani, 2017; Zebrowitz, Franklin, Hillman, & Boc, 2013), while sensitivity to untrustworthy information declines (Castle et al., 2012; Ebner, Bailey, Horta, Joiner, & Chang, 2015; Ruffman, Murray, Halberstadt, & Vater, 2012). Furthermore, older compared to young adults were more likely to show trust in interaction with others (e.g., more generous investment in trust games; Bailey, Ruffman, & Rendell, 2013; Bailey et al., 2015; 2016). These findings are in line with the well-documented age-related positivity bias, and more specifically reduced attention and memory to negative relative to positive information, in aging (Carstensen, & Mikels, 2005; Mather & Carstensen, 2005; Reed, Chan, Mikels, 2014). Also, older compared to young adults showed impaired ability in recognizing lies and had difficulty discriminating between trustworthy and untrustworthy faces (Castle et al., 2012; Ruffman et al., 2012). Furthermore, older compared to young adults showed reduced brain activation in insula to cues of untrustworthiness (Castle et al., 2012). In addition, older adults who had experienced financial exploitation in real life showed reduced cortical thickness in anterior insula when compared with individuals who had experienced attempts for financial exploitation but had not fallen for it (Spreng et al., 2016). Together, these findings suggest that age-related cognitive and functional brain changes contribute to an age-related decrease in sensitivity to untrustworthy information (see also Ebner et al., 2018; Gavett et al., 2017). Older compared to young adults may focus on potential gains proposed in phishing emails and may overlook cues of deception, rendering them more susceptibility to such attacks. This hypothesis is also supported by above-referenced evidence that reliance on deliberative processing during decision making decreases with age (Jones, Towse, Race, & Harrison, 2019; Peters et al., 2007).

In addition, age-related change in personality may make older compared to young adults more susceptible to phishing. Both cross-sectional and longitudinal evidence suggests an increase in conscientiousness with age (Helson & Kwan, 2000; Terracciano, McCrae, Brant, & Costa, 2005). Previous work has shown that individuals with higher conscientiousness were more susceptible to phishing (Halevi, Memon, & Nov, 2015). Thus, greater conscientiousness with age may render older adults more susceptible to phishing. A recent study found that older adults with a personality trait to trust things that are unproven or unlikely to be true (i.e., credulity; Pinsker, McFarland, & Pachana, 2010) were more likely to be persuaded and fall for fraud (Shao, Du, Lin, Li, Li, & Lei, 2019). Thus, older adults may be more likely to believe the information in phishing emails and to become victims of phishing.

Furthermore, it is plausible that older compared to young adults may be more persistent in their susceptibility to phishing. This assumption is based on evidence from a recent study showing that, in a trust game, older adults maintained high evaluations of trustworthiness for their partners (i.e., trustee in the game) even after they experienced those partners to breach their trust (Suzuki, 2016). In contrast, young adults more readily adjusted their trustworthiness evaluations based on their partners’ behavior. This reduced adjusting of trustworthiness evaluations in older adults is in line with research showing that age reduced the ability to update the value of objects in response to online feedback about the objects (Eppinger, Hammerer, & Li, 2011; Samanez-Larkin & Knutson, 2015). Therefore, it is possible that older compared to young adults are more persistent in their susceptibility to phishing and show less adjustment in their trusting behavior over time.

Considering the above evidence, the present research set out to address the following research question (Q1): Do young and older Internet users differ in their susceptibility to phishing emails? To address this question we tested the following hypotheses: (H1a) Compared to young users, older users will be particularly susceptible to phishing emails. (H1b) Young users will show a decrease in their susceptibility to phishing emails as the study proceeds, while older users’ susceptibility will persist throughout the study.

2.3. Principles (Weapons) of Influence

Humans often apply heuristics which can be beneficial in promoting fast and frugal actions, but can also result in faulty decision-making (Ferreira & Lenzini, 2015; Gigerenzer & Gaissmaier, 2011; Kahneman & Egan, 2011; Pachur, Suter, & Hertwig, 2017). In the context of persuasion, Cialdini (2007) proposed that humans have fixed behavioral patterns triggered by certain events, which may be considered the psychological correlate of sensorimotor reflexes. These triggered patterns of behavior can be effective for decision-making, but they can also be exploited by adversaries to lure individuals into a behavior that is counter their best interest. Cialdini called the events triggering these behavioral patterns psychological principles of influence. He defined six principles (called weapons of influence in this paper): authority, commitment, liking, reciprocation, scarcity, and social proof. A seventh principle, perceptual contrast, serves similar functions and was added based on Hadnagy’s work on social engineering (Hadnagy, 2010).

The principle of authority states that humans tend to comply with requests made by figures of authority, such as law enforcement personnel, lawyers, doctors, or politicians. Therefore, sending emails in the name of known authorities may be effective in luring recipients into clicking on a malicious link in an email.

The commitment principle proposes that once humans have taken a stand, they feel pressured to behave consistently with that stance. For example, Alice as the mother of two college students may feel devastated every time she learns about a new case of campus violence. Observing this, an adversary can target Alice by sending her an email asking her to click on a link to support a program to prevent campus violence.

The liking principle assumes that humans tend to comply with requests from people they like or with whom they share similarities. Consider Mary, an older adult who is also active in her local church. Per the liking principle, she will feel more at ease accepting a request from Dan, who is her age contemporary and is also a member of the same church, than a request from John, who is much young and attends a local university.

The perceptual contrast principle refers to the way people perceive the relative difference between two things that are presented in tandem. When the first is perceived as relatively worse than the second, people tend to perceive the second as better than it actually is. For example, a car salesperson may present the most expensive car in the lot first, whereby the subsequent car prices may seem more reasonable to the buyer than they would have been otherwise. The salesperson never intended to sell the first car, but wanted to sell the second car at a price that the buyer would not have otherwise considered. That is, with perceptual contrast, the attacker introduces a bias into the victim’s perception of the intrinsic value or cost of an action based on its relation to the initially presented option.

The reciprocation principle is based on the notion that humans tend to repay, in kind, what another person has provided them. An adversary can use this principle to lure a user into installing malware on a computer by offering a free gift attached (e.g., a pdf for a travel guide).

The principle of scarcity proposes that the perceived value of an object, experience, or opportunity is greatly increased when its availability is limited. Therefore, an adversary may tempt a user into clicking on a malicious link or website to get a “soon-to-expire” discount.

Finally, the social proof principle leverages the human tendency to avoid mistakes by acting according to what the majority of other people are doing. An adversary can exploit this principle by advertising, via a malicious link, an offer from a company “voted” as one of the top 10 in the country.

No previous study has systematically compared the relative effectiveness of these seven weapons of influence on phishing or has looked at age differences in their effectiveness. We, therefore, posed two research questions relevant to weapons of influence: (Q2a) Which weapon(s) of influence is/are particularly effective? (Q2b) Does the relative effectiveness of weapons of influence vary between young and older users? Given the lack of previous research, we did not formulate directional hypotheses regarding the relative effectiveness of weapons of influence.

2.4. Life Domains

To increase their impact on recipients, adversaries can construct a phishing email within a particular life domain context. Personal relevance of specific life domains varies by demographics and differs from person to person. In the present study, we adopted Baltes’ lifespan development theoretical framework (Baltes, 1987) and considered six life domains phishing emails can refer to financial, health, ideological, legal, security, and social.

Financial emails focus on money, discounts, or offers (e.g., an email offering a discounted vacation package). Health emails focus on mental and physical wellness, including sports and medication (e.g., an email offering a medication). Ideological emails focus on principles and beliefs (e.g., an email inviting someone to sign a petition). Legal emails focus on the law, such as emails about being arrested or sued, or an invitation to appeal a parking ticket. Security emails focus on safety, such as neighborhood watch or cybersecurity (e.g., an email invite to explore the Facebook page of a neighborhood). Finally, social emails focus on social interaction (e.g., an email inviting an individual to learn more about an upcoming concert in town).

Life-span developmental theory states that the relevance of specific life domains changes with age because individuals at different points in their lifespan pursue different goals (Baltes, 1987; Ebner, Freund, & Baltes 2006). In particular, within the meta-theoretical framework of life-span theory Socioemotional Selectivity Theory posits that the priority of personal goals changes as people age (Carstensen, Isaacowitz, & Charles, 1999; Isaacowitz, 2006; Mather & Knight, 2005), with older adults showing greater prioritization of social over information-seeking goals (Fredrickson & Carstensen 1990; Fung, Carstensen, & Lang, 2001). Further, older compared to young adults may be particularly interested in information relevant to health, as they experience a variety of health issues (Oh & Cho, 2015; Flynn, Smith, & Freese, 2006). Similar to weapons of influence, no previous study has systematically investigated the relative effectiveness of phishing emails focusing on these six life domains or has looked at age differences in their effectiveness. We therefore posed two research questions relevant to life domains: (Q3a) Which life domain(s) is/are particularly effective? (Q3b) Does the relative effectiveness of life domains vary between young and older users? Given the lack of previous research, we did not formulate directional hypotheses regarding the relative effectiveness of life domains.

2.5. Susceptibility Awareness

In addition to actual behavioral susceptibility to phishing emails, we were also interested in exploring age differences and effectiveness of weapons of influence and life domains in susceptibility awareness to phishing emails (i.e., the self-reported likelihood of users to click on the links embedded in phishing emails). Based on above reported evidence that age is associated with increased perceived trust (Zebrowitz et al., 2013; Zebrowitz et al., 2017), while sensitivity to untrustworthy information declines with age (Castle et al., 2012; Ebner et al., 2015; Ruffman et al., 2006; 2012), we posed the following research questions regarding susceptibility awareness: (Q4a) Do young and older Internet users differ in their susceptibility awareness to phishing emails? (Q4b) Does susceptibility awareness vary by weapon(s) of influence and does age moderate this variability? (Q4c) Does susceptibility awareness vary by life domain(s) and does age moderate this variability?

3. The Present Study

To address our research questions, we conducted a user study over a period of 21 days, which took place at the participants’ homes for increased ecological validity and allowed us to assess susceptibility to phishing emails behaviorally. During the 21-day study period, unbeknownst to them, we exposed young and older Internet users to experimentally controlled simulated phishing emails, which varied on the use of weapons of influence and life domains. Participants’ Internet activities on the computer they had registered for the study were monitored over the 21 study days via a pre-installed browser extension (plugin), which recorded (with participants’ consent) all URLs the participants visited during the study period, including URLs embedded in our simulated phishing emails. The clicking on the links in our simulated emails indicated that the participant would have fallen for the attack had it been real. Further, to assess susceptibility awareness to the phishing emails, on the last study day users reviewed 21 simulated phishing emails that they had not seen during the study intervention and indicated how likely they would click on the link in each email.

4. Methods

4.1. Participants

Our study comprised of 100 young (M = 21.7 years, SD = 4.1, 56.0% female) and 58 older (M = 61.7 years, SD = 6.8, 43.1% female) users. Table 1 summarizes participants’ demographics, health, and Internet usage by age group. Participants were recruited from North Central Florida via fliers (18 participants), handouts (10 participants), newspaper ads (10 participants), a lab-internal participant pool (12 participants), the university participant pool (58 participants), HealthStreet (a university-affiliated community recruitment and outreach program; 16 participants), and online venues (e.g., Facebook, Craigslist, ResearchMatch.org; 5 participants), and other mechanisms (e.g., word of mouth; 29 participants). Young participants recruited through the university participant pool were compensated with course credit; all other participants were compensated with a $50 gift card.

Table 1.

Demographic, health, and Internet usage information in young and older participants.

	Young	Older	Age-Group
	M (SD)/%	M (SD)/%	Differences
Years of Education (nyoung = 81, nolder = 33)	14.24 (3.57)	16.30 (2.79)	t(122) = −2.97
Annual Income (nyoung = 91, nolder = 54)
< $40,000	36.3	33.3	X2(3) = 61.83
$40,000 - $70,000	0.0	31.5
> $70,000	0.0	18.5
N/A	63.7	16.7
Race/Ethnicity (nyoung = 91, nolder = 52)
American Indian or Alaskan Native	0.0	5.8	X2(5) = 36.68
Asian	29.7	1.9
Black/African American	8.8	9.6
Native Hawaiian	0.0	1.9
Hispanic	19.8	1.9
White	39.6	76.9
Physical Health (nyoung = 91, nolder = 52)	7.69 (1.56)	7.58 (1.95)	t(141) = 0.39
Mental Health (nyoung = 91, nolder = 52)	8.02 (1.53)	8.54 (1,50)	t(141) = 1.86
Marital Status (nyoung = 91, nolder = 53)
Single	68.1	15.1
In a relationship, but not married	26.4	15.1
Married	4.4	47.2	X2(4) = 69.09
Divorced/Separated	1.1	15.1
Widowed	0.0	7.5
Internet Usage/Week (nyoung = 81, nolder = 49)
≤ 4 hours	21	20.4
4–9 hours	34.6	44.9	X2(2) = 1.56
≥ 10 hours	44.4	34.7
Average number of URLs Visited/Day (nyoung = 99, nolder = 57)	256.46 (166.14)	181.52 (115.85)	t(154) = 3.01

Note. Participants with negligible income/unemployed reported “N/A”. Physical Health “Please rate your general physical health” and Mental Health “Please rate your general mental health/mood” were measured by single self-report items on a scale ranging from 1 = poor to 10 = excellent.

For inclusion in the data analysis, participants needed to complete 21 days of study intervention and have at least 50% recorded daily email checking activities (see details below). These criteria excluded 33.1% (n = 78) of the total 236 volunteers originally enrolled. Excluded individuals were equally distributed across age group and gender. Among participants who did not complete the study, 22 gave an explicit reason for their discontinuation: Nine dropped out due to insufficient time for completing the study; ten experienced technical difficulties (e.g., plugin installation, audio response recording); and three became suspicious of the nature of the study.

4.2. Procedure

The university IRB approved the study protocol. We determined study eligibility through a brief phone interview, after informed verbal consent. To be eligible, participants had to be between 18–39 years (young participants) or 60–90 years (older participants). Eligible participants needed to engage in online activities (e.g., web browsing, email checking) regularly, but not necessarily daily. In addition, older users received the Telephone Interview for Cognitive Status (TICS; Brandt, Spencer, & Folstein, 1988) to screen for cognitive impairment (cut off score < 30). We also administrated the Brief Test of Adult Cognition by Telephone (BTACT; Tun & Lachman, 2006) to measure various cognitive abilities (see Ebner et al., 2018 for details).

For eligible participants, informed written consent was obtained via an online form, disclosing study procedures, the minimal study risk, and data protection mechanisms². To ensure naturalistic behavior, participants were told that the purpose of the study was to examine daily Internet use. Participants were not informed about the simulated phishing emails until debriefing at study closure. Next, with the help of a video tutorial, and as needed instructions over the phone, participants installed a plugin that allowed the recording of all URLs participants visited during the study, consequently recording whether participants clicked on the link embedded in each stimulated phishing email from the PHIT (see details in the Appendix). Each link embedded in each simulated phishing email was accompanied by a façade web page created for this study (see the Appendix for details). Clicking on the link embedded in the phishing emails constituted our central dependent variable, susceptibility to phishing emails.

On the first day of the 21-day intervention, participants filled out a demographic questionnaire via an online survey. Using an online survey link, on each study day, participants also completed the short Positive and Negative Affective Schedule (PANAS; Watson, Clark, & Tellegen, 1988) to access their daily mood before they engaged in the PHIT. For the PHIT, participants were asked to spend one hour of study-related Internet activities daily. These activities included: (i) reading an informative source (e.g., a news media website), (ii) reading entertainment/social network sources (e.g., Facebook), (iii) engaging in unstructured browsing, and (iv) time checking/handling emails from the email account the user had registered with for the study, with 15 minutes on each of these tasks. This variety of study-related activities assured capture of a range of possible everyday computer use, and most importantly, assured a sufficient base rate of email checking during the study period. During the 21-day study period, unbeknownst to them, participants received 21 simulated phishing emails (one per day) in the email account they had registered for the study (see the Appendix for details). A Phishing Manager was developed for this study and used to send the simulated phishing emails. This system was managed via five time-based process schedulers.

On the last study day, participants were asked to review 21 simulated phishing emails which they had not seen during the study intervention but which were comparable to the ones they had seen (see details below about counterbalancing). Participants were asked to evaluate, for each of these emails, how likely they would click on the link in the email (i.e., likelihood), how interested they were in what each email had to say (i.e., interested), and how convincing they found the content of the email (i.e., convincing). Participants used a 5-point scale (1 = not at all, 5 = very much) for their responses. We used self-reported likelihood of clicking on an email link as an indicator of users’ susceptibility awareness (e.g., higher self-reported likelihood of clicking on the link suggested higher susceptibility awareness). In addition, participants also completed a survey to measure socio-emotional functions including trust, state anxiety, and trait anxiety see Ebner et al., 2018 for details). During the debriefing, we disclosed the true purpose of the study. As part of the debriefing process, we asked participants about their thoughts regarding the study purpose and the need to use deception. Ninety-nine percent of the participants indicated that they understood the necessity of concealing the true study purpose.

4.3. Model Specification

Accommodating for the hierarchical structure of the data (i.e., email clicks nested within participants), we conducted three multilevel logistic regression models to address our research questions. Susceptibility, operationalized as the clicking on an email link during the 21-day study period (0 = not clicked, 1 = clicked), served as dichotomous outcome variable in all three models. The first multilevel logistic regression model tested the extent to which susceptibility differed between young and older users (Q1a-b). This model used age group (categorical variable; 0 = young, 1 = older), gender (categorical variable; 0 = male, 1 = female), time in study (continuous variable; range: 1–21), and the interactions between these variables as predictors.

The second multilevel logistic regression model tested the extent to which susceptibility varied as a function of the weapons of influence in young and older users (Q2a-b). This model used age group, gender, weapons of influence (categorical variable: 1 = Authority, 2 = Commitment, 3 = Liking, 4 = Perceptual Contrast, 5 = Reciprocation, 6 = Scarcity, 7 = Social Proof), and the interactions between these variables as predictors.

The third multilevel logistic regression model tested the extent to which susceptibility varied as a function of life domains in young and older users (Q3a-b). This model used age group, gender, life domains (categorical variable: 1 = Financial, 2 = Health, 3 = Ideological, 4 = Legal, 5 = Security, 6 = Social), and the interactions between these variables as predictors.

In an exploratory fashion, we conducted two multilevel regression models to test the susceptibility awareness as a function of age group, gender, weapons of influence, and life domains (Q4a-c). In these two models, susceptibility awareness, operationalized as the self-reported likelihood of clicking rating (1 = not at all, 5 = very much), served as continuous outcome variable. The first multilevel regression model tested the extent to which susceptibility awareness varied as a function of weapons of influence and considered the moderation of age group and gender. In particular, this model used age group, gender, weapons of influence, and the interactions between these variables as predictors. The second multilevel regression model tested the extent to which susceptibility awareness varied as a function of life domains and considered the moderation of age group and gender. That is, this model used age group, gender, life domain, and the interactions between these variables as predictors.

In all five models, the Wald statistics was used to indicate the significance of each effect. Adopting a top-down approach (i.e., from higher-order interaction to main effects), we removed non-significant effects from each model to determine the most parsimonious model to reflect the data. We used Stata 14.0 for all statistical analyses and applied p = 0.05 as the significance threshold.

5. Results

Overall, susceptibility to phishing was high, with 43.3% of users clicking on at least one of the 21 simulated phishing email links and 11.9% of users clicking on more than one link during the 21 -day study period.

5.1. Q1a-b: Susceptibility in Young and Older Internet Users

The interaction between age group, gender, and time in study (χ²(1) = 2.92, p = 0.09) and the interaction between gender and time in study (χ²(1) = 3.53, p = 0.06) were marginally significant. The interactions between age group and gender (χ²(1) = 5.41, p = 0.02) and between age group and time in study (χ²(1) = 4.77, p = 0.03) were significant, suggesting that the age-group difference in susceptibility varied depending on the participants’ gender and their time in the study (Table 2). The main effect of time in study (χ²(1) = 14.76, p < 0.001) was also significant, but the main effects of age group (χ²(1) = 2.49, p = 0.11) and gender (χ²(1) = .29, p = 0.59) were not significant.

Table 2.

Results of Multilevel Logistic Regression Model for Susceptibility to Phishing as a Function of Age Group, Gender, and Time in Study (Q1a-b).

Fixed Effects	B (SE)	Odds Ratio
Age Group	−1.21(0.46)	0.3
Gender	−0.36(0.29)	0.7
Time in Study	−0.1(0.03)	0.9
Age Group × Gender	0.98(0.49)	2.68
Age Group × Time in Study	0.09(0.04)	1.09
Random Effects	Variance (SE)
Intercept	0.35(0.2)

Notes. Dependent variable: Susceptibility (0 = not clicked, 1 = clicked). The final model included age group (0 = young, 1 = older), gender (0 = male, 1 = female), time in study, the interaction between age group and gender, and the interaction between age group and time in study as independent variables. Bold print indicates significant effects at p < 0.05.

In sum, as shown in Figure 1, older women (4.1%) constituted the most susceptible group to the simulated phishing emails compared to the other demographics (young men: 3.4%, young women: 2.4%, older men: 2.3%). Also, as shown in Figure 2, young users’ susceptibility declined with time in the study, while in older users’ susceptibility remained stable over the study period.

Figure 1.

Predicted susceptibility to phishing in young and older men and women. Error bars represent 95% confidence intervals.

Figure 2.

Predicted susceptibility to phishing in young and older users as a function of time in study (in days). Error bars represent 95% confidence intervals.

5.2. Q2a-b: Susceptibility as a Function of Weapons of Influence in Young and Older Users

The interaction between age group, gender, and weapons of influence (χ²(5) = 1.33, p = 0.93) and the interaction between gender and weapons of influence (χ²(6) = 8.37, p = 0.21) were not significant. The interaction between age group and weapons of influence was significant (χ²(6) = 12.61, p = 0.05), suggesting that young and older users differed in their susceptibility to specific weapons of influence. The interaction between age group and gender was also significant (χ²(1) = 6.37, p = 0.01). In addition, the main effect of weapons of influence was significant (χ²(6) = 26.18, p < 0.001), suggesting that users differed in their susceptibility to phishing depending on the specific weapons of influence applied (Table 3).

Table 3.

Results of Multilevel Logistic Regression Model for Susceptibility to Phishing as a Function of Age Group, Gender, and Weapons of Influence (Q2a-b).

Fixed Effects	B (SE)	Odds Ratio
Age Group	−0.9 (0.5)	0.41
Gender	−0.36 (0.29)	0.70
Age Group × Gender	0.99 (0.49)	2.69
Weapons of Influence (vs. Authority)
Commitment	−1.38 (0.58)	0.25
Liking	−1.38 (0.58)	0.25
Perceptual Contrast	−0.33 (0.39)	0.72
Reciprocation	−1.67 (0.57)	0.19
Scarcity	0.2 (0.32)	1.22
Social Proof	−1.38 (0.52)	0.25
Age Group × Weapons of Influence (vs. Authority)
Older × Commitment	0.65 (0.91)	1.92
Older × Liking	1.38 (0.81)	3.96
Older × Perceptual Contrast	0.14 (0.63)	1.15
Older × Reciprocation	1.98 (0.78)	7.21
Older × Scarcity	−0.03 (0.72)	0.97
Older × Social Proof	0.24 (0.82)	1.27
Random Effects	Variance (SE)
Intercept	0.37 (0.21)

Notes. Dependent variable: Susceptibility (0 = not clicked, 1 = clicked). The final model included age group (0 = young, 1 = older), gender (0 = male, 1 = female), weapons of influence (reference category = authority), the interaction between age group and gender, and the interaction between age group and weapons of influence as independent variables. Bold print indicates significant effects at p < 0.05.

In particular, as shown in Figure 3 users were significantly more likely to click on links in emails using scarcity (5.3%) than those using reciprocation (2.3%), liking (2.1%), commitment (1.5%), and social proof (1.3%). Also, users were significantly more susceptible to emails using authority (4.4%) than those using commitment and social proof. In addition, perceptual contrast (3.4%) was significantly more effective than social proof. Furthermore, young users showed greater susceptibility to scarcity (young = 6.1%, older = 4.2%) and, to a lesser extent, authority (young = 5.1%, older = 3.6%) than older users. In contrast, older users showed greater susceptibility to reciprocation (young = 1.0%, older = 4.8%) and, to a lesser extent, liking (young = 1.4%, older = 3.6%) than young users.

Figure 3.

Predicted susceptibility to phishing as a function of weapons of influence in young and older users. Error bars represent 95% confidence intervals.

5.3. Q3a-b: Susceptibility as a Function of Life Domains in Young and Older Users

The interaction between age group, gender, and life domains was not significant (χ²(5) = 0.60, p = 0.99). In addition, the interactions between age group and life domains (χ²(5) = 2.31, p = 0.81) and between gender and life domains (χ²(5) = 5.35, p = 0.37) were not significant. The interaction between age group and gender was marginally significant (χ²(1) = 3.35, p = 0.07), and when we removed from the model the non-significant interactions between age group and life domains and between gender and life domain, the interaction between age group and gender was significant (χ²(1) = 4.09, p = 0.04). The main effect of life domains was also significant (χ²(5) = 43.73, p < 0.001), suggesting that users differed in their susceptibility to phishing depending on the specific life domains the attack applied (Table 4).

Table 4.

Results of Multilevel Logistic Regression Model for Susceptibility to Phishing as a Function of Age Group, Gender, and Life Domains (Q3a-b).

Fixed Effects	B (SE)	Odds Ratio
Age Group	−0.42 (0.32)	0.66
Gender	−0.36 (0.29)	0.70
Age Group × Gender	1.00 (0.49)	2.71
Life Domains (vs. Financial)
Health	0.87 (0.46)	2.39
Ideological	1.13 (0.47)	3.09
Legal	1.91 (0.44)	6.73
Security	0.52 (0.53)	1.68
Social	0.61 (0.52)	1.85
Random Effects	Variance (SE)
Intercept	0.36 (0.21)

Notes. Dependent variable: Susceptibility (0 = not clicked, 1 = clicked). The final model included age group (0 = young, 1 = older), gender (0 = male, 1 = female), life domains (reference category = financial), and the interaction between age group and gender as independent variables. Bold print indicates significant effects at p < 0.05.

As shown in Figure 4, users showed greater susceptibility to legal emails (6.7%) than emails in any other life domain. Susceptibility was also greater for ideological emails (3.2%) than financial emails (1.1%). Susceptibility to health (2.5%), security (1.8%), and social (2.0%) emails was comparable and was not significantly different from either financial or ideological emails. This pattern of findings was equivalent across young and older users.

Figure 4.

Predicted susceptibility to phishing as a function of life domains in young and older users. Error bars represent 95% confidence intervals.

5.4. Q4a-c: Susceptibility Awareness as a Function of Weapons of Influence/Life Domains in Young and Older Users

The interaction between age group, gender, and weapons of influence (χ²(6) = 7.84, p = 0.25) was not significant. In addition, the interactions between age group and gender (χ²(1) = 0.33, p = 0.57), age group and weapons of influence (χ²(6) = 8.19, p = 0.22), and gender and weapons of influence (χ²(6) = 10.63, p = 0.10) were not significant. The main effect of gender was also not significant (χ²(1) = 0.01, p = 0.93). The main effect of weapons of influence was significant (χ²(6) =78.03, p < 0.001; Table 5); users showed the highest susceptibility awareness for emails using authority (2.11). Susceptibility awareness for authority emails was higher than susceptibility awareness of emails using any of the other weapons of influence except scarcity. In addition, users showed higher susceptibility awareness for emails using scarcity (2.03) and commitment (1.91) than for emails using perceptual contrast (1.75), reciprocation (1.78), and social proof (1.56). Susceptibility awareness for emails using social proof was the lowest and was lower than susceptibility awareness for emails using any of the other weapons of influence (Figure 5A).

Table 5.

Results of Multilevel Regression Model for Susceptibility Awareness to Phishing as a Function of Age Group, Gender, and Weapons of Influence (Q4a-b).

Fixed Effects	B (SE)
Age Group	−0.6 (0.32)
Gender	0.02 (0.29)
Weapons of Influences (vs. Authority)
Commitment	0.08 (0.11)
Liking	−0.29 (0.08)
Perceptual Contrast	−0.26 (0.09)
Reciprocation	−0.14 (0.09)
Scarcity	−0.12 (0.08)
Social Proof	−0.47 (0.08)
Random Effects	Variance (SE)
Intercept	3.29 (1.22)

Notes. Dependent variable: Susceptibility awareness. The final model included age group (0 = young, 1 = older), gender (0 = male, 1 = female), and weapons of influence (reference category = authority) as independent variables. Bold print indicates significant effects at p < 0.05.

Figure 5.

Predicted susceptibility awareness to phishing as a function of (A) weapons of influence and (B) life domains in young and older users. 1 = not at all; 5 = very much. Error bars represent 95% confidence intervals.

The interaction between age group, gender, and life domains (χ²(5) = 5.50, p = 0.36) was not significant. In addition, the interactions between age and gender (χ²(1) = 0.33, p = 0.56), age group and weapons of influence (χ²(5) = 9.11, p = 0.10), and gender and weapons of influence (χ²(5) = 3.85, p = 0.57) were not significant. In contrast, the effect of life domains was significant (χ²(5) =13.61, p = 0.02; Table 6). That is, users showed the highest susceptibility awareness for legal emails (2.04) and susceptibility awareness for legal emails was significantly higher than susceptibility awareness for financial (1.83), security (1.79), and health (1.74) emails (Figure 5B).

Table 6.

Results of Multilevel Regression Model for Susceptibility Awareness to Phishing as a Function of Age Group, Gender, and Life Domains (Q5a-b).

Fixed Effects	B (SE)
Age Group	−0.60 (0.32)
Gender	0.02 (0.29)
Life Domains (vs. Financial)
Health	−0.15 (0.09)
Ideological	−0.31 (0.08)
Legal	−0.18 (0.09)
Security	−0.25 (0.08)
Social	−0.21 (0.08)
Random Effects	Variance (SE)
Intercept	3.29 (1.22)

Notes. Dependent variable: Susceptibility awareness. The final model included age group (0 = young, 1 = older), gender (0 = male, 1 = female), and life domains (reference category = financial) as independent variables. Bold print indicates significant effects at p < 0.05.

Finally, the main effect of age group was significant in both models (χ²(1) = 3.59, p = 0.058 and χ²(1) = 3.60, p = 0.058, respectively). That is, older users showed lower susceptibility awareness than young users.

6. DISCUSSION

By adopting a method with high ecological validity and by using a pool of simulated phishing emails, which systematically manipulated email content pertaining to weapons of influence and life domains, the present study examined susceptibility to phishing in young and older users and determined the extent to which susceptibility varied as a function of weapons of influence and life domains. The study generated several novel findings as discussed in the following.

First, we observed a high overall susceptibility to phishing, as more than 40% of users clicked on the link in at least one of the simulated phishing emails during the 21-day study period. Note that clicking on one phishing email can affect an individual’s computer and through that computer multiple other computers. In line with our findings, Benenson et al. (2017) observed that 20% of email users and 43% of Facebook users clicked on simulated phishing links. Suggesting even higher susceptibility, Jagatic et al. (2007) found a high success rate (72%) of simulated phishing emails (seemingly sent from known acquaintances) among college students. Similarly, Halevi et al. (2015) observed that more than 62% of employees fell for simulated phishing emails they believed were sent from their company’s IT manager. Participants in the present study received simulated phishing emails from unknown entities/others (e.g., local farmers market, parking authority, etc.). This methodological variability between studies may explain differences in susceptibility rates.

Some previous studies reported lower susceptibility than found in our research. For example, Mohebzada Zarka, Bhojani, and Darwish (2012) observed very low susceptibility across two large-scale phishing experiments. In their study, phishing emails were sent to 10,917 institutional email addresses, associated with students, alumni, faculty, and staff from a specific university. The success rate of phishing was about 9% in their first experiment and 2% in their second experiment. Of note, about half of the email addresses targeted in Mohebzada et al. were from alumni, for whom it was not clear if they still checked their university email inbox regularly. Thus, it is possible that the actual number of individuals who saw the simulated phishing emails was smaller than 10,917. In the present study, phishing emails were sent to participants personal email inbox of the account they had registered with in the study. In addition, participants were asked as part of the study to engage in daily Internet activities, including checking their emails. These design features were implemented to increase the likelihood that participants would actually see the simulated phishing emails. Also, Mohebazada et al. defined susceptibility as either inputting correct personal credentials (in Experiment 1) or as completing an online survey (in Experiment 2). This operationalization of susceptibility was different (and representative of not just falling for an attack in the form of clicking on a link but actually providing personal information) from the measure used in the present study (i.e., clicking on the link embedded in the simulated phishing email). This difference in operational definition of susceptibility across the two studies may explain the different susceptibility rates. Finally, Mohebazada et al. only used one phishing email (per experiment) and their observation window was short (i.e., 10 days in Experiment 1; 18 hours in Experiment 2) -- compared to our 21-day study intervention in which 21 phishing emails were sent -- which may also have contributed to the low susceptibility rate in their investigation.

The high susceptibility to phishing found in our study, in line with similarly high or higher susceptibility reported in previous work, emphasizes the significant threat posed by phishing emails and warrants the development of efficient detection and prevention tools as well as effective phishing education protocols. Intriguingly, in contrast to this overall high (behavioral) susceptibility in the present study, participants’ self-reported susceptibility awareness was low. This disparity between behavior and self-report was especially pronounced among older users and highlights the importance of behavioral field experiments (vs. self-report) to study susceptibility to phishing, especially among older adults. The lower susceptibility awareness in older than young users observed in our study is in line with evidence of increased trust perceptions and decreased sensitivity to cues of untrustworthiness in older relative to young adults (Castle et al., 2012; Ebner et al., 2015; Ruffman et al., 2006; 2012; Zebrowitz et al., 2013; Zebrowitz et al., 2017). The results are also in accord with an age-related increase in overconfidence in performance on cognitive tasks in older compared to young adults (Crawford & Stankov, 1996). Taken together, this pattern of findings suggests that the older adult demographic may be at particular risk for phishing attempts and constitute an important target for anti-phishing education and decision- supportive interventions in cyberspace.

Different from our expectations, however, we did not observe a main effect of user age on susceptibility as measured in the PHIT task. Rather, we found an age-by-gender interaction on susceptibility. In particular, older women, compared to the other demographic groups, showed the highest susceptibility to phishing. This finding is in line with previous research that showed greater susceptibility to phishing in women than men (Halevi, Memon, & Nov, 2015; Sheng et al., 2010). In their study, the gender effect in susceptibility was partially due to the lesser amounts of technical training and technical knowledge of women compared to men. Further, there is evidence of greater computer/Internet usage by young vs. older adults (Morris, Goodman, & Brading, 2007). Thus, older women may be particularly at risk given their limited experience and knowledge about computers and the Internet. In the present study, we did not specifically inquire about computer literacy and, thus, we cannot speak to this explanation. Future research is warranted to uncover possible contributing factors and delineate underlying mechanisms for this particularly high susceptibility to email phishing in older women (see also Ebner et al., 2018).

Interestingly, when comparing the age groups in their susceptibility over time across the study period, we detected a decrement in the young group’s susceptibility to phishing, while the older groups’ susceptibility remained stable across the 21 days. This finding is consistent with the literature on economic games, in which older adults have been found to continue to trust their partners even after experiencing a breach of trust from their partners, while young adults adjust their perceptions on their partner’s behavior (Suzuki, 2016). The continued susceptibility to phishing in older Internet users that we observed across the study duration may indicate an age-related impairment in learning-based adjustment in decision-making (Eppinger et al., 2011; Samanez-Larkin & Knutson, 2015) and calls for continued, as opposed to one time, education as well as development of “on-the-spot” phishing detection and warning tools as effective approaches in older adults.

Further qualifying age-group differences in susceptibility to phishing, we observed an age-moderation of the effectiveness of weapons of influence on susceptibility. In particular, while scarcity and authority were more effective in young than older users, reciprocation and liking were more effective in older than young users. These age-group differences in the relative effectiveness of weapons of influence may reflect age-related changes in personal motivations and interests, as suggested by Socioemotional Selectivity Theory (Carstensen, Isaacowitz, & Charles, 1999; Isaacowitz, 2006; Mather & Knight, 2005). In particular, this theory proposes that, because of the perceived limitation of future time left in life, older compared to young adults focus more on emotionally meaningful goals. This motivational shift is in line with evidence that loneliness and social isolation are of great concern in older adults, with the potential to result in devastating effects on health and well-being (Shankar, McMunn, Banks, & Steptoe, 2011). Specifically relevant in the present context, there is evidence that loneliness and negative affect are one of the factors contributing to fraud victimization in the elderly (Alves & Wilson, 2008; Ebner et al., 2018; Lichtenberg et al., 2016). Thus, compared to the other weapons, reciprocation and liking may be particularly effective weapons in phishing of older individuals.

We did not observe age-group differences in the relative effectiveness of life domains on susceptibility. Though health-related information becomes increasingly important in old age (Baltes, Lindenberger, & Staudinger, 1998), phishing emails related to health were not particularly effective in older users. It is possible that because older adults are already inundated with medication advertisements or offers via email (usually seen as spam or annoying advertisement), they might have become desensitized to emails pertaining to these topics. This could explain the low effectiveness of health-related emails in our study. Supporting this possibility, in an independent study we showed that older compared to young Internet users were more likely to receive spam emails referring to health in their everyday lives (Oliveira et al., 2019).

Regarding the relative effectiveness of weapons of influence, overall, scarcity and authority were the two most effective, while commitment and social proof were the two least effective, weapons of influence, in line with previous work in young Internet users (Butavicius et al., 2016). However, in contrast to our findings, the effectiveness of scarcity in this previous study was lower than that of authority and was comparable to that of social proof. Methodological differences between these two studies may explain these discrepancies in results. While the present study measured susceptibility to phishing emails in young and older Internet users via an ecologically valid behavioral design (the PHIT task), Butavicius and colleagues assessed university students’ ability to self-report on the safety of links in simulated emails. It is possible that individuals may be able to remind themselves that an opportunity to receive a scarce object at low cost may be a suspicious offer, especially when not personally involved and only presented to them in the form of a scenario; however, when they are unprompted and must decide whether to act on an offer, they may still fall for it, as our data suggest. Consistent with this speculation, participants in the present study showed higher susceptibility awareness to scarcity emails than authority emails, while there was no difference in susceptibility between emails using these two types of weapons in our behavioral paradigm.

Regarding the relative effectiveness of life domains, we observed that legal emails were the most effective for all users. In contrast, somewhat surprisingly, the least effective life domain was financial. This small effectiveness for financial emails may be due to a familiarity effect. In particular, frequent experiences with financial scams in everyday life may have transferred knowledge into the study and could be underlying “resilience” to these types of attacks. Consistent with this speculation, in our independent study of naturally received spam emails by young and older Internet users, financial spam emails were most prevalent while legal spam emails were least prevalent (Oliveira et al., 2019).

Findings from the present study demonstrated that susceptibility to phishing emails varied as a function of the weapons of influence used and the life domains the emails referred to. For example, our finding of high susceptibility for emails using authority and referring to legal matters aligns with work by Halevi et al. (2015) who found very high susceptibility (62%) to simulated phishing emails seemingly sent by the company’s IT manager (i.e., an authority) to company employees asking them to download a plug-in per the company’s regulations (i.e., a legal matter). In contrast, Benenson et al. (2017) observed a much lower susceptibility (20%) to simulated phishing emails that asked recipients to retrieve party pictures (i.e., liking, social) via clicking on an email-embedded link, in line with our findings that those weapons of influence and life domains are less effective than authority and legal. Thus, our findings invite a novel interpretation of previous findings in light of the importance of email content (specifically the use of weapons of influence and life domains) on susceptibility to phishing.

Many previous research on susceptibility to phishing was conducted in laboratory settings, possibly affecting the validity of the observation (Hong, Kelley, Tembe, Murphy-Hill, & Mayhorn, 2013; Wang, Herath, Chen, Vishwanath, & Rao, 2012). Also, the few existing field experimental approaches were limited to specific contexts, such as the university (Jagatic et al., 2007; Mohebzada et al., 2012) or the company (Halevi et al., 2015) contexts. These contextualized investigations limit the generalizability of the previous findings. In contrast, our study was designed towards broader ecological validity and generalizability by randomly sampling individuals of different ages and gender from a wider population and using a varied set of emails (applying several weapons of influence and a range of life domains).

Also, the small number of simulated phishing emails used in previous studies (i.e., one email per study; Halevi et al., 2015; Jagatic et al., 2007; Mohebzada et al., 2012) did not allow analysis of varying email content on susceptibility. Our study comprised multiple simulated phishing emails with varying content, created based on a large set of authentic spam emails from an independent sample of young and older Internet users (Oliveira et al., 2019). This approach allowed systematic manipulation of some variables (weapons of influence and life domains in the present study) and their consideration in the statistical analysis.

Previous studies showed lower susceptibility in middle-aged than young adults (Halevi, Memon, & Nov, 2015; Sarno et al., 2017; Sheng et al., 2010). This low susceptibility to phishing in middle-aged adults may be related to this group being in the “golden age” of their decision-making, given still relatively intact cognitive ability combined with affluent life experience (Agarwal et al., 2009; Samanez-Larkin, 2013). In accord with life-span developmental theory (Baltes, 1987), it is, however, also possible that middle-aged adults would show a particular susceptibility to some phishing emails, aligned with the personal goals most relevant in this life phase. For example, previous work found that, compared to young and older adults, middle-aged adults were more likely to become victims of romance scams (Whitty, 2015, 2018). Moving forward, it will be interesting to test whether phishing emails using ‘liking’ as the weapon of influence and/or referring to the life domain ‘social’ would be the “Achilles’ heel” for middle-aged adults.

It will also be relevant in future research to address the extent to which previous exposure to phishing may affect susceptibility to future attacks among men and women of different ages as well as to determine the extent to which number of phishing emails received on a regular base in real life determines overall susceptibility. Thus, we propose for future work (with adequate privacy and data protection procedures in place) to capture information such as spam email box volume, computer literacy, and Internet usage in the attempt to determine the impact of these factors on susceptibility to email phishing.

7. CONCLUSION

This paper generated novel findings regarding susceptibility to phishing as a function of Internet user demographics (age and gender) and email characteristics (use of weapons of influence and life domains to lure users into clicking on email links). The data showed that older women were the most susceptible demographic to phishing. Furthermore, the effectiveness of phishing differed by weapons of influence and life domains and, directly comparing the age groups, young users were most susceptible to scarcity, while older adults were most susceptible to reciprocation. We also observed an intriguing discrepancy between participants’ high behavioral susceptibility while low self-reported susceptibility awareness, which was particularly prominent among older users.

Current security training and warning solutions for Internet threats affecting end-users operate under the implicit assumption that one-size-fits-all. However, our work suggests that this is not the case. In fact, susceptibility, and therefore the efficiency of phishing, varied by user demographic; with older women particularly vulnerable to the attacks and older compared to young Internet users little aware of these risks. Susceptibility was also affected by the weapons of influence and life domains applied in the phishing emails, partly in an age-differential fashion. Based on these findings, we propose that security communication and education take user demographics and email content into consideration. Security education and warnings which attempt to accomplish too much and lack personalization (e.g., consider individual variables like age and gender) may not be effective. The novel perspective adopted in our research has promise to benefit the development of the next generation of defense solutions for Internet users, also with an eye towards improving user-friendliness and compliance, with the goal to enhance effectiveness of security measures.

CCS CONCEPTS.

• Empirical studies in HCI

• Social aspects of security and privacy

• Seniors

• Phishing

A. APPENDICES: Details of Study Procedure

A.1. Terminology and Definitions

Cron Job - Time-based process scheduler in Unix-like operating systems. These processes can be scheduled to perform commands or shell scripts at specific times or intervals.

MySQL - An open-source relational database management system.

Plugin - a software component/extension that adds a specific feature to an existing computer program. Certain software allows for programming ‘entry points’ where developers can interface with features and information of an existing program.

URL - Uniform Resource Locator - a reference to a web resource, colloquially a ‘web address’. A URL is a specific case of URI (Uniform Resource Indicator).

Wordpress - An open-source website content management system that allows for visual editing of website content.

A.2. Server Architecture and Study Framework

For user convenience and to allow broader and more representative participant recruitment, we implemented a browser plugin for various types and versions of popular browsers: Internet Explorer, Chrome, Safari, and Firefox. This plugin tracked all URLs visited by participants during the study period together with access timestamps. As shown in Figure 6A below, the plugin continuously sent to the Log Manager all URLs visited with their corresponding timestamps. This information was recorded in log files in the Log Manager. The plugin did not collect sensitive and personal information from participants, such as bytes transferred in network connections and file accesses, keys typed, or names of files accessed. Participants were identified solely by an ID, which reflected their age group (young, older) and gender (female, male).

Using a controlled timeline and systematic counterbalancing scheme (see Sending Simulated Phishing Emails), as shown in Figure 1A below, (1) a set of cron jobs triggered a software module, called Phishing Manager, which (2) retrieved participant and schedule information and simulated phishing emails from the database, and (3) sent simulated phishing emails to the users. The Phishing Manager also recorded presence or absence of study activity, and, as needed, sent reminder emails for Internet activities, and recorded the day of the study.

A Linux machine with Intel Xeon CPU E5–2650 0 2.00GH and 4GB of memory was used to run the server and the study software framework. A MySQL database on the server stored the four sets of simulated phishing emails and participant information for email personalization (e.g., first and last name and county of residence). We used WordPress to deploy and host façade web pages that accompanied the links in the simulated phishing emails. Participant log data occupied 3.04 MB in total and 23.38 KB per participant on average.

Figure 6.

Phishing Internet Task (PHIT). A. Overall framework of PHIT. (1) Daily cron jobs invoked the phishing manager to (2) fetch participant, schedule, and spear-phishing emails from the database, and (3) send spear-phishing emails to the participants. B. Sample façade web-page created for the study to accompany the link embedded in the spear-phishing email. C. Overview of cron job implementation and triggered events in phishing manager.

A.3. Development of Simulated Phishing Emails and Maintenance of Fake Email Accounts

In the past, there was a propensity for spam and phishing emails to arrive from senders of unknown domains (e.g., @hotfax.com). Recent phishing, however, has become more targeted and sophisticated and leverages established domains, better grammar, spelling, and targeted interests (Fossi et al., 2011). The goal of our study was to examine (i) user susceptibility to and (ii) efficiency of email content of this new generation of phishing. We, therefore, used mainstream email providers for sending of our phishing emails. In particular, we created 22 fake email accounts (11 with common American female names and 11 with common American male names as fictional owners) hosted by Gmail, AOL, and Outlook/Hotmail. To prevent these accounts from being blocked and the study emails from being flagged by the email providers, before the study, each of the accounts were maintained over two months of regular interpersonal communication by our study team. Once the data collection was underway, we continued to create content for these email accounts.

For the development of our simulated phishing emails, we referred to a large set of real-life spam emails that we had collected in a pilot study from an independent sample of young and older Internet users (Oliveira et al., 2019). None of the original emails from this independent study were used in the present study. Each email had similar word length (between 50–150 words) and followed a similar structure. In particular, the emails contained contextual information, such as current events, relevant to the geographical area (e.g., city, county) where this study was located. The emails were personalized, addressing participants by their names, had a unique sender, and contained a link that directed the user to one of 70 façade web pages that we had created for this study (see Figure 6B above for an example of a façade web page). All façade web pages were harmless and were hosted on seven web domains purchased for this study.

Each email used one of the seven weapons of influence and one of the six life domains. Three independent research assistants reviewed all emails to assure representation of only one weapon of influence and one life domain per email. In total, we created 84 phishing emails, which covered all possible combinations of weapons of influence and life domains. We evenly separate those 84 emails into four comparable sets (i.e., 21 emails in each set). Each participant was randomly assigned to one of these sets during the three-week study intervention and to another set in the survey on the last study day. This counterbalancing scheme considered age group and gender of the participant. The full set of simulated phishing emails can be retrieved at: https://github.com/danielaoliveira/Counter-Balanced-Emails---Weapons-of-Influence-and-Life-Domains.git

A.4. The Plugin

We developed a plugin for the four most popular browsers at the time of this study: Chrome, Firefox, Safari, and Internet Explorer. At installation of the plugin, a popup window asked for the participant’s age group, gender, and email address. The plugin created a unique ID for each participant based on age group and gender only. The email address was required and used only for the outgoing phishing emails. URL logs were associated only with the unique participant ID. The plugin could not track web activity if the participant was browsing in private mode. Thus, as part of the study enrollment and consent form, we instructed participants not to browse in private mode.

A.5. Sending Simulated Phishing Emails

The Phishing Manager, implemented in Java, was controlled by five cron jobs (C 1 - C 5). Figure 6C above shows an overview of the cron job implementation. Depending on the day of the week, participants received emails either at a random time of the day, or at a time as close as possible to the time they were engaged in study-related browsing activities.

C 1 ran every Sunday, Monday, Wednesday, and Friday at 9am and invoked the Phishing Manager to randomly select a time between 9am and 5pm to send a personalized phishing email to each active participant based on the counterbalancing scheme. C 2 ran every Tuesday, Thursday, and Saturday and invoked the Phishing Manager to track a participant’s web activities and send the personalized phishing email as soon as it detected Internet activity from the participant (see the Procedure in the main text for a description of the scheduled Internet activities participants engaged in as part of the study). C 2 invoked the Phishing Manager every hour from 1am to 10pm to check a participant’s Internet activity. At 10:30pm, C 3 invoked the Phishing Manager to send a phishing email to all users who had not yet received a phishing email due to lack of web activity. C 4 ran daily at 9am to invoke the Phishing Manager to send a reminder email if users were on Day 1, 7, 14, or 21 of the study. Finally, C 5 ran each day at 11:59pm invoking the Phishing Manager to generate a participant’s activity report. These activity reports did not include any personal identifiable information; they represented a summary of data already present in the log files. Based on these reports, our team contacted participants in a timely manner in case of lapsed study activity. In particular, we contacted participants via phone and email if either their Internet activity or daily survey was absent for three or more days. All cron jobs code can be retrieved at: https://github.com/danielaoliveira/Counter-Balanced-Emails---Weapons-of-Influence-and-Life-Domains.git