Abstract
Many people learn about smoking cessation through information on the Internet. Whether people with severe mental illnesses, who have very high rates of smoking, are able to use currently available websites about smoking cessation is unknown. The study reported here assessed whether four smoking cessation websites met usability guidelines and whether they were usable by smokers with severe mental illnesses. Four websites that appeared first on a Google search and represented an array of sponsors were selected. First, five experts rated the websites on adequacy of content in six areas and usability in 20 areas. Second, 16 smokers with severe mental illnesses performed two search tasks on the websites with researchers observing their searches and interviewing them regarding usability. One of the websites was rated by experts as acceptable for content and usability, but most of the participants were unable to navigate this website. The only website that was navigable received poor content ratings by experts. Four easily accessible websites did not meet the needs of smokers with severe mental illnesses. Although the Internet is a promising strategy to provide education about treatments, website developers must attend to the needs and capacities of multiple user groups.
Introduction
Although public health strategies have successfully reduced the prevalence of smoking in the general population [1–3], cigarette smoking is still common among the 10 million Americans who have severe mental illnesses [4]. Rates are 45.3–88% among people with schizophrenia [5–9], 58.1–90% among people with bipolar disorder [7–11] and 36.6–73% among people with major depressive disorder [7–11], whereas the rate of smoking in the general population is about 20% [3]. Corresponding to the high rates of smoking, high rates of smoking-related diseases have been found among people with severe mental illnesses [12], and recent studies have found that people with severe mental illnesses die up to 25 years earlier than the general population, due in large part to cardiovascular disease [13–15].
Quitting smoking reduces the morbidity and mortality due to tobacco-related diseases: even a 60-year-old lifelong smoker can gain 3 years of life expectancy by quitting [16]. Smokers who quit before age 35 have the same life expectancy as non-smokers [16]. People with severe mental illness can quit smoking, especially when they use evidence-based treatments (combined behavioral and pharmacologic therapies) [17–26], but smokers with severe mental illnesses rarely use these treatments, due to lack of interest [27, 28], lack of access [29], need for tailoring of treatments or other yet unknown reasons. Strategies are needed to reach people with severe mental illnesses and engage them into evidence-based smoking cessation treatments.
Information about smoking cessation and cessation treatments is available on the Internet, which is increasingly used by people with severe mental illnesses [30, 31]. Additionally, computerized and web-based smoking cessation interventions have been shown to be effective [32, 33]. But many people with severe mental illnesses have minimal computer experience [30, 31] and problems with cognitive deficits [34] that could impair their capacity to use standard websites, as well as low education or reading levels [35, 36] that could reduce their ability to understand the information presented on typical websites. Research has demonstrated that many people with severe mental illnesses have poor capacities to navigate within standard website structures and to understand typical website language [37–39]. This ability to navigate and understand a website is referred to as usability.
The US government has published recommendations for website design to improve usability [40], but whether currently available websites have followed these guidelines is uncertain. Two studies reported content and usability of smoking cessation information websites [41] or websites designed to provide smoking cessation interventions [42], but the websites were assessed by experts, rather than tested by the intended users, as is recommended for truly adequate assessment of usability [43–45]. Whether people with severe mental illnesses are able to navigate and understand the information on currently available websites about smoking and cessation is unknown.
The study reported here assessed whether four easily accessible Internet smoking websites had adequate content, met usability guidelines and, further, whether they were usable and comprehensible by smokers with severe mental illnesses.
Methods
In order to identify websites that would typically be accessed by people with severe mental illness, the researchers identified and briefly reviewed the first four websites that came up with a Google search (on 17 March 2010) using the phrase ‘quit smoking’. The four websites (www.becomeanex.org, www.pmusa.com, www.smokefree.gov and www.whyquit.com) were created by an array of sponsors: private foundations with commitments to reduce cigarette smoking (becomeanex and whyquit), tobacco industry (pmusa) and government (smokefree). They were clearly designed to provide education about smoking and support for cessation. The researchers decided these four websites were broadly representative of educational sites that would be accessed in real-world computer searches and chose to study them. The researchers used two strategies to assess the usability of these four smoking cessation websites for people with severe mental illnesses: expert ratings of content and usability and patient performance on navigation tasks [46].
First, five experts rated each website on adequacy of content in six areas that are important to a smoker with mental illness seeking to make a decision about smoking cessation treatment [47]: general information about smoking, motivational information regarding quitting, cessation treatment information, information relevant to mental illness and smoking, information relevant for racial and ethnic minorities and the absence of misinformation. The researchers included a category on information relevant to minorities since they access treatment at lower rates than Caucasians [48] and are therefore an important target for educational outreach. The five experts then assessed the usability of each website using a 20-item checklist developed from published usability guidelines for people with cognitive deficits [40] and from research observing people with severe mental illnesses while they use computer programs [38, 39].
Second, the researchers assessed the practical usability of these websites among 16 smokers with severe mental illnesses. After providing written informed consent, seven to nine participants performed two simple search tasks on each website. The researchers then interviewed participants regarding their perceptions of the usability of the websites. Participants were paid $35 for participation. The Dartmouth Medical School Committee for the Protection of Human Subjects approved the study.
Procedures
Expert website rating
Five experts rated the content and usability of the four websites using an index that was created for this study based on published guidelines [40] and previous research [38, 39] (see Tables I and II). The five experts included three smoking cessation researchers (P.G., M.S. and S.P.) and two smoking cessation clinicians who worked with people with severe mental illness (W.M. and T.D.). They reviewed the index and then logged onto each website and navigated the site for 30 min in order to complete the index for that website. This rating system simulated real-world use of the site and was expected to lead to some variability in site ratings, as the five experts may not have reviewed the same pages within each site. One of the experts also used two standardized measures, the Flesch–Kincaid grade level and reading ease tests, to assess sites for readability.
Table I.
Content areas of smoking cessation websites
General information |
Effects of smoking on health |
What nicotine is and its effects |
Nicotine withdrawal |
How to cope with quitting |
Motivational information |
Risks of smoking |
Benefits of quitting |
Evaluates your own beliefs and/or attitudes |
Individualized assessment of smoking with user feedback provided |
Gives information designed for different level of readiness to quit |
Information tailored to age, gender, race or illness |
Evidence-based treatments information |
Cessation medicines |
Description of what the medicines do |
Description of how to take the medicines |
Pros and cons of each medicine (side effects, ease of use and efficacy) |
Group counseling or support for cessation |
Individual counseling or support for cessation |
Mental illness information |
Quitting when you have a mental illness |
Psychotropic medication when smoking is reduced or stopped |
Use of patient vignettes of people who have a mental illness |
Race/ethnicity information |
Menthol cigarettes |
Health risks of smoking to African Americans |
Health risks of smoking to Latinos |
Other information for African Americans |
Other information for Latinos |
Role of religion or family in smoking cessation |
Misinformation |
No misinformation was given |
Table II.
Usability index
1. | Homepage has three or fewer groupings of text/images |
2. | Homepage includes basic instructions on how to use a mouse |
3. | Homepage includes basic instructions on how to navigate the site |
4. | Text is black on light colored background |
5. | Primary navigation menu (or table of contents) is on left panel of page |
6. | Navigation buttons are obvious—button is labeled and font size is large |
7. | Site map is available on web site |
8. | Site map looks similar to table of contents |
9. | Important topic information appears at top of page |
10. | Descriptive headings provide cues about page organization |
11. | Scrolling for important content is not required |
12. | Easy and obvious return to homepage from any other page on the web site |
13. | Site logo is repeated on every page |
14. | Web site contains no more than two layers |
15. | Each page provides feedback so user knows where they are |
16. | Links are represented as text (rather than image) |
17. | Newly opened windows occupies no more than half of the screen |
18. | Windows for links have prominent control to close |
19. | Use of movement to draw attention does not last more than a few seconds |
20. | Image or a video loading time is less than 10 seconds |
Usability of websites by people with severe mental illnesses
Researchers recruited 16 adult smokers who were in treatment for a severe mental illness at a large urban community mental health center. Based on guidance from literature on usability testing, the researchers aimed to have eight smokers test each website [44]. These smokers had previously participated in a study in which they reported their computer use history. They were recruited to ensure that half had used the computer five or more times and half less than five times, yielding a range of computer experience.
The 16 study participants who tested the websites for usability were primarily middle-aged African American men with schizophrenia who had less than 12 years of education on average (see Table III). Demographics and diagnosis were not associated with computer experience. After providing informed consent, participants reviewed two randomly selected websites from the four under study (in order to minimize fatigue). Randomization was blocked according to the participant’s level of computer experience (less than five times or five or more times).
Table III.
Participants demographics
Total group | |
n = 16 | |
Mean ± SD or N (%) | |
Age | 49.9 ± 8.9 |
Gender (male) | 10 (62.5) |
Race (African American) | 15 (93.8) |
Highest grade completed | 11.5 ± 1.9 |
Used a computer <5 times | 8 (50) |
Diagnosis | |
Schizophrenia | 12 (75) |
Bipolar | 1 (6.3) |
Major depression | 1 (6.3) |
Other | 2 (12.5) |
Participant usability tasks and ratings
Usability task
Participants searched for information to answer two task questions. The task questions were presented to seven to nine participants in random order for each of two randomly selected sites. The tasks were designed to mimic a real-world search. The first task was to find information about how smokers can cope without smoking cigarettes. The second was to locate information about how the medicine varenicline can help a person stop smoking. These tasks were derived from questions commonly asked in clinical practice. For each task, researchers explained the task and encouraged participants to spend up to 5 min looking for the relevant information. (The experts were able to complete these search tasks in less than 1 min.) The researchers observed each participant use the computer to navigate through the sites, recorded whether they were able to complete the task and interviewed them briefly about their perception of the website. At the end of each session, the researcher debriefed participants and answered all of their questions about the website searches.
Measures
The content index contains 30 items in six categories (Table I). The first three and the final categories are important information for all users and the other two categories assess the websites’ relevance to disparity populations. Within a category, individual items assessed the extent of pertinent information. For example, under general information, items assessed the adequacy of information about the effect of smoking on health, nicotine and it’s effects, nicotine withdrawal and quitting advice (0 = no information on the topic, 3 = adequate information and 5 = extensive information on the topic). If at least three of the five raters scored a website three or higher on an item, the researchers designated that item as being adequately covered.
The usability index includes 20 items that measure a variety of characteristics that render a website easy to use (Table II) [40]. Examples of items include a clear navigation menu and important information visible without need for scrolling (response options included ‘present 0, 25, 50, 75 or 100% of the time’). The researchers defined an item as adequate if more than half of the experts rated it as present at least 75% of the time on the website.
The researchers used three items from the perceived usefulness and ease of use scale, an adapted 15-item semi-qualitative questionnaire [49], to assess participant-specific reactions to and opinions about the websites. The response options ranged from 1 = ‘totally disagree’ to 5 = ‘totally agree’.
Results
Expert reviews
The experts rated the informational content of the websites within six areas of information. Table IV shows the proportion of adequate items within each topic area. Researchers rated Smokefree.gov highest for content, but only three of the six topic areas were adequate (rated adequate or higher on a majority of items). The content of the other three websites was clearly inadequate. None of the websites had adequate content regarding mental illness and smoking nor on racial minorities and smoking. The experts rated the usability of the websites in 20 areas. They rated Smokefree.gov highest for usability, but only 50% of the items were adequate (Table IV). The usability of the other three websites was also clearly inadequate. All of the websites used Arial font sizes between 8.5 and 10 (large font is recommended). Smokefree.gov had the largest font (Arial size 10) and was the most readable of the websites based on lower grade level text and higher readability scores on the medication page. The home pages of becomeanex and PMUSA also scored high on readability.
Table IV.
Expert ratings of four websites for content, usability and readability
Becomeanex.org | Pmusa.com | Smokefree.gov | Whyquit.com | |
Adequacy of content (% of items that are adequate) | ||||
General information | 75.0 | 25.0 | 75.0 | 100.0 |
Motivational | 16.7 | 0.0 | 66.7 | 33.3 |
Evidenced-based treatments | 50.0 | 16.7 | 50.0 | 0.0 |
Mental illness | 0.0 | 0.0 | 0.0 | 0.0 |
Racial minority | 0.0 | 20.0 | 20.0 | 0.0 |
No misinformation | 100.0 | 100.0 | 100.0 | 0.0 |
Usability score (% of items that are adequate) | ||||
30.0 | 45.0 | 50.0 | 25.0 | |
Readability scorea | ||||
Homepage | ||||
Flesch–Kincaid grade level | 7.8 | 4.6 | 4.9 | 17.5 |
Flesch reading ease scoreb | 69.3 | 83.0 | 63.5 | 16.9 |
Medication information page | ||||
Flesch–Kincaid grade level | 11.4 | 11.5 | 6.9 | NA |
Flesch reading ease scoreb | 43.5 | 46.5 | 67.3 | NA |
NA, Not applicable because there was no medication information.
Sample text taken from websites on 4/23/10.
Higher indicates greater ease.
The researchers defined a website as usable if a majority of participants could complete the task (i.e. find the requested information). Among the participants with minimal computer experience, none of the websites was usable (see Table V). Among the participants who had used a computer five or more times, pmusa.com was usable and was rated highest on ease of use, but this website was rated poorly by participants with less computer experience and this was the website that experts rated poorly for content.
Table V.
Participant usability task scores and perception of four smoking cessation websites
Website |
Becomeanex.org
|
Pmusa.com
|
Smokefree.gov
|
Whyquit.com
|
||||
Computer use history | Novice | Casual | Novice | Casual | Novice | Casual | Novice | Casual |
N | 5 | 4 | 4 | 4 | 4 | 4 | 3 | 4 |
Task 1 (% achieved) | 0 | 50 | 50 | 75 | 0 | 25 | 33 | 25 |
Task 2 (% achieved) | 0 | 25 | 0 | 100 | 0 | 25 | 0 | 0 |
Easy to understand the information (% agreed) | 40 | 50 | 25 | 100 | 100 | 50 | 66 | 25 |
Easy to get it to work (% agreed) | 40 | 50 | 25 | 75 | 50 | 25 | 33 | 25 |
Easy to become skillful (% agreed) | 40 | 0 | 25 | 50 | 50 | 25 | 33 | 25 |
Novice, used computer <5 times, Casual, used computer ≥5 times. Task 1 Find information about how smokers can cope without smoking cigarettes. Task 2 Find information about how the medicine varenicline (Chantix) can help a person stop smoking.
Discussion
These data show that four easy-to-access smoking cessation websites do not meet the needs of smokers with severe mental illnesses. Experts rated one of the four websites as having acceptable content and usability, but the majority of smokers with severe mental illnesses were unable to navigate this website. The website that was navigable by participants received poor content ratings by experts. Of note, all were rated poorly for content related to minority smokers and smokers with mental illness, both groups of smokers with low rates of treatment engagement. The research reported here indicates that smokers with severe mental illnesses require simpler website design and usability features in order to obtain education from websites.
Our findings are similar to research on satisfaction with websites among smokers in the general population in 2005 [50]. In this survey, pmusa.com was the most frequently visited website, but the majority of users were not satisfied with the site. We are not aware of any publications that describe assessment of the usability of this website or of whyquit.com and becomeanex.org. Of note, the US government-sponsored website, smokefree.gov, was carefully tested for usability and modified to address problems that were identified in the process [45]. General population users rated 133 websites and rated smokefree.gov highest for quality (7.4 of 10) [50].
Our findings are in line with previous work demonstrating that people with schizophrenia require simple website designs [38]. Although others have assessed the quality of information about mental illness available on the web (e.g. [51]), we are not aware of previous research that has attempted to assess whether people with severe mental illnesses, many of whom have low computer experience and frequent cognitive deficits, are able to access and understand currently available health care information on the Internet. Although very little research has assessed how to make websites usable for this group, the careful work of Rotondi [38] and our own work [39] suggests that the following design features improve usability by people with severe mental illnesses:
explicit instructions on how to use the site
shallow hierarchy (reach destination within two clicks)
explicit labels that use longer concrete phrases to describe content
text at fifth grade reading level
large navigation buttons
pop-up menus that appear with hovering to reduce need for clicking
This study utilized a small number of volunteers from a large urban mental health center. A larger number of study participants may have resulted in different ratings of usability. Furthermore, because participants were not randomly selected to be a representative sample, these results may not generalize to other groups of people with severe mental illnesses. We did not assess cognition and do not know whether our group had typical levels of cognitive deficits. Additionally, these participants averaged 50 years of age. Older people with severe mental illnesses tend to be less likely to have used the Internet [30]. Our observations of usability testing with younger smokers with severe mental illnesses [39] suggested that younger people may have better computer and Internet skills—thus current websites may be more easily used by them. The expert reviewers were not blind to website sponsor but were not funded by any of these organizations. Finally, we did not test every smoking cessation website currently available on the Internet and we did not conduct full formal usability testing of all aspects of these websites. Our purpose for this study was to examine whether websites likely to be found by people with severe mental illnesses would be helpful to them. We did not have the capacity to test all smoking cessation websites. Despite these limitations, the validity of our findings are supported by their similarity to those of Etter [50], who found that smokefree.gov was very highly rated by users in the general population. But even this high quality site was not easily navigable by people with severe mental illnesses.
Researchers, policy makers and public health experts need to evaluate the efficacy of current strategies to educate and engage smokers into cessation treatments. The data reported here suggest that current efforts related to smoking cessation on the Internet are not likely to be effective for people with severe mental illnesses, who comprise a substantial proportion of smokers in the United States. Although the Internet is promising as a cost-effective strategy to reach high-risk populations, website developers must attend to the needs and capacities of the intended users by developing them with the user population characteristics in mind and testing them with these users [43, 44]. Specifically, in order for electronic interventions to be usable by people with severe mental illnesses, they must have design features that enhance usability for people with cognitive deficits and low computer experience. Further research to delineate designs that maximize website usability and efficacy for people with low computer experience and cognitive deficits, similar to the work of Rotondi [38], is needed to guide the development of web-based education and interventions for people with severe mental illnesses.
Funding
West Family Foundation; Bristol-Myers Squibb Foundation.
Conflict of interest statement
None declared.
References
- 1.Escobedo LG, Peddicord JP. Smoking prevalence in U.S. birth cohorts: the influence of gender and education. Am JPublic Health. 1996;86:231–36. doi: 10.2105/ajph.86.2.231. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 2.CDC. Cigarette smoking–United States, 1965–2008. MMWR. 2011;60:109–13. [PubMed] [Google Scholar]
- 3.Dube SR, McClave A, James C, et al. Vital signs: current cigarette smoking among adults aged >18 years –- United States, 2009. MMWR. 2010;59:1135–40. [PubMed] [Google Scholar]
- 4.Annonymous. The Numbers Count: Mental Disorders in America. National Institute of Mental Health; 2008. Available at: http://www.nimh.nih.gov/health/publications/the-numbers-count-mental-disorders-in-america/index.shtml. Accessed: 25 October 2010. [Google Scholar]
- 5.de Leon J, Diaz FJ. A meta-analysis of worldwide studies demonstrates an association between schizophrenia and tobacco smoking behaviors. Schizophr Res. 2005;76:135–57. doi: 10.1016/j.schres.2005.02.010. [DOI] [PubMed] [Google Scholar]
- 6.de Leon J, Tracy J, McCann E, et al. Schizophrenia and tobacco smoking: a replication study in another US psychiatric hospital. Schizophr Res. 2002;56:55–65. doi: 10.1016/s0920-9964(01)00192-x. [DOI] [PubMed] [Google Scholar]
- 7.Kotov R, Guey LT, Bromet EJ, et al. Smoking in schizophrenia: diagnostic specificity, symptoms correlates, and illness severity. Schizophr Bull. 2010;36:173–81. doi: 10.1093/schbul/sbn066. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 8.Lasser K, Boyd JW, Woolhandler S, et al. Smoking and mental illness: a population-based prevalence study. J Am Med Assoc. 2000;284:2606–10. doi: 10.1001/jama.284.20.2606. [DOI] [PubMed] [Google Scholar]
- 9.Vanable PA, Carey MP, Carey KB, et al. Smoking among psychiatric outpatients: relationship to substance use, diagnosis, and illness severity. Psychol Addict Behav. 2003;17:259–65. doi: 10.1037/0893-164X.17.4.259. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 10.Lawrence D, Mitrou F, Zubrick SR. Smoking and mental illness: results from population surveys in Australia and the United States. BMC Public Health. 2009;9:285–99. doi: 10.1186/1471-2458-9-285. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 11.Grant BF, Hasin DS, Chou SP, et al. Nicotine dependence and psychiatric disorders in the United States: results from the National Epidemiologic Survey on Alcohol and Related Conditions. Arch Gen Psychiatry. 2004;61:1107–15. doi: 10.1001/archpsyc.61.11.1107. [DOI] [PubMed] [Google Scholar]
- 12.Dickey B, Normand S-LT, Weiss RD, et al. Medical morbidity, mental illness, and substance use disorders. Psychiatr Serv. 2002;53:861–67. doi: 10.1176/appi.ps.53.7.861. [DOI] [PubMed] [Google Scholar]
- 13.Capasso RM, Lineberry TW, Bostwick JM, et al. Mortality in schizophrenia and schizoaffective disorder: an Olmstead County, Minnesota cohort: 1950–2005. Schizophr Res. 2008;98:287–94. doi: 10.1016/j.schres.2007.10.005. [DOI] [PubMed] [Google Scholar]
- 14.Colton CW, Manderscheid RW. Congruencies in increased mortality rates, years of potential life lost, and causes of death among public mental health clients in eight states. Prev Chronic Dis. 2006;3:1–14. [PMC free article] [PubMed] [Google Scholar]
- 15.Brown S, Inskip H, Barraclough B. Causes of the excess mortality of schizophrenia. Br J Psychiatry. 2000;177:212–17. doi: 10.1192/bjp.177.3.212. [DOI] [PubMed] [Google Scholar]
- 16.Doll R, Peto R, Boreham J, et al. Mortality in relation to smoking: 50 year's observations on male British doctors. BMJ. 2004;26:1519–28. doi: 10.1136/bmj.38142.554479.AE. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 17.Ferron JC, Alterman A, McHugo GJ, et al. A review of research on smoking cessation interventions for adults with schizophrenia spectrum disorders. Mental Health Subst Abuse. 2009;2:64–79. [Google Scholar]
- 18.Evins AE, Cather C, Deckersbach T, et al. A double-blind placebo-controlled trial of bupropion sustained-release for smoking cessation in schizophrenia. [see comment] J Clin Psychopharmacol. 2005;25:218–25. doi: 10.1097/01.jcp.0000162802.54076.18. [DOI] [PubMed] [Google Scholar]
- 19.George TP, Vessicchio JC, Termine A, et al. A placebo controlled trial of bupropion for smoking cessation in schizophrenia. Biol Psychiatry. 2002;52:53–61. doi: 10.1016/s0006-3223(02)01339-2. [DOI] [PubMed] [Google Scholar]
- 20.Gershon Grand RB, Hwang S, Han J, et al. Short-term naturalistic treatment outcomes in cigarette smokers with substance abuse and/or mental illness. J Clin Psychiatry. 2007;68:892–8. doi: 10.4088/jcp.v68n0611. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 21.Evins AE, Cather C, Culhane MA, et al. A 12-week double-blind, placebo-controlled study of bupropion SR added to high-dose dual nicotine replacement therapy for smoking cessation or reduction in schizophrenia. J Clin Psychopharmacol. 2007;27:380–6. doi: 10.1097/01.jcp.0b013e3180ca86fa. [DOI] [PubMed] [Google Scholar]
- 22.Evins AE, Culhane MA, Alpert JE, et al. A controlled trial of bupropion added to nicotine patch and behavioral therapy for smoking cessation in adults with unipolar depressive disorders. J Clin Psychopharmacol. 2008;28:660–6. doi: 10.1097/JCP.0b013e31818ad7d6. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 23.Baker A, Richmond R, Haile M, et al. A randomized controlled trial of a smoking cessation intervention among people with a psychotic disorder. Am J Psychiatry. 2006;163:1934–42. doi: 10.1176/ajp.2006.163.11.1934. [DOI] [PubMed] [Google Scholar]
- 24.Selby P, Voci SC, Zawertailo LA, et al. Individualized smoking cessation treatment in an outpatient setting: predictors of outcome in a sample with psychiatric and addictions co-morbidity. Addict Behav. 2010;35:811–7. doi: 10.1016/j.addbeh.2010.03.020. [DOI] [PubMed] [Google Scholar]
- 25.McClure JB, Swan GE, Catz SL, et al. Smoking outcome by psychiatric history after behavioral and varenicline treatment. J Subst Abuse Treat. 2010;38:394–02. doi: 10.1016/j.jsat.2010.03.007. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 26.Williams JM, Steinberg ML, Zimmermann MH, et al. Comparison of two intensities of tobacco dependence counseling in schizophrenia and schizoaffective disorder. J Subst Abuse Treat. 2010;38:384–93. doi: 10.1016/j.jsat.2010.03.006. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 27.Lucksted A, Dixon LB, Sembly JB. A focus group pilot study of tobacco smoking among psychosocial rehabilitation clients. Psychiatr Serv. 2000;51:1544–8. doi: 10.1176/appi.ps.51.12.1544. [DOI] [PubMed] [Google Scholar]
- 28.Morris CD, Waxmonsky JA, May MG, et al. What do persons with mental illnesses need to quit smoking? Mental health consumer and provider perspectives. Psychiatr Rehabil J. 2009;32:276–84. doi: 10.2975/32.4.2009.276.284. [DOI] [PubMed] [Google Scholar]
- 29.Weinberger AH, Reutenauer EL, Vessicchio JC, et al. Survey of clinician attitudes toward smoking cessation for psychiatric and substance abusing clients. J Addict Dis. 2008;27:55. doi: 10.1300/J069v27n01_06. [DOI] [PubMed] [Google Scholar]
- 30.Borzekowski DLG, Leith J, Medoff DR, et al. Use of the Internet and other media for health information among clinic outpatients with serious mental illness. Psychiatr Serv. 2009;60:1265–8. doi: 10.1176/ps.2009.60.9.1265. [DOI] [PubMed] [Google Scholar]
- 31.Khazaal Y, Chatton A, Cochand S, et al. Internet use by patients with psychiatric disorders in search for general and medical informations. Psychiatr Q. 2008;79:301–9. doi: 10.1007/s11126-008-9083-1. [DOI] [PubMed] [Google Scholar]
- 32.Myung S-K, McDonnell DD, Kazinets G, et al. Effects of web- and computer-based smoking cessation programs: meta-analysis of randomized controlled trials. Arch Intern Med. 2009;169:929–37. doi: 10.1001/archinternmed.2009.109. [DOI] [PubMed] [Google Scholar]
- 33.Shahab L, McEwen A. Online support for smoking cessation: a systematic review of the literature. Addiction. 2009;104:1792–804. doi: 10.1111/j.1360-0443.2009.02710.x. [DOI] [PubMed] [Google Scholar]
- 34.Keefe RS, Bilder RM, Harvey PD, et al. Baseline neurocognitive deficits in the CATIE schizophrenia trial. Neuropsychopharmacology. 2006;31:2033–46. doi: 10.1038/sj.npp.1301072. [DOI] [PubMed] [Google Scholar]
- 35.Baker A, Richmond R, Haile M, et al. Characteristics of smokers with a psychotic disorder and implications for smoking interventions. Psychiatry Res. 2007;150:141–52. doi: 10.1016/j.psychres.2006.05.021. [DOI] [PubMed] [Google Scholar]
- 36.Dixon L, Medoff DR, Wohlheiter K, et al. Correlates of severity of smoking among persons with severe mental illness. Am J Addict. 2007;16:101–10. doi: 10.1080/10550490601184415. [DOI] [PubMed] [Google Scholar]
- 37.Deegan PE. A web application to support recovery and shared decision making in psychiatric medication clinics. Psychiatr Rehabil J. 2010;34:23–8. doi: 10.2975/34.1.2010.23.28. [DOI] [PubMed] [Google Scholar]
- 38.Rotondi AJ, Sinkule J, Haas GL, et al. Designing websites for persons with cognitive deficits: design and usability of a psychoeducational interevention for person with severe mental illness. Psychol Serv. 2007;4:202–24. doi: 10.1037/1541-1559.4.3.202. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 39.Ferron JC, Brunette MF, Martin W, et al. The usability development of a quit smoking website for people with severe mental illness. Psychiatr Rehabil J. doi: 10.2975/35.2.2011.111.116. in press. [DOI] [PubMed] [Google Scholar]
- 40.Washington, DC: United States Department of Health and Human Services; Annonymous. Research-Based Web Design & Usability Guidelines. http://usability.gov/guidelines/index.html. [Google Scholar]
- 41.Cheh JA, RIbisl KM, Wildemuth BM. An assessment of the quality and usability of smoking cessation information on the Internet. Health Promot Pract. 2003;4:278–87. doi: 10.1177/1524839903004003012. [DOI] [PubMed] [Google Scholar]
- 42.Bock B, Graham A, Sciamanna C, et al. Smoking cessation treatment on the Internet: content, quality, and usability. Nicotine Tob Res. 2004;6:207–19. doi: 10.1080/14622200410001676332. [DOI] [PubMed] [Google Scholar]
- 43.Elwyn G, Kreuwel I, Durand MA, et al. How to develop web-based decision support interventions for patients: a process map. Patient Educ Couns. 2011;82:260–5. doi: 10.1016/j.pec.2010.04.034. [DOI] [PubMed] [Google Scholar]
- 44.Faulkner L. Beyond the five-user assumption: benefits of increased sample sizes in usability testing. Behav Res Methods Instrum Comput. 2003;35:379–83. doi: 10.3758/bf03195514. [DOI] [PubMed] [Google Scholar]
- 45.Stoddard JL, Augustson EM, Mabry PL. The importance of usability testing in the development of an internet-based smoking cessation treatment resource. Nicotine Tob Res. 2006;8(Suppl. 1):S87–93. doi: 10.1080/14622200601048189. [DOI] [PubMed] [Google Scholar]
- 46.Evans R, Elwyn G, Edwards A, et al. Toward a model for field-testing patient decision-support technologies: a qualitative field-testing study. J Med Internet Res. 2007;9:e21. doi: 10.2196/jmir.9.3.e21. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 47.Elwyn G, O'Connor A, Stacey D, et al. Developing a quality criteria framework for patient decision aids: online international Delphi consensus process. BMJ. 2006;333:417. doi: 10.1136/bmj.38926.629329.AE. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 48.Cokkinides VE, Halpern MT, Barbeau EM, et al. Racial and ethnic disparities in smoking-cessation interventions: analysis of the 2005 National Health Interview Survey. Am J Prev Med. 2008;34:404–12. doi: 10.1016/j.amepre.2008.02.003. [DOI] [PubMed] [Google Scholar]
- 49.Davis FD. Perceived usefulness, perceived ease of use, and user acceptance and information technology. MIS Q. 1989;13:319–40. [Google Scholar]
- 50.Etter JF. A list of the most popular smoking cessation web sites and a comparison of their quality. Nicotine Tob Res. 2006;8(Suppl. 1):S27–S34. doi: 10.1080/14622200601039923. [DOI] [PubMed] [Google Scholar]
- 51.Morel V, Chatton A, Cocharnd S, et al. Quality of web-based information on bipolar disorder. J Affect Disord. 2008;110:265–69. doi: 10.1016/j.jad.2008.01.007. [DOI] [PubMed] [Google Scholar]