Abstract
Background and Objectives
Decisions about long-term care and financing can be difficult to comprehend, consider, and communicate. In a previous needs assessment, families in rural areas requested a patient-facing website; however, questions arose about the acceptability of an online tool for older adults. This study engaged older adults and family caregivers in (a) designing and refining an interactive, tailored decision aid website, and (b) field testing its utility, feasibility, and acceptability.
Research Design and Methods
Based on formative work, the research team engaged families in designing and iteratively revising paper drafts, then programmed a tailored website. The field test used the ThinkAloud approach and pre-/postquestionnaires to assess participants’ knowledge, decisional conflict, usage, and acceptability ratings.
Results
Forty-five older adults, family members, and stakeholders codesigned and tested the decision aid, yielding four decision-making steps: Get the Facts, What Matters Most, Consider Your Resources, and Make an Action Plan. User-based design and iterative storyboarding enhanced the content, personal decision-making activities, and user-generated resources. Field-testing participants scored 83.3% correct on knowledge items and reported moderate/low decisional conflict. All (100%) were able to use the website, spent an average of 26.3 min, and provided an average 87.5% acceptability rating.
Discussion and Implications
A decision aid website can educate and support older adults and their family members in beginning a long-term care plan. Codesign and in-depth interviews improved usability, and lessons learned may guide the development of other aging decision aid websites.
Keywords: Caregiver, Consumer health informatics, Decision making, Health communication, Long-term care
Considering home versus residential long-term care options is a complex decision process for many families that often involves uncertain timelines, multiple decision makers, and significant cost considerations. Some families in the United States manage the financial burdens of care by aging in place, as evidenced by the estimated $230.1 billion (18.2 billion hours) of unpaid in-home care (Alzheimer’s Association, 2017). However, not every family has someone who can serve as a full-time caregiver, and caregiving needs may exceed available skills and resources. Families in rural or low-resource areas may find themselves weighing the trade-offs between what matters most to them (i.e., their “decision-making values”) and the available resources and costs. As a result, many families find themselves stuck in decisional conflict—a state of uncertainty and anxiety about the best course to take—that often delays care planning until a crisis occurs (O’Connor et al., 1998).
Caregivers articulate that being responsible for a family member’s medical and financial decisions is a primary source of burden that can contribute to premature placement in a nursing home (Etters, Goodall, & Harrison, 2008; Hirschman, Xie, Feudtner, & Karlawish, 2004; Roth, Fredman, & Haley, 2015; Stirling et al., 2010). Person-centered care planning has been shown to reduce premature institutionalization; however, only 46% of families create care plans, stating that the decisions can be overwhelming and give rise to family conflict (Dening, Jones, & Sampson, 2011; Thompson & Spilsbury, 1998). Evidence-based interventions are needed to support families who are facing these complex decisions.
The long-term goal of this research is to develop a suite of decision aids to facilitate shared decision making, high-quality decisions, and person-centered care for older adults and their families. Patient decision aids are tools that provide up-to-date, balanced, high-quality information, and evidence-based support to people who are deliberating between two or more medically relevant options (Stacey et al., 2017). The most recent Cochrane Collaboration review (89 randomized trials) reports that decision aids help patients and caregivers improve their knowledge of their options, form realistic expectations about risks and benefits, and actively communicate with their doctors as they engage in the decision-making process (Stacey et al., 2017).
Prior work found that patients and families in rural areas requested a website that they could use at home at any time, with information about the options, step-wise guidance for forming a personalized action plan, and the ability to share information with long-distance family members (Hoffman et al., 2013a). However, concerns arose regarding the feasibility, utility, and acceptability of a web-based tool for older adults (Chaudhuri, Le, White, Thompson, & Demiris, 2013). The guidelines of the International Patient Decision Aid Standards (IPDAS) Collaboration recommend engaging all potential users to design websites that are useful, usable, and acceptable for that population (Hoffman et al., 2013b). It also emphasizes the need to field-test health care websites as used on the Internet by real users prior to dissemination (Hoffman et al., 2013b). Therefore, the objectives of this study were first, to engage older adults and family members in codesigning a web-based decision aid focused on long-term care decisions, and, second, to field test its utility, feasibility, and acceptability as used on the Internet. In preparation for developing a larger suite of aging decision aids, an emergent objective was to explore more deeply the needs and preferences of older adults and family members when using a decision aid website.
Research Design and Methods
Conceptual Framework
Theories and approaches from decision science, cognitive science, human computer interaction, and health services research guided the design of the study and decision aid (Bandura, 1989; Elstein, 1999; Fagerlin et al., 2013; Hoffman et al., 2013b; O’Connor et al., 1998; Sepucha et al., 2018; Tunney & Ziegler, 2015; Tversky & Kahneman, 1974, 1981). For example, the interview guide purposefully assessed the role of uncertainty, and iterative testing assessed communication routes among family members. The Ottawa Decision Support Framework (O’Connor et al., 1998) postulates that evidence-based decision support interventions can address some of the modifiable barriers that create decisional conflict, such as feeling uninformed; uncertain about the risks and benefits; unclear about what is most important; unsupported; and unsure about the best course of action. Since long-term care can involve multiple decision makers over time, we applied this framework in two ways—first, to address the varying needs of each individual, and second, to recognize the varying roles each individual may have in the decision-making process (e.g., information gathering, deliberation, determining the decision, and/or implementing the decision).
We also recognized the differing needs of individuals who are planning for future care versus those who are making urgent decisions, particularly in regards to financing. We situated this adapted framework within the contexts of aging care and rural health care, to incorporate the potential role of cognitive, physical, functional, and financial limitations (Chesser, Burke, Reyes, & Rohrberg, 2016; Hirschman et al., 2004).
Following the international guidelines, we applied a modified user-centered design process to tailor decision support for older adults, their local family/caregivers, and their long-distance family members (Coulter et al., 2013; Dugas et al., 2017; Gustafson et al., 2016; Hoffman et al., 2014; Witteman et al., 2015). The primary research question was: Can engaging older adults and their family members in codesign produce a long-term care decision aid website that is useful, feasible, and acceptable?
Participants
Participants were adult patients, caregivers, and family members who were able to read, write, and understand English recruited from an Aging Resource Center that serves approximately 1,600 people per year from remote rural New England towns (e.g., 100 to 10,000 people). Unique groups of participants participated in each phase of study. All participants provided informed consent. The Dartmouth Committee for the Protection of Human Subjects reviewed and approved this study.
Study Design
A stakeholder advisory panel consisting of two older adults, two family caregivers, two decision scientists, four informaticians, three geriatric psychiatrists, and three memory care specialists (16 people total) guided the design and conduct of the study. We conducted three phases of research: (a) Determining the initial decision aid design and components; (b) Design testing—refining paper storyboards with iterative cognitive interviews; and (c) Field testing the acceptability of the decision aid website as used on the Internet (Figure 1).
Phase 1: Determining the Initial Design and Components
Several preliminary studies guided the initial content and design (Hoffman et al., 2013a). In brief, a scoping review of the literature assessed the current evidence, considerations, and costs for home and residential long-term care and financing. An environmental scan (review of existing resources) assessed the availability and quality of long-term care educational materials and decision aids. A series of focus groups with community members, needs assessment surveys, and interviews with selected clinical experts, long-term care counselors, and policy makers produced a preliminary list of potential decision support needs.
Recognizing that most individuals would need to seek location-specific details from their region’s aging resource center, the stakeholder panel and research team determined that the primary objectives of the decision aid should be: (a) to help viewers become well-informed and clarify which options were of most interest to them; and (b) to prepare a printable personal action plan that they can use to help them find services, address any remaining gaps in information, identify needed resources, and facilitate conversations as needed.
Based on the user-centered design approach, the design team then started by creating five personas—example scenarios representing potential decision aid users’ characteristics, reasons for using the decision aid, and specific decision-making needs. Using these personas, the design team created a Design Strategies Map (see Table 1 in Results)—a list of potential tasks (based on each persona’s needs for information and deliberative support) and proposed design features that could address each task. The Design Strategies Map guided the design of initial paper mock-ups (storyboards). The advisory panel reviewed the storyboards, which were revised, rereviewed, and approved for design testing.
Table 1.
Priority user needs | Proposed design strategies |
---|---|
Information needs | Information stem |
• What is long term, residential, home-based care? | • Definitions, examples, attributes & costs |
-Strengths/weakness of all options? | -Use recent update of Administration on Aging report |
-Costs of and financing for all options? | -Comparison chart |
• How might this affect our home, estate? | • Glossary |
• How do I find out what’s available in their community? | • Resources |
-Aging resource centers & decision counselors | |
• Process for initiating all options? | -Long-term care financial services |
Deliberation needs | Deliberative activities |
• Awareness • How do I compare the strengths/weaknesses of all options? • What matters most to each family member? • How do I know when “the right time” is to move a loved one? |
• Setting the stage—introduce the decisions and people involved, decision-making roles vs caregiving roles, etc. |
• Knowledge self-quiz | |
• Values clarification exercises | |
• Action planning exercises | |
Communication needs | Communication tools |
• How can I find out everyone’s priorities (i.e., “decision-making values”)? |
• Parallel information and deliberative activities for caregivers and family members • Web portal or printable/sharable summary? • Conflict resolution skills • Tips from previous families about decision making, communication, questions to ask, how to document |
• How can I share information with long-distance family members? | |
• How do I discuss differences between what’s best for the person versus what’s best for the family? | |
• Who can I talk to (doctor, senior center, lawyer) and what questions should I ask? | |
• How do I document all this information? | |
Usability needs | Accessible and tailored design |
• Accessibility: font, button, graph sizes |
• Increase font sizes, high/low contrast colors, Next/Back buttons, big buttons • Limit branching to 5.1 concept per page • Computer use tutorials (e.g., scrolling) • Caregiver-tailored versions of information and activities • Eighth grade language, glossary • Audio voice-over and visuals |
• Not easy to get lost in | |
• May not know how to scroll | |
• Parallel/tailored information for caregivers | |
• Lay language and support for low literacy, poor eyesight, shaking hands, etc. | |
• Under 30 min total |
Phase 2: Design Testing—Refining Storyboards with Iterative Cognitive Interviews
Design Testing included three iterative cycles of review focused on general content and features, followed by in-depth reviews of key components with older adults. The initial paper draft consisted of 86 pages describing three steps—learning, deliberating, action planning—and included interactive decision-making activities and a printable personal summary at the end. The research team conducted three rounds of cognitive interviews with unique samples of selected clinicians and family caregivers (n = 2–3 per design cycle) to confirm which elements were essential/nonessential, and to verify the overall flow of the deliberative steps.
In addition, 14 pages merited in-depth testing with older adults to evaluate wording, structure, navigation, and potential accessibility features (e.g., font size, size/location of buttons, ability to scroll, navigation of overlays/pop-ups, etc.). Semistructured interviews addressed three a priori research questions: “Is the wording, layout, and design usable and meaningful?”; “What levels of interactivity and personalization are needed and preferred?”; and, “Are there any additional types of information or accessibility features that are needed?”
A decision scientist and two human computer interaction designers cofacilitated the interviews (A.S.H., C.G., S.P.). The interviewers presented each page and engaged participants in reviewing and editing the paper storyboards. Participation was confidential (no identifiers were included in the data), notes were recorded by at least two people, and photos were taken of each page upon which a participant provided written feedback.
Two people transcribed the interviews and a third interviewer reviewed them for accuracy. Data was combined into a table comparing all sets of notes and photos for each page, and assessed by two researchers for convergent themes and unique recommendations for improvement. The design team revised the paper draft, added additional features requested by the participants, and reassessed the decision aid according to the IPDAS quality guidelines (Hoffman et al., 2013b; Joseph-Williams et al., 2014). The stakeholder advisory panel and research team reviewed and approved the design for website programming.
Phase 3: Production and Field Testing at an Aging Resource Center
Production of the website and database
Components of the final design are presented in Table 1 and described in the Results. The website contained four modular steps: Get the Facts, What Matters Most to You, Considering Your Resources, and Make an Action Plan. To allow viewers to select the amount of information they needed (e.g., for monitoring and blunting coping styles; Gordon & Hornbrook, 2016; Hoffman et al., 2013b; Miller, 1995), information was presented at an overview level, with embedded hyperlinks to definitions and additional detail. To create a printable My Decision Summary page, data collection items assessing users’ characteristics and decision-making process (e.g., knowledge, values clarification, leaning, and decisional conflict) were embedded within the flow of the website. The website designer (D.B.H, VeracityByDesign.com) designed the decision aid website using WordPress (v2015, www.wordpress.com) so that it would remain easily updateable by an Aging Resource Center.
Field-testing with older adults and family members
The research team pilot-tested the initial prototype with the staff and volunteers of the Dartmouth Aging Resource Center, then launched it live on the Internet. The Aging Resource Center provides Windows computers connected to external monitors, keyboards and mice, with text configured to “Larger—150%”. The staff of the Aging Resource Center recruited participants for the field test, and provided a secure study login and password to use the website. The human computer interaction researchers (C.G., S.P.) asked participants to “Think Aloud” as they viewed the website (Woodard et al., 2018). All sessions were audio-recorded, and participants’ interactions with the website were recorded using Morae screen capture software (v.2014, https://www.techsmith.com/morae.html).
After reviewing the decision aid website, questionnaires assessed knowledge, decisional conflict, acceptability, and a brief semistructured interview assessed suggestions for improvement. The research team had previously adapted the measures to the long-term care context, pilot-tested them, and retained items that were responsive in this population (Hoffman, A. S. (2019). Assessing the decision-making needs and preferences of older adults with dementia and their caregivers and clinicians: survey and qualitative interviews. Manuscript in preparation). The SURE Scale (Ferron Parayre, Labrecque, Rousseau, Turcotte, & Legare, 2014) is a four-item measure of decisional conflict; items are scored 1 for Yes and 0 for No; a total score of 3 or less indicates decisional conflict. The Ottawa Acceptability Scale (O’Connor, 1996 [Updated 2002]) is scored as the percentage of individuals who rate each item positively. The knowledge measure included three multiple-choice items (i.e., average need for long-term care, differences between home/residential care, and available financing options); and two True/False items about common misconceptions (i.e., home care is always more affordable than residential care, and health insurance covers all long-term care costs).
Data Analysis
This study was not powered to test hypotheses; however, the stakeholder advisory panel and the research team utilized several a priori criteria based on comparable decision support literature (Hoffman et al., 2014; Stacey et al., 2017). Feasibility of the web-based decision aid was assessed in terms of whether at least 75% of participants were able to view the 14 core pages and use at least 1 interactive activity. To confirm that the decision aid was not systematically biased, the team reviewed the distributions of participants’ decision-making values and preferences for home versus residential care services. Utility was assessed in terms of 80% or more of participants answering at least three of five knowledge items correctly, and responding Yes to at least three of four SURE Scale decisional conflict items (Ferron Parayre et al., 2014). Scores of 3 were included for this study, as the purpose of the decision aid is to educate and prepare for decision-making; families will need to meet with their local long-term care counselor for specific cost estimates and financing options before determining a decision. Acceptability was assessed in terms of at least 80% of participants rating at least six of nine acceptability items favorably (i.e., Yes, or Helpful/Very Helpful) (O’Connor, 1996 [Updated 2002]).
Interview notes, audio recordings, and screen recordings were reviewed to identify any potential systematic usability errors or barriers to use. The research team and Stakeholder Advisory Panel reviewed results and additional suggestions for improvement. The website designer made final revisions to optimize the website, and posted it for public use. The research team assessed the final website according to the IPDAS guidelines for high-quality patient decision aids (Joseph-Williams et al., 2014), and added a downloadable “source document” that describes development, original sources cited, and related resources in plain language.
Results
Overall, engaging older adults, family members, and stakeholders in the design, storyboarding, and field-testing process produced an interactive website that tailors information and decision-making support to older adults and their local and long-distance caregivers. Supplementary Appendix A provides the source document and selected screen shots of Planning for long-term care: choosing the best care and financial plan for your loved ones (v1.2, ©2017 Hoffman et al., Dartmouth Centers for Health and Aging).
Phase 1—Determining the Initial Design
Table 1 presents the Design Strategies Map, which provides proposed content and design elements to meet each need identified by older adults, caregivers, and providers. Notably, feedback from older adults and families split the decision into two so that they could focus their time and energy as needed. The opening page allows viewers to choose one or both topics to view:
Whether to receive home-based or residential-based long-term care services?
Which financial options to use to pay for long-term care?
Web-Based Decision Support System: Structure and Content
Four Decision-Making Steps were proposed to meet the identified needs, based on the decision support framework and evidence-based approaches to guiding patients towards making a well-informed, values-congruent decision among two or more treatment options (O’Connor et al., 1998; Stacey et al., 2013):
Step 1. Get the Facts—Presents up-to-date, high-quality information about each of the home and residential long-term care options, and the financing options. Interactive decision-making activities allow viewers to (a) select options to create a side-by-side comparison chart, and (b) complete the Essentials of Long-term Care self-quiz (a plain-language version of the knowledge measure, which included corrective feedback to facilitate understanding of concepts).
Step 2. What Matters Most to You?—Helps viewers understand the concept of “decision-making values”, i.e., the personal importance of various factors in making their decision, and provides a values clarification leaning scale where viewers can self-assess which factors are most important to each person involved in the decision.
Step 3. Consider Your Resources—Assists viewers in learning about and considering the personal, social, and financial resources they have available to assist them in aging in place or transitioning to residential long-term care.
Step 4. Make an Action Plan—Shows users their My Decision Summary of their responses from Steps 1–3 compared to the side-by-side chart of the long-term care options. It also reiterates the importance of assessing and weighing their care needs and preferences compared to their resources. Structured questions invite the user to identify gaps in information and support, and to plan their next steps (e.g., “I need more information about the cost of X”), with links to specific local resources for completing those steps (e.g., by providing contact information for selected assisted living facilities, local aging and disability resource councils, etc.).
Each decision-making step provides text and graphics summarizing the aim, importance, and facts related to that step, as well as an interactive activity to guide the user in completing that step for their personal decision. Through this five-step process, the user creates a My Decision Summary printout of their personalized information, responses, and questions to address with their clinical team and other advisors.
Phase 2—Storyboarding (design testing) with Paper Prototypes
The design team iteratively reviewed and refined sections of the initial design with 12 older adults and caregivers, and presented it in clinical and research meetings with key stakeholders. The advisory panel then reviewed, provided feedback, and approved a parsimonious draft of 64 core pages. Five additional older adults participated in reviewing the complete paper prototype. All interviews lasted from 1 to 2 hrs. Participants were primarily 65–84 years old, female, and Caucasian. They expressed varying levels of experience with computers (from “none at all” to “comfortable”) and all provided substantive feedback.
All participants affirmed the value of a tool to help families navigate this tough decision. They also approved of the majority of the text, layout, graphical elements, and interactive questions. The following paragraphs summarize their recommendations for improving the design of the initial website structure, content, and interactive features. Representative quotes and examples of feedback are provided in Table 2.
Table 2.
Provide linear yet flexible website structure |
• “I like that the website talks about the options and what’s important before talking about financing, but I might want to revisit the options and costs again after considering my financial resources.” |
• “You might want to ask what matters most to everyone in the family BEFORE comparing the options.” |
Provide variable levels of information detail and broad scope of topics |
• “National averages may be interesting to some, but time is limited. I wouldn’t want to go through 25 pages before getting to things that are meaningful locally”. |
• “It’s a lot of information, but I would want even more detail about some options”. |
• “Make transportation a separate item – it is an important and significant need (and not a simple part of receiving long-term care in this [rural] area)”. |
• “The list of options focuses on medical/physical needs; what about intellectual, social, and spiritual needs (e.g., access to books, hobbies, music, social/community activities) things that are satisfying and provide growth and well-being overall?” |
Address common misconceptions, particularly about costs |
• “The assumption is that as long as I can care for him, he should be at home, but maybe there are reasons it would be better to move sooner (while he still knows what’s going on).” |
• “Need to include something to help people figure out and compare their financial resources if they sold their house (the assumption is that moving is more expensive, but it may not be if you add up all the things that you would need to pay for at home like transportation, errands, nurses, etc.).” |
• “Need a way to directly compare the ‘real’ total costs to the family.” |
• “Make the hidden costs of staying at home clear.” |
Provide parallel information and decision support activities for caregivers |
• “Acknowledge up front the care that is needed for the caregiver” (seeFigure 2). |
• “Make the costs to caregivers a specific line item (and clarify between family caregivers versus paid caregivers).” |
• “There’s likely to be family disagreements at this point. Tips would be helpful.” |
• “Address the issue that many family members bring up ‘good ideas’ that need someone who has the time and expertise to make them happen.” |
• “I would want this chart [What Matters Most to You?] – for myself and for family members to complete so we can compare and discuss” |
• “I would print several copies out or perhaps could you provide columns for my husband to fill out too”. |
Tailor language for older adults |
• “Current wording seems scripted for someone who is new to thinking about long-term care; it may need adjustment for people who’ve already considered it (e.g., been a caregiver for their aging parents and are now looking for themselves).” |
• “Change ‘Who needs care?’ to ‘For whom will long-term care be needed? (to ensure proper grammar is present to support older adults’ trust in the website’s reliability)’”. |
Website Structure—Provide Linear and Open Architecture
Two themes emerged regarding the structure of the web-based decision aid. First, participants emphasized keeping cognitive structuring simple and familiar, based on social heuristics. For example, all participants preferred to be led through the tool in a stepwise fashion using Next/Continue buttons rather than exploring the website. Participants reported that they imagined the tool as an interactive worksheet similar to other questionnaires they receive when they visit the doctor, and that features such as the side-by-side comparison chart should look similar to other well-known websites. However, some individuals also recommended providing options to revisit or skip information as needed to streamline their searching depending on their familiarity with these decisions (e.g., whether they are confronting a new diagnosis and planning for the future, versus making acute advanced stage decisions). Second, when designing a tool for older adults with chronic disease, participants recommended purposefully addressing how caregivers and other family members could be involved.
Content—Provide Variable Breadth and Scope
Four themes emerged regarding the clinical information. First, participants requested variable levels of information detail. Some participants recommended limiting information to expedite their search time, others expressed interest in being able to access great detail. Participants also requested a broad scope of information on topics that affect their health (e.g., to include intellectual, social, and spiritual needs). Third, participants recommended purposefully addressing common misconceptions, particularly when comparing costs (e.g., when home care may not be more affordable). Finally, participants requested tailored, parallel streams of information for patients and caregivers.
Usability—Ensure Accessibility for Older Adults
Participants made several suggestions about even larger font sizes, larger buttons for individuals with shaking hands, options for auto-play audio voiceover for visual impairments, and short video tutorials on the homepage for individuals who were less familiar with how to navigate a website (e.g., How to Use a Mouse, How to Scroll, etc.). Notably, most participants preferred learning to scroll to keep similar information on one page rather than having to move back and forth between pages. Lastly, several comments focused on the trade-offs between the more familiar language style that may be expected by younger caregivers and the more formal language style that conveyed a sense of importance and validity to older adults (see Table 2 and Figure 2).
Phase 3—Field Test
A new group of 12 older adults (4 individuals and 4 couples) field-tested the decision aid website and rated its acceptability. The majority of participants were female (83.3%), over 80 years old (66.7%, min. 61, max 89), White/Caucasian (91.7%), somewhat comfortable with computers (41.7%), and had social security (83.3%) plus other financial means that could provide $1000/month for long-term care (75%). Six were caregivers and all lived in small rural towns. They were considering long-term care services within the next year (50%) for a spouse (33.3%) or both of them (33.3%). Most sought household care (58.3%) to support them with moderate limitations in activities of daily living (58.3%), and several had memory problems (41.7%) or chronic conditions (50%) that may need special care.
Feasibility, Utility, and Acceptability
Tables 3 and 4 summarize the results of the field test. All participants were able to use the website, interacted with at least one activity, and printed a copy of their My Decision Summary page in under 50 min (including doing the ThinkAloud process). All participants chose to complete the entire site and two individuals went back to revisit pages.
Table 3.
Postdecision Aid Measures | n | % |
---|---|---|
Feasibility | ||
Time spent on website, mean minutes (min, max) | 26.3 (12.3, 49.5) | |
Viewed all pages | 12 | 100% |
Viewed pages linearly/sequentially | 12 | 100% |
Viewed pages dynamically/iteratively (revisited pages) | 2 | 16.7% |
Interacted with at least 1 activity | 12 | 100% |
Printed My Decision Summary page | 12 | 100% |
Observed errors or expressed difficulty using website | 1 | 8.3% |
Utility | ||
Knowledge of Long-term care services and financing, mean total percent correct (min, max) |
83.3% (66.7%, 100%) | |
On average, how many people over 65 years old need long-term care? | 12 | 100% |
On average, how many years do people need long-term care? | 9 | 75.0% |
Care at home is the cheapest option. | 8 | 66.7% |
Which of the following are correct average costs? | 9 | 75.0% |
Medicare and Health Maintenance Organizations (HMOs) pay for: | 12 | 100% |
SURE Decisional Conflict Scale, mean total score (min, max) | 3 (2, 4) | |
Do you know enough about the pros and cons of each option | 7 | 87.5% |
Are you clear about which pros and cons matter most to you? | 11 | 91.7% |
Do you have enough support and advice from others to make a choice? | 11 | 91.7% |
Do you feel sure about the best choice for you? | 7 | 87.5% |
Acceptability, mean total favorable rating (min, max) | 87.5% (50.0%, 100%) | |
Did the decision aid help you: | ||
Learn about the options for long-term care services? | 11 | 91.7% |
Learn about the options for long-term care financing? | 5 | 62.5% |
Sort out which services may be best for you and your family? | 9 | 75% |
Choose the options that best match your preferences? | 10 | 83.3% |
Start planning for how to receive the care you want? | 10 | 83.3% |
Was the decision aid: | ||
Useful? | 11 | 91.7% |
Easy to understand? | 10 | 83.3% |
The right length (not too long or too short)? | 10 | 83.3% |
Interactive enough? | 9 | 75% |
Table 4.
Preferences | n | % |
---|---|---|
Leaning Towards (select all that apply) | ||
Asking a family member to be my/their caregiver | 4 | 33.3% |
Using services at the senior center | 3 | 25.0% |
Hiring a home health aide | 2 | 16.7% |
Hiring a personal aide, e.g., “companion” | 4 | 33.3% |
Moving into senior housing | 1 | 8.3% |
Moving into an assisted living facility | 3 | 25% |
Moving into a nursing home | - | - |
Moving into a continuing care retirement community | 1 | 8.3% |
What community services are you interested in using? (select all that apply) | ||
Transportation Support—buses, trains, taxis, or shuttles to the hospital, stores | 7 | 87.5% |
Respite Care—temporary care to let the caregiver have a break | 10 | 83.3% |
Groceries and Meals—grocery delivery, Meals on Wheels, dinners at the senior center | 8 | 66.7% |
Social Activities—senior center programs, museum passes, movies, organized day trips | 9 | 75.0% |
Volunteering—opportunities to mentor, teach, or share your skills and wisdom | 6 | 50.0% |
Spiritual support—churches and spiritual programs that support aging | 5 | 62.5% |
Exercise—classes for strength, aerobics, balance, yoga | 10 | 83.3% |
Learning—classes at the senior center, library, and local colleges | 8 | 66.7% |
In terms of utility, 11 out of 12 participants scored greater than 60% correct on the knowledge items (min. 3, max. 5), for an average of 83.3% correct. On the SURE Scale, 8 out of 12 (67%) selected Yes on 3 or more questions (min.1, max. 4). In terms of acceptability, 10 out of 12 (83%) rated the decision aid favorably on at least 6 of 9 items. Notably, a majority of participants did not provide favorable ratings for the item about the helpfulness of the website for learning about the financial options for long-term care, stating that they wanted the cost ranges of each care provider/facility.
While viewing the website, a slight majority (33%) of participants indicated a leaning towards home-based care options compared to residential options (25%). None indicated a leaning towards moving to a nursing home. The majority also indicated an interest in using community services for transportation, respite care, social activities, and exercise.
In open-ended items assessing what gaps remained, participants indicated they would like to meet with their local aging resource center for more information about: home equity/reverse mortgages, Veteran’s benefits, long-term care insurance, Medicare/Medicaid coverage, senior housing options, and how to raise more funds. Several responses expressed a need for help figuring out at what point their loved one would need more care than they could provide at home. Suggestions for improving the decision aid website included requests for: “more tips”, “more personalized information opportunities”, and “more local information and examples”.
Final assessment of the development process using the IPDAS quality checklist indicated that it met 23 of 23 relevant criteria (additional criteria about screening, side effect probabilities, and efficacy were not relevant).
Discussion and Implications
Overall, engaging all stakeholders in the development produced a long-term care decision aid website for older adults and family members that was feasible, useful, and acceptable to older adults in a rural aging resource center. Older adults were able to use the website, individually and as couples, and create a personal action plan for addressing gaps in information, support, financing, or communication. After viewing the website, they scored moderately high on knowledge items, moderately low on decisional conflict, and indicated clear decision-making values and an initial leaning towards options. They rated the website favorably in terms of acceptability, with requests for more interactivity, additional resources about how to finance care, and integrated links to specific costs for facilities in their local area.
Field-test results met the a priori criteria for feasibility, utility, and acceptability. These results are comparable to similar web-based patient decision aids, and to the effects seen in the Cochrane Collaboration review of patient decision aids (Hoffman et al., 2014; Hoffman et al., 2013b; Stacey et al., 2017; Woodard et al., 2018). Postdecision aid treatment preferences were higher for home care options, as expected, but no dominant preference was observed, suggesting the decision aid did not bias towards a particular care option or community service.
Several decision aids are emerging about home versus residential long-term care. The Ottawa Hospital Research Institute Place of Care patient decision aid is a paper worksheet for patients who are considering where they would like to receive palliative care (Murray, 2010). Veterans Affairs created two worksheets—for individuals and for caregivers (available at: https://www.va.gov/GERIATRICS/Guide/LongTermCare/Shared_Decision_Making.asp). Healthwise and The Mayo Clinic provide web-based decision aids to help caregivers of people with Alzheimer’s consider moving a loved one into residential care (Healthwise, 2012; Mayo Clinic, 2012). This long-term care decision aid contains similar elements based on the Ottawa Decision Support Framework (O’Connor et al., 1998), and expands the scope to tailor information for (a) individuals, caregivers, and other family members, (b) considering long-term care services and/or financing options, and (c) immediate decisions or planning for the future.
Notably, multiple participants mentioned the role of uncertainty, time, and values in making these decisions. Some requested advice on how to know when a loved one would need to be moved to residential care and others remarked on the trade-offs between moving while young enough to be active versus remaining in their home as long as possible. Previous studies have reported on the importance of uncertainty and affective forecasting in care planning decisions (Schapira et al., 2016; Tversky & Kahneman, 1974). These observations also parallel the literature on advanced care planning regarding the trade-offs and tensions caregivers experience when considering who’s values should determine a decision (e.g., substitutive judgment—what would the individual have wanted, versus distributive judgment—what’s best for the family) (Dening et al., 2011; Etters et al., 2008; Marks & Arkes, 2008; Shalowitz, Garrett-Mayer, & Wendler, 2006; Stirling et al., 2010). Given the significant time, money, and impact of providing long-term care in the U.S. health care system, these observations echo the continued call for more tools to help families navigate these complex decisions.
Engaging Users in the Design
This study adds to the literature as an example of applying user-centered design to codesign, refine, and test a patient decision aid with older adults. There is an established literature on designing health technologies for older adults and user-centered design is an established approach in human factors research (Feather et al., 2016; Gustafson et al., 2016; Witteman et al., 2015). Integrating this multidisciplinary approach may improve decision aids designed for aging care. Recent reports of the Patient-Centered Outcomes Research Institute and the National Quality Forum call for increased stakeholder involvement in the design and development of patient decision aids (National Quality Forum, 2016; Patient-Centered Outcomes Research Institute, 2017). A recent systematic review indicates an increasing number of patient decision aid development studies are engaging stakeholders in the development process; however, the level of engagement still varies widely (Dugas et al., 2017; Witteman et al., 2015). Best practices are currently being identified.
Additionally, this study demonstrates the importance of meaningfully engaging multiple types of users in designing patient decision aid websites for aging care (Dugas et al., 2017). Results from this study complement that literature, and add potential design considerations specific to development of multiuser decision aids. Patient decision aid designs for multiple users face the challenge of balancing the needs of each users’ characteristics, decision-making role, deliberative style, and decision-support needs. For example, guidelines for designing websites for older adults generally recommend “shallow” hierarchies, that is, limiting each branch of webpages to four to five links deep (Chaudhuri et al., 2013; Norman & Skinner, 2006; Smith, 2014; Tennant et al., 2015). However, nested/branching architecture may provide information and decision-making support tailored to each voice (first-, second-, and third-person), role (patient, family caregiver, surrogate, and legal proxy), and for the needs of multiple users (values clarification vs communication management and conflict resolution).
Engaging all potential users also highlighted the need for extensive user-testing of the wording of multiple-user decision aids. For example, participants drew attention to the generational-heuristics of the language, with an emphasis on formal wording and traditional grammar over common/familiar language, as an indication that the site was well-crafted and worth trusting. In this case, combining elements of both language styles was preferred over providing another level of tailoring. Previous studies observed that older adults may be particularly vulnerable to limited health literacy and challenges associated with eHealth literacy, which is “the ability to seek, find, understand, and appraise health information from electronic sources and apply the knowledge gained to addressing or solving a health problem” (McCaffery et al., 2013; Tennant et al., 2015). Since older adults are increasingly seeking health information on the Internet (Chaudhuri et al., 2013; Smith, 2014), engaging older adults in strategies to support eHealth literacy may enhance older adult use and engagement in Internet-based patient decision aids. Finally, significant iterative testing was also needed to provide parallel interactive activities for caregivers, and to optimize how information was presented back to each user in the printable summary.
Several limitations must be considered. The sample of individuals involved in the design of this decision aid focused on rural health and is representative of the dissemination area; however, a more racially/ethnically diverse sample will be needed to inform scalability. As a website, the decision aid may be limited for use in some populations; however, the intended users specifically requested providing the information and support on a website to extend access to shut-in elders, caregivers who needed support available after business hours, and families that wanted to be able to coordinate information and communication with local and long-distance family members. Finally, patient advisors noted the limitations of the tool for providing locally specific information and costs; however, long-term care counselors on our stakeholder panel advised us that this information is proprietary, not publically available, and would need to be updated too often to be practical. Providing accurate cost predictions would require viewers entering their personal financial information online, which was not acceptable to participants. Hence, the purpose of the tool remains to help families become informed and facilitate discussions with their doctors, local aging resource center, lawyers, care facilities, and long-term care counselors, etc.
In conclusion, a decision aid website that provides tailored information and deliberative support is feasible, useful, and acceptable for older adults and families considering long-term care options. Engaging a stakeholder advisory panel and a variety of potential users enhanced this research and the design of the decision aid website, and provided additional insights for developers of decision aids for aging care. A suite of patient decision aids designed with and for older adults and their families could reduce the decisional burden for caregivers and improve decision making, coordination, and communication for families.
Supplementary Material
Supplementary data are available at The Gerontologist online.
Appendix A: Supplementary Material—Patient Decision Aid Source Document.
Funding
This work was supported by the Administration on Aging (GS23F9840H) through a contract from the Lewin Group (TLG100545269), and by a National Institute for Mental Health National Research Scholars Award with the Health Services Research Training Collaboratory (T32MH073553 to A.S.H.).
Acknowledgments
The authors would like to thank the participants, clients, and staff of the Dartmouth Aging Resource Center, Memory Clinic, and Memory Café, as well as the annual Jeanne Anderson Alzheimer’s Conference, who provided valuable insight and advice to improve the study design and the long-term care decision aid. Dr. Hoffman would also like to acknowledge and thank Drs. Stephen J. Bartels, Dawn Stacey, Hilary Llewellyn-Thomas, and Annette O’Connor for their expertise and generous mentorship, and Drs. Martha Bruce, Jurgen Unutzer, Fred Blow, George Neiderehe, Jovier Evans, Bob Santulli, Susan Mitchell, and the faculty of the Summer Research Institute of the Health Services Research Training Collaboratory for their expertise and guidance on the design of this study. This manuscript reports results according to the International Patient Decision Aid Standards (IPDAS) Standards for UNiversal reporting of Decision Aid Evaluation studies (SUNDAE Reporting Checklist) (Hoffman et al., 2018; Sepucha et al., 2018).
Author Contributions: All authors contributed substantially to this paper, including the conceptualization (A.S.H., S.J.B.) and conduct of the study (A.S.H., C.G., S.P., L.D., H.P., C.T.); design and programming of the intervention (A.S.H., D.B.H., C.G., S.P.), interpretation of the results (A.S.H., D.B., A.J.H., A.B.); and drafting (A.S.H., D.B., A.J.H., A.B.), and editing and final review of the manuscript (all authors).
Conflict of Interest
Planning for Long Term Care was developed at the Dartmouth Centers for Health and Aging. The website is provided free of charge and the Decisions in the Grey workgroup declares that they receive no direct financial benefit from the website or from the decisions women make based on using the decision aid. A.S.H. is a decision scientist who specializes in designing websites to help people make well-informed decisions. S.J.B. and D.R.B. are geriatric psychiatrists who have helped many families face this decision. No funding was received from any commercial, for-profit businesses.
References
- Alzheimer’s Association (2017). 2017 Alzheimer’s disease facts and figures. Alzheimer’s & Dementia, 13, 325–373. doi:10.1016/j.jalz.2017.02.001 [DOI] [PubMed] [Google Scholar]
- Bandura A. (1989). Human agency in social cognitive theory. The American Psychologist, 44, 1175–1184. doi:10.1037/0003-066x.44.9.1175 [DOI] [PubMed] [Google Scholar]
- Chaudhuri S., Le T., White C., Thompson H., & Demiris G (2013). Examining health information-seeking behaviors of older adults. Computers, Informatics, Nursing, 31, 547–553. doi:10.1097/01.NCN.0000432131.92020.42 [DOI] [PMC free article] [PubMed] [Google Scholar]
- Chesser A., Burke A., Reyes J., & Rohrberg T (2016). Navigating the digital divide: A systematic review of eHealth literacy in underserved populations in the United States. Informatics for Health & Social Care, 41, 1–19. doi:10.3109/17538157.2014.948171 [DOI] [PubMed] [Google Scholar]
- Coulter A., Stilwell D., Kryworuchko J., Mullen P. D., Ng C. J., & van der Weijden T (2013). A systematic development process for patient decision aids. BMC Medical Informatics and Decision Making, 13 (Suppl 2), S2. doi: 10.1186/1472-6947-13-S2-S2 [DOI] [PMC free article] [PubMed] [Google Scholar]
- Dening K. H., Jones L., & Sampson E. L (2011). Advance care planning for people with dementia: A review. International Psychogeriatrics, 23, 1535–1551. doi:10.1017/S1041610211001608 [DOI] [PubMed] [Google Scholar]
- Dugas M., Trottier M. È., Chipenda Dansokho S., Vaisson G., Provencher T., Colquhoun H.,…Witteman H. O (2017). Involving members of vulnerable populations in the development of patient decision aids: A mixed methods sequential explanatory study. BMC Medical Informatics and Decision Making, 17, 12. doi:10.1186/s12911-016-0399-8 [DOI] [PMC free article] [PubMed] [Google Scholar]
- Elstein A. S. (1999). Heuristics and biases: Selected errors in clinical reasoning. Academic Medicine, 74, 791–794. doi:10.1097/00001888-199907000-00012 [DOI] [PubMed] [Google Scholar]
- Etters L., Goodall D., & Harrison B. E (2008). Caregiver burden among dementia patient caregivers: A review of the literature. Journal of the American Academy of Nurse Practitioners, 20, 423–428. doi:10.1111/j.1745-7599.2008.00342.x [DOI] [PubMed] [Google Scholar]
- Fagerlin A., Pignone M., Abhyankar P., Col N., Feldman-Stewart D., Gavaruzzi T.,…Witteman H. O (2013). Clarifying values: An updated review. BMC Medical Informatics and Decision Making, 13 (Suppl 2), S8. doi:10.1186/1472-6947-13-s2-s8 [DOI] [PMC free article] [PubMed] [Google Scholar]
- Feather J. S., Howson M., Ritchie L., Carter P. D., Parry D. T., & Koziol-McLain J (2016). Evaluation methods for assessing users’ psychological experiences of web-based psychosocial interventions: A systematic review. Journal of Medical Internet Research, 18, e181. doi:10.2196/jmir.5455 [DOI] [PMC free article] [PubMed] [Google Scholar]
- Ferron Parayre A., Labrecque M., Rousseau M., Turcotte S., & Legare F (2014). Validation of SURE, a four-item clinical checklist for detecting decisional conflict in patients. Medical Decision Making, 34, 54–62. doi:10.1177/0272989X13491463 [DOI] [PubMed] [Google Scholar]
- Gordon N. P., & Hornbrook M. C (2016). Differences in access to and preferences for using patient portals and other eHealth technologies based on race, ethnicity, and age: A database and survey study of seniors in a large health plan. Journal of Medical Internet Research, 18, e50. doi:10.2196/jmir.5105 [DOI] [PMC free article] [PubMed] [Google Scholar]
- Gustafson D. H. Jr, Maus A., Judkins J., Dinauer S., Isham A., Johnson R.,…Atwood A. K (2016). Using the NIATx model to implement user-centered design of technology for older adults. JMIR Human Factors, 3, e2. doi:10.2196/humanfactors.4853 [DOI] [PMC free article] [PubMed] [Google Scholar]
- Healthwise (2012). Alzheimer’s or other dementia: Should I move my relative into long-term care? Retrieved from https://www.healthwise.net/ohridecisionaid/Content/StdDocument.aspx? DOCHWID=aa58684 [Accessed October 25, 2019] [Google Scholar]
- Hirschman K. B., Xie S. X., Feudtner C., & Karlawish J. H (2004). How does an Alzheimer’s disease patient’s role in medical decision making change over time? Journal of Geriatric Psychiatry and Neurology, 17, 55–60. doi:10.1177/0891988704264540 [DOI] [PubMed] [Google Scholar]
- Hoffman A. S., Bateman D. R., Bartels S. J., Blandin K., Santulli R. B., Lee H., & Tang C. (2013a). Patient and caregiver needs and preferences for decision support interventions in Alzheimer’s disease. American Journal of Geriatric Psychiatry, 21, S98–S99. doi: 10.1016/j.jagp.2012.12.130 [Google Scholar]
- Hoffman A. S., Llewellyn-Thomas H. A., Tosteson A. N., O’Connor A. M., Volk R. J., Tomek I. M.,…Bartels S. J (2014). Launching a virtual decision lab: Development and field-testing of a web-based patient decision support research platform. BMC Medical Informatics and Decision Making, 14, 112. doi: 10.1186/s12911-014-0112-8 [DOI] [PMC free article] [PubMed] [Google Scholar]
- Hoffman A. S., Sepucha K. R., Abhyankar P., Sheridan S., Bekker H., LeBlanc A.,…Thomson R (2018). Explanation and elaboration of the Standards for UNiversal reporting of patient Decision Aid Evaluations (SUNDAE) guidelines: Examples of reporting SUNDAE items from patient decision aid evaluation literature. BMJ Quality & Safety, 27, 389–412. doi:10.1136/bmjqs-2017-006985 [DOI] [PMC free article] [PubMed] [Google Scholar]
- Hoffman A. S., Volk R. J., Saarimaki A., Stirling C., Li L. C., Härter M.,…Llewellyn-Thomas H. A. (2013b). Delivering patient decision aids on the internet: Definitions, theories, current evidence, and emerging research areas. BMC Medical Informatics and Decision Making, 13 (Suppl 2), S13. doi:10.1186/1472-6947-13-S2-S13 [DOI] [PMC free article] [PubMed] [Google Scholar]
- Joseph-Williams N., Newcombe R., Politi M., Durand M. A., Sivell S., Stacey D.,…Elwyn G (2014). Toward minimum standards for certifying patient decision aids: A modified Delphi consensus process. Medical Decision Making, 34, 699–710. doi:10.1177/0272989X13501721 [DOI] [PubMed] [Google Scholar]
- Marks M. A., & Arkes H. R (2008). Patient and surrogate disagreement in end-of-life decisions: Can surrogates accurately predict patients’ preferences? Medical Decision Making, 28, 524–531. doi:10.1177/0272989X08315244 [DOI] [PubMed] [Google Scholar]
- Mayo Clinic (2012). Alzheimer’s: Consider options for long-term care. Retrieved from https://www.mayoclinic.org/healthy-lifestyle/caregivers/in-depth/alzheimers/art-20047171 [Accessed October 25, 2019] [Google Scholar]
- McCaffery K. J., Holmes-Rovner M., Smith S. K., Rovner D., Nutbeam D., Clayman M. L.,…Sheridan S. L (2013). Addressing health literacy in patient decision aids. BMC Medical Informatics and Decision Making, 13 (Suppl 2), S10. doi:10.1186/1472-6947-13-s2-s10 [DOI] [PMC free article] [PubMed] [Google Scholar]
- Miller S. M. (1995). Monitoring versus blunting styles of coping with cancer influence the information patients want and need about their disease. Implications for cancer screening and management. Cancer, 76, 167–177. doi: 10.1002/1097-0142(19950715)76:2<167::aid-cncr2820760203>3.0.co;2-k [DOI] [PubMed] [Google Scholar]
- Murray M. A. (2010). When you need extra care, should you receive it at home or in a facility? Ottawa, Canada: Ottawa Hospital Research Institute. [Google Scholar]
- National Quality Forum (2016). National standards for the certification of patient decision aids Washington DC: National Quality Forum; Retrieved from http://www.qualityforum.org/Publications/2016/12/National_Standards_for_the_Certification_of_Patient_Decision_Aids.aspx [Accessed October 25, 2019] [Google Scholar]
- Norman D. C., & Skinner A. H (2006). eHealth literacy: Essential skills for consumer health in a networked world. Journal of Medical Internet Research, 8, e9. doi:10.2196/jmir.8.2.e9 [DOI] [PMC free article] [PubMed] [Google Scholar]
- O’Connor A. M., Cranney A. (1996. [Updated 2002]). Acceptability - user manual Ottawa, Canada: Ottawa Hospital Research Institute; Retrieved from https://decisionaid.ohri.ca/eval_accept.html [Accessed October 25, 2019] [Google Scholar]
- O’Connor A. M., Tugwell P., Wells G. A., Elmslie T., Jolly E., Hollingworth G.,…Drake E (1998). A decision aid for women considering hormone therapy after menopause: Decision support framework and evaluation. Patient Education and Counseling, 33, 267–279. doi:10.1016/s0738-3991(98)00026-3 [DOI] [PubMed] [Google Scholar]
- Patient-Centered Outcomes Research Institute, M. C (2017). The PCORI methodology report Retrieved from https://www.pcori.org/sites/default/files/PCORI-Methodology-Report.pdf [Accessed October 25, 2019]
- Roth D. L., Fredman L., & Haley W. E (2015). Informal caregiving and its impact on health: A reappraisal from population-based studies. The Gerontologist, 55, 309–319. doi:10.1093/geront/gnu177 [DOI] [PMC free article] [PubMed] [Google Scholar]
- Schapira M. M., Aggarwal C., Akers S., Aysola J., Imbert D., Langer C.,…Fraenkel L (2016). How patients view lung cancer screening. The role of uncertainty in medical decision making. Annals of the American Thoracic Society, 13, 1969–1976. doi:10.1513/AnnalsATS.201604-290OC [DOI] [PMC free article] [PubMed] [Google Scholar]
- Sepucha K. R., Abhyankar P., Hoffman A. S., Bekker H. L., LeBlanc A., Levin C. A.,…Thomson R (2018). Standards for UNiversal reporting of patient decision aid evaluation studies: The development of SUNDAE checklist. BMJ Quality & Safety, 27, 380–388. doi:10.1136/bmjqs-2017-006986 [DOI] [PMC free article] [PubMed] [Google Scholar]
- Shalowitz D. I., Garrett-Mayer E., & Wendler D (2006). The accuracy of surrogate decision makers: A systematic review. Archives of Internal Medicine, 166, 493–497. doi:10.1001/archinte.166.5.493 [DOI] [PubMed] [Google Scholar]
- Smith A. (2014). Older adults and technology use Pew Research Center, Internet & American Life Project; Retrieved from http://www.pewinternet.org/2014/04/03/older-adults-and-technology-use/ [Accessed October 25, 2019] [Google Scholar]
- Stacey D., Kryworuchko J., Belkora J., Davison B. J., Durand M.-A., Eden K. B.,…Street R. L (2013). Coaching and guidance with patient decision aids: A review of theoretical and empirical evidence. BMC Medical Informatics and Decision Making, 13 (Suppl 2), S11. doi:10.1186/1472-6947-13-S2-S11 [DOI] [PMC free article] [PubMed] [Google Scholar]
- Stacey D., Légaré F., Lewis K., Barry M. J., Bennett C. L., Eden K. B.,…Trevena L (2017). Decision aids for people facing health treatment or screening decisions. The Cochrane Database of Systematic Reviews, 4, CD001431. doi:10.1002/14651858.CD001431.pub5 [DOI] [PMC free article] [PubMed] [Google Scholar]
- Stirling C., Andrews S., Croft T., Vickers J., Turner P., & Robinson A (2010). Measuring dementia carers’ unmet need for services–an exploratory mixed method study. BMC Health Services Research, 10, 122. doi: 10.1186/1472-6963-10-122 [DOI] [PMC free article] [PubMed] [Google Scholar]
- Tennant B., Stellefson M., Dodd V., Chaney B., Chaney D., Paige S., & Alber J (2015). eHealth literacy and Web 2.0 health information seeking behaviors among baby boomers and older adults. Journal of Medical Internet Research, 17, e70. doi:10.2196/jmir.3992 [DOI] [PMC free article] [PubMed] [Google Scholar]
- Thompson C. C. A., & Spilsbury K (1998). Support for carers of people with Alzheimer’s type dementia. Cochrane Database of Systematic Reviews, CD000454. doi:10.1002/14651858.CD000454 [DOI] [PMC free article] [PubMed] [Google Scholar]
- Tunney R. J., & Ziegler F. V (2015). Toward a psychology of surrogate decision making. Perspectives on Psychological Science, 10, 880–885. doi:10.1177/1745691615598508 [DOI] [PubMed] [Google Scholar]
- Tversky A., & Kahneman D (1974). Judgment under uncertainty: Heuristics and biases. Science (New York, N.Y.), 185, 1124–1131. doi:10.1126/science.185.4157.1124 [DOI] [PubMed] [Google Scholar]
- Tversky A., & Kahneman D (1981). The framing of decisions and the psychology of choice. Science (New York, N.Y.), 211, 453–458. doi:10.1126/science.7455683 [DOI] [PubMed] [Google Scholar]
- Witteman H. O., Dansokho S. C., Colquhoun H., Coulter A., Dugas M., Fagerlin A.,…Witteman W (2015). User-centered design and the development of patient decision aids: Protocol for a systematic review. Systematic Reviews, 4, 11. doi:10.1186/2046-4053-4-11 [DOI] [PMC free article] [PubMed] [Google Scholar]
- Woodard T. L., Hoffman A. S., Covarrubias L. A., Holman D., Schover L., Bradford A.,…Volk R. J (2018). The Pathways fertility preservation decision aid website for women with cancer: Development and field testing. Journal of Cancer Survivorship: Research and Practice, 12, 101–114. doi:10.1007/s11764-017-0649-5 [DOI] [PubMed] [Google Scholar]
Associated Data
This section collects any data citations, data availability statements, or supplementary materials included in this article.