Abstract
Purpose
The-Optimal-Lymph-Flow health IT system (TOLF) is a patient-centered, web-and-mobile-based educational and behavioral health IT system focusing on safe, innovative, and pragmatic self-care strategies for lymphedema symptom management. The purpose of this study was to evaluate usability, feasibility, and acceptability of TOLF among the end-user of breast cancer survivors.
Methods
Two types of usability testing were completed with 30 breast cancer survivors: heuristic evaluation and end-user testing. Each participant was asked to think aloud while completing a set of specified tasks designed to explicate and freely explore the system features. A heuristic evaluation checklist, the Perceived Ease of Use and Usefulness Questionnaire, and The Post Study System Usability Questionnaire were used to evaluate usability of the system. Open-ended questions were used to gather qualitative data. Quantitative data were analyzed using descriptive statistics and qualitative data were summarized thematically.
Results
Breast cancer survivors were very satisfied with the system: 90% (n = 27) rated the system having no usability problems; 10% (n = 3) noted minor cosmetic problems: spelling errors or text font size. The majority of participants 96.6% (n = 29) strongly agreed that the system was easy to use and effective in helping to learn about lymphedema, symptoms and self-care strategies. Themes from the qualitative data included empowerment, high quality information, loving avatar simulation videos, easy accessibility, and user-friendliness.
Conclusions
This usability study provided evidence on breast cancer survivor's acceptance and highly positive evaluation of TOLF's usability as well as feasibility of using technologically-driven delivery model to enhance self-care strategies for lymphedema symptom management.
Abbreviation: TOLF, The-Optimal-Lymph-Flow health IT system
Keywords: Self-care, Lymphedema symptoms, Health IT, Usability, Feasibility, Breast neoplasms, Breast cancer survivor, Lymphedema, Symptoms
Highlights
-
•
Many breast cancer survivors suffer from daily distressing symptoms related to lymphedema, requires ongoing symptom management.
-
•
Health information technology (IT) plays a significant role in improving self-care, patient-clinician communication, and access to health information by patients.
-
•
The-Optimal-Lymph-Flow health IT system (TOLF) is a patient-centered, web-and-mobile-based educational and behavioral health IT system focusing on safe, innovative, and pragmatic self-care strategies for lymphedema symptom managemet.
-
•
This usability study provided evidence that supports the usefulness of TOLF in terms of breast cancer survivor’s acceptance, user-friendliness, and feasibility of using technologically-driven delivery model to enhance self-care strategies for lymphedema symptom management.
1. Introduction
Each year, 1.38 million women worldwide and > 230,000 women in the United States are diagnosed with breast cancer (American Cancer Society (ACS), 2016, Jemal et al., 2011). Many breast cancer survivors suffer from daily distressing symptoms related to lymphedema [hereafter, lymphedema symptoms (Armer et al., 2003, Fu and Rosedale, 2009, Fu et al., 2015a)]. Lymphedema, an abnormal accumulation of lymph fluid in the ipsilateral body area or upper limb, remains an ongoing major health problem affecting > 40% of 3.1 million breast cancer survivors in the United States (Armer and Stewart, 2010, Petrek et al., 2001). Lymphedema symptoms include arm swelling, breast swelling, chest wall swelling, heaviness, firmness, tightness, stiffness, pain, aching, soreness, tenderness, numbness, burning, stabbing, tingling, arm fatigue, arm weakness, and limited movement in shoulder, arm, elbow, wrist/fingers in the ipsilateral body area or upper limb (Armer et al., 2003, Fu and Rosedale, 2009, Fu et al., 2015a). Even breast cancer survivors without a diagnosis of lymphedema suffer from pain (40%), tenderness (47.3%), aching (30%), or soreness (32.7%), tightness (34.7%), limited shoulder movement (28%), arm firmness (24%), arm swelling (17.3%), arm heaviness (14.4%) while significantly more breast cancer survivors with lymphedema experience pain (45.2%), tenderness (52.4%), aching (61.9%), soreness (31%), tightness (71.4%), limited shoulder movement (52.4%), arm firmness (69%), arm swelling (100%), arm heaviness (71.4%) in the ipsilateral upper limb or body (Fu et al., 2015a). The experience of lymphedema symptoms has been linked to clinically relevant and detrimental outcomes, such as disability and psychological distress, both of which are known risk factors for breast cancer survivors' poor quality of life (QOL) (Fu and Rosedale, 2009, Fu et al., 2013). More importantly, lymphedema symptoms may indicate an early stage of lymphedema in which changes cannot be detected by current objective measures of limb volume or lymph fluid level (Armer et al., 2003, Armer and Stewart, 2010). Without timely intervention in this early disease stage, lymphedema can progress into a chronic condition that no surgical or medical interventions at present can cure (Fu et al., 2014a, Fu et al., 2014b).
Patient-centered care is recognized as an essential aim of healthcare, and is fundamental to healthcare quality (IOM, 2001). Unfortunately, patient-centered care related to lymphedema symptom management is often not adequately considered in clinical research and practice. The lack of patient-centered care for lymphedema symptoms has been evidenced by > 40% of breast cancer survivors never receiving information about lymphedema (Fu et al., 2010a, Ridner et al., 2011a). In spite of the growing body of evidence linking the experience of lymphedema symptoms to poor QOL which are critical to cancer survivorship (Fu et al., 2013, Passik and McDonald, 1998, Ridner, 2005, Radina et al., 2015), the experience of lymphedema symptoms is often not valued as an essential factor that impact breast cancer survivors' QOL, therefore is not considered essential for patient-centered outcome, assessment or inclusion in clinical research and practice. In addition, oftentimes clinicians' intentions for helping breast cancer survivors are impeded by lack of information about lymphedema symptoms, lack of guidance in how to assess lymphedema symptoms, and lack of standardized and effective interventions for managing lymphedema symptoms (Fu et al., 2012, Ryan et al., 2012). Routine assessment and effective interventions, are key to patient-centered care to improve lymphedema symptom management. Interventions using health information technology (IT), which improves information access, hold potential to address these key issues of lymphedema symptom management.
One of the most promising ways to support the delivery of patient-centered care is through the use of health IT. Health IT is broadly defined as “the use of information and communication technology in health care to support the delivery of patient or population care or to support patient self-management” (Finkelstein et al., 2012, Shortliffe and Sondik, 2006). Health IT plays a significant role in improving self-care, patient-clinician communication, and access to health information by patients. A growing body of evidence has confirmed the positive impact of health IT which supports patient-centered care on health care outcomes (Hoffman et al., 2014, Shortliffe and Sondik, 2006). The-Optimal-Lymph-Flow health IT system (TOLF) is a patient-centered, web-and-mobile-based educational and behavioral health IT system focusing on safe, innovative, and pragmatic self-care strategies for lymphedema symptom managemet (Fu et al., 2016). TOLF uses a technologically-driven delivery model to enhance self-care for managing lymphedema symptoms. The purpose of this study is to evaluate usability, feasibility, and acceptability of TOLF among the end-user of breast cancer survivors.
2. Methods
2.1. Development of The-Optimal-Lymph-Flow health IT system (TOLF)
The development of TOLF was guided by the Model of Self-Care for Lymphedema Symptom Management based on our prior research (Fu, 2010b, Fu et al., 2014a, Fu et al., 2014b, Ridner et al., 2012). Fig. 1. The model views symptoms as subjective phenomenon that indicates abnormal changes in body functioning or side effects from cancer treatment (Fu et al., 2004, Deng et al., 2014). This definition of lymphedema symptoms is supported by research evidence that lymphedema symptoms are significantly associated with lymphedema defined by > 200 ml or 10% limb volume increase (Cormier et al., 2009, Fu et al., 2015a). Self-care for lymphedema symptoms refers to activities that individuals initiate and perform for themselves, without professionally administered treatments (e.g., by lymphedema therapists or nurses) (Fu, 2010b, Ridner et al., 2012). Since the experience of lymphedema symptoms leads to poor QOL (performance outcome) in breast cancer survivors (Fu et al., 2013, Passik and McDonald, 1998, Ridner, 2005, Radina et al., 2015) and the increased limb volume and lymph fluid level (physiological outcomes) (Cormier et al., 2009, Fu et al., 2015a), the goal of self-care is to prevent, relieve or decrease lymphedema symptom occurrence (i.e. number and severity of symptoms) so as to improve QOL and achieve optimal limb volume and lymph fluid level.
Fig. 1.
Model of Self-Care for Lymphedema Symptom Management.
While more research is needed to explore the exact etiology of lymphedema symptoms after breast cancer treatment, physiologically, the accumulation of lymph fluid in the affected area or limb can create undue pressure on nerves, producing feelings of pain, aching, tenderness, soreness, burning, tingling, stabbing, and numbness as well as inducing sensations of swelling, heaviness, tightness, and firmness (Fu et al., 2015a, Stanton et al., 2009). Accumulated lymph fluid in the affected area or limb also leads to stiffness and limited limb movement of arm, shoulder, elbow, wrist, and fingers (Fu et al., 2015a, Stanton et al., 2009). Significant associations are found between lymphedema symptoms and accumulation of lymph fluid in the ipsilateral upper limb (Cormier et al., 2009, Fu et al., 2015a). Breast cancer survivors are known to have a compromised lymphatic system due to cancer surgery, dissection of lymph nodes and vessels, and radiation, leading to ineffective lymphatic drainage, thus accumulated lymph fluid in the affected area or limb (Mak et al., 2008, Norman et al., 2010, Tsai et al., 2009). In addition to the risk factor of compromised lymphatic drainage from cancer treatment, higher BMI is also an established risk factor for the accumulation of lymph fluid as well as predictor of lymphedema (Fu et al., 2015b, Mak et al., 2008, Ridner et al., 2011b). Physiologically, a larger body mass creates a disproportion in lymph transport and capacity, resulting in excess extracellular fluid (Stanton et al., 2009). A prospective study (Ridner et al., 2011b) followed 138 breast cancer survivors for 30 months after cancer diagnosis. Individuals with BMI of 30 or higher at the time of diagnosis were 3.6 times more likely to develop lymphedema. Another prospective study shows that women are 1.1 times more at risk for developing lymphedema with every increase of 1 kg/m2 in their BMI (Mak et al., 2008).
Although the known risk factors for lymphedema symptoms directly from cancer treatment cannot be avoided (such as removal of lymph nodes, surgery, radiation, chemotherapy, and hormonal therapy) (Mak et al., 2008, Norman et al., 2010, Tsai et al., 2009), some personal factors (such as compromised lymphatic drainage and higher BMI) can be modified through self-care strategies (Fu et al., 2014a, Fu et al., 2014b). The-Optimal-Lymph-Flow self-care program features an innovative, safe, feasible and easily-integrated-into-daily-routine self-care strategies (Fu et al., 2014a, Fu et al., 2014b). By targeting compromised lymphatic system and BMI, self-care to promote lymph flow aims to lessen lymphedema symptom burden through daily lymphatic exercise and keeping a healthy weight (Fu et al., 2014a, Fu et al., 2014b). Daily lymphatic exercises (i.e. muscle-tightening breathing and muscle-tightening pumping exercises) are needed to promote lymph flow since lymph fluid accumulates every 3 s and lymph fluid only drains during deep breathing that stimulates lymphatic ducts and during muscle movements that create muscle milking and pumping action (Stanton et al., 2009). Building self-care skills to promote lymph flow through daily lymphatic exercises and keeping a healthy weight has been shown to be effective in reducing the risk of lymphedema and relieving symptoms (Fu et al., 2014a, Fu et al., 2014b). Table 1 presents self-care strategies, rationales, & actions.
Table 1.
The-Optimal-Lymph-Flow: self-care strategies, rationales, and actions (Fu et al., 2014a, Fu et al., 2014b).
| Strategies | Rationales | Actions | |
|---|---|---|---|
| Promote lymph flow |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| Keep a healthy weight |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
Building self-care skills for managing lymphedema symptoms is a process of making a health habit in which breast cancer survivors initiate and perform activities to prevent, relieve or decrease lymphedema symptom occurrence (i.e. number and severity of symptoms) so as to improve QOL and achieve optimal limb volume difference and lymph fluid level. Patients' evaluation of lymphedema symptoms as a risk for the onset or worsening of lymphedema is the key for initiation and sustained self-care activities; while sustained self-care behavior change is maintained by the contextual personal factors identified by the participants in our research: patients' perceived effectiveness of the intervention (“I personally feel the exercises helped with the pain and heaviness”), perceived ease to integrate the self-care strategies into daily life (“The exercises were easy and could be completed anywhere,” “I love the simple 75% full for each meal and more fruits and vegetables…I can do it”) and performance of self-care behaviors (Fu, 2010b, Fu et al., 2014a, Fu et al., 2014b). Thus, interventions developed based on the model aim to build self-care skills that are easy to integrate into breast cancer survivors' daily lives and that is effective to relieve symptoms. Self-care skills for lymphedema symptom management include symptom evaluation, daily lymphatic exercises, strategies for optimal BMI, and situational self-care strategies. TOLF hosts the essential health information needed for building self-care skills for managing lymphedema symptoms (Fu et al., 2016). The health information includes: lymphedema and symptoms, lymphedema diagnosis and measurement, lymphatic system, risk of lymphedema, self-care to promote lymph flow, symptom assessment, daily lymphatic exercises to promote lymph flow, ask experts, and self-care strategies based on conventional arm precaution recommendations and limb mobility exercises which can serve as standard care or control treatment.
2.2. TOLF system design
We designed TOLF to enhance training and motivating self-care to promote lymph flow among breast cancer survivors. Since understanding the meaning of the experience of lymphedema symptoms can motivate patients to practice self-care strategies (Fu, 2010b, Ridner et al., 2012), we designed symptom evaluation using web-and-mobile based health IT system and research-based, valid, and reliable instrument for breast cancer survivors to evaluate and monitor their lymphedema symptom experience virtually anytime and anywhere (Fu et al., 2016). Upon the submission of their symptom report, the patients will receive a symptom evaluation in terms of fluid accumulation and recommended self-care strategies immediately. Using avatar technology (Fig. 2), we designed a novel training system to assist breast cancer survivors in building self-care skills by visually showing how lymph fluid drain in the lymphatic system when performing lymphatic exercises, a request from breast cancer survivors in our prior face-to-face, nurse-delivery study. Patients can learn the daily lymphatic exercises by watching and following the avatar simulation videos. The avatar model, a fit and healthy woman of more than one race background was chosen for the videos by 35 breast cancer survivors to embrace patients with different backgrounds and healthy image that breast cancer survivors strive for. This not only enhances the fidelity and transparency of the intervention delivery but also the reproducibility of the intervention. To minimize potential technical and internet barriers for patients to use TOLF, we iteratively refined TOLF to ensure that even patients with minimal technical skill in using computer or any electronic devices or smartphones can download TOLF to computers, laptops, iPad or other electronic devices or smart phones; patients can use the system by just scrolling up and down and clicking on or touching icons denoting specific videos or intervention information. The homepage of TOLF with an introduction of the system, avatar simulation video of lymphatic system and patients' testimonies can be accessed through this link (http://optimallymph.org).
Fig. 2.
Screenshots from Avatar Simulation Videos.
2.3. Preliminary heuristic evaluation of TOLF
We completed heuristic evaluation with experts in human-computer interaction (HCI). Heuristic evaluation is a usability inspection method in which a relatively small group of experts examine the extent to which a user interface meets Nielsen's principles for usability (Nielsen & Mack, 1994; Kreuter, 2000). These include visibility of system status, the match between system and the real world, user control and freedom, recognition rather than recall, and flexibility and efficiency of use (Kreuter, 2000, Nielsen and Landauer, 1993). We had 15 usability experts (formally prepared in HCI and experienced in the design of interfaces) to participate in the preliminary heuristic evaluation of the initial TOLF prototype. Each expert completed a set of specified tasks designed to explicate system features and also freely explore the prototype. They completed a heuristic evaluation checklist that includes ratings of the severity of heuristic violations (no usability problem, cosmetic problem, minor usability problem, major usability problem, usability catastrophe) and provide additional comments regarding the interface (Bright et al., 2006). Only minor cosmetic problems were identified from the experts, such as making the font bigger, the identification of spelling errors, and comments on repeated information. We iteratively refined TOLF system based upon the feedback of heuristic evaluation before we conducted the end-user testing to evaluated usability, feasibility, and acceptability of TOLF among breast cancer survivors.
2.4. Usability evaluation
2.4.1. Study participants
Our general pool of potential subjects was chosen as they would be representative of the future end-users of breast cancer survivors. After the approval of Institutional Review Board from the study institute in the metropolitan area of New York, U.S., we recruited 30 English-speaking breast cancer survivors over 21 years of age who had the various experience of using the internet to evaluate TOLF from October to December 2014. Participants provided written informed consent. Strategies to protect human subjects were ensured by following the guidelines set forth by the Institutional Review Board and successful recruitment procedures used in our prior studies (Fu et al., 2010a, Fu et al., 2014a, Fu et al., 2014b, Fu et al., 2015a, Fu et al., 2015b).
2.4.2. Study procedure
We conducted two types of usability testing on TOLF: heuristic evaluation and end-user testing with 30 breast cancer survivors, adequate sample size for usability testing (Nielsen and Landauer, 1993). At the start of each session, a researcher explained to each participant about the study procedures. Each participant was asked to think aloud while completing a set of specified tasks designed to explicate TOLF features and also to freely explore TOLF either using a laptop or any electronic devices they preferred. We recorded screen shots and participants' utterances using Morae software™ (Techsmith Corporation, Okemos, MI), which allowed to record and analyze the audio recording and screen shots that were captured during the evaluation. Following completion of the tasks, participants were asked to complete a heuristic evaluation checklist that included ratings of severity of heuristic violations (no usability problem, cosmetic problem, minor usability problem, major usability problem, and usability catastrophe) and provided additional comments regarding the interface. Participants were also asked to complete two brief end-user questionnaires regarding their perceptions of information quality and system quality, and their behavioral intention to use the system using The Perceived Ease of Use and Usefulness Questionnaire (Davis, 1989, Dillon et al., 1998) and The Post Study System Usability Questionnaire (Lewis, 1995). Finally, participants were asked to provide narrative responses to the open-ended questions, “what do you like about the system,” “what do you dislike about the system,” and “what can be improved?”
2.4.3. Evaluation instruments
2.4.3.1. Demographic information and internet experience
Prior to the usability evaluation, each participant completed a short questionnaire regarding demographic information (age, education, ethnicity, and English capability) and computer/internet experience and use.
2.4.3.2. The Perceived Ease of Use and Usefulness Questionnaire
The Perceived Ease of Use and Usefulness Questionnaire is a tool to evaluate users' acceptance of a new information system with a strong reliability for perceived usefulness and perceived ease of use (Davis, 1989, Dillon et al., 1998). We used the modified eight-items with 5-point Likert Scales ranging from 2 (strongly agree) to − 2 (strongly disagree) for this usability evaluation.
2.4.3.3. The Post Study System Usability Questionnaire
The Post Study System Usability Questionnaire originally is a 19-item survey instrument developed at IBM to assess user satisfaction with system usability on a scale ranging from 1 (strongly agree) to 7 (strongly disagree). We used modified 14-item tool from the original survey that focus on system usefulness and information quality (Lewis, 1995). The survey consists of three subscales accounted for 87% of the total variance: system usefulness, information quality and interface quality. The overall reliability of the survey was 0.97, and ranged from 0.91 to 0.96 for the three subscales (Lewis, 1995).
2.5. Data analysis
Audio recordings of the think-aloud protocols were verbatim transcribed and summarized thematically. Qualitative data from heuristic evaluation and responses to the open-ended questions were also summarized thematically. Demographic and quantitative data were analyzed using descriptive statistics. For the Post Study System Usability Questionnaire, items 7 (The system gave error messages that clearly told me how to fix problems), 8 (Whenever I made a mistake using the system, I could recover easily and quickly), and 9 (The information such as on-line help, [onscreen messages and other documentation] provided with this system was clear) were eliminated from the analysis as all participants marked “N/A (Not Applicable)” rather than assigned a numeric rating since the system does not require such functions for end-users to use the system.
3. Results
3.1. Participants
A total of 30 breast cancer survivors completed the usability study. Participants aged from 31 to 78 years, with mean age of 58.6 years and Standard Deviation (SD) of 11.39. Among the 30 participants, 73% (n = 22) self-identified as White or Caucasian, 20% (n = 6) as Black or African American, and 6.7% (n = 2) as Asian. Among the 30 participants, 13 participants (43.3%) had graduate degree or above; 13 had Bachelor's degree (43.3%); 3 (10%) had partial college and one had high school education (3.3%). The mean ratings for English language capacity to understand, read, speak and write, using a scale of 1–5, where 1 = almost none, 2 = some, 3 = functional, 4 = very good, and 5 = outstanding, were between very good to outstanding level with a mean of 4.8 for understanding, 4.98 for reading, 4.67 for speaking, and 4.70 writing, respectively. Participants reported weekly Internet use ranging from 5 to 70 h with a mean of 26.23 h (SD = 16.3). Using the same 1–5 scale noted above, 16.7% (n = 5) rated their computer skills as functional; 70% (n = 21) as very good; and 13.3% (n = 4) as outstanding. Table 2 provides more detailed information of the participants.
Table 2.
Demographic characteristics of the participants (N = 30).
| Participants' characteristics | Mean (Standard Deviation; range) |
|---|---|
| Age (in years) | 58.6 (11.4; 31–78) |
| Internet use (weekly hours) | 26.2 (16.3; 5–70) |
| Computer/internet skills | n (%) |
| Almost none | 0 (0.0) |
| Some | 0 (0.0) |
| Functional | 5 (16.7) |
| Very good | 21 (70.0) |
| Outstanding | 4 (13.3) |
| Highest level of education | n (%) |
| High school | 1 (3.3) |
| Partial college | 3 (10.0) |
| Bachelor's degree | 13 (43.3) |
| Master's degree or above | 13 (43.3) |
| English capability- understanding | n (%) |
| Almost none | 0 (0.0) |
| Some | 0 (0.0) |
| Functional | 1 (3.3) |
| Very good | 4 (13.3) |
| Outstanding | 25 (83.3) |
| English capability- reading | n (%) |
| Almost none | 0 (0.0) |
| Some | 0 (0.0) |
| Functional | 1 (3.3) |
| Very good | 4 (13.3) |
| Outstanding | 25 (83.3) |
| English capability- speaking | n (%) |
| Almost none | 0 (0.0) |
| Some | 0 (0.0) |
| Functional | 2 (6.7) |
| Very good | 6 (20.0) |
| Outstanding | 22 (73.3) |
| English capability- writing | n (%) |
| Almost none | 0 (0.0) |
| Some | 0 (0.0) |
| Functional | 2 (6.7) |
| Very good | 5 (16.7) |
| Outstanding | 23 (76.7) |
| Ethnicity | n (%) |
| African American or Black | 6 (20.0) |
| White or Caucasian | 22 (73.3) |
| Asian | 2 (6.7) |
3.2. Usability evaluation
3.2.1. Heuristic evaluation
The responses to Neilson's heuristics (Nielsen and Landauer, 1993) indicated no major usability problems or usability catastrophe with TOLF: 90% of participants (n = 27) rated the system having no usability problems; 10% (n = 3) noted minor cosmetic problems, such as spelling errors (Table 3). We iteratively refined TOLF based on the feedback from the participants.
Table 3.
Rating of the severity of heuristic violations.
| n (%) | |
|---|---|
| No usability problem | 26 (90%) |
| Cosmetic problem | 3 (10%) |
| Minor usability problem | 0 (0%) |
| Major usability problem | 0 (0%) |
| Usability catastrophe | 0 (0%) |
3.2.2. End-user testing
For all items of the Perceived Ease of Use and Usefulness Questionnaire, agreement for ease of use exceeded 93.3% (n = 28). There was no disagreement reported for ease of use except one participant reported neutral. The same participant reported neutral on Item 3, 4, 6, 7, 8 (Table 4).
Table 4.
The Perceived Ease of Use and Usefulness Questionnaire.
| N = 30 | Strongly agree n (%) | Agree n (%) | Neutral n (%) | Disagree n (%) | Strongly disagree N (%) |
|---|---|---|---|---|---|
| 1. I find the system easy to use. | 18 (60.0) | 11 (36.7) | 1 (3.3) | 0 (0) | 0 (0) |
| 2. Learning to operate the system is easy for me. | 22 (73.3) | 8 (26.7) | 0 (0) | 0 (0) | 0 (0) |
| 3. Interaction with the system is difficult. | 0 (0) | 0(0) | 1 (3.3) | 8 (26.7) | 20 (66.7) |
| 4. I find it easy to get the system to do what I want it to do. | 21 (70) | 8 (26.7) | 1 (3.3) | 0 (0) | 0 (0) |
| 5. The system is flexible to interact with. | 18 (60.0) | 12 (40) | 0 (0) | 0 (0) | 0 (0) |
| 6. It is easy for me to remember how to perform tasks using the system. | 15 (50) | 14 (46.7) | 1 (3.3) | 0 (0) | 0 (0) |
| 7. Interacting with the system requires a lot of mental effort. | 0 (0) | 1 (3.3) | 1 (3.3) | 12 (36.6) | 16 (53.3) |
| 8. My interaction with the system is clear and understandable. | 21 (70) | 8 (26.7) | 1 (3.3) | 0 (0) | 0 (0) |
For the Post Study System Usability Questionnaire, average ratings on all the items ranged from 1.1 to 1.4 on a scale ranging from 1 (strongly agree) to 7 (strongly disagree). Importantly, there were no neutral and disagreement scores (> 3) on any item (Table 5).
Table 5.
The Post-Study System Usability Questionnaire (N = 30).
| Items | Mean (SD)b | Rangea | ||
|---|---|---|---|---|
| System usefulness | 1 | Overall, I am satisfied with how easy it is to use this system. | 1.4 (0.36) | 1–2 |
| 2 | It was simple to use this system. | 1.4 (0.36) | 1–2 | |
| 3 | I could effectively find all the information using this system. | 1.4 (0.36) | 1–2 | |
| 4 | I felt comfortable using this system. | 1.3 (0.45) | 1–2 | |
| 5 | It was easy to learn to use this system. | 1.3 (0.45) | 1–2 | |
| 6 | I believe I can learn how to reduce the risk of lymphedema and manage lymphedema symptoms using this system. | 1.1 (0.25) | 1–2 | |
| Interface quality | 7 | The system gave error messages that clearly told me how to fix the problems. | Not applicable | |
| 8 | Whenever I made a mistake using the system, I could recover easily and quickly. | Not applicable | ||
| 9 | The information (such as on-line help, on-screen messages and other documentation) provided with this system was clear. | Not applicable | ||
| Information quality | 10 | It was easy to find the information I needed. | 1.1 (0.25) | 1–2 |
| 11 | The information provided for the system was easy to understand. | 1.2 (0.44) | 1–2 | |
| 12 | The information was effective in helping me to learn about lymphedema, symptoms and self-care strategies. | 1.1 (0.25) | 1–2 | |
| 13 | The organization of information on the system screens was clear. | 1.3 (0.45) | 1–2 | |
| 14 | Overall I am very satisfied with the system. | 1.1 (0.25) | 1–2 | |
On a scale of 1 to 7, 1 represents strongly agree and 7 represents strongly disagree.
SD: Standard Deviation.
3.3. Qualitative findings
Several themes emerged from the qualitative data analysis. These themes include empowerment, high quality information, loving avatar simulation videos, easy accessibility, user-friendly. Table 6 provides representative quotes for the identified themes.
Table 6.
Summary of Themes from Qualitative Data 30 (100%) participants Provided Qualitative Evaluation of The-Optimal-Lymph-Flow Health IT System.
| Themes | Representative quotes |
|---|---|
| Empowerment | ** I am so happy that I came for the study. It empowers me, any breast cancer patients should come. ** The website is empowering for the women who want to prevent lymphedema, as well as for those who already have lymphedema. ** It is a fabulous program. Women will love it. I learned so much about risk reduction. The videos are fabulous to show how to exercise & I love the why I need to do the exercise. This will be a great asset to patients. ** I liked the clarity of the explanations & the simplicity of using the website. The website is empowering for the women who want to prevent lymphedema, as well as for those who already have lymphedema. |
| High quality information | ** High quality information. Easy to understand, details included in each sections are clear to read and view. ** Excellent explanation of lymphedema-best I've seen. ** The system is excellent when I compare to the time years ago when I had breast cancer and was given Xerox copies of exercises to do. ** Lymphatic system, lymphedema and self-care are explained very clearly. It did not create panic but clearly stated why it is important to do the lymphatic exercises. ** Very well rounded/comprehensive approach to helping in reducing the risk of lymphedema. ** Love the self-care tips. ** Looked at many lectures on lymphedema but none as clear as this system. ** The option to read along, or just listen, while watching the videos. Great idea! ** Invaluable information. Great videos!— pictures and movies help me understand both the lymphatic system and the lymphatic exercises. ** Simple language provided clear understanding. The information was effective…very much so. The diagrams & process guides provided understanding & learning on a higher level. ** Very effective exercises. ** Love the information about what, when and how to recover and managing symptoms. |
| Loving avatar simulation videos | ** I like the exercises!!! After I finished learning the exercises by following the videos, my pain and soreness were much better. ** Videos are very helpful in teaching how to do the exercises. It is nice that I can go back and watch it again & again. I am glad that patients can have it at home. I love the contents & hope patients can get it sooner. I enjoyed the videos a lot. ** I love the videos that show the anatomy & fluid flow & deep breathing. I also love the lymphatic system video. ** The videos of how to perform the lymphatic exercises were very easy to follow. ** Videos are excellent-you could do as you watched. ** Love the avatar videos. You can follow the videos and do the exercises. |
| Easy accessibility | ** With website all can be solved & one can access at the time one wants. ** Not need to commute to hospital but stay at home to go through the videos and information to understand the important of managing lymphedema symptoms and reduce my risk. ** I enjoyed the study. I am glad that patients can have this at their fingertips. ** Can go back and review repeatedly; Portable!!! ** When you forget, easy to go back at anytime and anywhere. ** Self-paced; Can repeat & review particular sections ** I can repeat & review particular sections whenever and wherever I want. |
| User-friendly | ** User friendly and easy to navigate. ** Very simple, clear, approachable. ** It was easy to find the information. It covered the subject in a comprehensive method. I like having written instructions with the videos for the exercises. ** Layout is clear and concise. ** It was very easy to navigate the system. The video presentations were very clear and helpful. ** Nice layout- User friendly and easy to navigate. ** It was user friendly, easy to follow. It flowed at a reasonable rate and the diagrams were very easy to follow as well as the tone & temperament was calming thereby making it useable by all levels of individuals. It was not frustrating like other systems I have used. ** It was very clear, thorough, and informative. System is clear. I understand where to click and scroll. |
3.4. Pilot testing of TOLF
Currently, a parallel randomized clinical trial (RCT) (Fu et al., 2016) with a control-experimental, pre- and post-test, repeated-measures design is undergoing to recruit 120 breast cancer survivors with chronic pain and lymphedema symptoms. The goal of the RCT is to evaluate the effectiveness of using TOLF to manage chronic pain and lymphedema symptoms among breast cancer survivors. Preliminary effectiveness of using TOLF for pain management among breast cancer survivors was supported by the data from 33 patients who completed the RCT. Participants in the intervention group received the self-care to promote lymph flow while participant in the control group had self-care for arm precaution using TOLF. Participants were encouraged to enhance their learning by accessing TOLF and following the daily exercises during the study period. Participants also had monthly online self-report of pain and symptoms at 4 and 8 weeks post intervention. At study endpoint of 12-week after intervention, more participants who received the self-care to promote lymph flow (n = 17) reported significantly no pain (n = 6, 35%) and pain reduction (n = 16, 94%) than those who received self-care for arm precaution (n = 16) no pain (n = 1; 6%) and pain reduction (n = 4; 25%) (p < 0.00). Furthermore, the daily 5-minute routine avatar simulation video of lymphatic exercises provides a unique way of helping breast cancer survivors to establish their own self-care routine by following the video. As our patients remarked, “the video helped to complete the exercises.” This ongoing pilot clinical trial using TOLF will continue providing needed evidence for future research and clinical implementation using TOLF (Table 7). Qualitative Evaluation of TOLF from breast cancer survivors in the RCT.
Table 7.
Qualitative Evaluation of TOLF from breast cancer survivors in the ongoing Randomized Clinical Trial (Fu et al., 2016).
| 1. Summary of themes Thirty-three participants who completed the RCT provided unanimous highly positive qualitative feedback. | |
|---|---|
| Themes | Representative quotes |
| Skills building | ** It made me realize that I can manage the loss of strength in my right arm and may be able to manage the numbness and tingling. I will be able to help myself. ** Being on this study allowed me to do something for myself, really take care of myself and focus on being healthy. ** It helped me realize that I had excess fluid. My arms got lighter each time I did the exercises. My arms began to feel less heavy. It noticed it in my clothes as well. |
| Perceived ease | ** This is a very easy study and the videos helped to complete the exercise. ** I can repeat & review particular sections whenever and wherever I want. ** I like the fact that I can go to the site at any time even when I travel. ** The [lymphatic] exercises were easy and could be completed anywhere. According to my measurements there was a decrease in fluid. That was good news. ** The exercises were easy to do and remember. If I noticed my arm was more stiff than usual, I would do the exercises more and they helped. ** It is not about whether I can do it (self-care) or I feel I can do it. The breathing and pumping exercises (daily lymphatic exercise) are easily to do and I feel better after doing them. So, I do it every day. |
| Perceived effectiveness | ** I personally feel the exercises helped with the pain. ** The exercises made my arm feel a lot better. ** The exercises really helped increase my range of motion and was effective for decreasing my pain. I do the exercises every day. ** The exercises definitely helped reduce pain and increase mobility. ** The exercises really helped increase my range of motion and was effective for decreasing my pain. I do the exercises every day. |
| Psychological benefits: fear and anxiety relief |
** It afforded me the opportunity to feel better about myself. I was actually doing something to alleviate the anxiety about having symptoms and now I feel more in control of managing my physical and mental state. |
| Enjoy doing the exercises | ** I enjoyed the exercises! ** The exercises that I have been doing in this study helped me to gain strength and mobility in my arm even after I stopped physical therapy. I also found the exercises fun and enjoyable. |
4. Discussion
4.1. Principal findings
We conducted usability testing to optimize the design of The-Optimal-Lymph-Flow health IT system (TOLF), a patient-centered, web-and-mobile-based educational and behavioral health IT system focusing on safe, innovative, and progmatic self-care strategies for lymphedema symptom management (Fu et al., 2016). High quality information regarding effective self-care strategies for lymphedema symptom management is a key for patient-centered care. TOLF was designed to address the long neglected challenges to lymphedema symptom management in research and clinical practice: lack of information about lymphedema and symptoms, lack of guidance in how to assess lymphedema symptoms, and lack of standardized and effective interventions for managing lymphedema symptoms and reduce the risk of lymphedema (Fu et al., 2012, Ryan et al., 2012). Participants in our study rated the usability high in terms of quality of information. All the participants agreed that the information provided by TOLF regarding lymphedema symptom management is clear, easy to understand, high quality, and empowering. As one of them remarked, “I liked the clarity of the explanations & the simplicity of using the website. The website is empowering for the women who want to prevent lymphedema, as well as for those who already have lymphedema.” Easy access to high quality health information for lymphedema symptom management is essential for patient-centered care in the electronic era. Participants loved the fact that patients can access TOLF at anytime and anywhere and learn about lymphedema, symptoms, and self-care strategies at their own pace. As our participants commented, “portable,” “patients can have this at their fingertips,” “self-paced; can repeat & review particular sections.”
Fidelity and transparency of any educational and behavioral intervention using traditional face-to-face and in-person delivery model is a concern for intervention delivery and reproducibility. Using technologically-driven delivery model, TOLF was designed to enhance reproducibility of intervention delivery for lymphedema symptom management by providing a novel training system using avatar simulation videos to assist breast cancer survivors in building self-care skills through visual demonstration of how lymph fluid drain in the lymphatic system when performing lymphatic exercises. This innovation was based on the request from the research participants in our prior research that used face-to-face, nurse-delivery model of intervention for lymphedema symptom management. Participants in this usability study loved avatar simulation videos that help them to understand lymphatic system and learn daily lymphatic exercises. As our participants remarked, “I love the videos that show the anatomy & fluid flow & deep breathing. I also love the lymphatic system video.” Avatar simulation videos hosted by TOLF was able to train our participants do the daily lymphatic exercises, as our participants remarked, “I like the [lymphatic] exercises!!! After I finished learning the exercises by following the videos, my pain and soreness were much better.” Using the technologically-driven delivery model, TOLF, thus, is able to enhance the patients' self-care skill building given that patients can review the self-care strategies on their own schedule and pace virtually anytime and anywhere.
The ease of using TOLF is vitally important to enhance patients' engagement in using the system. All our participants rated the ease of use high for TOLF. The use of user manual that lists the tasks to explore the system for technical skill training enhanced the perceived ease for our participants to use the system. It should be noted that older breast cancer survivors with limited experience of internet use and education may need more guidance in using the system and learning self-care strategies. Our oldest participant of age 78 who had only high school education and only learned how to use internet a few days prior to the study. However, by guiding her with the user manual, she was able to access and navigate the system freely and learn the self-care strategies. For the majority of our participants, TOLF was “user friendly and easy to navigate,” “It was very clear, thorough, and informative. System is clear. I understand where to click and scroll.”
4.2. Limitations
Our participants in this usability study represented general breast cancer survivor population in terms of age, education, and ethnicity in the study institute. Yet, our participants had relatively high education level and were familiar with internet use. Usability tests of TOLF are needed for breast cancer survivors with less education level or limited experience of using internet.
5. Conclusions
This usability study provided evidence that supports the usefulness of TOLF in terms of breast cancer survivor's acceptance, user-friendliness, and feasibility of using technologically-driven delivery model to enhance self-care strategies for lymphedema symptom management. As a feasible, user-friendly, patient-centered, web-and-mobile-based educational and behavioral health IT system focusing on safe, innovative, and pragmatic self-care strategies, TOLF provides a much-needed foundation for advancing the science of self-care for symptom management by transformation of healthcare from reactive and hospital-centered to preventive, proactive, evidence-based, patient-centered and focused on well-being rather than disease, a nursing research and practice focus.
Statement of financial interest
All the authors have no financial interest or commercial association with information submitted in manuscript.
List of products used
No products, devices, or drugs were used or identified in the manuscript.
Acknowledgments
This study was supported by National Institutes of Health (NIMHD Project # P60 MD000538-03), Pfizer Independent Grants for Learning & Change (IGL&C) (grant #13371953), Judges and Lawyers Breast Cancer Alert (JALBCA), and New York University Research Challenge Fund (grant # R4198) with Mei R. Fu as the principal investigator. Its contents are solely the responsibility of the authors and do not necessarily represent the official views of the NIH and other funders. The funders had no role in the study design, data collection and analysis, decision to publish, or preparation of the manuscript.
References
- American Cancer Society (ACS) Cancer facts and figures 2015–2016. 2016. http://www.cancer.org/Research/CancerFactsFigures/CancerFactsFigures/index accessed 20.02.2016.
- Armer J.M., Stewart B.R. Post-breast cancer lymphedema: Incidence increases from 12 to 30 to 60 months. Lymphology. 2010;43(3):118–127. [PMC free article] [PubMed] [Google Scholar]
- Armer J.M., Radina M.E., Porock D., Culbertson S.D. Predicting breast cancer-related lymphedema using self-report symptoms. Nurs. Res. 2003;52 doi: 10.1097/00006199-200311000-00004. [DOI] [PubMed] [Google Scholar]
- Bright T.J., Bakken S., Johnson S.B. Heuristic evaluation of eNote: an electronic notes system. AMIA Ann. Symp. Proc. 2006:864. [PMC free article] [PubMed] [Google Scholar]
- Cormier J.N., Xing Y., Zaniletti I., Askew R.L., Stewart B.R., Armer J.M. Minimal limb volume change has a significant impact on breast cancer survivors. Lymphology. 2009;42:161–175. PMID: 20218084. [PMC free article] [PubMed] [Google Scholar]
- Davis F.D. Perceived usefulness, perceived ease of use, and user acceptance of information technology. MIS Q. 1989;13(3):319–340. [Google Scholar]
- Deng J., Fu R.M., Armer J., Cormier J., Radina E., Thiadens S., Dietrich M., Weiss J., Tuppo C., Ridner S.H. Self-care status, symptom burden, and reported infections in individuals with lower-extremity primary lymphedema. J. Nurs. Scholarsh. 2014;47(2):126–134. doi: 10.1111/jnu.12117. Epub 2014 Dec 4. PMID: 25475008. [DOI] [PubMed] [Google Scholar]
- Dillon T.W., McDowell D., Salimian F., Conklin D. Perceived ease of use and usefulness of bedside-computer systems. Comput. Nurs. 1998;16(3):151–156. [PubMed] [Google Scholar]
- Finkelstein J., Knight A., Marinopoulos S. Enabling patient-centered care through health information technology. Evid. Rep. Technol. Assess. 2012;206:1–1531. [PMC free article] [PubMed] [Google Scholar]
- Fu M.R. Cancer survivors' views of lymphoedema management. J. Lymphoedema. 2010;5(2):39–48. [PMC free article] [PubMed] [Google Scholar]
- Fu M.R., Rosedale M. Breast cancer survivors' experience of lymphedema related symptoms. J. Pain Symptom Manag. 2009;38(6):849–859. doi: 10.1016/j.jpainsymman.2009.04.030. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Fu M.R., Axelrod D., Cleland C.M., Guth A.A., Qiu Z., Kleinman R., Scagliola J., Haber J. Symptom reporting in detecting breast cancer-related lymphedema. Breast Cancer Targets Ther. 2015;7:345–352. doi: 10.2147/BCTT.S87854. #1825502, PMID: 26527899. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Fu M.R., Axelrod D., Guth A., Cartwright-Alcarese F., Qiu Z., Goldberg J., Kim J., Scagliola J., Kleinman R., Haber J. Proactive approach to lymphedema risk reduction: a prospective study. Ann. Surg. Oncol. 2014;21(11):3481–3498. doi: 10.1245/s10434-014-3761-z. Online First. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Fu M.R., Axelrod D., Guth A., Fletcher J., Qiu J.M., Scagliola J., Kleinman R.E., Ryan C.E., Nicholas C., Haber J. Patterns of obesity and lymph fluid level during the first year of breast cancer treatment: a prospective study. J. Personal. Med. 2015;5:326–340. doi: 10.3390/jpm5030326. PMID: 26404383. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Fu M.R., Axelrod D., Guth A., Scagliola J., Rampertaap K., El-Shammaa N., Fletcher J., Zhang Y., Qiu J.M., Schnabel F., Hiotis K., Wang Y., D'Eramo Melkus G. A web-and mobile-based intervention for women treated for breast cancer to manage chronic pain and symptoms related to lymphedema: randomized clinical trial rationale and protocol. JMIR Res. Protoc. 2016;5(1) doi: 10.2196/resprot.5104. 2016, URL: http://www.researchprotocols.org/2017/1/e7/ [DOI] [PMC free article] [PubMed] [Google Scholar]
- Fu M.R., Chen C.M., Haber J., Guth A.A., Axelrod D. The effect of providing information about lymphedema on the cognitive and symptom outcomes of breast cancer survivors. Ann. Surg. Oncol. 2010;17(7):1847–1853. doi: 10.1245/s10434-010-0941-3. [DOI] [PubMed] [Google Scholar]
- Fu M.R., Deng J., Armer J. Cancer-related lymphedema: evolving evidence for treatment and management from 2009 to 2014. Clin. J. Oncol. Nurs. 2014;18(Supplement):68–79. doi: 10.1188/14.CJON.S3.68-79. PMID: 25427610. [DOI] [PubMed] [Google Scholar]
- Fu M.R., LeMone P., McDaniel R. An integrated approach to an analysis of symptom management in cancer patients. Oncol. Nurs. Forum. 2004;31(1):65–70. doi: 10.1188/04.ONF.65-70. PMID: 14722589. [DOI] [PubMed] [Google Scholar]
- Fu M.R., Ridner S., Hu S.H., Steward B., Cormier J.N., Armer J.M. Psychosocial impact of lymphedema: a systematic review of literature from 2004 to 2011. Psycho-Oncology. 2013;22(7):1466–1484. doi: 10.1002/pon.3201. Epub 2012 Oct 9. PMID: 23044512. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Fu M.R., Ryan J.C., Cleland C.M. Lymphedema knowledge and practice patterns among oncology nurse navigators. J. Oncol. Navig. Surviv. 2012;3(4):9–15. [Google Scholar]
- Hoffman A.J., Brintnall R.A., Brown J.K., von Eye A., Jones L.W., Alderink G., Holland D., Enter M., Patzelt L., VanOtteren G.M. Virtual reality bringing a new reality to postthoracotomy lung cancer patients via a home-based exercise intervention targeting fatigue while undergoing adjuvant treatment. Cancer Nurs. 2014;37(1):23–33. doi: 10.1097/NCC.0b013e318278d52f. PMID: 23348662. [DOI] [PubMed] [Google Scholar]
- IOM . Institute of Medicine; Washington, D.C.: 2001. Crossing the Quality Chasm: A New Health System for the 21st Century. [Google Scholar]
- Jemal M., Bray F., Center M.M., Ferlay J., Ward E., Forman D. Global cancer statistics. Cancer J. 2011;61(2):69–90. doi: 10.3322/caac.20107. [DOI] [PubMed] [Google Scholar]
- Kreuter M. Lawerence Eribaum Associates, Inc.; Mahwah, NJ: 2000. Tailoring Health Messages: Customizing Communication With Computer Technology. [Google Scholar]
- Lewis J.R. IBM computer usability satisfaction questionnaires: psychometric evaluation and instructions for use. Int. J. Hum. Comput. Interact. 1995;7(1):57–78. [Google Scholar]
- Mak S.S., Yeo W., Lee Y.M., Mo K.F., Tse K.Y., Tse S.M., Ho F.P., Kwan W.H. Predictors of lymphedema in patients with breast cancer undergoing axillary lymph node dissection in Hong Kong. Nurs. Res. 2008;57:416–425. doi: 10.1097/NNR.0b013e31818c3de2. PMID: 19018216. [DOI] [PubMed] [Google Scholar]
- Nielsen J., Landauer T.K. Proceedings of the INTERACT'93 and CHI'93 Conference on Human Factors in Computing Systems; Apr 24–29; Amsterdam, The Netherlands. Association for Computing Machinery; New York: 1993. A mathematical model of the finding of usability problems. [Google Scholar]
- Norman S.A., Localio A.R., Kallan M.J., Weber A.L., Torpey H.A., Potashnik S.L., Miller L.T., Fox K.R., De Michele A., Solin L.J. Risk factors for lymphedema after breast cancer treatment. Cancer Epidemiol. Biomark. Prev. 2010;19(11):2734–2746. doi: 10.1158/1055-9965.EPI-09-1245. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Passik S.D., McDonald M.V. Psychosocial aspects of upper extremity lymphedema in women treated for breast carcinoma. Cancer. 1998;83(S12B):2817–2820. doi: 10.1002/(sici)1097-0142(19981215)83:12b+<2817::aid-cncr32>3.0.co;2-2. [DOI] [PubMed] [Google Scholar]
- Petrek J.A., Senie R.T., Peters M., Rosen P.P. Lymphedema in a cohort of breast carcinoma survivors 20 years after diagnosis. Cancer. 2001;92(6):1368–1377. doi: 10.1002/1097-0142(20010915)92:6<1368::aid-cncr1459>3.0.co;2-9. [DOI] [PubMed] [Google Scholar]
- Radina E., Fu M.R., Horstman L., Yang K. Breast cancer-related lymphedema and sexual experiences: a mixed-method comparison study. Psycho-Oncology. 2015 doi: 10.1002/pon.3778. Epub March 2015. PMID: 25777358. [DOI] [PubMed] [Google Scholar]
- Ridner S.H. Quality of life and a symptom cluster associated with breast cancer treatment-related lymphedema. Support Care Cancer. 2005;13(11):904–911. doi: 10.1007/s00520-005-0810-y. [DOI] [PubMed] [Google Scholar]
- Ridner S.H., Dietrich M., Kidd N. Breast cancer treatment-related lymphedema self-care: education, practices, symptoms, and quality of life. Support Care Cancer. 2011;19(5):631–637. doi: 10.1007/s00520-010-0870-5. [DOI] [PubMed] [Google Scholar]
- Ridner S.H., Dietrich M.S., Stewart B.R. Body mass index and breast cancer treatment-related lymphedema. Support Care Cancer. 2011;19(6):853–857. doi: 10.1007/s00520-011-1089-9. 2011. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Ridner S.H., Fu M.R., Wanchai A., Stewart B.R., Armer J.M., Cormier J.N. Self-management of lymphedema: a systematic review of the literature from 2004 to 2011. Nurs. Res. 2012;61(4):291–299. doi: 10.1097/NNR.0b013e31824f82b2. [DOI] [PubMed] [Google Scholar]
- Ryan J.C., Cleland C.M., Fu M.R. Predictors of practice patterns for lymphedema care among oncology advanced practice nurses. J. Adv. Pract. Oncol. 2012;3(5):307–318. doi: 10.6004/jadpro.2012.3.5.4. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Shortliffe E.H., Sondik E.J. The public health informatics infrastructure: anticipating its role in cancer. Cancer Causes Control. 2006;17(7):861–869. doi: 10.1007/s10552-006-0028-4. [DOI] [PubMed] [Google Scholar]
- Stanton A.W., Modi S., Mellor R.H., Levick J.R., Mortimer P.S. Recent advances in breast cancer-related lymphedema of the arm: Lymphatic pump failure and predisposing factors. Lymphat. Res. Biol. 2009;7(1):29–45. doi: 10.1089/lrb.2008.1026. PMID: 19302022. [DOI] [PubMed] [Google Scholar]
- Tsai R.J., Dennis L.K., Lynch C.F., Snetselaar L.G., Zamba G.K. The risk of developing arm lymphedema among breast cancer survivors: a meta-analysis of treatment factors. Ann. Surg. Oncol. 2009;16(7):1959–1972. doi: 10.1245/s10434-009-0452-2. PMID:19365624. [DOI] [PubMed] [Google Scholar]