Initial Development of the Mental Health Assessment and Dynamic Referral for Oncology (MHADRO)

Abstract

The Mental Health Assessment and Dynamic Referral for Oncology (MHADRO) is a program that conducts a computerized assessment of physical, psychological and social functioning related to oncology treatment, prints personalized summary reports for both the patient and the provider, and for those who provide consent, faxes a referral and assessment summary report to a matched mental health treatment provider (i.e., dynamic referral). The functionality, feasibility, and end user satisfaction of the Mental Health Assessment and Dynamic Referral for Oncology (MHADRO) were tested in a comprehensive care center. Of the 101 subjects enrolled, 61 (60%) exhibited elevated distress on at least one of the mental health indices, and, of these, 12 (20%) chose a dynamic referral for mental health services. Patients and health care providers exhibited high levels of satisfaction with the program. The MHADRO has potential for assisting in meeting the psychosocial needs faced by individuals with cancer and should be tested further for its facilitation of mental health treatment initiation.

More than 11 million Americans have been diagnosed with cancer, and 1.3 million more are diagnosed each year (Centers for Disease Control and Prevention [CDC], 2009). Psychological distress is common during the diagnosis, treatment, and post-treatment phases of cancer. Depression, anxiety, post traumatic stress disorder (PTSD), physical side effects, and social isolation adversely impact quality of life and functional capacity for at least half of all cancer patients (Derogatis et al., 1983; Massie & Holland, 1990). Health care providers and researchers have begun routinely address psychological distress in oncology populations. Particular attention has been paid to the experience of depression in patients with cancer. This area of inquiry is controversial, with estimates of depression in patients with cancer varying widely (Coyne & Palmer, 2005) and a lack of agreement regarding how to best assess and manage depressive symptoms in patients with cancer.

The need for screening and proactive management of psychological distress in patients with cancer is widely recognized. The Institute of Medicine (IOM) and the National Comprehensive Cancer Network (NCCN) have called for universal screenings for psychological distress among all cancer patients (IOM, 2007; NCCN, 2007). The National Cancer Institute has recommended that oncology care providers screen for depression and provide further assessment, monitoring, and, if appropriate, treatment for patients that screen positive. Nonetheless, oncology care providers often do not comply with these recommendations and problems continue to go undetected (NCCN, 2007). Research suggests that oncologists diagnose depression in only 15% to 30% of their patients who have a depressive disorder (IOM, 2007; NCCN, 2007). Even when a mental health problem is properly diagnosed, cancer care providers may still not actively treat it (Cohen, et al., 2003; IOM, 2007; Jacobsen & Jim, 2008).

Reasons for inattention to psychological distress have not been systematically studied. Providers may not feel they have the expertise to accurately assess and monitor psychological distress in their patients. Some feel ill equipped to provide appropriate referrals for care when psychological disturbances are identified. Finally, and perhaps most importantly, providers experience many competing clinical demands and avoid using their time to explore psychosocial concerns unless the patient raises them.

This last barrier is being addressed by researchers studying the use of information technology to improve healthcare delivery, including the use of computerized psychosocial assessments for medical patients (Clark, Bardwell, Arsenault, DeTeresa, & Loscalzo, 2008). A recent study by Fann, et al. (2009) tested the feasibility of using a computerized depression screening, the Patient Health Questionnaire-9, to detect depressive symptoms in individuals with cancer who attended a medical oncology clinic, a radiation oncology clinic, or a hematopoietic stem cell transplantation (HSCT) clinic. Results suggested that the computerized assessment was practical and useful in detecting depression in an oncology population. Fann’s study is an excellent example of how computerized assessments can be integrated into clinical oncology practice. A computerized program might be widely adopted if it also helped providers decide among treatment options and make appropriate referrals when mental health needs are identified.

The Mental Health Assessment and Dynamic Referral for Oncology (MHADRO), presented in the present article, was designed to serve this purpose. The MHADRO is a computerized program that a) assesses psychosocial variables in patients with cancer, b) provides feedback to patients and providers regarding patient psychological symptoms as compared to a normative sample of cancer patients, c) provides tailored feedback reports for patients and providers to review together, and d) provides patients with a dynamic referral option. The dynamic referral is an electronic facsimile (FAX) which is transmitted to a mental health professional who has been matched to the individual’s behavioral health profile, insurance status, and residential location. This dynamic referral contains an assessment summary report and patient contact information and is only sent to mental health professionals at the specific request of the cancer patients. The mental health professional uses the information embedded in the referral to contact the patient and schedule an initial assessment, if appropriate.

The purpose of the present study was to evaluate the MHADRO’s functionality, feasibility, and end-user satisfaction. The study was funded as a Phase I project through the National Institutes of Health Small Business Technology Transfer (STTR) program. Phase I studies are designed to demonstrate feasibility, or proof of concept, prior to a Phase II demonstration of the impact of the system on clinical end-points, such as psychological distress and mental health treatment engagement.

Methods

Participants

The sample of this study was one based on convenience. Patients were recruited for the study if they (a) had received a cancer diagnosis from a physician, (b) were seeking treatment at the Cancer Institute at Cooper Hospital, (c) were over the age of 18, and (d) were physically and mentally capable of consenting and participating on their own. One hundred and three (N = 103) individuals diagnosed with various types of cancer and their oncology providers were recruited from two sites, one urban and one suburban, that comprise a comprehensive cancer care center. If a patient requested a dynamic referral, the mental health care provider was also included. There were no exclusion criteria based on type of cancer, stage of cancer, length of treatment, type of treatment, stage of treatment, or time since diagnoses.

We aimed to enroll a heterogeneous sample of individuals with cancer. Figure 1 summarizes the sample recruitment. The participants’ demographics are summarized in Table 1. Of the 103 participants who consented to participate, 101 (98%) successfully completed all components of the protocol. The two that did not had to leave before their assessment was complete. The sample was younger and consisted of a greater percentage of females (61% female, M_age = 62 years) when compared to the demographics of a sampling of patients treated in our cancer center (N = 13,607).

FIGURE 1.

Enrollment flow and mental health treatment initiation within 30 days of baseline.^a

Table 1.

Descriptive Statistics

Variable	Statistic Count (%), except where noted
Demographics (N = 103)
Age, in years, mean (SD)	52.5 (12.1 yrs)
Sex
Male	22 (22)
Female	79 (78)
Race
While	83 (83)
Black	15 (15)
Hispanic	3 (3)
Other	0 (0)
Insurancea
None	3 (3)
Medicare	20 (20)
Medicaid	14 (14)
Private insurance	75 (74)
Health Status and Cancer Indicators
General health status
Poor	2 (2)
Fair	24 (24)
Good	46 (46)
Very good	24 (24)
Excellent	5 (5)
Type of cancer
Breast	46 (46)
Lung	18 (18)
Colon	12 (12)
Prostate	3 (3)
Lymphoma	9 (9)
Other	13 (13)
When did you receive the diagnosis?
Today	6 (6)
Within the past week	1 (1)
1–4 weeks ago	7 (7)
1–3 months ago	23 (23)
4–6 months ago	14 (14)
>6 months ago	50 (50)
Are you currently in remission?
No	80 (80)
Yes	17 (17)
Current or recent treatments
Surgery	14 (14)
Chemotherapy	66 (66)
Radiation	21 (21)
Oral chemotherapy	6 (6)
Hormone therapy	7 (7)
Other	10 (10)
Physical symptoms and side effects (15 total possible)	3.8 (σ = 2.95)

^adoes not equal 101 because some patients have Medicare and private insurance

Materials

The MHADRO is web-based and designed for self-report on a desk-top, laptop, or tablet personal computer (PC). It is comprised of three integrated modules: (a) a computerized assessment, (b) a report generator, and (c) a referral generator.

MHADRO Assessment

The assessment is subdivided into sections that oncology providers reported were the most important for clinical care. It contains 150 questions, though the response-adaptive programming logic ensures that an individual is presented only with the questions appropriate to his or her situation. On average, patients complete about 120 items. According to the Flesch-Kincaid Grade Level analysis, the reading level is 6^th grade. In addition to socio-demographics (e.g., age, sex, insurance, ZIP code) and health status (e.g., cancer type, time since diagnosis, phase of treatment), the following major domains are assessed:

• Mental health/psychosocial (63 items). The core measure of the MHADRO mental health domain is the Behavioral Health Status (BHS) scale (Grissom, Lyons, & Lutz, 2002). The BHS is a 45-item composite of three subscales reflecting three dimensions: Subjective Well-being, Psychological Symptoms, and Functioning. Each of the subscales has good internal consistency reliability (α > 0.80) and has been validated against one or more established scales including the Symptom Checklist-90-R (Derogatis, 1977), General Well-Being scale (Dupuy, 1977), Positive and Negative Affect scale (Watson & Tellegen, 1985), Beck Depression Inventory (Beck, Steer, & Garbin, 1988), and the Social Adjustment scale (Weissman & Bothwell, 1976). The composite BHS has good reliability (0.88), concurrent validity, and sensitivity to change (effect size = 0.60; p < .001; Skarstein, Aass, Fossa, Skovlund, & Dahl, 2000).

  An abbreviated (9 items) version of the BHS was developed as one of the scales of the MHADRO, incorporating items from each of the three original scales by selecting items with good clinical utility for oncology and high item-total correlations. In addition to the BHS, the Polaris Depression and Anxiety scales were created to be used as primary indicators of mental health functioning. The Depression Scale (DS) and Anxiety Scales (AS) were constructed by recasting the symptoms associated with a Major Depressive Episode and Generalized Anxiety Disorder (17), respectively, as self-report items. They have well-established psychometric properties, including strong reliability, concurrent validity, and discriminate validity (18). In addition to the three primary symptom measures, the MHADRO also includes subscales that assess a variety of other psychosocial domains. The domains and the number of items relating to each are: psychiatric diagnostic and treatment history (4 items), interest in psychiatric treatment (2 items), suicidal ideation/intent (2 items), cancer-related post-traumatic stress disorder (PTSD) (3 items), will to live (3 items) (20), acceptance of disease (2 items) (21), social support (8 Items) (22), tobacco use (6 items) (23, 24), alcohol use (7 items) (23, 25), and drug use (4 items) (23, 25). Additional detail is available from the authors.

  The MHADRO includes items or scales assessing a variety of other psychosocial domains. When possible, we used scales with proven reliability and validity. The domains and the number of items relating to each are: (a) psychiatric diagnostic and treatment history (four items); (b) interest in psychiatric treatment (two items); suicidal ideation/intent (two items); (c) cancer-related post-traumatic stress disorder (PTSD) (three items); (d) will to live (three items) (Grissom & Lyons, 2006); (e) acceptance of disease (two items) (Sperry, Brill, Howard, & Grissom, 1996); (f) social support (eight items) (Grissom, Lyons, & Lutz, 2002); (g) tobacco use (six items) (Boudreaux, et al., 2009; Kozlowski, Porter, Orleans, Pope, & Heatherton, 1994); (h) alcohol use (seven items) (Boudreaux, et al., 2009; McLellan, Luborsky, O’Brien, & Woody, 1980); and (i) drug use (four items) (Boudreaux, et al., 2009; McLellan, et al., 1980).

• Physical symptoms/side effects (13 items). The presence of common disease and treatment-related physical symptoms are assessed, along with the frequency and functional impact of the symptoms endorsed. The items included in this section were derived through a literature review and interviews with oncology physicians and nursing staff.

• Patient-provider partnership (nine items). Focus groups with oncology staff and advice from our consultant (Martha Gaines, Executive Director, Center for Patient Partnerships, University of Wisconsin) suggested the patient-provider partnership was a particularly important domain to assess, because early identification of breakdowns in the partnership can help to repair the relationship. Items were derived from an existing, validated scale assessing interpersonal aspects of medical care (DiMatteo, et al., 1993) and used with permission.

• Barriers to treatment (16 items). This domain includes three subsections derived from validated scales (DiMatteo, et al., 1993). Perceived Disease Severity (four items) inquires about the patient’s beliefs regarding the seriousness of his or her cancer. Perceived Treatment Utility (eight items) taps into the individual’s beliefs about the usefulness of treatment. Perceived Supports and Barriers (four items) gathers information about the patient’s perception of the obstacles to adhering to the cancer treatment plan.

• Adherence (13 items). The MHADRO adherence subscale assesses patient compliance with medical office visits and cancer-related treatments such as chemotherapy, radiation, and oral hormone therapy. It also assesses compliance with lifestyle recommendations made by the patient’s treatment team, such as smoking cessation and dietary changes.

• Referrals and psychosocial resources (13 items). Each patient is offered a dynamic referral to a mental health provider, regardless of their answers to the questions on the MHADRO. Further, all individuals who identify themselves as smokers are queried about interest in dynamic referrals to stop-smoking programs. Patients who select the dynamic referral option are asked for their personal contact information, the best time to call, and if it is acceptable to leave a message if the patient is unavailable to answer the call. Additionally, all patients are presented a menu of the support groups and complementary medicine classes offered at the site. Patients indicate which groups, if any, they would like to know more about.

• Functional impairment. The MHADRO assessment also includes an eight-item, global functional impairment scale. Two of the items are derived from the SF-12 (Stewart, Hays, & Ware, 1988) while the other six were either created for the purposes of the MHADRO or used in other Polaris products (Fann, et al., 2009).

Report Generator

The Report Generator piece of the MHADRO system draws upon data collected during the assessment to produce three reports: (a) a Healthcare Provider Report for the oncology treatment team; (b) a Referral Provider Report for the mental health provider (provided only when a dynamic referral is requested); and (c) a Patient Feedback Report.

• Healthcare Provider Report. The two-page Healthcare Provider Report summarizes the patient’s global functioning and provides specific feedback regarding physical symptoms/side effects, patient/provider relationship, barriers to treatment, adherence, and functional impairment. The Mental Health Functioning subsection presents the severity of the patient’s psychological condition through the use of cancer patient norms on the BHS, DS and AS scales. This summary is supported by a distress thermometer (0 = low distress to 100 = high distress) representation of the individual’s percentile rank on each of the three scales. This thermometer also identifies the corresponding percentile rank from the mean of a sample of over 4,000 patients who are actively engaged in mental health treatment (NCCN, 2007). This provides a clinically useful comparison that contributes to the oncology team’s decision about whether the patient’s distress is elevated enough to warrant mental health treatment.

  In addition to the continuous scaling, the percentile scores were used to classify the patient’s severity as Normal (0–60^th percentile), Elevated (61^st to 79^th percentile), or High (≥ 80^th percentile). We also highlighted any reported problems with labile mood, suicidal ideation/intent, PTSD symptoms, tobacco, alcohol, and drugs for the providers, as these are all problems that may warrant immediate medical attention. Finally, the report concludes with a Referral Summary that is comprised of a personally tailored mental health referral list, psychosocial support groups, and complementary medicine classes (e.g., yoga, meditation, nutrition) requested by the patient during the assessment.

• The Patient Feedback Report. The Patient Feedback report, which can be up to three pages long, provides personally tailored motivational messages and suggestions for action. It addresses six areas that the healthcare providers, mental health clinicians, and oncology patients felt were most important: (a) mental health functioning, (b) psychosocial resources and tailored mental health referral lists, (c) social support, (d) pain management, (e) tobacco use, and (f) adherence to oral medications.

• The Referral Provider Report. The Referral Provider Report is an abbreviated version of the Healthcare Provider Report and is produced only when patients request a dynamic referral. It includes patient contact information (e.g., phone number, best time to call) and is automatically faxed to a mental health provider matched to the client on several dimensions (see Referral Generator below). Both the Patient Feedback Report and the Referral Provider Report were automatically printed on a printer accessible only by staff.

Referral Generator

Patients are able to request dynamic referrals for counseling services or smoking cessation treatment. The referral provider options are determined by the specifics of the site during installation. In some cases, in-house providers are available to handle all mental health referrals. In others, community-based providers are contacted. A library is created using existing community-based provider lists compiled by the site or other publicly available provider registries, such as those maintained by state licensing agencies. The referral is sent to a provider matched to the patient’s residential ZIP code, insurance, and clinical characteristics (e.g., whether co-morbid substance abuse is present).

For the field testing described herein, all dynamic referrals were sent to two in-house referral groups within the cancer institute. The location of the oncology treatment facility where the participant was enrolled dictated which of the two groups received the referral. If the group was unable to provide treatment because of geographic or clinical reasons, such as co-morbid substance abuse, they assisted the patient with accessing alternate resources.

Procedure

This study was approved by two Federally Wide Assured Institutional Review Boards and was in full adherence to the guidelines of human subjects’ treatment as outlined by the National Institutes of Health.

A 5-phase development and field testing plan was executed, with each phase progressively informing the next. First, we conducted focus groups and semi-structured interviews with end-users to develop the core design principles. Participants were 23 oncology professionals, including clinical nurses, nurse coordinators, oncologists, mental health counselors, administrators, and the director of the cancer institute. For brevity, the design principles that emerged from these interviews and groups are not included here but are available from the authors. Great care was taken to design the MHADRO assessment to be user friendly and to require little or no computer literacy. A heavy emphasis was placed on ensuring that it was practical and useful in the clinical setting. Using the core design principles, the project team wrote the initial technical specifications, which were modified subsequent to the results from each phase of field testing.

Second, we administered the primary mental health scales (BHS, DS, AS) to 54 mixed cancer patients to establish cancer-specific norms for Version 1.0. Third, we pilot tested Version 1.0 using 6 participants to ensure that all of the components of the MHADRO, such a printing the reports and faxing the referrals, were functioning properly in the clinical setting. Fourth, we made adjustments to the MHADRO assessment using feedback from the pilot test to create Version 2.0, and enrolled 51 participants for a pilot test of Version 2.0. Finally, using feedback from the Version 2.0 participants, we enhanced the patient report (Version 3.0) and enrolled 50 additional participants to test the acceptability of the revisions using Version 3.0. At each phase, the participants were added to the normative database for the subsequent phase. The final two phases (Version 2.0 and 3.0) represent the primary focus of the present paper.

Two sites participated in recruitment, one urban and one suburban. Participants were recruited during office visits, chemotherapy infusion, and radiation treatment visits. Research staff reassured individuals that they did not have to have had any previous experience with computers to participate. The MHADRO assessment was self-administered using a desktop or laptop PC with high-speed Internet access. Participants were instructed by research staff on how to take the assessment using the mouse or numeric keypad, based on the participants’ preference. They were closely observed throughout the process and were encouraged to point out items they found confusing and to report any difficulties navigating the interface. All difficulties observed, questions asked, and the type of assistance provided was recorded on a process log. All problems with the MHADRO software or hardware that were identified were rectified immediately. The MHADRO’s automatic time stamps were used to calculate assessment completion times.

Patients and healthcare providers rated their satisfaction with the MHADRO. The patient satisfaction evaluation consisted of three components. First, the last four questions of the MHADRO assessment evaluated patient’s perception of the program, including ease of using the computer, preference of computer vs. paper, appropriateness of the questions, and length of the assessment. Second, the research assistant completed a semi-structured interview assessing likes, dislikes, suggestions for improvement, and overall impression of the assessment, the Patient Feedback Report, and the referrals given. Finally, patients provided structured ratings across 14 domains using a five-point scale: 1 = Very Poor, 2 = Poor, 3 = Fair, 4 = Good, 5 = Excellent. Sample items include: clarity of instructions, ability to read the words on the screen, comfort in answering honestly, organization of the report, understandability of the information, usefulness of the information, accuracy of the report, and usefulness of referrals provided.

After the treating oncologist and nurse reviewed the Healthcare Provider Report, they each completed structured ratings across four domains, using the same five-point scale described above. Items include: understandability of the report, usefulness of the report, length of the report, and appropriateness of the referral (if the participant received a referral). Providers were also asked about whether the Healthcare Provider Report added important information to their clinical assessment, whether the information impacted their management of the patient, whether the MHADRO had provided referrals that would otherwise not have been provided during the course of routine clinical care. The questionnaire provided space for open-ended comments about the administration process, reports, and referrals.

Data analytic plan

For the field tests (Version 2.0, 3.0), descriptive statistics, including counts, proportions, means, standard deviations, medians, and interquartile ranges (IQR), were calculated for all variables. Our primary quantitative outcomes consisted of (a) end-user satisfaction ratings, (b) completion times for the assessment module, and (c) the proportion of distressed patients that chose to receive a dynamic referral. We compared the mean scores on the three mental health scales (BHS, DS, AS) and satisfaction ratings (patient, provider) between Versions 2.0 and 3.0 using independent samples t-tests. We planned to combine their results for reporting if there were no statistically significant differences observed. This was a descriptive study and statistical tests were not conducted beyond those used to establish the psychometric properties of the three primary mental health scales.

Fostering mental health treatment initiation and reducing distress is an important goal of the MHADRO. However, because this study was designed primarily to establish the MHADRO’s functionality, feasibility, and acceptability, and not to establish its efficacy, treatment initiation and decreased distress at 1-month follow-up were considered secondary outcomes.

Results

Participants

Comparisons revealed no statistically significant differences between the Version 2.0 and 3.0 field tests across mental health indices or satisfaction scores for patients or providers (all p > 0.05). Therefore, the results from Versions 2.0 and 3.0 were aggregated (N = 101).

Functionality

Table 2 summarizes barriers to implementation recorded on the process logs as well as the solutions applied to overcome said barriers. The most common process barrier consisted of a delay in the Healthcare Provider Report being reviewed by the nurse or physician. Many reports were reviewed by the healthcare providers at a later time (e.g., at the end of the day when they were completing charting) because of other clinical demands. The next most common barrier pertained to patients requesting clarification of the meaning of items. While we responded by modifying some items to clarify their meaning, some of the problematic items were derived from standardized instruments, which prevented revision. Of note, reverse scored questions from the Dimetteo scales (Kozlowski, et al., 1994) were particularly problematic. We plan to test these items further to determine if the reverse scored items can be omitted while retaining the psychometric strengths of the scale and clinical utility. Other technical issues were noted, including problems printing, unreliable internet connection, and the assessment “freezing” up. These issues occurred early in the Version 2.0 field test and were rectified by Version 3.0.

Table 2.

Barriers to Implementation Noted on the Process Log and Solution Applied

Barriers	n	Solutions
1. Setting up the computer and introducing the system to the patient
Patient indicated that he/she was not familiar with computers.	3	Staff enrolled and assisted patient.
Technical difficulties (e.g., internet not working, log-in screen “freezing”).	2	Research staff identified problem and applied solution.
Medical interruptions.	1	Assessment began after interruption.
2. Completing MHADRO assessment
Difficulty understanding specific items.	24	Questions clarified by research staff
Technical difficulties (e.g., screen “freezing,” problems maintaining internet connection).	11	Research staff identified problem and applied
Medical interruptions.	9	Assessments resumed after interruption.
Difficulty understanding dynamic referral process.	3	Research staff answered questions.
3. Printing the reports
Auto-print not working.	9	Research staff identified problem and applied solution.
Report inaccurate.	1	Technical team identified programming logic error and rectified it.
4. Nurse reviewing the Healthcare Provider Report
Nurse unable to review report while the patient was still present at the facility (i.e., delayed review).	30	Nurse reviewed report later.
5. Oncologist reviewing the Healthcare Provider Report
Oncologist unable to review report while the patient was still present at the facility (i.e., delayed review).	47	Oncologist reviewed report later.

Because of response-adaptive branching logic, participants completed an average of 121 items. The median completion time was 24 minutes (IQR = 20 – 28 minutes).

Satisfaction ratings

Patient satisfaction

Eighty-eight (88%) participants stated that the length of the assessment was “just right,” with 10 (10%) reporting it was “a little too long” and 2 (2%) stating it was “much too long.” Four (4%) reported having difficulty using the computer. Seventy-four (74%) reported they preferred answering the questions on the computer as opposed to paper-and-pencil, 22 (22%) reported that they had no preference and 4 (4%) reported preferring paper. All (100%) participants believed that the software was appropriate for use with cancer patients. Figure 2 summarizes the patient satisfaction scores across 14 domains. Every item fell between 4.00 “Good” and 5.00 “Excellent.”

FIGURE 2.

Mean satisfaction scores for patients (N = 101).

Oncology provider satisfaction

We obtained physician or nurse evaluations on 90 (90%) of the participants’ Healthcare Provider Reports. Figure 3 summarizes the healthcare provider satisfaction scores. Every item fell between 4.00 “Good” and 5.00 “Excellent.” For 48 (48%) participants, their oncology providers stated that the Healthcare Provider Report contained important information that had not been assessed during their medical evaluation. For 22 (24%) participants, providers stated that the new information changed the way they managed the patient, such as starting a new psychotropic medication. According to their providers, 30 (49%) of the 61 participants who were in the Elevated or Highly distressed range (> 60^th percentile on BHS, DS, or AS) received referral information that they likely would not have received under routine clinical care.

FIGURE 3.

Mean satisfaction scores for oncology providers (n = 90).^a

Mental health indices

We chose to focus on the three primary mental health scales for this paper because they are the most well-developed. Figure 1 shows the proportion of patients within each BHS distress category who requested a dynamic referral for a mental health professional. Sixteen (16%) participants chose a dynamic referral for mental health, with 12 (75%) of these being from the elevated or highly distressed subset. Of the 20 smokers enrolled, four (20%) chose a dynamic referral for tobacco counseling. All of the faxed referrals were received successfully and judged to be “appropriate referrals” by the mental health and tobacco counselors. Clinicians were able to successfully contact 17 (85%) dynamically referred participants, 9 (45%) attended an initial assessment within the 30 day follow-up window, and 2 (10%) had pending appointments.

Discussion

Our findings suggest that a computerized mental health outcomes management system like the MHADRO can be employed to support integration of behavioral health in a clinical oncology setting. Although there were some limitations, the integration of this computerized psychosocial assessment went quite smoothly in the oncology clinical practice. We found that satisfaction ratings across patients and health care providers were uniformly high, with all ratings in the “Good” or “Excellent” range. For the patients, we were able to recruit 101 participants, some of whom were very ill, to participate in the MHADRO. All of these patients reported that they believed the questions were appropriate to be asked of individuals with cancer and the majority of the patients reported they felt the length of the assessment was appropriate and that the computer was comfortable to use.

Oncology providers overall reported that the clinical utility of the MHADRO system was largely favorable. They stated that the reports summarized clinically relevant information that had not been assessed during their medical assessment in over half of the participants. Further, the summary reports allowed the oncologists to rapidly determine the participant’s overall psychosocial functioning, as well as derive additional detail for problem areas. Finally, the providers reported actually changing their management of nearly one-quarter of their patients as a result of the reports, and also noted that nearly one-half (49%) of distressed patients received referrals that would probably not have been received during routine care.

The MHADRO system also appears to have utility in identifying patients with psychological distress and assisting those who are interested in seeking mental health services. Of the 61 patients classified as elevated or high in terms of emotional distress, 12 (20%) chose the dynamic referral option. The significance of this proportion is difficult to evaluate, since a control group was not included in the study, and there are no good estimates of spontaneous referral requests by patients under treatment as usual conditions against which to compare. As mentioned before, many of these individual’s healthcare providers stated that they did not believe the participant would have received such a referral under routine clinical care. Those who received the dynamic referrals were appreciative of the option and viewed it as a valuable aspect of the program. Most were successfully contacted by the mental health treatment staff and more than half (55%) had either entered treatment or had a pending appointment by the 1-month follow-up. Further randomized controlled studies with the MHADRO will need to focus on the examination of improved clinical outcomes as a result of the dynamic referral program.

As illustrated in Figure 4, the MHADRO system could be implemented into clinical practice a couple of ways with each option having the benefit of expedited identification of patients with depression. Further, early identification with as system such as the MHADRO would also provide opportunities for early treatment initiation of interventions. Finally, the MHADRO has the capability of being programmed to prompt follow up assessments by oncologists for those patients who have reported high levels of depression or distress while receiving care.

FIGURE 4.

An example of two clinical implementation strategies.

Study Limitations

The present study had a number of limitations. First, the research staff was heavily involved in recruitment of patients and administration of the MHADRO assessment. Therefore, our procedure is not an accurate reflection of how the MHADRO would be integrated into a clinical practice without research staff. Our trial, because it was an early field test and required intensive evaluation and ongoing modification, was not designed to specifically test whether implementation was feasible in the real world. This kind of full systems integration awaits future studies. Second, the sample was restricted and might not represent all oncology patients. It included only patients who were attending a comprehensive cancer institution, had a disproportionate number of patients who were young, female, and with breast cancer as well as other solid tumors, and patients from a racial/ethnic minority background were underrepresented. Third, this study was not an efficacy trial. Future randomized controlled studies are planned to determine if the MHADRO can improve treatment engagement, lead to decreased distress, and increase quality of life in patients with cancer.

A fourth limitation was that 40% of patients who were approached to participate in the study declined to participate, most commonly due to feeling poorly. This limitation is notable in terms of the validity of our findings, but also in terms of the integration of such a program into clinical practice. Fifth, although the MHADRO assessment measured a number of important variables such as perceived social support, post-traumatic stress disorder symptoms, and will to live, we were unable to score and interpret the patients’ responses at this time. This is because there is a lack of normative data for these subscales, which were created for the MHADRO, and further, given the scope of the present paper, we did not have the time or space to address the results. Future research that will attend to these relevant areas is presently underway and will be disseminated in future manuscripts.

Finally, there are some potential barriers to clinical implementation of the MHADRO. Technical barriers, such as difficulty with auto-printing or the system freezing up during an assessment, were noted and rectified during the field tests. However, several of the barriers noted during the field tests cannot be remedied by technical revisions, because they are inherent in the nature of providing oncology care. They are important to consider when translating the MHADRO into clinical settings. For example, assessments were frequently interrupted by medical care, a barrier likely to persist in any clinical setting. When this occurred, the patients, with the exception of two who signed consent forms but left prior to completing the assessment, simply continued the assessment where they left off once the medical care was complete.

Delayed review of the Healthcare Provider Reports by the nurse or physician also was common because they were otherwise occupied by clinical tasks and could not review the report immediately after the patient completed the assessment. The vast majority reviewed the reports later, usually at the end of the day when they were completing their medical charting. Busy clinical settings may have trouble establishing an efficient method for ensuring administration of a computerized assessment during routine work flow. Ultimately, adoption of any new system is dependent upon willing staff who recognize its importance.

As we enter a new era of cancer care, one which is marked by attending to all aspects of a person’s life and not just the treatment of a disease process, providers will have to consider the costs and benefits of implementing a program such as the MHADRO into their clinical practice. A proposal for further testing and development of the MHADRO, including an RCT to address these issues, has been approved for funding by NIH. The research will contribute to our understanding of the role of psychosocial factors in treatment of cancer patients, and the utility of computerized clinical decision support systems for integration of behavioral healthcare in oncology practices.