Randomized Control Trial to Test a Computerized Psychosocial Cancer Assessment and Referral Program: Methods and Research Design

Abstract

The National Cancer Coalition Network, National Cancer Institute, and American College of Surgeons all emphasize the need for oncology providers to identify, address, and monitor psychosocial needs of their patients. The Mental Health Assessment and Dynamic Referral for Oncology (MHADRO) is a patient-driven, computerized, psychosocial assessment that identifies, addresses, and monitors physical, psychological, and social issues faced by oncology patients. This paper presents the methodology of a randomized controlled trial (RCT) that tested the impact of the MHADRO on patient outcomes at 2, 6, and 12 months. Patient outcomes including overall psychological distress, depression, anxiety, functional disability, and use of psychosocial resources will be presented in future publications after all follow–up data is gathered. Eight hundred and thirty six cancer patients with heterogeneous diagnoses, across three comprehensive cancer centers in different parts of the United States, were randomized to the MHADRO (intervention) or an assessment-only control group. Patients in the intervention group were provided detailed, personalized reports and, when needed, referrals to mental health services; their oncology provider received detailed reports designed to foster clinical decision making. Those patients who demonstrated high levels of psychosocial problems were given the option to authorize that a copy of their report be sent electronically to a “best match” mental health professional. Demographic and patient cancer-related data as well as comparisons between patients who were enrolled and those who declined enrollment are presented. Challenges encountered during the RCT and strategies used to address them are discussed.

1. Introduction

Changes in quality of life, anxiety, and depression are all psychosocial consequences related to cancer [1]. Researchers have studied interventions that reduce such negative effects [2, 3, 4], with recent studies focusing on telephone [5, 6, 7] and computer based interventions [8, 9, 10]. For example, Loiselle and colleagues tested the impact of unlimited access to an eight week, computer-based, psychosocial program [11]. They found improvement in female cancer patients’ quality of life when compared to the treatment as usual control condition. They note that they were able to successfully incorporate their program into the clinical care for newly diagnosed patients at four ambulatory oncology clinics. They argue their findings, as well as others like them [12, 13], support the use of technological interventions in oncology settings. Similarly, Erharter and colleagues [14] had over one hundred patients with brain cancer complete a computerized assessment that evaluated overall quality of life; physical, social, role, emotional, cognitive, and global functioning; and common physical and neurological symptoms of brain cancer and brain cancer treatment. The assessments were done while awaiting neuro examinations at three outpatient oncology clinics. Patients completed an average of 4.74 assessments over the course of the approximately three year study. Results further supported the feasibility of integrating the monitoring program into the routine clinical care flow of their centers, and it proved to be effective in identifying psychosocial symptoms in patients attending the clinic.

The Mental Health Assessment and Dynamic Referral for Oncology (MHADRO) is a computerized psychosocial assessment program that assists oncology providers in identifying, monitoring, and managing psychosocial issues in individuals with cancer [15]. The MHADRO provides reports for the oncology provider, the patient, and, when appropriate, a mental health provider to which the patient is referred. In addition to the reports, which are described in detail in Section 2.5.1.2 below, the MHADRO has the unique capability to provide patients with a dynamic referral to a mental health provider if they are experiencing high levels of distress or sexual dysfunction and are interested in receiving professional help. Unlike a standard mental health referral, which typically consists of either a general recommendation to see a clinician or a pre-printed list of mental health providers, a dynamic referral matches the patient’s zip code and insurance carrier to a mental health specialist in his or her area and the referral is sent electronically to the mental health provider, who then reaches out proactively to contact the patient, perform a brief telephone screening, and set up an initial appointment, if appropriate. The MHADRO’s referral capabilities are described in greater detail in Section 2.5.1.3. The MHADRO can be completed during outpatient appointments and during chemotherapy infusions in facilities that can provide for patient-facing computing. A prototype was well accepted by oncology providers and patients alike during a field test [15]. This paper presents the methodology of the randomized controlled trial (RCT) designed to test the impact of the MHADRO on patients’ psychosocial and medical outcomes.

2. Method

2.1. Hypotheses

Our primary hypothesis for the clinical trial is that, compared to assessment only, the MHADRO will result in greater reductions in psychological distress among cancer patients at 2, 6, and 12 months after the initial assessment (Figure 1). The hypothesized mechanisms of action include the initiation of mental health counseling, psychotropic medications, and psychosocial support group participation. We anticipate that these actions will be promoted both directly through the patient reports and dynamic referral functions, as well as indirectly through prompting clinical action by the individual’s oncology provider. Secondary objectives include evaluating the MHADRO’s effect on the patient-provider partnership, medical regimen and lifestyle change, and health-related outcomes. We completed baseline enrollment in February 2012 and are currently completing the follow-up assessments. Final results will be published after all of the follow-up data is complete in another publication.

Figure 1.

Model Depicting Group Assignment, Hypothesized Mechanisms of Action, and Other Factors Predicting Psychological Distress

Although there are no hypotheses for the present study, we aim to present a detailed analysis of this large, multi-site RCT with emphasis on challenges encountered and overcome during the course of the three year study. We will present demographic and cancer related descriptive data of all patients enrolled as well as differences in demographic and cancer related variables across patients who accepted and declined enrollment.

2.2. Study Design

The study design is a parallel group, 1:1 allocation, randomized, single-blind clinical trial. The baseline assessment was completed by participants during their oncology outpatient visit. They were re-assessed at 2, 6, and 12 months by a centralized, blinded research assistant by telephone. Participants who preferred to complete their assessments on-line rather than by phone where provided an encrypted link to a secure website. The website only contained the questions to the assessment, not any personal information or previous assessment data. The study was approved by the Institutional Review Boards of all three institutions.

2.3. Participants and Research Sites

Participants consisted of 836 cancer patients recruited from three comprehensive cancer centers: the University of Massachusetts Medical School Cancer Center (n=581; 70%), the Cancer Institute of New Jersey at Cooper Hospital (n=126; 15%), and the University of Texas MD Anderson Cancer Center (n=129; 15%). Because the program is designed to operate across all cancers in a general oncology setting, inclusion criteria were kept deliberately broad. Any patient with a past or current cancer diagnosis who was 18 years old or older was considered for enrollment. Patients were excluded if they had any of the following: altered mental status (e.g., psychosis, delirium, and disorientation), hostile or agitated behavior, severe illness that would preclude conversation or computer use (e.g., persistent vomiting, severe pain), or factors precluding follow-up (e.g., transient residence or lack of a telephone). Patients were recruited regardless of type, duration of illness, stage of cancer or phase of treatment.

2.4. Study Procedures

Participants arriving for routine oncology (treatment or follow up) appointments or chemotherapy infusions were approached in an exam room or an infusion chair after their treating oncology provider’s permission to approach the individual was obtained. Each patient was given information about the study’s purpose and participant requirements. They were informed that their participation would not delay their care, and that they could interrupt or discontinue the assessment at any time. Individuals with transient symptoms that precluded enrollment during a given appointment, such as nausea or pain, were re-approached during later appointments. Once a participant signed the informed consent, the research staff entered the individuals’ identifying information into the MHADRO system to begin the assessment. The participant was randomly assigned to one of two study conditions, which are detailed below. Randomization was completed by an internal random number generator programmed into the software. The research assistant was blind to study assignment until the patient completed the assessment, whereupon it was necessary to determine the participant’s group assignment to carry out the rest of the protocol, also described below.

Participants were contacted by phone or email (based on stated preference) at 2, 6, and 12 months after enrollment to complete a follow-up MHADRO assessment, as well as other outcome measures. The research assistant performing the telephone assessments was in a centralized location and was blind to group assignment.

2.5. Study Conditions

2.5.1. Intervention group

Participants assigned to the intervention group completed the MHADRO assessment, during which they were given the option of a dynamic referral if they met certain criteria, which are described in Section 2.5.1.3. Upon completion of the assessment, the research assistant printed the oncology provider’s report and presented it to the provider. The protocol stipulated that this should be done, when possible, prior to the participant’s clinical appointment. However, this was not always feasible because of the clinic logistics, competing clinical priorities, and because some assessments were not completed until after the clinical encounter. In these cases, the research assistant would give the report to the provider at the next feasible time and, after the provider had reviewed the report, would have the report scanned and placed on the electronic health record. Oncologists were trained on how to read and interpret the provider reports. To preserve ecological validity we did not mandate any specific action from the provider, nor did the report provide specific suggestions to the health care provider regarding counseling or clinical management of the patient. The providers managed the patients based on their own usual and customary practice.

In addition to printing the provider reports, the research assistant printed the patient reports and reviewed the sections briefly with the participant. While this would ordinarily be done by clinical staff in the clinical environment, the internal validity of the study demanded that all participants in the intervention condition receive their report, and we decided the only way to ensure this was to make it the responsibility of the research staff. The research assistant did not provide any counseling, discuss the content of the report, or make any other recommendations beyond encouraging the individual to read the report.

2.5.1.1. Oncology provider report (see appendix A)

The oncology provider report was one page long and was designed in consultation with practicing oncologists to convey the information in a format that was intuitive and did not require extensive training or orientation, but provided the essential clinical information needed to assist in decision making [15]. It contained information on psychosocial functioning, psychiatric history, cancer symptoms and side effects, and summarized the referrals and other educational resources provided to the patient by the system. It was reviewed by the oncologist, signed, and placed in the patient’s medical chart. Oncologists were provided with a report after each follow-up assessment (2, 6, and 12 months) for those in the intervention group. The follow-up reports tracked psychosocial symptoms over time and helped to identify important changes, such as deterioration in psychosocial indices.

2.5.1.2. Patient report (see appendix B)

Each patient in the intervention group received a tailored report, which ranged from 5 to 7 pages. The reports were designed in consultation with oncology patients to provide information in a way that was intuitively understandable and maximally useful. It provided information on emotional distress, relationships, concerns relating to sexuality, health management, tobacco use, alcohol use, side effects and physical symptoms, mental health referrals, and an action plan. A key feature of the reports was the presentation of the severity of the participant’s overall psychological distress, depression, anxiety, and functional disability based on a normative database of cancer patients. This normative database consisted patients with a wide range of cancer diagnoses. As the data collection progressed we continued to update the normative analyses on the core measures (i.e., depression, anxiety, behavioral health status, functional disability) until the normative sample include all of 836 enrolled cancer patients. Based on the cutoffs established by the normative database, participants who scored at the 70^th percentile or greater were categorized as the elevated distress group, whereas participants at 30^th – 70^th percentiles were in the average or normal group, and participants scoring anything less than the 30^th percentile were in the low distress group.

2.5.1.3. Dynamic referral

Participants in the intervention group received the option for having a faxed referral sent to a matched mental health provider if they were not already in treatment; responded “yes” or “not sure” when asked if seeing a counselor or therapist would be beneficial; and any of the following applied: (1) a score in the elevated range (>70^th percentile) on the overall distress scale or the depression subscale, OR (2) a rating of 9 or 10 on the NCCN distress thermometer OR (3) sexual difficulties were endorsed. A two-step process was used to confirm that the individual wanted the dynamic referral. First, the dynamic referral process was explained in text on the screen to participants meeting the criteria, and they were asked if they were interested. Second, if the participant replied “yes,” a consent statement explaining that the system would transmit their personal contact information to a designated mental health provider was presented and the individual was given the option of “agree” or “disagree.” When participants agreed, their personal information was automatically faxed to a mental health provider based on the parameters at each site set up upon initial installation. Dynamic referrals could be sent to a mental health specialist from the community at large or ‘in-house’ to psychiatry/counseling providers within the cancer centers. If sent outside of the cancer center, the dynamic referral was matched to the patient’s zip code and insurance carrier.

The building of the referral library included several steps. First, using internet searches and the white pages of the phone book, we crafted a list of all the mental health care specialists within a 30 mile radius of each cancer center. Second, we called the specialists to recruit dynamic referral providers and to gathering information (e.g., perceived competency to treat participants with cancer specific needs, insurance carriers accepted, fax number). Third, we obtained their agreement to respond to referrals within five days of receiving them (for those interested in being dynamic referral providers). Fourth, we constructed the computer database of mental health care providers and coded which ones agreed to be a dynamic referral provider. Finally, we validated the MHADRO’s ability to send appropriate referral to each dynamic referral provider through using mock patients.

2.5.2. Control group

Participants assigned to the control condition completed the MHADRO baseline and follow up assessments in the same manner as those assigned to the intervention group and received standard care for psychosocial issues, which was determined by each oncology provider. The participants in the control group did not receive any reports or the option for a dynamic referral. Their oncologist did not receive a healthcare provider report.

2.6. Measures

2.6.1. Baseline assessment

The MHARDO assessment is web based and is accessed through a computer with an internet connection (e.g., PC, laptop, tablet). Baseline assessments were completed on a handheld tablet with a stylus. Table 1 provides details of the baseline assessment constructs. A maximum of 80 items could be presented, though participants answered an average of 62.34 (SD = 3.03) questions, because branching logic ensured that participants only answered applicable items. In a laboratory setting, the baseline assessment was completed by naïve confederates in 15 – 20 minutes; however, RCT participants were sometimes interrupted by a doctor or nurse. When paused, the assessment would save progress with the option to resume, but the timer did not stop until assessment completion. For this reason, completion time for the RCT assessment was longer, averaging 28.17 (SD = 17.13) minutes.

Table 1.

Description of the constructs assessed at baseline and at follow up from which tailored reports were created

Construct	Description
Demographic and Cancer Information	Age, sex, marital status, education level, race, ethnicity, insurance provider, cancer history (e.g., type of cancer, time since diagnosis, treatment received, number of times in remission, duration of treatment)
Mental Health Assessment	History of mental health diagnoses; if positive a drop down menu was presented that listed 11 common mental health diagnoses (i.e., depression, bipolar, alcohol abuse, drug abuse, anxiety, panic attacks, PTSD, ADD/ADHD, anorexia, bulimia, schizophrenia)
*Depression	Feelings of sadness; decreased pleasure in activities; feelings of worthlessness; hopelessness; trouble concentrating
*Anxiety	Worry; tension or anxiety; irritability or easily angered; keyed up or on edge; trouble concentrating
*Functional Disability	Time had to cut down on work and spent activities as a result of any emotional problems; physical health limitations (e.g., carrying groceries, climbing stairs); managing day-to-day life; getting along with others; work, school, or household performance
*Subjective Well Being	How well getting along emotionally and psychologically
*Behavioral Health Status	An average of the anxiety, depression, functional disability, and subjective well-being scores; this scale is psychometrically validated [28].
*Self-Reported Distress	National Comprehensive Cancer Network’s (NCCN) distress thermometer [29]
Social Support	Help and advice from others; emotional support, comfort, and understanding; people to help with difficult time.
General Health Information	General self-assessment of physical health
*Alcohol Use	Alcohol Use Disorders Identification Test (AUDIT); frequency consuming alcoholic beverages; number of drinks typically consumed when drinking; and frequency of binge drinking. [30]
*Tobacco Use	History of tobacco use; tobacco products used in the 30 days prior to enrollment; quit attempts; number of cigarettes per day; how many minutes after waking first cigarette (the Heavy Smoking Index) [31]
Total Number of Symptoms	Pain, tiredness or fatigue, nausea or vomiting, insomnia or sleep difficulties, difficulty with bowel movements, and sexual difficulties or lack of interest in sex
*Patient-Provider Partnership	Treated the patient in a friendly and courteous manner, cared about the patient as a person, listened to patient, answered all questions, and had good communication with each other.
*Behavioral Health Recommendations	Whether or not oncologist had made specific recommendations (i.e., quit smoking, exercise daily, reduce alcohol use, go to support groups or counseling services, increase fluid intake, eat nutritious foods). Participants chose from a checklist of recommendations, indicating which recommendations had been made explicitly by their oncologist.
*Counseling/therapy Status	Whether or not presently in therapy
*Perception of Benefit of Therapy	If not in counseling, perception of potential benefit to engaging in therapy at present time
* Interest in DR (Intervention Group only)	Whether or not the patient is interested in having their information, in the form of a tailored report, sent to a mental health counselor, matched to their insurance and zip code, who will contact them to set up an initial therapy appointment.

2.6.2. Follow-up assessments

Each follow-up assessment included 94 possible items. These items covered many of the same domains as the baseline assessment with a few additional areas to assess outcomes of interest (see Table 2).

Table 2.

Description of the additional constructs assessed at follow up from which updated reports were created and sent to oncology providers

Construct	Description
Health Care Utilization	Number of times since the previous assessment they: visited an emergency room, were admitted to a hospital, spent a night in a hospital, saw their primary care doctor, saw their oncology doctor, needed to schedule an emergency oncology visit because there was a problem, and felt the need to call their oncology team because of an immediate concern.
Health Behavior Recommendations	Health habits were discussed (e.g., exercise levels and fruit, vegetable, red meat, and processed meat servings consumed), as well as whether or not these health behaviors had been discussed during oncology appointments (yes, no, I don’t know/can’t remember).
Return to work	Participants had taken time off from work for cancer treatment, they specified how much time they took off, whether they applied for disability, and whether they’d returned by the time of the assessment
Use of groups/classes	Amount of support groups and classes offered by cancer related organizations they had attended in the two months prior to the follow up assessment, through “yes or no” options and numerical answers
Oncology provider discussion of mental health	This domain inquired as to whether an oncologic doctor or nurse talked to participants about mental health or mental health treatments, gave participants educational materials about mental health or mental health treatment, or reviewed the MHADRO’s report with the patient
Mental health provider treatment initiation	How many times since the last assessment, if any, they had seen or been contacted by a counselor or mental health professional and whether they had an appointment scheduled for the future. Participants who accepted a Dynamic Referral at the previous assessment but failed to make an appointment were asked why they were not currently in treatment (e.g., “don’t think I need treatment,” “not ready to enter treatment,” “did not like the program(s) I was referred to,” “want to contact a different program,” “not feeling well enough lately,” “have too many appointments already,” “other reason”).
Psychotropic medications	Participants who had been prescribed a psychotropic medication for an emotional problem since the last assessment described who prescribed the medication (e.g., psychiatrist, oncologist, primary care doctor, other) and whether or not they were taking the prescribed medication.

2.6.3. Follow up assessment for mental health providers

Whenever a dynamic referral was sent on behalf of a participant, the mental health provider involved was contacted four weeks later. After being faxed a copy of the authorization to release personal health information form that participants completed during the baseline assessment, mental health providers indicated whether they had successfully contacted the participant, whether the participant had completed an initial evaluation and whether they began treatment with the provider. If so, the provider indicated how many appointments the patient had attended. The provider was asked if the participant was an appropriate referral for the program. The researchers did not have access to information about the participant’s treatment.

2.6.4. Medical chart review

The chart review examined participant’s (intervention and control groups) oncology records for the seven months after enrollment, allowing time for the reports from the baseline, two month, and six month assessments to be shared with the oncology team and charted on the medical record. The research staff that completed the chart- reviews was blind to the participants’ group assignment. The purpose of conducting chart reviews was to assess whether providers had conducted and documented discussions pertaining to mental health and health behaviors with the participants. Chart reviews included: participant’s age, sex, race, ethnicity, zip code, type of cancer, stage of cancer, date of diagnosis, and whether they were in active treatment or survivorship when they completed the MHADRO baseline assessment. We noted oncology team documentation of mental health indicators (i.e., depression, anxiety, sleep disturbance, sexual/intimacy difficulties, social support issues, marital problems, drug use/abuse, alcohol use, tobacco use, PTSD, and serious mental illness), referrals and recommendations for psychosocial resources/treatment, change in psychotropic medications, and healthcare utilization.

2.7. Treatment Fidelity

We assessed treatment fidelity in the intervention group. First, the MHADRO recorded whether the subject completed the assessment and whether the fax for those that chose a dynamic referral was sent successfully. Second, the research assistant completed a process log on each participant documenting whether critical tasks occurred, along with a description of any barriers, solutions applied, and outcomes. Critical tasks included: (1) printing the reports, (2) review of the oncology provider report by the participant’s treatment team, and (3) giving the patient-tailored report to the patient. The process log was updated after each follow-up assessment.

2.8. Data Analytic Plan

To take full advantage of the data collected, we have chosen to use several different but complementary analytic approaches. Given that distress scores are measured more than once for each participant and that these measures are correlated over time, most likely with unequal variance, two methods extending the generalized linear model (GLM) to longitudinal data will be used to test the primary hypothesis: Generalized Estimating Equations (GEE) for proportions and Linear Mixed Modeling (LMM) for means. Both methods can adjust for data obtained from multiple enrollment sites and can account for unbalanced designs that sometimes arise from attrition, including different numbers of repeat measurements available, different intervals between assessments or both. Additionally, like GLM, both methods can also handle non-normally distributed outcome variables. All analyses will be repeated using the global distress score and the depression subscale score to evaluate the robustness of the findings. Additionally, all analyses will be repeated using only those participants who scored in the Elevated distress range at baseline (i.e., 70^th percentile or higher), which represents the subsample of distressed cancer patients most likely to benefit from the MHADRO. Finally, because of the established links between some of our demographic and disease variables (e.g., age, gender, type of cancer) and psychological distress, such variables will be evaluated during preliminary analyses and those found to be related to the outcome variables will be treated as confounding variables and controlled for in all primary analyses. Also, demographic and cancer related variables that were found to differ between the intervention and control group will also be statistically addressed in all analyses.

Intention to treat principles will be applied. All participants will be analyzed in the group to which they are assigned. Participants lost to follow up will be assumed not to have improved (i.e., no change from baseline level of distress) and to be non-initiators of mental health treatment.

2.8.1. Hypothesis testing

GEE will be used to test the difference in the proportions of participants in the intervention and control groups who achieve non-elevated distress defined as a score below the 70^th percentile (after adjusting for confounders). The model will include a nested term for site of enrollment as a random effect. The difference in proportions between groups at or below this 70^th percentile threshold will be summarized as an odds ratio and tested for inequality between the intervention and control groups using a non-central Wald chi-square. Results will provide a measure of whether a greater proportion of patients in the intervention group have achieved a norm-referenced “non-elevated” distress score, which, clinically, can be translated into experiencing distress equal to or less than the vast majority of cancer patients.

LMM is comparable to GEE however it is more routinely used to evaluate means. Mean differences between the intervention and control groups will be analyzed using LMM with two random effects: time and a nested term for site of enrollment. The fixed component consists of distress scores entered stepwise after adjusting for confounding variables. Results will provide another view of the data from the perspective of a continuous measure and will indicate whether mean distress levels were significantly lower in the intervention group compared to the control group at each time-point.

2.8.2. Mechanisms of action

We are interested in examining a potential mechanism of action to include initiation of mental health counseling, psychotropic medications, and psychosocial support group participation in distressed patients who were not receiving mental health treatment at baseline or enhancement of existing treatment in patients already receiving mental health treatment at baseline. Enhancements might include adding another treatment modality, such as psychotherapy being added to a medication regimen, or adjusting their treatments, such as increasing the dose of an anti-depressant. We will create an aggregated variable called “Treatment Initiation/Augmentation” that represents any mental health treatment initiation or augmentation during the monitoring period. Participants will be coded “Yes” if they (1) were not receiving mental health treatment at baseline and subsequently initiated counseling, psychotropic medications, or psychosocial support group participation; (2) if they were receiving treatment at baseline but added another modality (e.g., if they were already taking psychotropic medications at baseline but initiated psychotherapy); or (3) if they were initially receiving treatment but had it enhanced in some way (e.g., if their dose was increased or if they were receiving outpatient therapy on a monthly basis and this was increased to a weekly basis). Subjects will be considered a “No” if none of these conditions apply. We will create dummy coded variables to represent the cross classification between group assignment and Treatment Initiation/Augmentation. All patients will be classified into one of four groups: control group X “No” Treatment Initiation/Augmentation (0,0 or reference group); Intervention group X “Yes” Treatment Initiation/Augmentation (1,1); intervention group X “No,” Treatment Initiation/Augmentation (1,0); and, control group X “Yes,” Treatment Initiation/ Augmentation (0,1). The three later groups will be coded as contrast variables against the reference group, yielding three variables to be tested for significance in the GEE and LMM models. Beta coefficients for these contrasts will allow inferences about the combined influence of group assignment (intervention, control) and exposure to the mechanisms of action (Treatment Initiation/Augmentation, Yes/No) on distress over time.

2.8.3. Power analysis

As our primary outcomes of interest are dichotomous in nature (e.g. a score in the elevated range (>70^th percentile) on the overall distress scale), minimum sample size calculations for GEE analyses are based on a parallel group design with a baseline and two follow-up measurements (t = 3) using the global distress score, equal allocation, and a 20% loss to follow-up, most of which we assume will occur in the first 6 months. We use the method of Diggle et al. [16] to determine the required sample size for the longitudinal case of determining a consistent difference in proportions between two groups across several timepoints. Sample sizes required to attain 80% power for a two-sided test at alpha = 0.05, assuming an exchangeable covariance structure and autocorrelation of 0.50 to 0.70, range from 646 to 776 (see Figure 3). The difference tested corresponds to a conservative estimated difference of proportions of 0.1 (0.5 vs. 0.4). The corresponding odds ratio of 1.5 can be converted [17] to a Cohen’s D value of 0.22 representing a relatively small effect size [18]. Our attained sample size of 836, allowing for 20% attrition, should yield approximately 670 subjects for analysis, which falls within the target range of 646 to 776.

Figure 3.

Sample size estimates (Rho = 0.2 – 0.7)

3. Results

In this paper, we present only descriptive statistics on the enrolled sample, since the study is still ongoing. Patient outcomes results will be published in future papers.

3.1. Descriptive statistics and group differences

The sample consisted of predominately white/Caucasian (n = 742, 89%) women (n = 718, 86%) who were married (n = 542, 65%). The average age was 59 years and participants had typically completed at least some college (n = 579, 69%). Although there were over 12 types of cancer represented (e.g., breast, ovarian, endometrial, colorectal, lung, cervical, lymphoma, prostate, leukemia, head-neck, thyroid, kidney), the most common diagnoses were breast cancers (n = 410, 49%) and gynecological cancers such as endometrial, cervical, and ovarian (n=179, 21.4%). The majority of patients had received their diagnosis more than 6 months prior to the baseline assessment (n=552, 66%) and 443 (53%) patients were in remission when they were enrolled. To determine if the randomization worked to equalize the intervention and control groups, we also conducted group comparison analyses on the demographic and cancer related variables including age, race, gender, marital status, education, cancer diagnosis, time since diagnosis. No significant differences were found.

3.2. Enrolled vs non-enrolled

Enrollment is summarized in Figure 2. Enrolled (n=836) and non-enrolled (n=1,424) patients were compared to determine overall representativeness of the sample. Enrolled participants were more likely to be female (86% vs. 78%, χ²(1)=21.99, p=.000), younger (x=58.9 (s.d.=11.7) vs. 62.6 (s.d.=13.1), F(1,2235)=45.74, p = .000), and White/Caucasian (92% vs. 86%, compared to Black/African American and to all other races combined, χ²(2)=29.09, p=.000). Type of cancer also varied significantly for enrolled compared to non-enrolled patients, with a higher percentage of enrolled participants having breast cancer (49% vs. 37%, χ²(1)=31.43, p=.000).

Figure 2.

Enrollment Flow into RCT Across 3 Sites

4. Discussion

Identifying cancer patients who are experiencing significant psychosocial distress, monitoring their status over time, providing interventions to help mitigate the impact on their quality of life, and helping individuals who need specialized mental health treatment to identify and access such treatment, are important therapeutic goals in oncology settings. These activities, however, require time, resources, and training that many oncology providers simply do not have. As a result, the field has begun to explore how technological solutions can help make these tasks more efficient, effective, and standardized. The MHADRO was designed with these goals in mind and this large RCT had a few unconventional design features. For example, many clinical trials face problems regarding external validity or generalization into “real world” situations. The design of the present study was crafted with this in mind and efforts were made to study the intervention under conditions that would mimic the “real world” as closely as possible. Also, we worked closely with the end-users (i.e., cancer patients, oncology providers, mental health providers, and oncology staff) during the building and implementing of the program. This also ensured that our clinical trial would be able to be more easily integrated into busy clinical practices and that the program would be useful to all those involved in the study.

On the other hand, testing such technological interventions using a clinical trial can raise important methodological challenges. We encountered a number of key challenges during this study and made thoughtful attempts to solve or mitigate them. First, although we did emphasize the importance of “real world” experience, if the MHADRO were to be integrated into clinical care, it is very likely that oncology providers would be trained on the nature of the program and how to read the reports but would not engage in extensive training on management of mental health conditions or be expected to perform counseling. Rather, they would be encouraged to manage conditions as per their best clinical judgment, based on the clinical data provided on the reports. The site would also have to institute a process to ensure that patients received their reports after completing the assessment. It is unlikely that the oncologist would do this personally. It would probably be delegated to a paraprofessional or administrative personnel, and it is unlikely that the individual who ultimately gave the report to the individual would review it in depth with him or her. Instead, it is likely they would simply hand it to the patient and reinforce the importance of reading it and following through on the referrals, if any were given.

We attempted to mimic these conditions but it was impossible to rely entirely upon the clinical staff to manage all parts of the process. Clinical staff may often be reluctant to adopt new procedures and modify clinical protocols to implement experimental interventions if they have not yet demonstrated efficacy. Consequently, the research assistant was the one who guided the “paper process” by making sure the provider and patient both received their respective reports. This departed from what would likely happen in the clinical setting since this would have to be done by an employee of the site but we judged that this minimal level of consistency in protocol implementation was deemed indispensable for the success of the project. We felt the protocol balanced internal and external validity by implementing a process very similar to clinical implementation but with safeguards to ensure standardization and fidelity to the intervention.

Second, once again to mimic the real world, we enrolled a heterogeneous sample of cancer patients regardless of cancer type, severity of cancer, or phase of treatment. The benefit of this approach is that our results are more likely to be applicable to the real world. However, even though we did not exclude based on cancer type, some selection bias occurred, reducing the heterogeneity of the sample. Almost half of our sample was comprised of cancer survivors, as defined as patients who were no longer in active treatment, some for more than 15 years. In addition, the majority of our sample was female. We believe these over-representations were the result of a few factors. First, we speculate that survivors were more likely to be feeling physically better and psychologically less anxious as compared to the active treatment patients, resulting in more willingness to take the time to participate. Second, the large percentage of women enrolled in the study is likely related to type of oncology providers that agreed to participate in this study. The breast and gynecological cancer providers were highly receptive to study recruitment from their clinics and many noted to us that their female patients tended to have psychosocial difficulties, which they would appreciate assistance in addressing. Thus, when considering the psychological status of our sample in future papers, the large number of patients in remission should be noted as previous research has demonstrated that cancer patients’ physical and mental health status improves significantly as time since diagnosis increases [21]. Similarly, the gender bias towards females should be noted in interpreting future analyses as women have consistently reported greater levels of psychological distress as compared to men with cancer [22– 24].

It is also important to note that we did not use stratification and instead used a simple randomization approach for enrollment. This choice was made as we felt the enrollment of almost 900 patients would minimize likelihood of differences between control and intervention group participants. The data comparing the two groups in terms of demographic and cancer related variables showed no differences between the two groups, which suggest the randomization was somewhat successful in equalizing across these variables. However, there are a number of variables such as age, gender, stage of cancer, that have been clearly linked to psychological distress. It is a limitation that we did not stratify for some of the variables known to impact psychological distress and emphasize that a stratified randomization approach would have been useful for ensuring such differences would not impact final results. In place of the stratification, we will statistically control for all of the variables that are linked with psychological distress in our primary analyses.

We also compared enrolled versus non-enrolled patients and found differences, which we outlined in the results section above. Specifically, the enrolled patients were more likely to have breast disease and to be White/Caucasian. This brings up an important issue that cancer researchers continue to face; that is, an under-representation of people of color in clinical trials. We do not know the reason this happened. It is reasonable to hypothesize that Black/African American patients distrust of research [25] could have decreased enrollment rates or that the predominantly White/Caucasian research staff unintentionally were more active in recruiting patients from their own race. Previous research has clearly documented such unconscious biases in laboratories [26] as well as real-life situations [27]. While these explanations are speculative, the net effect is that results from our study will have to be interpreted within the light of under-representation of minority patients.

Fourth, recruiting participants to participate in a trial of a technology intervention can introduce a bias towards recruiting younger, more computer literate individuals. Considering the fact that cancer patients tend to be older than the general population, and the inverse association between age and computer literacy, one might expect this bias to be even more important in oncology settings. As a result, the MHADRO assessment was designed to require no computer literacy, and our recruitment procedures emphasized the ease of the assessment and assured potential participants that computer familiarity or experience was not needed. While we cannot state whether our sample was more or less computer literate than the population from which it was drawn, we do know that our enrolled sample was significantly younger than the population. It is possible that one of the contributors to this age-related bias was computer literacy [19, 20]. Consequently, the results will be more generalizable to a younger, more computer literate cancer population.

Finally, another notable challenge in this kind of research relates to the magnitude of the impact that brief, intermittent interaction with a simple computer program is likely to make on outcomes as resistant to change as psychological distress and depression. The MHADRO represents a minimal intervention. The clinical effects are likely to be modest and exerted through intermediate process outcomes such as those we will examine as mechanisms of action. It is possible that the MHADRO may have significant impact upon process outcomes, such as prompting the oncology provider to begin an antidepressant or helping the individual connect with a mental health provider, without a commensurate impact upon psychological distress. It is important to consider whether there is benefit to improving process outcomes if there is only modest improvement in clinical outcomes. In addition, a practical consequence of the likely small effect is that sample sizes must be large, resulting in the need for multiple centers and large budgets. In our study, we helped to mitigate the costs of performing follow-up interviews on our 836 subjects by allowing subjects to complete the follow-up assessments on-line; at both the two and six month follow-up, approximately 50% of patients chose the on-line option. Although we are still completing the 12 month follow up data collection, we anticipate a similar level of interest in on-line assessments.

In conclusion, studying technology-based interventions for psychosocial issues in clinical oncology settings poses many challenges to internal and external validity. We have discussed some strategies to mitigate these challenges. The results of this study should help advance our understanding of the role of computerized psychosocial interventions in identifying, managing, and ultimately improving psychosocial outcomes among cancer patients.