Factors Identified by Experts to Support Decision Making for Post Acute Referral

Abstract

Background

Each year, more than 13 million post acute referral decisions are made for Medicare recipients, yet there are no national, empirically derived decision support tools to assist in making these important decisions.

Objectives

The aim of this study was to elicit expert knowledge about factors important to referral decision making and identify the characteristics of hospitalized patients who need a post acute referral.

Methods

This was a retrospective and prospective mixed-methods study of the referral decisions made by discharge planning experts for 355 hospitalized older adults. Variables included sociodemographics, living arrangement, insurance, diagnosis, comorbid conditions, adverse events, medications, home care use, hospitalization in last 30 days or 6 months, patients’ perception of need for and use of assistive devices or post acute services, length of stay, cognition, self-rated health, depression, functional status, and post acute referral decision.

Results

The final model identified six factors associated with the need for a post acute referral. A cutpoint was derived with a sensitivity and specificity of 87.6% and 63.2%, respectively. Experts were more likely to refer patients who had no or intermittent help available (odds ratio [OR] = 3.0), major walking restrictions (OR = 6.5), less than excellent self-rated health (3.1 and 4.0 times more likely with good and fair–poor health, respectively), remained in the hospital longer (OR = 1.2), and had higher depression scores (OR = 1.1) or number of comorbidities (OR = 1.2).

Discussion

This study begins to identify information useful to clinicians caring for hospitalized older adults who may benefit from post acute services. By assuring the systematic, valid, and reliable collection of these items, the multidisciplinary team is alerted to patients who may benefit from post acute services. Further work is needed to increase the specificity and generalizability of the model and to test its effects on patient and clinician outcomes.

Annually, more than 13 million post acute referral decisions are made for Medicare recipients (U.S. Department of Health and Human Services, 2006). Although the volume of decisions is high, there are no national, empirically derived decision support tools to assist in making these important decisions. For example, in a document titled Discharge Planning for the Older Adult (Zwicker & Picariello, 2003), a list of what should be assessed is offered. However, this document is not useful as a decision support tool because of its length and lack of synthesis or explicit recommendations to the user. Decision support in nursing is an understudied but newly developing area of science. As evidence-based practice develops, research-based methods to support decision making, such as the one described in this report, will become more common place. The purposes of this study were to elicit expert knowledge about the factors considered important to the referral decision-making process and to use this knowledge to build a decision model associated with experts’ recommendations for post acute care (PAC) referral. A PAC referral is a clinician recommendation to refer the patient for post acute services such as skilled home care, outpatient rehabilitation, or admission to a nursing home, inpatient rehabilitation center, or skilled nursing facility.

The PAC referral process requires careful, comprehensive assessment to determine patients’ current needs, reassess for changing needs, anticipate future needs, make appropriate referral decisions, and coordinate follow-up services. Shortened lengths of stay, inconsistent assessment criteria, inaccessible information, work overload, and varying levels of expertise, knowledge, and risk tolerance in decision making may affect the quality of decision-making accuracy (Parker et al., 2002; Shepperd, Parkes, McClaren, & Phillips, 2004).

Several studies describe disparities in referral decision making and point to the importance of standardization and decision support for this important function. Patients living with a spouse were less than half as likely to be referred for skilled home care than those living with nonspouse caregivers (Diwan, Berger, & Manns, 1997). Living alone assured 94% of the patients a visit by discharge planners, whereas only 40% of those who lived with someone were visited (Furstenberg & Mezey, 1987). Disparities also exist by race and gender. African American patients received significantly fewer hours of home care services (Chadiha, Proctor, Morrow-Howell, Darkwa, & Dore, 1995), nursing home admissions (Morrow-Howell, Chadiha, Proctor, Hourd-Bryant, & Dore, 1996), and referrals to cardiac rehabilitation (Gregory, LaVeist, & Simpson, 2006) than White patients did. Women with the same functional limitations as men were only one fourth as likely as men to be referred for home care (Pohl, Collins, & Given, 1995). High-risk patients and patients with unmet needs were not referred for home care as expected (Bowles, Naylor, & Foust, 2002; Bowles et al., 2008). Multiple studies show that geographic, financial, structural, personal, socio-demographic, and attitudinal factors affect PAC access regardless of disability, diagnosis, or age (Buntin, 2007; Ottenbacher & Graham, 2007). Of the 25% of Medicare beneficiaries who are readmitted within 30 days, 64% receive no PAC services (Medicare Payment Advisory Commission, 2007a). Discharge planning, transitions, and care coordination are important quality-of-care topics (National Quality Forum, 2006); therefore, studies that illuminate the issues and test interventions for improvement are needed.

Clearly, this important and common decision to refer older adults for PAC services may benefit from standardized, expert decision support. No study has reported the development and testing of a knowledge-based model of factors that synthesizes the input of multidisciplinary, gerontology experts to support PAC referral decision making. This study was designed to begin to address this gap in knowledge.

Methods

Specific Aim

This study addressed two specific aims: to elicit expert knowledge about factors important to referral decision making and to identify the characteristics of hospitalized patients who need a PAC referral.

Design

A mixed-method comparative and exploratory analysis of the characteristics of hospitalized older adults and the decisions of experts to refer them for PAC was conducted on a combined retrospective and prospective sample. The independent variables were derived through various methods of knowledge elicitation, including case study analysis, Delphi rounds, focus groups, and data mining. The details of these methodologies are described below. The dependent variable was the experts’ yes–no referral decision. Institutional review board approval was obtained for all study phases.

Theoretical Framework

The Self-care Deficit Theory of Orem (1985) guided the identification and organization of factors that affect the patient’s ability for self-care. Nursing care is appropriate when the person is not able to engage in self-care. Orem notes that basic conditioning factors are internal or external factors that affect the ability of an individual to engage in self-care or influence the amount of self-care required. These basic conditioning factors fall into 10 categories: age, gender, developmental state, health state, sociocultural orientation, healthcare system factors, family systems factors, patterns of living, environmental factors, and socioeconomic factors (Orem, 1985). The Orem conditioning factors provided the framework to categorize information abstracted from patients’ records and to organize the ontology and the discussion during expert focus groups.

Sample

The sample contained 355 older adults admitted to one of six hospitals (urban, suburban, and rural). Data came from two sources: existing records and a convenience sample. The original plan was to analyze only existing records, but the experts requested additional cases to add variety to types of diagnoses represented. The data sets were combined because they contained measures of the same variables and the analysis achieved the same goal for both data sets. Their retrospective or prospective nature did not affect the study design.

Existing Records

The existing records were taken from among the control group patients enrolled during three completed randomized clinical trials (RCT) (Naylor et al., 1999, 2004, 1994). These records were used because they had a comprehensive database of variables that described the characteristics of older adults while hospitalized and 12 weeks after discharge. The same variables were collected across all three studies, and the control group was used so that the effects of the RCT intervention did not influence the 12-week outcomes. A power analysis, using a standard error of 0.045 associated with the area under the curve (AUC) statistic and assuming an anticipated concordance of 0.80, was performed to calculate the minimum number of cases, which was found to be 100 (50 referral + 50 nonreferral; Hanley & McNeil, 1982). A list of subject identification numbers (N = 443) was generated from the control groups of the three previous studies. The sample for this study randomly drawn from this list. On the basis of pilot work, where most patients were referred by experts, a larger sample (than the power analysis indicated) was purposely drawn to assure enough nonreferral types. Therefore, 245 participants were chosen randomly using a table of random numbers. After reviewing these records, 37 were not used because of missing data (n = 21), withdrawal or moved (n = 6), not readable (n = 3), cases too similar (n = 2), or cases used to train the abstractors (n = 5). The remaining 208 records were used in this study.

Convenience Sample

Data on the remaining 147 older adults were collected prospectively from a convenience sample enrolled from an academic medical center (n = 50) and a rural community hospital (n = 97), both sites used in the previous studies.

All participants were 65 years and older, English speaking, and cognitively intact and expected to be discharged home. In addition, patients enrolled in Studies 2 and 3 met at least one criteria associated with risk for poor discharge outcomes (Naylor et al., 1999, 2004). Participants in Study 1 were admitted for heart failure, angina pectoris, myocardial infarction, valve replacement, or coronary artery bypass surgery (Naylor et al., 1994). Study 2 patients had those diagnoses and respiratory infection, major small and large bowel surgery, or lower extremity orthopedic surgery (Naylor et al., 1999). All Study 3 patients were admitted for heart failure (Naylor et al., 2004).

Participants enrolled prospectively met the same inclusion criteria except for diagnoses and the criteria associated with poor outcomes. They were sought because the experts requested cases with more diversity in diagnosis and severity. These cases represented cancer, diabetes, infection, genitourinary conditions, and traumatic injury and were mixed with the others before presentation to the experts. All patients had the same database of information about their characteristics available for analysis (i.e., function, number of conditions, caregiver availability).

Data Collection

Case studies were developed from the patients’ medical records and interviews conducted during the index hospitalization. Data were collected on age, race, gender, income, education, living arrangement, insurance, medical diagnosis, comorbidities, adverse events, admission and discharge medications, length of stay (LOS), previous home care or hospitalization in the last 6 months, use of assistive devices or services, and patient’s perceived need for devices or services after discharge. Measures of self-rated health, cognition, functional status, and depression were obtained using standardized instruments. Discharge disposition was recorded to know whether the patient received a referral for PAC services and subsequent rehospitalization and emergency department use up to 12 weeks after discharge. All PAC services were verified by subsequent medical record review. Trained nursing graduate student research assistants (RAs) collected all data in-person during hospitalization and by telephone at 12 weeks.

Instruments

Self-rated health status is the patient’s perception of overall health measured using a single question: How would you rate your overall health at the present time? Is it excellent, good, fair, or poor (Maddox & Douglass, 1973)?

The Short Portable Mental Status Questionnaire (SPMSQ) was used to measure the presence and degree of intellectual impairment. It is a valid and reliable measure of mental status in the elderly (Roccaforte, Burke, Bayer, & Wengel, 1994). For construct validity, the SPMSQ showed good correlation with the Mini-Mental State Exam and a test–retest reliability k value of .45 was reported, with a sensitivity of .74 and specificity of .91 for detecting dementia (Roccaforte et al., 1994).

Functional status was measured using the Enforced Social Dependency Scale (Moinpour, McCorkle, & Saunders, 1981). Enforced social dependency is defined as needing assistance to perform activities or roles that adults can usually do alone. The instrument describes the patient’s function regarding eating, dressing, walking, traveling, bathing, and toileting; home, work, and recreational activities; and communication. Scores range from 10 to 51, with higher scores indicating more dependency. McCorkle and Benoliel (1981) reported a .8 reliability coefficient for the total scale with cardiac patients, and test–retest reliability for the revised scale was .62.

Depression was measured using the Centers for Epidemiologic Studies Depression Scale (Radloff, 1977). Depression was not measured in Study 1 and partially measured in Study 3 patients. Participants rate the occurrence of 20 items during the last week; scores >16 indicate depressive symptoms. The scale has high internal consistency (.85) and adequate test–retest reliability (average of .53 for different samples, including the elderly; Callahan, Hui, Nienaber, Musick, & Tierney, 1994; Datto, Thompson, Knott, & Katz, 2006).

Development of Case Studies

The principal investigator (PI) used five records to train two RAs in abstracting information from the research and medical records of each patient. The abstractors read the entire case to become familiar with the content and context. During a second reading, information about the assessments and interventions documented in the progress notes, discharge plans, and summaries was collected and summarized into a case study as shown in Table 1. The 10 Orem basic conditioning factors provided the organizing framework to standardize the format of the case studies. The PI reviewed all 355 cases for completeness and grammatical and medical accuracy before they were sent to the experts.

TABLE 1.

An Example of a Case Study Sent to the Experts

Age, race, gender: The patient is a 78-year-old White man.

Health state: The patient was admitted for shortness of breath,     coughing, and chest tightness. A chest X-ray was performed,     and his diagnosis was pneumonia and exacerbation of chronic     obstructive pulmonary disease (COPD). He was admitted and     treated with IV antibiotics and nebulizer treatments. Past     medical history is significant for coronary artery disease, atrial     fibrillation, COPD, hypercholesterolemia, and diabetes mellitus     secondary to steroid use. Medications prior to this admission     include aspirin, Cartia, Combivent, Coumadin, Digoxin,     Flovent, Nitrostat, and Augmentin (for the last 2 days).     Discharge medications included Levaquin and Glucotrol.     Length of stay was 3 days.

Developmental state: The patient attended school until the 10th      grade. He is retired. He scored 9/10 on the mental status      examination and 9 on the depression scale (16–60 =      depression). He is independent in all activities of daily living.      He has no health aids and does not feel that he needs any. He      has no dietary restrictions. He reports fatigue and shortness of      breath while dressing, walking, and bathing. He rated his      health as fair at the present time. He has not used any formal      health services and does not feel he needs any.

Health care system factors: The patient has had no      hospitalizations in the past 6 months. He saw his doctor twice      in the past 6 months.

Family system factors: The patient is married and lives with his      spouse. He considers his wife and son to be his primary      caregivers. They are available whenever needed.

Environmental factors, patterns of living: The patient lives in a      single-family home. There is a bathroom on the level of the      bedroom and kitchen.

Socioeconomic factors: The patient reports his income as      $5,000–$10,000/year and is insured by Medicare Parts A and      B and private health insurance.

The Experts

Four nationally recognized scholars and four local clinicians were selected as experts. The four national scholars were chosen based on their published record of scholarly work in discharge planning and the care of the elderly. They held either a PhD or MD degree. The four clinical experts were recommended by their acute care managers for their discharge planning skill and had at least 5 years of experience in discharge decision making for older adults. They held at least an MS or MD degree. This multidisciplinary group contained two of each of the major disciplines involved in discharge planning: physicians, nurses, social workers, and physical therapists. They were paid $55/hour and each case took 15–30 minutes to review.

Expert Judgments and Data Management

The cases were posted online where the experts anonymously and independently evaluated each case, provided a “yes” or “no” referral decision, and identified which patient characteristics (factors) influenced their decision and, if a referral was suggested, to which PAC service. Their findings (recorded on the Web site) were entered automatically into a relational database for analysis. Cases were posted for the experts in batches of 50. When all eight experts entered their judgments, the round was considered complete. All cases for which there was no agreement for the yes–no referral decision and the site were posted for Delphi rounds. Three Delphi rounds enabled the experts to review the case again and judge again. The goal was to seek consensus. The experts could see each other’s decisions and reasons, but not their identity. In this subsequent review, the experts considered the various viewpoints of their colleagues and, if they changed their mind, indicated their new decision and the reasons for it. After three rounds, if total agreement was not reached, the majority decision (reflected by at least five experts) was accepted. This occurred for 91 of the cases (26%). There were 12 cases with a tie (four said “yes,” four said “no”) that were discussed in the final in-person session and resolved. The experts achieved statistically significant correlations among their decisions regardless of discipline (r = .291–.517, p = .01).

An ontology was developed in collaboration with the experts to standardize the reasons for referral. An ontology is a formal representation of a set of concepts within their domains and the relationships between those concepts. It is used to define the domain. By coding each of the factors systematically, the terms used were standardized by the experts to describe the reasons for referral or not and prepare the qualitative data for more robust analyses. For example, one expert might have referred because the patient has difficulty walking, another might have referred because of impaired mobility. These two terms were each coded with the same code within the ontology so they could be categorized and counted. The study investigators created the ontology using the domain headings from the Orem Self-care Deficit Theory and created subcategories under them that described the terms used by the experts. Techniques used are described further by Castro et al. (2006). For example, the Orem category of Developmental State contained ontology code 1.0 functional status, with subcategories of 1.1 walking ability, 1.2 bathing ability, and so forth, as subcategories within that domain. Prior to coding the reasons for referral, the study team presented the ontology categories to the experts for approval. The experts and study investigators coded several cases together and the experts affirmed that the ontology domains and subcodes adequately captured their meaning. The PI was the primary coder, with member checks by the experts on every 50th case, and an interrater reliability of 92% was achieved with a coinvestigator on a random sample of 10% of the codes. When coders did not agree or were unclear, they were discussed with the experts. These data were analyzed using data mining techniques, recursive partitioning, and decision tree algorithms (Witten & Frank, 2005). These exploratory analyses helped to identify the most common factors related to the decisions. The results were discussed in the focus groups described below.

Focus Groups

Over the course of the study, seven in-person sessions were held with the local experts, and national experts participated by e-mail, telephone, and one in-person session. During these sessions, the ontology was validated, cases were reviewed, and the factors identified by the data mining were discussed. The eight experts reviewed the rules generated by the data mining and systematically considered each factor in relation to the decision to refer or not. Factors identified by the experts as important were included in the logistic regression analyses (Table 2).

TABLE 2.

Variables Considered in the Logistic Regression

Variable	n (%)	Mean ± SD
Function: bathing
No restriction	219 (61.7)
Minor restriction	49 (13.8)
Help of equipment	55 (15.5)
Daily person assist	32 (9.0)
Function: eating
No restriction	117 (33.0)
Minor restriction	178 (50.1)
Occasional help	20 (5.6)
Daily assistance	40 (11.3)
Function: walking
No restriction	106 (29.9)
Minor restriction	117 (33.0)
Major restriction	132 (37.2)
Function: dressing
No restriction	209 (58.9)
Restrictions	146 (41.1)
Mental status
Normal mental status	343 (96.6)	9.56 ± 0.769
Cognitively impaired	12 (3.4)
Who lives with patient?
Spouse	175 (49.3)
Other person	81 (22.8)
No one	98 (27.6)
Subjective health rating
Excellent	44 (12.4)
Good	128 (36.1)
Fair/poor	183 (51.5)
Rehospitalization in past 6 months
None	166 (46.8)
One prior hospitalization	122 (34.4)
Two or more prior hospitalizations	62 (17.5)
Length of stay (days)	355	7.08 ± 6.05
Complicated hospital stay
No	297 (83.7)
Yes	58 (16.3)
Surgery occurred during admission
No	249 (70.1)
Yes	106 (29.9)
Patient is at risk for falls
No	343 (96.6)
Yes	12 (3.4)
Help available
No help or do not know	30 (8.5)
Yes	325 (91.5)
How often is help available
None or intermittent	102 (28.7)
Whenever needed	253 (71.3)
Patient perspective on total number     of resources needed
No resources needed	252 (71.0)
One or more resources needed	101 (28.5)
Age (years)	354	74.25 ± 5.7
Income level
≥$20,000	203 (57.2)
<$20,000	152 (42.8)
Total number of comorbid diagnoses	354	4.71 ± 2.3
Number of medications on discharge	351	5.73 ± 3.3
Depression score admission (0–60)	259	11.44 ± 10.6
Imputed depression score admission	355	12.08 ± 9.5

Statistical Analysis

Descriptive statistics revealed the distribution of the variables. Two cases were not posted accidentally for the experts, so the data set contained 353 cases. Because depression was not assessed in Study 1 and was assessed partially in Study 3, 27.0% of the participants did not have depression scores. Missing depression scores were imputed using multiple imputation via the EM method (Schafer, 1997). Imputed variables were generated based on the known participant demographic and functional information at baseline, including age, race, gender, marital status, self-rated health, number of comorbidities, and functional status variables. Categorical variables were collapsed as necessary to ensure sufficient numbers (>5%) in any one category. Due to the number of variables considered, terms were added using forward selection, adding those factors whose addition caused a significant change in the model fit. Variables that were selected in the final models were reviewed to ensure that there was no confounding. The final model contained all of the independent variables able to sustain statistical significance (p < .05) or those whose removal affected the estimate for another significant variable by more than 10% (Hosmer & Lemeshow, 2000). Receiver operating characteristic curves were used to determine the optimal cut-point for classification and prediction by the regression model (Hanley & McNeil, 1982). Finally, Monte Carlo cross-validation was run to obtain overall predictive value, using 500 replications and 20% of the data as the validation set (Efron & Tibshirani, 1993).

Results

Sample characteristics are the following: 54% were female, 74% were White, 26% were Black, average age was 74 years (range = 65–90 years), 50% were married, 36% were widowed, 30% had less than a high school education, 43% had an annual income <$20,000, and 52% had at least one hospitalization in the previous 6 months.

Experts’ Referral Decisions

The experts recommended referral for 183 additional patients than were referred by hospital clinicians. Although the experts’ referral recommendations were predominantly for skilled home care services (88%), a few patients were referred also to outpatient (8%) and inpatient rehabilitation (4%). This distribution precluded examination separately by site; however, identification of who should be flagged for referral, regardless of site, remained clinically important. Therefore, the presence or absence of a discharge referral, regardless of site, remained the dependent variable.

Logistic Regression Results for Expert Decision

Of the 20 variables, 6 were found to be statistically significant (p < .05; Table 3), and the patient’s age was an effect modifier; it changed the estimates for self-rated health by >10%. Experts were more likely to refer patients who had no or intermittent help (odds ratio [OR] = 3.0, p = .018, 95%confidence interval [CI] = 1.2–7.3), had major walking restrictions (OR = 6.5, p = .002, 95% CI = 2.0–20.9), had less than excellent self-rated health (good health: OR = 3.1, p = .017, 95% CI = 1.2–7.7; fair–poor health: OR = 4.0, p = .005, 95% CI = 1.5–10.5), had longer LOS (OR = 1.2 per day, p = .004, 95% CI = 1.0–1.3), had higher depression scores (OR = 1.1, p = .011, 95% CI = 1.0–1.1), or had more comorbidities (OR = 1.2, p = .010, 95% CI = 1.1–1.5).

TABLE 3.

Final Logistic Regression Model for the Experts’ Referral Decision

					95% CI for odds
Variable (n = 255)a	Estimate	SE	p	Odds of referral	Lower	Upper
How often is help available?			.018
Never or intermittent	1.084	0.458	.018	2.96	1.204	7.263
Whenever needed (ref)b
Walking function			.044
No restriction (ref)b
Minor restriction	0.437	0.363	.229	1.55	0.760	3.153
Major restriction	1.868	0.598	.002	6.47	2.004	20.899
Subjective health rating			.016
Excellent (ref)b
Good	1.117	0.470	.017	3.05	1.216	7.671
Fair or poor	1.387	0.493	.005	4.00	1.523	10.523
Length of stay (days)	0.137	0.048	.004	1.15	1.043	1.259
Depression score	0.061	0.024	.011	1.06	1.014	1.115
Age (years)	0.050	0.030	.102	1.05	0.990	1.115
Number of comorbidities	0.209	0.081	.010	1.23	1.051	1.446

Note. CI = confidence interval.

^aSample decreased to 255 because of missing depression scores.

^bReference group used in the model.

Similar results were obtained when participants with imputed depression scores were excluded from the model, except that age obtained statistical significance (OR = 1.1, p = .037, 95% CI = 1.0–1.2). Combination factors (interactions) identified by the experts were examined. However, they either did not occur in a large enough quantity to include in the models or were not statistically significant.

The receiver operating characteristic curves were used to determine the optimal cut-point for classification. The optimal cut-point was 0.69. This corresponded to a sensitivity and specificity of 87.6% and 65.2%, respectively. The AUC was 86.3%. In general, an AUC greater than 80% indicates a good model for classifying participants with an outcome of interest against those without the outcome (Hanley & McNeil, 1982). The overall predictive value for the model was 83.2%, with a cross-validated predictive value of 80.1%.

Discussion

Study findings address an important component of care coordination, which is an Institute of Medicine and National Quality Forum priority (Adams & Corrigan, 2003; National Quality Forum, 2006). The period surrounding hospital discharge is a busy time, involving multiple disciplines, often with little time for collaboration and careful deliberation (Shepperd et al., 2004). In this study, experts in discharge planning provided the knowledge to build this classification model. Providing experts with comprehensive, high-quality information from actual clinical cases and the time to consider their decisions carefully contributed to the identification of additional patients who might have benefited from referrals. It is important to note that 23% of the additional patients identified by experts but not referred by hospital clinicians were rehospitalized by 12 weeks (Bowles et al., 2008).

The resulting six-factor model may provide guidance to busy clinicians about some key patient characteristics that are associated with experts’ decisions to refer. For example, the model suggests careful evaluation of walking ability because those with major walking restrictions were 6.5 times more likely to be referred. Multiple sources agree that physical function, which includes walking, is an important predictor of the need for PAC and outcomes (Cornette et al., 2006; Nsameluh, Holland, & Gaspar, 2007).

Length of stay was identified as a predictor of a referral. The mean LOS was 7 days, which corresponds to the mean LOS during the time in which most of the study data were collected (mean = 6.8 days in 1995). By 2005, the average LOS fell to 5.4 days (Medicare Payment Advisory Commission [MedPac], 2007b). However, even though the experts knew that the current LOS is likely to be lower, they still identified it as an important factor as a proxy for severity of illness, complicated hospital course, (Fogel, Hyman, Rock, & Wolf-Klein, 2000) or predictor of poor outcomes (Cornette et al., 2005).

Advanced age (Holland, Harris, Leibson, Pankratz, & Krichbaum, 2006) and number of comorbidities (Naylor et al., 1999, 2004) commonly are associated with risk for poor outcomes. Usually, these factors are documented and readily available in the medical record. However, other factors such as depression and self-rated health are not measured routinely in acute care. Study findings suggest that it may be important to add these variables to routine assessment. In addition, usually, it is noted whether a patient has a caregiver or not. But these findings and others suggest that further probing into their availability is important. Nearly forty percent of older adults discharged from the hospital reported unmet needs needed help from a caregiver within the first week (Mistiaen, Duijnhouwer, Wijkel, deBont, & Veeger, 1997).

The model is quite proficient (87.6% accurate) at predicting who should be referred but is suboptimal (65.2%) at classifying those who do not need a referral. Use of this model may run the risk of promoting over-referral and requires further study to assess quality and cost implications. The experts referred 183 additional patients than were referred in real life. Notably, in the 12-week outcome analysis, these patients were rehospitalized at a rate of 23% (Bowles et al., 2008). With a rehospitalization rate that high, it may be cost-effective to provide PAC services for more patients if those services result in decreased future costs. Several recent studies suggest the clinical and economic value of correctly identifying patients and assuring appropriate follow-up care (Naylor et al., 1994, 1999, 2004; Steeman et al., 2006). This remains an important area for future study.

Limitations

Because of the following limitations, the model is preliminary and, although helpful at this stage, is not fully developed. This study was limited to subjects in Southeastern Pennsylvania. The types of medical conditions and treatments were limited and the sample was largely a high-risk group. In a more balanced sample, additional factors that affect the referral decision may be identified. In addition, the sample lacked representation of some characteristics in large enough frequency to explore combinations of factors fully. For example, only 3% of the sample was cognitively impaired. The study was limited also to the opinion of eight multidisciplinary experts.

Finally, the experts were instructed to make their referral decisions without regard for reimbursement or Medicare criteria such as being homebound, which may have increased their willingness to refer. This approach is supported in a report to Congress by MedPac (2005): “Providers should base their decision about where beneficiaries receive post-acute services on patient characteristics and resource needs, not on Medicare payments” (p. 105). These findings, if replicable and linked to improved outcomes, may have important policy implications.

Conclusions

This study suggests a preliminary model that can be used to identify hospitalized older adults in need of PAC services. It is currently being tested in three hospitals. From this data, the research team has developed models for the emergency department and with and without depression. By assuring the systematic collection of these six items, the multidisciplinary team is alerted to patients who may benefit from PAC services.

The experts identified many more patients for referral than were referred in practice. The experts had information from standardized, reliable research instruments; time; and ability to consider each other’s viewpoints while making their decisions in contrast to the hectic hospital environment. This study is a step in the direction toward better PAC referral decision making. Future studies will use a larger, more diverse sample to validate the model externally and increase the specificity; determine the impact on referral rates, workload, and outcomes; and examine factors related to site of care such as home care, nursing home, or rehabilitation.