Skip to main content
PMC home page
BMC Health Services Research logoLink to BMC Health Services Research
. 2011 Jul 5;11:160. doi: 10.1186/1472-6963-11-160

Reliability of INTERMED Spanish version and applicability in liver transplant patients: a cross-sectional study

Elena Lobo 1,2,✉, MªJosé Rabanaque 1, MªLuisa Bellido 3, Antonio Lobo 2,3,4
PMCID: PMC3142204  PMID: 21729313

Abstract

Background

Integrated delivery methods in healthcare systems have been proposed to confront the increasing complexity in general health care. INTERMED is an empirically derived, observer-rated instrument to measure case complexity. It was intended as a visualized, action-oriented decision-support tool for the early assessment of bio-psychosocial health risks and health needs. This study aims to document the reliability and applicability of the Spanish version of INTERMED in liver transplant patients.

Methods

Cross-sectional study of patients who had been included in the liver transplant waiting list. Two nurses interviewed the patients with INTERMED, and scored the instrument blind to each other. Kappa and w-kappa, Spearman, Kendall and intraclass correlation coefficients, and Cronbach's alfa were calculated.

Results

No patient refused the interview. Satisfactory coefficients were documented in most INTERMED items. Kappa was = 0.858 for the categorization of patients as "complex", and 21 of them (48.8%) were classified in this category, and were considered to need integrated treatment.

Conclusions

The Spanish version of INTERMED is reliable. Its applicability in liver transplant patients adds to its generalizability.

Background

Complexity in general health care is increasingly prevalent because of the growing number of patients who have multi-morbid conditions, and the increased technical possibilities of medicine [1]. Furthermore, social problems [2], and particularly behavioural health disorders, which may affect 40% or more of medical inpatients, interfere with medical care adding up to this complexity [3]. Social problems and mental health disorders are usually managed independently from general somatic disorders, and this care fragmentation has been shown to result in medical and behavioural disorders more persistent with increased complications and greater disability, leading to higher total health care and disability costs [3].

Integrated delivery methods in healthcare systems have been proposed to confront the complex problems found in present day medicine [4]. In the context of theoretic approaches to case complexity in the medically ill [1], a new instrument was empirically derived, INTERMED [5], its conceptual model being based on observations that comprehensive understanding of patients' needs increases effectiveness of healthcare delivery [6], The concept measured is case complexity [7-9]. Specifically, INTERMED classifies information into four domains (biological, psychological, social and healthcare), and was intended to universally screen at admission to medical wards, for the early identification of patients with multiple care needs. The utility of this method has been shown in different cultures [10] and in different types of both ambulatory populations and in-patients [11,12], including elderly patients [13]. INTERMED provides a quick overview of the patient's vulnerabilities [14], so that an integrated treatment plan can be formulated to counteract fragmentation of medical care [3]. It also provides a model to communicate these potential problems to the medical staff and to facilitate an early referral [15].

The reliability of INTERMED, which may be applied by nurses, was reported in a trauma center [7] and then in a heterogeneous sample of patients with somatic conditions [16]. The validity of this instrument has been documented in patients admitted to internal medicine [9], and also in specific diagnostic groups including patients in dialysis [17], with low back pain [18], terminal cancer [19], diabetes [2], rheumatoid arthritis [20], multiple sclerosis [11] and pulmonary patients [21]. It has also been shown that compared with usual care, nurse-led interventions based on INTERMED scores resulted in improvements in quality of life at discharge in a sample of general medical patients [22].

However, the applicability of INTERMED in transplant patients has not been reported until now. The early evaluation of health risks may be essential for optimal care in such patients, since complexity is anticipated in view of common medical co-morbidities and complications of medical and surgical procedures, and the fact that they often require a life-long regimen of immunosuppressive medication and dependency on the transplant team [23]. The burden of psychosocial problems in transplant patients has been considered to be associated with potential morbidity and mortality [24], and the improvement of the accuracy and meaningfulness of assessment, by incorporating biological, psychological, and contextual factors that may contribute to negative adjustment has been emphasized [25].

A Spanish version of INTERMED has been developed [21]. However, the reliability of this version has not been adequately documented. This is the first report intended to document the applicability of INTERMED in transplant patients, and the first one to document the reliability of the Spanish version.

Methods

Setting, study design and sample

Patients who had been included in the liver transplant waiting list of the Hospital Clinico Universitario in Zaragoza, Spain were invited to participate by a research nurse.

The study participants were interviewed by this research nurse and a consultation-liaison nurse, alternating the position of interviewer and observer. Following the interview the patients were scored by both nurses, blind to each other and using the INTERMED anchor points. Once the scores were computed, a between-rater consensus score for each INTERMED item was decided, to be used in some statistical analysis.

Instrument

INTERMED is based on a review of the medical chart and a patient semi-structured interview, which is designed like a medical anamnesis but complemented by psychosocial and health care-related information. It classifies the data into the four domains related to biopsychosocial and health care aspects of disease, and each domain has five variables, related to "history," "current state," and "prognoses". The twenty resulting variables are rated 0 to 3 [15] according to a manual with clinical anchor points, the potential score ranging 0 to 60, obtained by adding-up the individual variables, indicating the patient's level of care needs [5,7].

The instrument was previously translated into Spanish by an experienced consultation-liaison psychiatrist along with the research nurse, and went through the forward and backward translation process. Then, in a pilot study among transplant patients in the same hospital, 19 patients were interviewed with INTERMED by two trained nurses in a procedure similar to the one used in this study. The method was considered to be feasible and acceptable to patients, physicians and nurses. Inter-rater agreement coefficients were in general satisfactory (kappa >0.6), but refining the training was recommended, as it has been done for this study [26].

For the present report patients were classified by INTERMED as "complex" and "non-complex" on the basis of the optimal cut-off score (20/21) for the need of integrated treatment documented in previous studies [27,21].

Statistical analysis

Inter-rater reliability was calculated for individual items in INTERMED, as well as for the categorization of patients as "complex" or "non-complex". Both, kappa and weighted-kappa (w-kappa) coefficients were used. To further explore reliability, both Spearman and intraclass correlation coefficients between the raters were calculated for the INTERMED total score and the four domain scores. Kendall's coefficient was calculated for the categorical variable, namely the distinction between "complex" and "non-complex" patients. The consensus score for each INTERMED item was used for calculating the internal consistency of the instrument, and Cronbach's alfa was applied for these calculations. As in the original report, we further studied the structure of INTERMED by calculating Spearman correlation coefficients among domain scores [27]. Data were analyzed using Epidat and SPSS14.0 software.

This study was approved by the Ethical Committee Research Comission of the Hospital Clinico Universitario.

Results

Forty three patients were assessed with INTERMED (mean age 52.4 ± 8.7; female sex 30%). No patient refused the interview, which was completed in an average of 20 minutes per patient.

Table 1 shows inter-rater reliability coefficients for each INTERMED item. Satisfactory agreement was observed in most items. Kappa values below 0.5 were only observed in psychological and social prognoses, and w-kappa values were better in almost all items. Satisfactory agreement was also observed in the categorization of patients as "complex" or "non-complex" in INTERMED (kappa = 0.858).

Table 1.

Inter-rater reliability coefficients, by INTERMED item

DOMAINS ITEMS Kappa Kappa Weighted*
Chronicity 1 1
Diagnostic dilemma 0.76 0.87
BIOLOGICAL Severity of symptoms 0.69 0.73
Diagnostic challenge 0.65 0.89
Prognosis - Complications life threat 0.86 0.87
Restrictions in coping 0.75 0.93
Psychiatric dysfunctioning 0.69 0.85
PSYCHOLOGICAL Resistance to treatment 0.61 0.69
Psychiatric symptoms 0.75 0.84
Prognosis - Mental health threat 0.42 0.31
Restrictions in integration 0.62 0.78
Social dysfunctioning 0.75 0.80
SOCIAL Residential instability 0.57 0.69
Restrictions of network 0.63 0.66
Prognosis - Social vulnerability 0.48 0.56
Intensity of treatment 0.71 0.69
Treatment experience 0.79 0.84
HEALTH CARE Organisation of care 0.83 0.91
Appropriateness of referral 1 1
Prognosis - Coordination 0.52 0.74
Open in a new tab

*Kappa weighted with quadratic wheights.

Table 2 shows the mean scores by domain and mean total scores, which were similar in both raters. Means of the consensus scores were: biological = 7.64 (± 1.49); psychological = 4.63 (± 2.41); social = 1.95 (± 1.86); health care = 6.67 (± 1.86). Between-rater correlation coefficients for the INTERMED total score and for the four domain scores were also quite acceptable (table 2). For the distinction between "complex" and "non-complex" patients Kendall's coefficient was 0.867. Cronbach's alfa coefficient for INTERMED was 0.71. Between-domain correlations are shown in table 3. Positive correlations are observed in general, the exception being correlations with the social domain.

Table 2.

Mean scores by INTERMED domain, mean total scores, and between-rater correlation coefficients

Domain Rater 1 Rater 2 Spearman Intraclass
Mean Min Max SD Mean Min Max SD
BIOLOGICAL♦ 7.7 6 12 1.3 7.7 6 12 1.5 0.91* 0.92*
PSYCHOLOGICAL♦ 4.2 0 9 2.3 4.6 1 9 2.2 0.87* 0.87*
SOCIAL♦ 1.4 0 6 1.6 2.0 0 8 2.1 0.91* 0.76*
HEALTH CARE♦ 6.6 3 10 1.8 6.3 3 11 1.7 0.91* 0.89*
INTERMED Score♦♦ 19.9 13 30 4.4 20.5 12 32 4.8 0.92* 0.91*
Open in a new tab

♦Range 0-15; ♦♦Range 0-60; *p < 0.001

Table 3.

Spearman correlations among INTERMED domain scores

BIOLOGICAL PSYCHOLOGICAL SOCIAL
Spearman Spearman Spearman
PSYCHOLOGICAL 0.32* - -
SOCIAL 0.05 0.08 -
HEALTH CARE 0.28 0.38* 0.24
Open in a new tab

*p < 0.05

In relation to applicability of INTERMED in liver transplant patients, 21 of them (48.8%) scored above the cut-off score (20/21). The highest scores were observed in the biological and health care domains, but high scores were also given to the psychological domain (table 2). Five (11.6%) patients scored more than one standard deviation above the mean in the biological domain; eight (18.6%) patients in the psychopathological domain; ten (23.2%) patients in the social problems domain; and the same proportion in the healthcare domain.

Discussion

One objective of this paper was to assess the reliability of the Spanish version of INTERMED, and the overall results were satisfactory. The raters' agreement in the categorization of patients as "complex" or "non-complex" was high (k = 0.858). Similarly, inter-rater kappa (k) coefficients were also satisfactory for most INTERMED individual items. The measurement of agreement is difficult, particularly when the variables of interest have more than two possible values. However, there is widespread consensus regarding the advantages of the stringent kappa (k) coefficient, which takes into account chance agreement. The main weakness of k is that it measures the frequency of exact agreement, rather than the degree of approximate agreement. However, to take this into account we have also calculated weighted kappa in this particular study, the agreement coefficients being even better. Furthermore, the reliability of this Spanish version of INTERMED was also supported by satisfactory Spearman and intraclass correlation coefficients, when the measures were close to a continuous distribution, namely, INTERMED total scores and individual domain scores.

In one of the original studies, DeJonge et al reported Spearman coefficients ranking from 0.91 to 0.96 in assessing the different domains, and considered "very good" the agreement in the categorization of patients as "complex" or "non-complex, the only variable they assessed with kappa (k = 0.85) [16]. Therefore, this study on the Spanish version of INTERMED is in agreement with and supports the results reported with the original version.

The original authors have suggested that inexperienced raters may need more training. Consequently, the satisfactory results with the Spanish version of INTERMED are remarkable, since one of the raters was a research nurse without clinical experience. Moreover, the agreement of experienced and inexperienced raters supports the "procedural validity" of INTERMED. Procedural validity speaks only of the evaluation procedure and not to the validity of the construct itself, and is defined as the extent to which a new diagnostic procedure yields results similar to the results of an established diagnostic procedure that is used as a criterion [28]. In the absence of an established "gold standard," the experienced clinicians' judgement has been used as a comparative criterion when researchers with little clinical experience are trying to use a new diagnostic instrument [28,29]. It might be argued that further training and/or refinement of items related to psychosocial prognosis should be considered, since k values obtained were below 0.5. In fact, in the first reliability study by Huyse et al inconsistent results were reported for these items, the agreement coefficients being low in some (ICC were 0.49 and 0.84 for the psychological prognosis; and 0.80 and 0.42 for the social prognosis; Kendall's coefficients were 0.52 and 0.36 for the psychological prognosis and 0.25 and 0.70 for the social prognosis) [7]. However, that might be the closest one can come with a prognostic item, since prognosis in medicine is always difficult (with regard to any outcome) in view of the uncertain future.

Internal consistency of the Spanish INTERMED may also be considered to be acceptable (alfa = 0.71). Slightly higher coefficients have been reported with the original version of INTERMED (alfa >0.78) [27]. However, as suggested by De Jonge et al, values higher than 0.8 can not be demanded, not even desired, in view of the heterogeneity of domains included in INTERMED [16]. In support of this view, between-domain correlations were statistically significant, but modest (Sperman's around 0.3), and the correlations with the social domain were non-significant. In fact, common clinical sense does not require in liver transplant patients that, for example, the severity of the biological condition should have a proportional counterpart of social problems.

In relation to applicability, the second objective in this report, no patient refused the interview. It is noticeable that nurses, even with limited clinical experience in patient care were able to detect in a 20 minutes interview, early after inclusion of patients in the transplant waiting list, that almost half of them had high levels of potential complexity. "Complex" patients in INTERMED have been considered to need integrated treatment [30]. The needs of psycho-social attention in this particular sample are supported by the proportion of patients scoring above the 20/21 threshold point proposed (48.8%). Previous studies have also shown considerable proportions of complex patients according to the same INTERMED criteria: 62.3% among diabetic patients [2] and 12.8% or 21.4 (%) among multiple sclerosis patients [11]. Physicians are well prepared to detect the severity of the medical condition, but under-detection of psychopathological and social problems is common in medicine and has been associated with negative outcomes [3]. Specifically, psychopathological problems in liver transplant patients have been considered to be associated with poor prognosis [24]. Therefore, the early evaluation and then the appropriate management of complex transplant recipients might be essential for optimal patient care [23].

Fragmentation of care into biological, psychopathoplogical and social care may be even more detrimental in liver transplant patients than in other, less complex medical patients [3]. The early detection with an instrument such as INTERMED, intended to enhance the communication between patients and the health providers as well as between different professions and disciplines [31] should facilitate the early development of individualized, integral care planning. Coordination between professionals, which may also be facilitated by the use of INTERMED, has been suggested to affect patients' clinical outcomes and satisfaction with their care [32,33]. Efforts have been done to improve health service utilization and costs with new models of guided, comprehensive healthcare provided for people with complex health conditions [34,35]. There is some evidence of improved health outcomes following an intervention targeted for complex medical patients identified by INTERMED [36].

In support of the applicability of the use of INTERMED, this study was completed in a socio-cultural environment different from most previous studies. Patients in this sample were recruited in a rather typical public hospital in Spain, which covers a geographical health area and populations with a wide range of socio-economical background. Therefore, the results support its future use in Spanish speaking populations.

Conclusions

The Spanish version of INTERMED is reliable. Moreover, since the validity of this instrument was previously documented in different types of medical patients, this study adds to the applicability of INTERMED in special populations such as the liver transplant patients, with high levels of complexity as shown in this study. The usefulness of INTERMED in studies such as the ongoing multi-national, European study in liver transplant patients [37] is now further supported. However, its utility in improving health care delivery for complex liver transplant still needs to be evaluated.

Competing interests

The authors declare that they have no competing interests.

Authors' contributions

EL participated in the design, carried out the study, interviewed the patients, prepared the dataset, analysed and prepared all the results and drafted the manuscript. MJR participated in the design of the study and revised critically the manuscript. MLB participated in the design of the study, carried out interviews with patients and helped draft the manuscript. AL conceived the study and designed it, helped to draft the manuscript and made important final modifications. All authors read and approved the final manuscript.

Pre-publication history

The pre-publication history for this paper can be accessed here:

http://www.biomedcentral.com/1472-6963/11/160/prepub

Contributor Information

Elena Lobo, Email: elobo@unizar.es.

MªJosé Rabanaque, Email: rabanake@unizar.es.

MªLuisa Bellido, Email: mlbellido@hotmail.com.

Antonio Lobo, Email: alobo@unizar.es.

Acknowledgements

Funding was provided by the Spanish Ministries of Health and of Science and Innovation.

References

  1. de Jonge P, Huyse FJ, Stiefel FC. Case and care complexity in the medically ill. Med Clin North Am. 2006;90(4):679–92. doi: 10.1016/j.mcna.2006.04.005. [DOI] [PubMed] [Google Scholar]
  2. Fischer CJ, Stiefel FC, De Jonge P, Guex P, Troendle A, Bulliard C, Huyse FJ, Gaillard R, Ruiz J. Case complexity and clinical outcome in diabetes mellitus. A prospective study using the INTERMED. Diabetes Metab. 2000;26(4):295–302. [PubMed] [Google Scholar]
  3. Kathol R, Saravay SM, Lobo A, Ormel J. Epidemiologic trends and costs of fragmentation. Med Clin North Am. 2006;90(4):549–72. doi: 10.1016/j.mcna.2006.04.002. [DOI] [PubMed] [Google Scholar]
  4. Enthoven AC. Integrated delivery systems: the cure for fragmentation. Am J Manag Care. 2009;15(10 Suppl):S284–90. [PubMed] [Google Scholar]
  5. Huyse FJ, Herzog T, Lobo A, Lyons JS, Slaets JPJ, Fink P, Stiefel F, De Jonge P. Detection and treatment of mental disorders in general health care. Eur Psychiatry. 1997;12(2):70–8. [Google Scholar]
  6. Silverstone PH. Prevalence of psychiatric disorders in medical inpatients. J Nerv Ment Dis. 1996;184(1):43–51. doi: 10.1097/00005053-199601000-00008. [DOI] [PubMed] [Google Scholar]
  7. Huyse FJ, Lyons JS, Stiefel FC, Slaets JP, de Jonge P, Fink P, Gans RO, Guex P, Herzog T, Lobo A, Smith GC, van Schijndel RS. "INTERMED": a method to assess health service needs. I. Development and reliability. Gen Hosp Psychiatry. 1999;21(1):39–48. doi: 10.1016/S0163-8343(98)00057-7. [DOI] [PubMed] [Google Scholar]
  8. Stiefel FC, de Jonge P, Huyse FJ, Guex P, Slaets JP, Lyons JS, Spagnoli J, Vannotti M. "INTERMED": a method to assess health service needs. II. Results on its validity and clinical use. Gen Hosp Psychiatry. 1999;21(1):49–56. doi: 10.1016/S0163-8343(98)00061-9. [DOI] [PubMed] [Google Scholar]
  9. De Jonge P, Huyse FJ, Stiefel FC, Slaets JP, Gans RO. INTERMED--a clinical instrument for biopsychosocial assessment. Psychosomatics. 2001;42(2):106–9. doi: 10.1176/appi.psy.42.2.106. [DOI] [PubMed] [Google Scholar]
  10. Kishi Y, Meller WH, Kathol RG, Swigart SE. Factors affecting the relationship between the timing of psychiatric consultation and general hospital length of stay. Psychosomatics. 2004;45(6):470–6. doi: 10.1176/appi.psy.45.6.470. [DOI] [PubMed] [Google Scholar]
  11. De Jonge P, Hoogervorst EL, Huyse FJ, Polman CH. INTERMED: a measure of biopsychosocial case complexity: one year stability in Multiple Sclerosis patients. Gen Hosp Psychiatry. 2004;26(2):147–52. doi: 10.1016/j.genhosppsych.2003.09.001. [DOI] [PubMed] [Google Scholar]
  12. Huyse FJ, Lyons JS, Stiefel FC, Slaets JP, De Jonge P, Fink P, Gans RO, Guex P, Herzog T, Lobo A, Smith GC, van Schijndel RS. "INTERMED": a method to assess health service needs. I. Development and reliability. Gen Hosp Psychiatry. 1999;21(1):39–48. doi: 10.1016/S0163-8343(98)00057-7. [DOI] [PubMed] [Google Scholar]
  13. Wild B, Lechner S, Herzog W, Maatouk I, Wesche D, Raum E, Muller H, Brenner H, Slaets J, Huyse F, Sollner W. Reliable integrative assessment of health care needs in elderly persons: the INTERMED for the Elderly (IM-E) J Psychosom Res. 2011;70(2):169–78. doi: 10.1016/j.jpsychores.2010.09.003. [DOI] [PubMed] [Google Scholar]
  14. De Jonge P, Bauer I, Huyse FJ, Latour CH. Medical inpatients at risk of extended hospital stay and poor discharge health status: detection with COMPRI and INTERMED. Psychosom Med. 2003;65(4):534–41. doi: 10.1097/01.PSY.0000077504.01963.1B. [DOI] [PubMed] [Google Scholar]
  15. Huyse FJ, Lyons JS, Stiefel F, Slaets J, De Jonge P, Latour C. Operationalizing the biopsychosocial model: the intermed. Psychosomatics. 2001;42(1):5–13. doi: 10.1176/appi.psy.42.1.5. [DOI] [PubMed] [Google Scholar]
  16. De Jonge P, Latour C, Huyse FJ. Interrater reliability of the INTERMED in a heterogeneous somatic population. J Psychosom Res. 2002;52(1):25–7. doi: 10.1016/S0022-3999(01)00309-9. [DOI] [PubMed] [Google Scholar]
  17. De Jonge P, Ruinemans GM, Huyse FJ, ter Wee PM. A simple risk score predicts poor quality of life and non-survival at 1 year follow-up in dialysis patients. Nephrol Dial Transplant. 2003;18(12):2622–8. doi: 10.1093/ndt/gfg453. [DOI] [PubMed] [Google Scholar]
  18. Stiefel FC, De Jonge P, Huyse FJ, Slaets JP, Guex P, Lyons JS, Vannotti M, Fritsch C, Moeri R, Leyvraz PF, So A, Spagnoli J. INTERMED--an assessment and classification system for case complexity. Results in patients with low back pain. Spine (Phila Pa 1976) 1999;24(4):378–84. doi: 10.1097/00007632-199902150-00017. discussion 385. [DOI] [PubMed] [Google Scholar]
  19. Mazzocato C, Stiefel F, De Jonge P, Levorato A, Ducret S, Huyse FJ. Comprehensive assessment of patients in palliative care: a descriptive study utilizing the INTERMED. J Pain Symptom Manage. 2000;19(2):83–90. doi: 10.1016/S0885-3924(99)00156-6. [DOI] [PubMed] [Google Scholar]
  20. Koch N, Stiefel F, De Jonge P, Fransen J, Chamot AM, Gerster JC, Huyse F, So AK. Identification of case complexity and increased health care utilization in patients with rheumatoid arthritis. Arthritis Rheum. 2001;45(3):216–21. doi: 10.1002/1529-0131(200106)45:3&#x0003c;216::AID-ART251&#x0003e;3.0.CO;2-F. [DOI] [PubMed] [Google Scholar]
  21. Lobo E, De Jonge P, Huyse FJ, Slaets JP, Rabanaque MJ, Lobo A. Early detection of pneumology inpatients at risk of extended hospital stay and need for psychosocial treatment. Psychosom Med. 2007;69(1):99–105. doi: 10.1097/PSY.0b013e31802e46da. [DOI] [PubMed] [Google Scholar]
  22. De Jonge P, Latour CH, Huyse FJ. Implementing psychiatric interventions on a medical ward: effects on patients' quality of life and length of hospital stay. Psychosom Med. 2003;65(6):997–1002. doi: 10.1097/01.PSY.0000097332.77685.C8. [DOI] [PubMed] [Google Scholar]
  23. Surman OS, Cosimi AB, DiMartini A. Psychiatric care of patients undergoing organ transplantation. Transplantation. 2009;87(12):1753–61. doi: 10.1097/TP.0b013e3181a754d4. [DOI] [PubMed] [Google Scholar]
  24. Heinrich TW, Marcangelo M. Psychiatric issues in solid organ transplantation. Harv Rev Psychiatry. 2009;17(6):398–406. doi: 10.3109/10673220903463259. [DOI] [PubMed] [Google Scholar]
  25. Mosher CE, Redd WH, Rini CM, Burkhalter JE, DuHamel KN. Physical, psychological, and social sequelae following hematopoietic stem cell transplantation: a review of the literature. Psychooncology. 2009;18(2):113–27. doi: 10.1002/pon.1399. [DOI] [PMC free article] [PubMed] [Google Scholar]
  26. Lobo E, Bellido M, Campos R, Saz P, Huyse FJ, De Jonge P, Lobo A. Primera validación en español del método INTERMED: Un sistema de temprana detección de problemas biopsicosociales y de consumo de servicios en pacientes médico-quirúrgicos. Cuadernos De Medicina Psicosomática y Psiquiatría De Enlace. 2003;67/68:89–98. [Google Scholar]
  27. De Jonge P, Stiefel F. Internal consistency of the INTERMED in patients with somatic diseases. J Psychosom Res. 2003;54(5):497–9. doi: 10.1016/S0022-3999(02)00441-5. [DOI] [PubMed] [Google Scholar]
  28. Spitzer RL, Williams JBW. In: Comprehensive Textbook of Psychiatry. 4. Kaplan HI, editor. Sadock BJ. Baltimore, MD: Williams & Wilkins; 1985. Classification of Mental Disorders and DSM-III; pp. 591–613. [Google Scholar]
  29. Lobo A, Campos R, Perez-Echeverria MJ, Izuzquiza J, Garcia-Campayo J, Saz P, Marcos G. A new interview for the multiaxial assessment of psychiatric morbidity in medical settings. Psychol Med. 1993;23(2):505–10. doi: 10.1017/S0033291700028592. [DOI] [PubMed] [Google Scholar]
  30. Huyse FJ, Stiefel FC, De Jonge P. Identifiers, or "red flags," of complexity and need for integrated care. Med Clin North Am. 2006;90(4):703–12. doi: 10.1016/j.mcna.2006.05.003. [DOI] [PubMed] [Google Scholar]
  31. Stiefel FC, Huyse FJ, Sollner W, Slaets JP, Lyons JS, Latour CH, van der Wal N, De Jonge P. Operationalizing integrated care on a clinical level: the INTERMED project. Med Clin North Am. 2006;90(4):713–58. doi: 10.1016/j.mcna.2006.05.006. [DOI] [PubMed] [Google Scholar]
  32. Weinberg DB, Gittell JH, Lusenhop RW, Kautz CM, Wright J. Beyond our walls: impact of patient and provider coordination across the continuum on outcomes for surgical patients. Health Serv Res. 2007;42(1 Pt 1):7–24. doi: 10.1111/j.1475-6773.2006.00653.x. [DOI] [PMC free article] [PubMed] [Google Scholar]
  33. Wynia K, Annema C, Nissen H, De Keyser J, Middel B. Design of a Randomised Controlled Trial (RCT) on the effectiveness of a Dutch patient advocacy case management intervention among severely disabled Multiple Sclerosis patients. BMC Health Serv Res. 2010;10:142. doi: 10.1186/1472-6963-10-142. [DOI] [PMC free article] [PubMed] [Google Scholar]
  34. Leff B, Reider L, Frick KD, Scharfstein DO, Boyd CM, Frey K, Karm L, Boult C. Guided care and the cost of complex healthcare: a preliminary report. Am J Manag Care. 2009;15(8):555–9. [PubMed] [Google Scholar]
  35. Maiers MJ, Westrom KK, Legendre CG, Bronfort G. Integrative care for the management of low back pain: use of a clinical care pathway. BMC Health Serv Res. 2010;10:298. doi: 10.1186/1472-6963-10-298. [DOI] [PMC free article] [PubMed] [Google Scholar]
  36. Stiefel F, Zdrojewski C, Bel Hadj F, Boffa D, Dorogi Y, So A, Ruiz J, De Jonge P. Effects of a multifaceted psychiatric intervention targeted for the complex medically ill: a randomized controlled trial. Psychother Psychosom. 2008;77(4):247–56. doi: 10.1159/000129658. [DOI] [PubMed] [Google Scholar]
  37. INTERMED Foundation. Literature. http://www.intermedfoundation.org/homepage

Articles from BMC Health Services Research are provided here courtesy of BMC

RESOURCES