No service is an island: towards an ecosystem approach to mental health service evaluation

Johnson et al ¹ provide an overview of the huge transformation occurring in a­­cute mental health care during the last two decades. The authors enumerate and discuss an extensive array of novel alternatives, while underscoring the lack of ro­­bust evidence to support their implementation. We provide here some complementary in­­formation to further understand the context of this reform and the current challenges related to its evaluation.

The accelerated reform of acute mental health care should be framed within the broader shift from hospital to communi­ty care occurring in the health sector as a whole. The development of vanguard services include enhanced health care at home, multispecialty community providers, integrated primary and acute care systems, and blended systems encompassing real world and digital health care ² . The combined effect of these innovations is inexorably displacing care from hospitals to community in general health care and not only in the mental health field.

Awareness is increasing that acute health care improvement cannot be attained without adopting a whole system approach to the design, implementation and evaluation of new models of care. A health care ecosystem includes four main domains: the places and communities in which we live; the wider social and demographic characteristics; health lifestyles; and the health care provision at the different levels of the ecosystem: nano (patient‐professional level), micro (service level), meso (local area level) and macro (region/country level) ³ . This whole system perspective is particularly relevant in the mental health field.

Johnson et al’s paper describes how integrated community care models, including acute care, started in the mental health field decades before being adopted by gen­eral health care. Note that most general Hospital in the Home research was preceded by several generations of randomized controlled trials of integrated home‐delivered mental health care ⁴ . Breakthrough innovations in mental health included the first integrated models of care such as the community/hospital care systems ⁵ , and the “balance of care” across hospital and community, and across different sectors (health and social care) ³ .

The mental health field also contributed the first ecological model for the assessment of the production of care (the Care Matrix ³ ), the first integrated standards defining all sites of acute mental health care (Area Integrated Mental Health Service Stan­­dards ‐ AIMHS ³ ), and the instruments for assessing mental health care in catchment areas developed by the European Psy­chi­atric Care Assessment Team (EPCAT) in 2000 ⁶ .

However, the pioneering contribution made by the mental health field may drop behind advances in other areas of medicine due to a restrictive focus on acute care and the methodological challenges of its evaluation in real world conditions. Acute mental health services are typical­ly analyzed in isolation, disregarding a whole system’s perspective. For example, demands for more emergency rooms and hospital beds in Australia are made without even considering a system perspective to mental health crises ⁴ . We need to emphasize that continuity of care (e.g., in continuing day centres, rehabilitation programmes, assertive community treatment teams, community respite and supported accommodation, often with their own internal crisis response capabilities) may prevent relapses, provide early intervention, and avoid need for acute care.

The lack of current evidence on new ser­vices and interventions in acute mental health care is attributed to the practical and ethical challenges in recruiting participants experiencing a crisis, but it is not only this. The evidence‐based medicine approach may not suffice to generate evidence on the efficiency of new models of acute care. These complex systems are non­linear, and operate under conditions of uncertainty. Therefore, realistic priority‐setting requires the incorporation of systems thinking, standard classification of services, new data analytics techniques, modelling tools, and decision‐support systems that incorporate domain expertise ³ .

Terminological ambiguity and lack of com­parability are key problems in mental health service research. As first reported by Leginski et al ⁵ and widely corroborated by our service mapping research ⁶ , the nominal definition of a service does not correspond to its function. For example, the variation in target response times of crisis resolution and home treatment teams (CRHTTs) described in England and in Norway ¹  may indicate that very different services are grouped under this heading.

“Service” is an umbrella term and not an operational unit of analysis. The European Service Mapping Schedule (ESMS) and its extension beyond mental health, called the European Description and Evaluation of Ser­­vices and DirectoriEs (DESDE), have been extensively used for mapping services across health conditions (mental health, chronic care, disability, ageing) and care sectors (health, social, employment, education) in over 34 countries ⁶ .

The disambiguation process facilitated by ESMS/DESDE is not limited to service types. It provides an operational definition of acute care: assessment and initial treatment in response to a crisis – deterioration in physical or mental state, behaviour or social functioning – which is related to a health condition, that can usually be provided on the same day or at least within 72 hours after the care demand. Standard definitions of related services and acute care categories such as crisis, emergency, disaster and catastrophe are also needed as part of a common terminology in this field ⁷ .

The comparable description of services in catchment areas is critical to establish the local availability of services, their capacity (e.g., in individual “places” or in bed occupancy) and workforce provided. Once collected, this information can be used to assess the evolution of a care system, for gap and equality analysis, quality assessment, and modelling the effect of the implementation of new services or the needs of staff. Thereby, mapping of a care system has been used to estimate the optimal workforce in full time equivalents in acute wards and acute day care in the Basque Country (Spain), and the relative technical efficiency of service provision in catchment areas, including both acute and non‐acute services ⁶ .

Impact analysis is another key compo­nent of the evaluation in mental health care. This should not be limited to end‐point re­sults on individuals. Major attention should be paid to the process of implementation and the analysis of the readiness, usability, adoption and penetration of a new service in real world environments ⁸ . The emphasis on fidelity should be balanced with the need for adaptation to local con­texts ⁹ .

Additional mention should be made of the role of international networks in promoting new models of care and implementation. Relevant examples are the Crisis Now/Recovery International globally growing network of facilities, which provides welcoming, peer‐partnership and firmly community‐based service facilities, not backed as yet by published rigorous research; the I‐CIRCLE consortium, that promotes community models in urban environments; and the EUCOM model of community care in Eu­­rope.

The broader bio‐psycho‐socio‐cultural innovations have evolved with an emphasis on complexity science, co‐design with lived experience and family expertise, human rights facilitation and community‐based recovery approaches. Attempts to fragment and undo cost‐effective community‐based reforms are often accompanied by demands for ever‐more hospital beds ⁴ . These hospital‐centric views should no long­­er prevail over responsive, wholistic ecosystems, integrating community and hospital components.

Transforming acute mental health care towards community models exceeds men­tal health systems, heralding broader reform of general acute health care and support systems towards community care. To keep on‐track with previous advances, the evaluation of the mental health sector acute care should adopt a health care ecosystem ­perspective, including systematic assessment of the service delivery systems, their impact on processes, outcomes, workforce, and especially service users and families, val­­orizing lived experiences.