Initial steps in addressing the challenges of elder mistreatment evaluation: Protocol for evaluating the Vulnerable Elder Protection Team

Abstract

Introduction

Although many programmes have been developed to address elder mistreatment, high-quality, rigorous evaluations to assess their impact are lacking. This is partly due to challenges in conducting programme evaluation for such a complex phenomenon. We describe here the development of a protocol to mitigate these challenges and rigorously evaluate a first-of-its-kind emergency department/hospital-based elder mistreatment intervention, the Vulnerable Elder Protection Team (VEPT).

Methods and analysis

We used a multistep process to develop an evaluation protocol for VEPT: (1) creation of a logic model to describe programme activities and relevant short-term and long-term outcomes, (2) operationalisation of these outcome measures, (3) development of a combined outcome and (4) design of a protocol using telephone follow-up at multiple time points to obtain information about older adults served by VEPT. This protocol, which is informing an ongoing evaluation of VEPT, may help researchers and health system leaders design evaluations for similar elder mistreatment programmes.

Ethics and dissemination

This project has been reviewed and approved by the Weill Cornell Medicine Institutional Review Board, protocol #20-02021422. We aim to disseminate our results in peer-reviewed journals at national and international conferences and among interested patient groups and the public.

STRENGTHS AND LIMITATIONS OF THIS STUDY.

• We have developed a protocol to rigorously evaluate an intervention to address elder mistreatment, mitigating the challenges associated with programme evaluation for this complex phenomenon.

• We used a multistep process to develop an evaluation protocol for an emergency department/hospital-based interdisciplinary elder mistreatment response team.

• This protocol may serve as a model to help researchers and health system leaders design evaluations for similar elder mistreatment and other complex issues.

• The programme evaluation design we describe will not be able to fully attribute impact on outcomes to the intervention because of the absence of a comparison group.

Introduction

Importance of programme evaluation

Conducting systematic, comprehensive, rigorous programme evaluation is critical to assessing a programme’s impact on stakeholders and effectiveness in achieving its aims. This evaluation may also identify a programme’s unintended negative consequences. The results of a programme evaluation are used to inform creation, modification or elimination of programme components. Programme evaluation can demonstrate a programme’s value and cost-effectiveness, which is often necessary for securing future support for a programme. It is also helpful for dissemination, offering other communities valuable information about the potential impact of launching the programme as well as resources needed for implementation. Programme evaluation is often a requirement for continued funding for programmes supported by grants.¹

Evaluation is particularly important for programmes aiming to address a complex multidimensional phenomenon such as elder mistreatment (EM). The Centers for Disease Control and Prevention defines EM as ‘an intentional act or failure to act by a caregiver or another person in a relationship involving an expectation of trust that causes or creates a risk of harm to an older adult’.² Different types of EM include: physical abuse, sexual abuse, neglect, psychological abuse or financial exploitation.³ Notably, despite this definition, even measuring whether mistreatment has occurred (eg, differentiating between substandard care of an older adult and caregiver neglect) can be challenging, as can measuring the severity of ongoing mistreatment.⁴

Many additional complex issues must be considered when determining how to assess the impact of an EM intervention programme. For example, the decision of an older adult with decision-making capacity to remain in an unsafe situation must be respected, the safety of an older adult must be balanced with their quality of life, and the goal is often to reduce harm, minimising rather than eliminating mistreatment. Different types and circumstances of EM may require different approaches to intervention with varied expected outcomes and measurement strategies. Further, preliminary research examining some EM programmes found that intervention and reporting may actually increase future mistreatment risk,⁵ raising questions about the unintended impact of interventions.

Challenges in evaluating programmes developed to address EM

Despite the importance of evaluating programmes developed to address EM and the large number of programmes that have been launched, very few high-quality, rigorous evaluations have been published or conducted. In fact, a comprehensive, systematic literature review of EM intervention programmes in 2019 that focused on integration of hospitals found that 115 programmes had been described, but only 2% had an impact evaluation using a high-quality study design.⁶

The absence of rigorous programme evaluation is due in part to the significant challenges in designing and conducting evaluations of EM programmes, which are summarised in figure 1. These include challenges in: (1) identifying appropriate outcomes to examine and (2) operationalising and measuring these outcomes.

Figure 1.

Challenges in designing and conducting evaluations of elder mistreatment (EM) programmes.

Challenges identifying appropriate outcomes to examine

Identifying appropriate outcomes to examine a programme’s impact is challenging, given the complexities of each individual case and the differences in goals of older adults experiencing EM as well as the goals of other stakeholders. Many categories of outcomes are possible, including safety, health related, functional, psychosocial and legal.⁷

Key goals for an older adult experiencing EM may be in conflict with each other. For many older adults, reducing or eliminating contact with the perpetrator may be necessary to ensure safety. Often, however, the older adult wishes to continue contact with the perpetrator, who may be a close family member. As a result, harm reduction, which aims to reduce mistreatment and optimise safety while maximising an older adult’s quality of life within the context of their goals, is often the focus of interventions. While an older adult experiencing EM is the primary stakeholder for most EM programmes, other relevant stakeholders also exist. These include family, concerned non-family members, healthcare providers, community-based service providers, adult protective service (APS), law enforcement, the community and even the perpetrator.⁸ An ideal programme evaluation should consider impact on all stakeholders, but conflict may exist between the goals of these different stakeholders. For example, law enforcement and APS may want the perpetrator to be arrested and charged as a criminal, while an older adult and the family may be strongly against this.

The ability of a programme to intervene and the scope of interventions offered for older adults without cognitive impairment may be limited by the wishes of the older adult, which may change over time. In addition, if cognitive impairment occurs and progresses to a stage where an older adult can no longer make decisions, approaches to management of ongoing EM and impact substantially change. Notably, determining whether an older adult has capacity to make specific decisions requires expertise and can often be challenging.

The older adult may be dependent in many ways on the perpetrator, including physically, financially and emotionally. The perpetrator may be providing essential personal care as well as assistance with activities of daily living and instrumental activities of daily living. Cultural norms, religious beliefs and fear of isolation may also contribute. Any intervention that removes the perpetrator or reduces their interactions with the older adult must also have an acceptable solution to replace the care and support they provide. Further, many older adults care deeply about the perpetrator and what happens to them due to the intervention.

Challenges operationalising and measuring these outcomes

Understanding the impact of a programme usually requires follow-up with older adults who have received the intervention, in person or via telephone or other technology. Dementia, which affects 10.8% of US adults aged ≥65 and 33.3% of those age ≥85,^{9 10} may preclude the older adult from accurately reporting. Further, though 61% of adults aged ≥65 have smartphones and 45% use social media,¹¹ some may be unfamiliar with, uncomfortable using or lack access to technology. Recontacting victims may jeopardise their safety and increase risk of harm in some cases, as occurs with other types of family violence and mistreatment including intimate partner violence.¹ Therefore, programme sponsors and staff may be reluctant to recontact victims, and this limits the information that may be obtained to evaluate a programme.

Highly relevant outcomes, such as changes in quality of life, may be difficult to accurately measure by self-report in older adults experiencing EM, particularly those with dementia. A baseline measurement is essential for comparison, and given that cognitive impairment is typically progressive, the ability during follow-ups of an older adult to engage and report as well as their interpretation of the questions may be affected by differences in the severity of their brain disease at different time points. Additionally, the ability to discern the specific impact of an EM intervention on desired outcomes independent of other factors that change over time may be challenging. Short-term outcomes, examining, for example, an older adult’s living situation 1 month after intervention are typically easier to measure than long-term outcomes, 1 year or more after intervention. Long-term outcomes are more meaningful, however, given that they suggest the impacts of a programme are durable over time. As a result, many evaluations examine ‘process’ outcomes, such as feasibility and acceptability of a programme or changes in attitudes of professionals serving older adults.⁶ Although informative, measuring these process outcomes does not adequately demonstrate the actual impact of the programme on older adults and other stakeholders.

We describe here the development of a protocol to mitigate these challenges and rigorously evaluate a first-of-its-kind emergency department (ED)/hospital-based EM response team, the Vulnerable Elder Protection Team (VEPT).

Methods and analysis

VEPT programme overview

VEPT is an interdisciplinary ED/hospital-based consultation service available 24/7 to assess, treat, and ensure the safety of older adults experiencing EM while also collecting forensic evidence when appropriate and working closely with the authorities. The design and development of the programme have been described in detail elsewhere.^{7 12} VEPT was launched on 3 April 2017 after a multistep design process and accompanied by training for ED and hospital providers from multiple disciplines (eg, physicians, nurses, social workers).

VEPT is activated via pager whenever an ED/hospital provider has concern that a patient may be a victim of EM. The VEPT provider on-call, who is a physician or advanced practice provider with EM and geriatrics expertise, elicits information about the victim and the case dynamics and typically comes into the ED/hospital to perform a face-to-face assessment. When appropriate, this investigation may include a forensic evaluation including for sexual assault/abuse with comprehensive documentation and photographs.

The VEPT provider assesses the likelihood of victimisation (high/moderate/low) for each type of EM (physical abuse, sexual abuse, neglect, verbal/emotional/psychological abuse and financial exploitation). Among patients with moderate or high likelihood of victimisation, the providers also assess whether the victim is in immediate danger, which impacts the management approach. The VEPT provider advises the primary ED/hospital medical team about next steps to provide appropriate care from the EM perspective and supports the victim during the process.

VEPT also includes social workers who have specialised expertise in EM. These VEPT social workers (1) provide supportive counselling to victims; (2) obtain collateral from and work with family members, caregivers, primary care physicians and other concerned persons; (3) involve law enforcement and APS as appropriate; (4) coordinate with the primary medical team and hospital resources, including psychiatry, patient services, ethics and security; (5) identify which community-based services may be appropriate to offer the victim and (6) collaborate with community-based agencies.

The VEPT social worker follows a case longitudinally throughout an older adult victim’s hospitalisation and continues to provide supportive counselling to the victim, offer services and referrals, involve community resources and assist the medical team in case management and designing a safe discharge plan. The team also includes a VEPT inpatient medical specialist, a geriatrician with EM expertise, who collaborates closely with the VEPT social eorker. The VEPT inpatient medical specialist monitors and periodically reassesses all hospitalised EM victims and provides any necessary follow-up mistreatment-related guidance from a medical/forensic perspective. Online supplemental figure 1 is a flow chart describing the intervention.

Summary of multistep process to develop evaluation protocol

To evaluate VEPT, we used a multistep process. We: created a logic model to optimally describe the programme and relevant outcomes (step 1), operationalised these outcome measures (step 2), created a combined outcome to assess how many older adults were impacted in any important way from the programme (step 3) and developed a protocol for telephone follow-up to obtain information about cases at multiple time points (step 4). This approach was designed to attempt to address the two major categories of programme evaluation challenges described above: identifying appropriate outcomes to measure (step 1) and operationalising and measuring these outcomes (steps 2–4). This connection is depicted visually in online supplemental figure 2. For each of these steps, to develop this optimised evaluation approach, we initially conducted multiple meetings involving VEPT clinical and research team members from multiple disciplines (emergency physicians, advanced practice providers, geriatricians, social workers, an epidemiologist and a programme manager). Additionally, we involved five national EM experts from other institutions, conducting two in-person meetings for their perspective and incorporating their feedback. Ultimately, all participants agreed about the final evaluation approach developed and reported on here.

Creation of a logic model

Developing a logic model is a helpful preliminary step in programme evaluation. This widely used tool^13–15 visually summarises and describes the main programme activities and outcomes. We have expanded on this by creating a formal logic model focusing on activities (assessments, interventions and referrals) that the VEPT programme provides for older adults with moderate or high suspicion for EM after initial evaluation. The logic model (figure 2) includes inputs and activities as well as outputs (direct products of a programme’s activities, including types, levels and targets of services delivered by the programme), and outcomes (expected changes in the older adults served and other stakeholders that result from the programme’s activities).¹⁶ As part of creating this model, we included a comprehensive list of all of the activities that the VEPT programme may provide for older adults (online supplemental figure 3). We believe this list of activities may be useful to others seeking to replicate the model. In the programme evaluation, we plan to compare recommendations made vs carried out as important process outcomes.

Figure 2.

VEPT logic model. ED, emergency department; VEPT, Vulnerable Elder Protection Team. *complete, detailed list of Activities (assessments, interventions, referrals) included as Online Supplemental Material

As part of the logic model, we identified five critical outcomes to operationalise and measure: (1) continued safety/freedom from (or reduction of) mistreatment by identified perpetrator, (2) safety/freedom from mistreatment by other perpetrator(s), (3) reduction in mistreatment-related ED visits/hospitalisations, (4) improvement in quality of life and (5) long-term permanent living situation optimised to minimise potential for EM.

Operationalisation of outcome measures

Our team operationalised 4 of the 5 long-term outcomes into 20 measurable items. The measurable items related to each of the long-term outcomes are shown in table 1. Notably, these 20 measurable outcomes include ones focused on the domains of safety, medical and legal. For safety, we measure the status of contact with abuser, living situation and establishment of home healthcare services. For medical, we measure whether the patient has returned to ED/hospital for mistreatment-related issue or any issue. For legal, we measure the current status of any APS case as well as whether guardianship has been initiated or obtained.

Table 1.

Key long-term outcomes for VEPT programme evaluation operationalised into measurable items

Measurable item (as of follow-up)	Long-term outcomes
	Continued safety, freedom from mistreatment by identified perpetrator	Safety, freedom from mistreatment by new/other perpetrator(s)	Reduction in mistreatment-related ED visits/ hospitalisations	Long-term/permanent living situation optimised to minimise potential for elder mistreatment
				No longer living with identified perpetrator/eviction of perpetrator	Increased home health assistance	Increased connection to community-based services	Decreased isolation
No contact with abuser	✓			✓
Reduced contact with abuser	✓			✓
Abuse no longer occurring	✓
Abuse potentially still occurring but reduced	✓
New living situation	✓
Permanent new living situation	✓	✓		✓			✓
Temporary new living situation	✓	✓		✓			✓
New home healthcare services	✓	✓			✓		✓
Additional home healthcare services	✓	✓			✓		✓
Change in existing home healthcare services	✓	✓
Guardianship initiated/obtained after hospitalisation	✓	✓
APS opened case and remains open	✓	✓
APS opened case and closed case	✓	✓
APS evaluated and found older adult ineligible for services	✓	✓
Return to ED/hospital for mistreatment-related issue			✓
Return to ED/hospital for any issue			✓
% with additional VEPT consultation when returned to ED/hospital			✓
At least one resource recommended by VEPT used						✓

ED, emergency department; VEPT, Vulnerable Elder Protection Team.

We recognise that many of the measurable items imperfectly measure the long-term outcomes themselves. For example, ‘new living situation’ is not necessarily ‘long-term/permanent living situation optimised to minimise potential for EM’. It is possible that a new living situation may not represent an improvement in safety and/or may represent a worsening in other domains. We have chosen this rather than attempting to directly measure ‘optimisation’ of living situation because of the difficulty in reliably doing so due to variation in this definition based on individual patient preferences and the relative nature of the concept of optimisation, the definition of which may shift as contextual circumstances change. How would it be determined whether an older adult’s living situation was optimised? For example, if they currently had 4 hours daily of assistance from a home health aide, would increasing it to 8 hours daily represent optimisation? Would increasing it to 24/7 coverage represent optimisation but increasing to 8 hours daily would not? What would happen if the older adult’s son then fired the home health aides or refused to let them into the home—would the living situation no longer be optimised? For cases where the older adult is unsafe in the home because of chronic physical abuse by an adult daughter, is moving them to a skilled nursing facility optimising their living situation? Is moving them (temporarily) into the home of another relative optimising? As these questions are very challenging to answer Given the complexity of defining, let alone measuring, living situation optimisation(and that experienced EM professionals working together on the same case may have very different opinions about living situation optimisation), we think it is more appropriate to use ‘new living arrangement’ with a clear, operationalisable definition as a measurable outcome demonstrating impact and change, despite its limitations. As with the other measurable items, the change that ‘new living arrangement’ represents is seldom universally positive in all domains. In some cases, it may represent improvement in some domains (such as safety) but worsening in others (connection to family and community). In others, despite the best intentions of the programme team and other professionals, it may represent worsening in most domains. Additionally, whether a change in living situation represents an improvement or worsening for the older adult in various domains may change over time as circumstances change. Despite this, we believe that it is useful to measure. A universally acknowledged challenge for most community-based EM programmes is their inability to effect substantial changes for an older adult, such as a change in living situation, even when these changes are essential and emergent. Therefore, the association of VEPT programme involvement with substantial change is important to assess and describe.

Notably, we recognised that change in quality of life is an important outcome but acknowledged that accurately measuring it is challenging, particularly without a baseline measurement for comparison obtained before VEPT intervention. Therefore, quality of life measures were not included in the initial evaluation strategy. We describe a planned exploratory analysis below. Also, given the large differences between cases, we have planned an exploratory analysis to assess individualised goals and VEPT’s ability to achieve them.

Creating a combined outcome to assess fraction of older adults impacted in any important way by the programme

Cases of EM managed by VEPT are varied and complex, and goals for each older adult and for other stakeholders often differ. Also, reporting on several individual outcomes may not fully characterise the impact of a programme. It is possible that the percentage with each positive outcome either represents the same or different older adults. For example, if, after 1 year, 40% of older adults had no contact with perpetrator or reduced contact with the perpetrator and 40% had new or additional/changed home services, it is possible that anywhere from 40%–80% of older adults had either. Assessing the amount of this ‘overlap’ is critical to accurately understand how many older adults were impacted in any meaningful way by VEPT activities. We, therefore, created a combined outcome to assess whether any substantial changes in the domain of safety occurred. In creating this combined outcome, we attempted to identify outcomes which, even in the absence of any others, would have alone indicated that VEPT had a substantial impact. We discussed and ultimately agreed on five elements to include, which are shown in box 1. We recognise that these outcomes may not always represent unequivocal improvements across all domains for the older adult or other stakeholders, but we believe that they represent meaningful change, and each is worthy of measurement. This is not an overall combined score but rather an approach allowing assessment of: (1) whether VEPT has had any impact on an individual case and (2) whether, when reporting the fraction of cases for which each outcome has occurred, the occurrence of various outcomes is for the same cases or for different cases.

Box 1. Items in Vulnerable Elder Protection Team combined outcome.

Item

Mistreatment no longer occurring or reduced.

No contact with perpetrator or reduced contact.

Permanent or temporary new living situation.

New or additional/changed home services.

Adult protective services has opened a case, remains open.

The combined outcome is considered to be positive if any of these items are positive. We believe that it is essential, when reporting this combined outcome, to also report results for each of the elements. This will allow readers to appreciate which outcomes are driving the results. Notably, examining the relationship between these outcomes as well as, particularly, between stopping or reducing the abuse and each of the other outcomes, is an opportunity for future research that may be highly valuable to the field.

Protocol for telephone follow-ups

We recognised that active follow-up was necessary to obtain reliable data about these long-term outcomes. We developed a telephone protocol for a VEPT social worker to obtain these data. In this protocol, the VEPT social worker attempted to speak with patient, family, concerned community member(s), APS, community agencies and the nursing/rehab facility as appropriate for this update. This social worker has deep experience and expertise interviewing older adults to assess for whether mistreatment is occurring. They also have substantial knowledge of the case and the specific circumstances surrounding the older adult as well as their family and caregivers. Therefore, rather than requiring the use of a standard set of questions or assessment tool, they use their interview skills to conduct a conversation to establish whether specific types of mistreatment are occurring as well as other relevant information. Not using a standardised tool also allows for circumstances where information is obtained exclusively from sources other than the older adult. The VEPT social worker used a semistructured list of questions (online supplemental figure 4) we developed to guide the interview.

Within the protocol, the VEPT social worker introduces him/herself and explains that the reason for the call is to check-up on the older adult who was evaluated by the VEPT team previously when in the ED/hospital. Then, they attempt to ask questions, with appropriate follow-ups and probes based on the responses. Notably, asking questions using exactly the language within the semi-structured list may be awkward and problematic. These questions serve as placeholders to demonstrate the type of information the VEPT social worker obtains during follow-up calls using their interview skills. During the conversation, the social worker records the updated information so that it may be entered into the tracking database and the electronic health record (EHR). The VEPT social worker makes three attempts to reach the older adult and APS and two attempts to reach all other potential reporters before stopping. We have mapped the question responses to the measurable outcomes we plan to examine (table 2).

Table 2.

Using VEPT telephone follow-up protocol to operationalise outcome measurement

Question*	Measurable item
Question 1: Living Situation Narrative: Are there any changes to the home situation since discharge? (eg, Does the older adult still live in the home? Does the older adult have a home health aide? Does the perpetrator live in the home?)	No contact with perpetrator Reduced contact with perpetrator Mistreatment no longer occurring Mistreatment potentially still occurring but reduced Permanent new living situation Temporary new living situation New home healthcare services Additional home healthcare services Change in existing home healthcare services
Question 2: Perpetrator Contact: Does older adult have contact with perpetrator? Is mistreatment still occurring?	No contact with perpetrator Reduced contact with perpetrator Mistreatment no longer occurring Mistreatment potentially still occurring but reduced
Question 3: Has the older adult had contact with Adult Protective Services (APS)? When? What did APS do? (Document dates of visit/case closure if available)	Guardianship initiated/obtained after hospitalisation APS opened case and remains open APS opened case, patient refused services APS opened case and closed case APS evaluated and found older adult ineligible for services
Question 4: Did older adult make use of the resources VEPT provided?	At least one resource recommended by VEPT used

*Phrasing of questions depending on whether follow-up with conversation with older adult, family, concerned community member(s), APS, community agencies and the nursing/rehab facility.

APS, adult protective service; VEPT, Vulnerable Elder Protection Team.

Notably, we believe that whether the older adult made use of the resources VEPT provided (question 4) is a relevant outcome for the programme as well as a potential mediator of other outcomes. Examining it has the potential to add insight into the mechanism of VEPT’s impact. Also, examining concordance and discordance between the responses of older adults in follow-up and those of other reporters, particularly around whether mistreatment is no longer occurring, is an opportunity for future research that may improve understanding of the experiences of older adults.

We also added to the protocol a review of the EHR for any healthcare encounters since the initial discharge, including older adult patient has returned to the ED since discharge, how many times, whether they were admitted and whether VEPT reconsulted.

Follow-up time frames

As the impact of interventions may evolve over time, we have conducted multiple follow-ups for each case. Though we recognise that longer-term outcomes are often difficult to measure and attribute to a single programme or intervention, we believe that assessing the potential contribution of the VEPT programme to meaningful changes in an older adult’s life is important to do. We used recommendations within the programme evaluation literature^{17 18} to determine the optimal strategy for long-term follow-up. These included (1) basing our approach and rationale for follow-up time intervals on a logic model for programme impact and (2) taking into account the likelihood of other factors affecting measurement over the follow-up period and (3) the practicality of obtaining accurate follow-up information. Given that, though multifaceted, the VEPT programme is an ED/hospital-based intervention without postdischarge follow-up, we recognise that the attributability of outcomes to the programme will decrease over time. Additionally, many older adults experience progressive functional and/or cognitive decline as they age, which may have an impact on outcomes of interest. This also may impact our ability to follow up successfully and obtain accurate information. Therefore, We conducted a protocolised follow-up on VEPT cases at 1 month, 6 months and 12 months after ED/hospital discharge, similar to other studies assessing the impact of hospital-based geriatric programmes.¹⁹

Given that older adults with high or moderate concern for mistreatment are those most likely experiencing EM, we focused our follow-up strategy on these cases. We also followed up, however, on cases with low concern at 6 months. As VEPT did not typically provide interventions for these older adults, this follow-up was conducted primarily to ensure that the initial VEPT assessment of low suspicion was not inaccurate and that they had not experienced EM. Notably, if an older adult represents to the ED/hospital again and VEPT is consulted during any follow-up period (0–30 days, 31–120 days, 121–365 days), no future follow-ups are conducted based on the previous discharge date and follow-ups are conducted from the new discharge date.

Exploratory analysis to assess change in quality of life

Despite the challenges of accurately measuring change in quality of life attributable to the VEPT programme, we plan to attempt to measure it in the future as part of our programme evaluation strategy. To do so, we will need to assess quality of life at baseline (soon after programme intake) and then again during the telephone follow-ups. We plan to assess overall quality of life using a modified version of a frequently used single-item measure, the Kemp Quality of Life Scale.²⁰ This question is: ‘Taking everything in your life into account: On a scale from 1 to 7, how would you rate your overall quality of life during the 2 weeks before you came to the ED? (1=means life is very distressing, 4=life is so-so, 7=life is great)’. A modified version of the single question has been used by other investigators to incorporate mistreatment into the assessment of quality of life.²¹ We chose this rather than longer multi-item measures of life satisfaction to ensure feasibility of initial and follow-up interviews and to not over-burden older adults. We recognise that assessing self-reported quality of life in an ED setting in the context of an acute medical or social emergency may not yield accurate results. Therefore, we plan to ask the baseline question a few days later when appropriate. We also recognise that it may be particularly challenging for those with more advanced dementia to understand and respond, but we believe attempting to measure this important domain is essential. To evaluate any changes in quality of life potentially attributable to the VEPT programme, we plan to use: ‘Taking everything about the mistreatment discussed in the hospital into account: On a scale from 1 to 7, how would you rate your overall quality of life over the past 2 weeks?’

Exploratory approach to assess individualised goals and programme’s ability to achieve them

Recognising the complexity and uniqueness of EM cases, we also plan to explore an evaluation approach assessing individualised goals for each case and the programme’s ability to achieve them. In developing these goals for each case, we will consider: (1) what is important to the older adult and other stakeholders, (2) what is realistically achievable and (3) what would be unlikely to occur without VEPT involvement. Progress towards these goals may be measured using goal attainment scaling, the use of which has previously been described for EM intervention programmes.^{21 22} In goal attainment scaling,²³ progress towards each goal is measured on a five-point ordinal scale (−2 to +2). The levels of the scale correspond to: −2=much less than expected, −1=somewhat less than expected, 0=expected, 1=somewhat better than expected, 2=much better than expected. The goal(s) and progress corresponding to each of the scale levels are ideally developed by the team working with the older adult and/or other stakeholders. Though the initial application of goal attainment scaling involved the collaborative development of each goal and related levels of progress from scratch,²³ this was found to be challenging to do in busy clinical settings.²⁴ Therefore, a common approach, which has been previously used in EM, allows for the selection of relevant goals and corresponding prepopulated five-point goal scales from a comprehensive menu.^{21 25 26} This approach, which we will use, also offers flexibility to edit the wording to align with an older adult’s circumstances and progress expectations.

Patient and public involvement

Patients and/or the public were not involved in the design, or conduct, or reporting, or dissemination plans of this research.

Limitations

The approach to evaluating VEPT we describe here has several limitations. We recognise that the operationalisation of outcomes we have proposed here is imperfect, and that a gap remains between specific, concrete, measurable items that we identify and the outcomes about which we care most. Other than stopping or reducing the mistreatment, the validity of these measurable items is less clear. For example, change in living situation may improve safety by reducing mistreatment by the identified perpetrator, but it may also have a negative component if an older adult is removed from their home against their wishes and may lead to mistreatment by another perpetrator. Also, we have focused on outcomes that improve older adult safety and quality of life rather than other areas, such as gathering forensic evidence that is actually used by the criminal justice system. Nevertheless, we believe this work represents an initial step in developing consensus around optimal measures. All outcomes, including whether mistreatment is still occurring or reduced, are dependent on self-report by the older adult as well as information gleaned from other stakeholders, which may not be reliable. The VEPT programme currently only provides services to an older adult while they are in the ED or hospital. While long-term outcomes (1–12 months after discharge) may be at least partially attributable to VEPT, other causes may also contribute. Despite its semistructured approach, the follow-up protocol uses subjective assessments made by the VEPT social worker. To optimally and accurately assess the impact of a programme, it is important when possible, to compare outcomes for programme recipients to a non-intervention control group. The protocol we have described here does not include the identification, recruitment or data gathering for a comparison group. Despite these limitations, we believe that our findings will provide important insight into the impact of our programme and a structure for systematic evaluation of similar programmes.

Discussion and implications

The multistep process we describe here to evaluate VEPT represents, to our knowledge, the first published detailed description of the development of a protocol to evaluate an EM intervention. Although high-quality evaluation of programmes developed to address EM is essential, very few have been designed or conducted. As a result, the effectiveness and impact of the vast majority of EM programmes is not known.

The lack of high-quality evaluation is partly due to the many challenges designing and conducting them for such a complex phenomenon. In developing a protocol to evaluate VEPT, we have attempted mitigate these challenges. We believe that our multistep approach may serve as a model for how to design and conduct an evaluation of an intervention to address EM. This approach will be useful for others planning to evaluate interventions as well as for those reviewing proposed and conducted evaluations to assess quality, rigour and appropriateness of outcomes measured. We are currently using the framework we describe here to evaluate VEPT and plan to report comprehensively on our findings about its impact.

We also believe that, when possible, standardising programme evaluation is optimal, as this facilitates the comparison of one programme to others. When it launched in 2017, VEPT was, to our knowledge, the first and only programme of its kind. In 2020, colleagues at the University of Colorado School of Medicine launched a programme modelled after VEPT with funding from the Colorado Office for Victims Services. To allow for comparison, the teams have collaboratively developed the evaluation protocol and have worked closely together to ensure alignment of the data captured and analytic approach. Members of the Colorado team (EB, DL and ST) serve as coauthors on this manuscript.

Additional innovation is needed to further optimise evaluation protocols, particularly for large, multifaceted programmes. This may include incorporating techniques to measure the impact of a programme on an older adult’s individualised goals. Preliminary examination of the potential for goal attainment scaling has been described and is promising.²²

Ultimately, high-quality, rigorous evaluation will advance understanding of the true impact of EM interventions. This will help ensure that existing programmes can develop improvement plans, facilitate comparison between programmes and identify the optimal programmes to expand and replicate.

Ethics and dissemination

This project has been reviewed and approved by the Weill Cornell Medicine Institutional Review Board, protocol #20-02021422. We aim to disseminate our results in peer-reviewed journals at national and international conferences and among interested patient groups and the public.

Ethics statements

Patient consent for publication

Not applicable.