Skip to main content
NCBI home page
Wiley Open Access Collection logoLink to Wiley Open Access Collection
. 2020 Feb 4;67(3):250–259. doi: 10.1111/1440-1630.12648

Using social return on investment analysis to calculate the social impact of modified vehicles for people with disability

Claire Hutchinson 1,2,✉, Angela Berndt 3, Jenny Cleland 1,2, Susan Gilbert‐Hunt 3, Stacey George 2, Julie Ratcliffe 1,2
PMCID: PMC7317535  PMID: 32017155

Abstract

Introduction

Returning to driving is often a goal for people with acquired disabilities. Vehicle modifications make it possible for people with both acquired and lifelong disabilities to drive yet can be costly. There has been no financial evaluation of vehicle modifications in Australia or internationally.

Methods

A social return on investment analysis of vehicle modifications was undertaken. Primary data were collected via qualitative interviews with consumers and other stakeholders (e.g. driver‐trained occupational therapists, rehabilitation physicians, driving instructors, vehicle modifiers) (n = 23). Secondary data were collected from literature searches and used to identify suitable financial proxies and make estimations of the proportion of drivers with vehicle modifications experiencing each outcome. A co‐investment model was adopted to estimate social return on investment and payback period for funder and consumer. Five scenarios were developed to illustrate social return for low‐cost modifications (Scenario 1) through to high‐cost modifications (Scenario 5).

Results

Social return on investment ratios was positive for funder and consumer investment in all five scenarios. Social return on investment calculations based on co‐investment ranged from $17.32 for every $1 invested (Scenario 1) to $2.78 for every $1 invested (Scenario 5). Consumers' payback periods were between 5.4 and 7.1 months, and funders between 3.5 weeks and 2 years 8.4 months.

Conclusion

Vehicle modifications represent sound investments for both funders and consumers. Given the short payback periods, funders should reconsider age restrictions on vehicles considered suitable for modifications, especially for low‐ to medium‐cost modifications.

Keywords: driver rehabilitation, evaluation, social return on investment, vehicle modifications, assistive technology

1. BACKGROUND

Returning to driving is often a rehabilitation goal for people with acquired disabilities. A return to driving is associated with a range of positive outcomes including economic and social participation (Norweg, Jette, Houlihan, Ni, & Boninger, 2011; Ramakrishnan, Chung, Hasnan, & Abdullah, 2011; Tsai, Graves, & Lai, 2014). The cost and complexity of vehicle modifications can be wide ranging, although the latest advanced technology can support many drivers with low levels of limb function for an investment of $200,000 Australian dollars (AUD) (Eley, 2016).

Despite these potentially high levels of investment, to date, there have been no formal evaluations of the cost and benefits associated with vehicle modifications provided by recognised funding bodies conducted in Australia or internationally. This paper seeks to address this gap by presenting a social return on investment analysis of vehicle modifications. Social return on investment is an innovative methodology which has been used to measure and value a wide range of personal, social and community outcomes not typically valued in other types of evaluation methodology, such as: improved family functioning, reduced loneliness and isolation, promoting a sense of identity, and improved self‐esteem and confidence (Arvidson, Battye, & Salisbury, 2014; McGrath & Stevens, 2019; Willis, Semple, & de Waal, 2016). Unlike other social return on investment studies, this study adopts a co‐investment model that recognises the financial and time contributions of consumers as well as investment by funders. The details of our approach, the stages of our analysis, the payback periods and social return on investment for every $1 invested into vehicle modifications by consumers and funders are documented.

1.1. Vehicle modifications

The process of obtaining a vehicle modification, is one that involves many professional groups; such as driver‐trained occupational therapists, rehabilitation physicians, specialist driving instructors and vehicle modifiers. Modifications may be off‐the‐shelf, tailored or entirely bespoke. Modifications can be as little as AUD$80 for a basic spinner knob to as much as AUD$200,000 dollars for comprehensive electronic driving controls (Eley, 2016). Although there has been considerable research on the return to driving for different populations (e.g. stroke, spinal cord injury), there has been less attention on drivers' investment in their own return to driving and the outcomes that are achieved from vehicle modification use. However, a small number of survey studies conducted internationally have identified a range of positive outcomes from vehicle modifications use including: return to employment and education, enhanced ability to enact family roles, greater independence, better mental health and wellbeing, and improved access to health services and sports and recreation (Carpenter, Forwell, Jongbloed, & Backman, 2007; Darcy & Burke, 2018; Di Stefano, Stuckey, McDonald, & Lavender, 2015; Kiyono, Hasizume, Matsui, Ohtsuka, & Takaoka, 2001; Norweg et al., 2011; Ramakrishnan et al., 2011; Tsai et al., 2014).

Several organisations and agencies currently fund vehicle modifications in Australia (e.g. insurance companies, Department of Veteran Affairs, and state agencies such as the Lifetime Support Authority in South Australia). However, it is expected that the National Disability Insurance Scheme (NDIS), managed by the National Disability Insurance Agency (NDIA), will now become the largest single funder of vehicle modifications in Australia. In their Assistive Technology Strategy Document for the scheme, the NDIA predicts expenditure of AUD$56.8 million on vehicle modifications in 2019‒2020 (NDIS, 2015).

In Australia, state laws govern medical licensing and there are no publicly available data at either the state or national level regarding how many people use vehicle modifications, or what type of modifications are used. However, a recent Australia survey study conducted in the state of Victoria (n = 97) identified that the most common vehicle modifications were hand controls and steering aids (Di Stefano et al., 2015). These findings align with those of a larger (n = 793) but older Swedish study (Henriksson & Peters, 2004). However, these modification types are common prescriptions for people with spinal cord injuries and the majority of participants in these studies reported having sustained a spinal cord injury (Di Stefano et al., 2015; Henriksson & Peters, 2004).

1.2. Social return on investment approach

Social return on investment (SROI) analysis was developed in 2000 by the Roberts Enterprise Development Fund in the US (Millar & Hall, 2013). The methodology was subsequently tested and refined by the UK‐based New Economics Foundation (Millar & Hall, 2013). The guide to social return on investment analysis (Nicholls, Lawlor, Neitzert, & Goodspeed, 2012) remains the most cited reference in both the academic and grey literature for the conduct of SROI analysis internationally, and was the framework used in the current study. The stages of the analysis will be outlined in detail in the methodology section but, in brief are: 1) Establish scope and identify stakeholders, 2) Map outcomes, 3) Evidence outcomes and give them value, 4) Establish impact, 5) Calculate the SROI and 6) report findings (Nicholls et al., 2012).

SROI methodology has been used most extensively in the UK. This is largely due to the establishment of the Social Enterprise Investment Fund to provide support and funding to organisations seeking to conduct SROI studies (Millar & Hall, 2013). However, the methodology has also been taken up internationally, including in Australia, where several SROI studies have been published in recent years (see, e.g., McGrath & Stevens, 2019; Muyambi et al., 2017).

SROI methodology places a strong emphasis on engagement with stakeholders and can be used as a management tool to assist with organisational learning, to reinforce mission and to attract further funding (Pathak & Dattani, 2014). Furthermore, SROI can be used to measure outcomes that are not typically measured in other evaluation methodologies (such as cost benefit analysis and cost utility analysis), and the simple SROI ratio is easily understood, for example, $4 of social return realised for every $1 spent (Banke‐Thomas, Madaj, Charles, & van den Broek, 2015).

However, SROI studies can be time and resource intensive for organisations to conduct (Millar & Hall, 2013) and can be implemented with differing levels of resources and expertise (Moody, Littlepage, & Paydar, 2015). Furthermore, outcomes can be difficult to quantify, especially ‘soft outcomes’ or those experienced at the societal level (Arvidson & Lyon, 2014; Banke‐Thomas et al., 2015; Moody et al., 2015). SROI ratios are context specific and therefore not generalisable and cannot be compared (Pathak & Dattani, 2014). Researchers have called for a more standardised approach to conducting SROI studies (Arvidson, Lyon, McKay, & Moro, 2010; Krlev, Münscher, & Mülbert, 2013) as well as for greater involvement by academics to further develop and refine the methodology (Arvidson et al., 2010; Banke‐Thomas et al., 2015).

1.3. The current study

This study investigates the social return on investment of vehicle modifications for people with disability. Unlike other SROI studies, the current study investigates a type of intervention rather than a specific intervention where costs are already known. Therefore, the analysis required the additional step of identifying the costs of the vehicle modification process. Given the wide range of costs associated with vehicle modifications, a further adaptation was made to the usual SROI methodology by introducing five scenarios which represented low‐cost to high‐cost vehicle modifications. A SROI ratio and payback period was calculated for each scenario. It is anticipated that the adoption of this analytical approach may make this study more generalisable, as SROI studies are usually based on a single intervention or program.

Finally, SROI studies do not typically include costs incurred by the consumer, whether directly or indirectly, such as their financial or time investment. As vehicle modifications are a capital asset in which consumers invest (consumers usually purchase the vehicle themselves), a co‐investment model was adopted in this study to estimate SROI ratio and payback period for both funder and consumer, individually and combined.

2. METHODS

2.1. Ethics approval

Ethics approval was provided by the Human Ethics Committee at the University of South Australia (applications 200304 and 200351). Written consent was obtained from all participants following their receipt of participant information about the study. Interviews were guided by a semi‐structured interview protocol. Interviews were audio recorded with the permission of participants and transcribed in full by a reputable transcription company under a signed confidentiality agreement with the University.

2.2. Participants

Primary data were collected from stakeholders identified in stage 1 of the social return on investment analysis (Stage 1: Establishing Scope and identifying stakeholders). Stakeholders included in the analysis were: consumers, funders, driver‐trained occupational therapists, rehabilitation physicians, driving instructors, vehicle modifiers and rehabilitation engineers (See Supplementary Information A: Included and excluded stakeholders).

Eight consumers participated in semi‐structured interviews. Consumers had to be at least 18 years of age, have a disability, and own a modified vehicle. Fifteen semi‐structured interviews were completed with other stakeholders, some of whom were dual qualified, for example, a driver‐trained occupational therapist who was also qualified as a specialist driving instructor. Face‐to‐face interviews were conducted with stakeholders in Adelaide, South Australia and over the phone for stakeholders located regionally or in other states of Australia.

Data gathered from consumers included a discussion of the process of obtaining their modified vehicle, identification of their inputs into the vehicle modification process (financial and time investment), and a discussion of the outcomes they and their families experienced from having a modified vehicle. Consumers were also asked how long they thought each outcome had lasted or would last, and what transport options they would use if they did not have access to a modified vehicle. Data gathered from other stakeholders included details of their professional expertise, costs associated with their part of the vehicle modification process, how they typically worked with consumers, examples of complex and straightforward vehicle modifications, and identified outcomes for consumers and others from access to modified vehicles.

2.3. Procedure—social return on investment stages

A project advisory group was established at study commencement that included a vehicle modification consumer, as well as representatives from a funder, a vehicle modifier and a disability advocacy group. The purpose of the advisory group was to provide accountability and transparency, to inform the analysis and to support the dissemination of findings. Based on the interview data from consumers and other stakeholders, and engagement with the project advisory group, a theory of change was developed (Stage 2 of social return on investment analysis) which identifies the inputs, activities and outcomes of the vehicle modification process (Figure 1).

Figure 1.

Figure 1

Open in a new tab

Theory of change

As the main objective was to estimate social value for a type of intervention rather than a specific intervention by one organisation, an estimate of the costs of inputs (investment) was required. Five scenarios were developed by the research team and agreed with industry experts and the project advisory group. These scenarios were based on a low‐cost vehicle modification (scenario 1) through to a high technology complex vehicle modification (scenario 5).

All the scenarios were based on the person with disability being an independent driver, that is, not requiring an accompanying carer. The inputs were costed in Australian dollars (AUD) based on data provided in the stakeholder interviews, as well as additional data provided by three vehicle modification companies. The costs included driver‐trained occupational therapist assessment, evaluation and reporting time and driving instruction time based at National Disability Insurance Scheme published rates (NDIS, 2017). A 12% estimation for administrative overhead was made based on data from the Australian National Audit Office (2017) and was applied to the costs of vehicle modifications for each scenario. This administrative overhead was to account for the costs associated with establishing participant eligibility, as well as assessing, approving, overseeing vehicle modifications and follow‐up with a funded participant over the lifetime of the vehicle modification.

Outcomes were evidenced (Stage 3) from interview data and from published vehicle modification studies (Darcy & Burke, 2018; Di Stefano et al., 2015; Kiyono et al., 2001; Lee, Hasnan, & Engkasan, 2018; Ramakrishnan et al., 2011). Given there were several published vehicle modification survey studies with larger sample sizes than the current study, survey data were predominantly used to weight outcomes based on what proportion of drivers could be reasonably expected to experience each outcome. This was necessary as the number of people using vehicle modifications in Australia or even within South Australia was unknown. SROI analysis typically multiplies the value of benefits by the number of people experiencing the intervention. In this study, if it was expected that 75% of vehicle modification participants would experience a particular outcome, the value of that outcome was multiplied by 0.75. Suitable financial proxies were identified via the research team as well as extensive searches of existing literature to identify how similar outcomes had been valued in previous SROI studies. A social value database developed by the UK‐based Housing Associations' Charitable Trust (HACT) was also accessed. This database is an initiative to support SROI projects and contains many values for personal, social and community outcomes. For some outcomes, it was possible to determine a monetary value that represented a direct valuation, for example, the costs of access taxis or counselling services. A benefit period of 10 years was agreed as being a conservative estimate of how long a modified vehicle would provide utility to its owner.

In stage 4 of the analysis, identified discounts were applied to the value of outcomes in order to avoid over claiming. In SROI discounts are calculated on the basis of: what would have happened without the intervention (deadweight), what outcomes were displaced by the intervention (displacement), who else has contributed to the outcomes aside from the funder (attribution), and whether experience of the outcomes declines over time (drop off). It was identified that no positive outcomes were displaced by vehicle modifications and that, due to the ongoing presence of the vehicle, outcomes did not drop‐off over time. Therefore, we estimated discount values for deadweight and attribution only. These values were estimates based on interview findings as well as input from our advisory group and research team.

The calculations to produce the SROI ratio were then performed (Stage 5). Outcomes were weighted to reflect the proportion of people expected to experience each outcome, and discounts applied for deadweight and attribution to produce the social value for year 1. Outcomes values were then projected into the future with a 2.5% discount for years 2 to 10 to account for inflation (Australian Bureau of Statistics, 2018). This produced the net present value of outcomes; that is, the value of outcomes in today's dollars. The total value of the outcomes was then divided by the cost of inputs for each scenario to produce the SROI ratios. This calculation was conducted based on (a) funder inputs, (b) consumer inputs and (c) co‐investment (funder and consumer inputs combined). Payback periods were also calculated based on funder investment, consumer investment and co‐investment for each of the five scenarios.

Finally, sensitivity analysis was performed. Sensitivity analysis is an accepted methodology for testing how sensitive the SROI ratio is to changed assumptions in the calculation. This analysis tested assumptions regarding the five outcomes with the highest proxy values, and we focussed only on making more conservative adjustments rather than more favourable ones. The value of each proxy was reduced by 50%, and attribution and deadweight values were reduced by 50%. Each of the five highest proxy value outcomes were then removed in turn from the SROI calculation to investigate the impact on the ratio. Finally, the estimated benefit period was reduced by 50% from 10 years to 5 years. For each assumption changed, the new social return on investment ratio was calculated for scenario 1 (low cost) and scenario 5 (highest cost) based on funder investment, consumer investment and co‐investment.

3. RESULTS

3.1. Identifying inputs

Funder inputs were calculated based on five scenarios outlined in Table 1. Costs to the funder included the costs of the vehicle modifications (product and installation) as well as the costs of professional services to support a return to driving. Additional costs were assumed in scenario 5, as high end technology is not currently available via South Australian vehicle modifiers. Additional costs included day return flights from Adelaide to Sydney for the consumer and driver‐trained occupational therapist to test modifications, and car transportation from New South Wales to South Australia. It was assumed that the electronic ramp and wheelchair docking modifications were installed by a South Australian vehicle modifier. Flights and vehicle transportation costs were based on internet searches (Table 1).

Table 1.

Scenarios and funders inputs

Scenario Modification

Occupational therapist

(hr)

Driving instructor

(hr)

 Other costs Total AUD
1 Left accelerator pedal, spinner knob 5 3   $3,652.29
2 Hand controls, electronic lifter to store wheelchair 5 4   $21,419.07
3 Electronic ramp for wheelchair, 6 way Ricon seat, hand controls 5 4   $35,766.27
4 Electronic ramp, wheelchair docking in driver and passenger positions, removable seat, hand controls 5 7   $103,621.01
5 Four‐way joy stick driving controls (sourced from NSW), Voice command for auxiliary controls (sourced from NSW), Electronic ramp and docking for wheelchair in driver position (sourced from SA) 20 20 Day return flights X 2, vehicle transportation. $144,999.40
Open in a new tab

Driver Trained Occupational Therapist and Driving Instructor hourly rates based on National Disability Insurance Scheme published rates, items 15_048_0128_1_3 and 07_004_0132_8_3 respectively (NDIS, 2017). Vehicle modification costs based on estimates from three vehicle modification companies.

To estimate consumer inputs, we estimated the cost difference between a 2‐year‐old vehicle and a 10‐year‐old vehicle; that is, a vehicle that meets the National Disability Insurance Scheme's policy vehicle age requirements (NDIS, 2018) as opposed to an average age vehicle in Australia (Australian Bureau of Statistics, 2017a). Ongoing vehicle maintenance, insurance, etc. was excluded from the analysis as these are costs incurred by other Australian drivers and are not specific to consumers of vehicle modifications. A conservative estimate of the time consumers invest in the vehicle modification process was made and was costed at the average hourly rate for South Australian full‐time workers (ordinary time) (Australian Bureau of Statistics, 2017b) (Supplementary Information B).

3.2. Valuing outcomes

Outcomes identified are shown in the theory of charge (Figure 1). Thirteen outcomes were identified, 11 related to consumers, including educational and employment opportunity, increased confidence and self‐esteem, increased access to health services, and improved mental health and wellbeing. Two additional outcomes identified were reduced burden on caregivers and increased community awareness.

The financial proxies used in the analysis were a combination of replacement valuations and direct valuations. For example, increased confidence and self‐esteem was based on the cost of attending an assertiveness and self‐confidence training course and having six life coaching sessions (replacement valuation), whereas the value for increased access to health services was based on the costs of attending appointments in an access taxi (direct evaluation). See Supplementary Information C: Financial proxy values.

3.3. Weighting outcomes

Weightings were based on interview data and expert opinion from our research team and advisory group. Data from vehicle modification survey studies were then used to validate or adjust our initial estimates. Where there was more than one source of data we could suitably apply as a weighting, we selected the most conservative estimate for the SROI calculation.

For example, we identified that all our consumers reported experiencing independence as a result of having a modified vehicle. Our initial weighting of the financial proxy value for independence was therefore 100%, that is, $22,825 over the 10‐year benefit period. However, in their survey study, Di Stefano et al. (2015) reported that 96% of their sample reported that independence was a key outcome of having a modified vehicle. We therefore adjusted our estimation down to 96%, that is, $21,912.00 over the 10‐year benefit period. See Supplementary Information D: Outcome weightings.

3.4. Discounting values

The assigned values for deadweight and attribution were agreed by the research team members based on data from the interviews, input from our advisory group and data from vehicle modification survey studies. For example, all of the consumers in this study reported that independence was the most significant benefit they derived from their modified vehicle, using other transportation methods was undertaken reluctantly, and temporary loss of their vehicle (e.g. due to repairs), resulted in a considerable loss of independence. Deadweight for independence was therefore estimated at 100%, indicating that this outcome would not have occurred without the modified vehicle. For the enactment of family roles outcome, it was assumed that people were fulfilling family roles without their vehicle, just as other Australians do. However, the consumers interviewed identified a number of ways in which their modified vehicle enhanced their ability to perform family roles, for example, teaching adult children to drive, picking up adult children from nights out, and driving an older parent to medical appointments, shopping trips and social engagements. The vehicle therefore allowed consumers to broaden the ways in which they enacted family roles relevant to their life stage. We therefore estimated deadweight for this outcome at 60%, indicating that this outcome would have been experienced in part without the modified vehicle.

An attribution discount is a way of acknowledging that interventions do not operate in isolation, as other organisations and agencies may be working with people with disabilities on the same outcomes. For example, rehabilitation activities, physical therapy and capacity building programs also assist people with disabilities to increase their confidence and self‐esteem. Confidence and self‐esteem were strongly linked to independence in our sample; a key outcome of modified vehicles expressed by all consumers. Therefore, the attribution value for confidence and self‐esteem was estimated at 60%, indicating that modified vehicles were slightly more likely to be responsible for the outcome than other interventions working to achieve the same outcome. In relation to social participation, a higher preservation value was applied (80%) as our interview data showed that temporary loss of their vehicle resulted in a significant decline in consumers' social participation (see Supplementary Information E: Discounting social value).

3.5. Social return on investment ratios

The SROI calculation involved three steps. In step 1, proxy values were weighted to reflect the proportion of vehicle modification consumers expected to experience each outcome. In step 2, discounts are applied (deadweight, attribution) to produce the discounted social value for year 1. Finally, in step 3, social value was projected into the future (years 2 to 10) by applying a 2.5% discount to account for inflation (Australian Bureau of Statistics, 2018). This final step represents net present value; that is, the total social value in today's dollars.

For example, the proxy value for improved mental health and wellbeing was $3,048.75. A weighting of 75% was applied to this outcome (i.e. we expected 75% of consumers to experience this outcome). A weighting of 80% was applied for deadweight (i.e. we expected this outcome would not have happened to any considerable extent without the presence of the modified vehicle) and 60% for attribution (i.e. we expected that modified vehicles contributed towards this outcome somewhat more than other interventions aimed at addressing the same outcome). Therefore, the social value of mental health and wellbeing in year 1 was $1,097.55 ($3,048.75 × 0.75 × 0.8 × 0.6). With a 2.5% discount applied, a social value of $1,070.11 was calculated for year 2, and $1,043.36 for year 3, etc. Over the entire 10‐year benefit period, the social value of improved health and wellbeing generated by use of modified vehicles was calculated at $9,819.58. These calculations were conducted for every outcome and total outcome value was estimated to be AUD$493,091.15 over the 10‐year benefit period (see Supplementary Information F: Discounted values over benefit period for the full net present value calculation).

Three sets of calculations were then performed for each scenario (a) social return for funders, (b) social return for consumers and (c) social return based on consumer and funder co‐investment. For example, the present net value for all the outcomes was $493,091.15 and the total funder inputs for scenario 5 were $144,999.40. Therefore, the SROI ratio for Scenario 5 was 3.40:1, that is, $3.40 of social value was generated for every $1 invested (Table 2).

Table 2.

Social return on investment by scenario and stakeholder

Scenario Funders Consumers Co‐investment
  Return for every $1 invested
1 (low cost) $135.01 $19.86 $17.32
2 $23.02 $15.98 $9.43
3 $13.79 $15.94 $7.39
4 $4.76 $15.90 $3.66
5 (high cost) $3.40 $15.27 $2.78
Open in a new tab

3.6. Payback period

Payback period demonstrates how long it takes for investment to be paid off in accumulated social value. Payback period is an indication of the risk of investment, as short payback times are considered less risky than those that are longer term. Payback periods for funders varied more than for consumers, as consumers' investment was relatively consistent across scenarios. Payback periods were calculated against each of our scenarios and identified that the lowest cost scenario only took 3.5 weeks to payback funders' investment and 5.4 months to payback consumers' investment. The payback period for co‐investment was 6.2 months (Table 3). Even the most costly scenario took only 3 years and 3.9 months to payback co‐investment in accumulated social value; well within the conservative 10‐year benefit period.

Table 3.

Payback periods by scenario and stakeholder

Scenario Funders Consumers Co‐investment
1 (low cost) 3.5 weeks 5.4 months 6.2 months
2 4.7 months 6.8 months 11.4 months
3 7.8 months 6.8 months 1 year 2.7 months
4 1 year 10.9 months 6.8 months 2 years 6 months
5 (high cost) 2 years 8.4 months 7.1 months 3 years 3.9 months
Open in a new tab

3.7. Sensitivity analysis

In the sensitivity analysis, it was observed that changes to the assumptions for social participation impacted the SROI ratios the most. However, even when this outcome was removed from the analysis all together, the social return on investment for the most costly scenario (scenario 5) was positive: $1.58 (funder), $7.34 (consumer) and $1.34 (co‐investment). Reducing the benefit period to five years reduced the co‐investment social return on investment for scenario 5 to $1.47 for every $1 invested. See Supplementary Information G: Sensitivity analysis for full details.

4. DISCUSSION

To our knowledge this study represents the first study in Australia and internationally to employ SROI methodology to estimate the social value of vehicle modifications to consumers, other stakeholders and the wider community. This study has demonstrated the utility of this methodology in evaluating vehicle modifications, although some adaptations were required to make the methodology fit for purpose. Most notably, the use of scenarios (to reflect input variance), weightings (to replace quantity of consumers effected by the intervention, which was unknown), and the addition of consumers financial inputs.

This study sought to determine the social value of a type of intervention rather than a specific intervention, where inputs were already known. This adaptation might be useful to other researchers who wish to assess the social value of intervention types more broadly, however, it should be acknowledged that the value of inputs represents an estimate only, and other types of interventions may include a broader range of costs than those included in our estimation for vehicle modifications.

As the intended audience of SROI outputs is usually the funder, only funder investments are typically considered when determining social return ratios. In the current study, social return ratios based on funder investment, particular for the first three scenarios (low‐ to mid‐cost vehicle medications), were high compared to those of other social return ratios reported in the academic and grey literature. However, our additional analysis suggests that this is because consumers are bearing a significant proportion of the investment costs for modified vehicles. For example, for the low cost scenario 1, consumers are estimated to be covering 87% of the overall investment. When the investment by the consumer is included in the SROI calculation, the ratio drops from $135.01 of social value for every $1 invested, to $17.32 for every $1 invested. Therefore, unlike many interventions that have been evaluated using SROI methodology, vehicle modifications represent co‐investment between the funder and the consumer requiring some adaptation to the methodology.

It is important to highlight that some funders place restrictions on the age of vehicles they consider suitable to be modified (NDIS, 2018). Given the relatively short payback periods for funders—especially in the low‐ to medium‐cost range—funders may wish to reconsider such policies. Even the highest costs scenario suggested that older vehicles could provide enough utility over the vehicle life to repay the investment made in social value.

Although this study may be more generalisable than other SROI studies, as with all research, this study has a number of limitations. Given the vast and fast changing nature of vehicle modifications, the five scenarios will not be relevant to all types and combinations of modifications. Furthermore, as with other SROI projects, the analysis represents a snapshot in time and is not future proofed; product offerings, costs of professional services and other market conditions may change over time and different funders may have different cost bases. Furthermore, bias towards independent driving resulted in negativity towards the use of public transportation in this sample of consumers. This bias impacted to some extent upon the outcomes identified and assumptions made in the analysis.

In the future it is expected that self‐driving cars will negate the need for vehicle modifications. Although mainstream use of Level 5 fully automated driverless vehicles may still be 20 years away, motor vehicles with a significant amount of automation should be available in the next five years (Walker, 2018). However, three factors remain to be determined. Firstly, whether Level 3 (partial automation) and Level 4 (high automation) automated vehicles will meet the driving needs of people with disabilities. Secondly, whether funders will be early adopters of such technology when costs are likely to be high. Thirdly, in the absence of funding sources, how affordable self‐driving cars will be to people with disabilities, who typically have lower levels of employment and household income than Australians without disability (Australian Bureau of Statistics, 2015, 2016).

Overall, this study does show the broad social impact of vehicle modifications to consumers, their families and the wider community, and highlights the number of professional groups, organisations and agencies that support a person with disabilities return to driving. The study also demonstrates the significant investment people with disabilities make in their own return to driving. A co‐investment models of social return may be relevant to other types of interventions aimed at people with disabilities.

5. CONCLUSION

Using social return on investment methodology to evaluate a type of intervention required some additional considerations to ensure that outputs were meaningful to the intended audience of funders, consumers and other vehicle modification stakeholders. Addressing these factors with the use of scenarios (to reflect input variance), weightings (to replace quantity of consumers effected by the intervention, which was unknown), and the addition of consumers financial inputs, represents innovations to the methodology in order to make it fit for purpose. This study may therefore assist other researchers, funding bodies and policy makers in adopting SROI methodology for the evaluation of types of intervention rather than specific interventions. Such adaptations to SROI methodology may make results more generalisable and relevant to audiences beyond individual funding organisations.

KEY POINTS FOR OCCUPATIONAL THERAPY

  • This study applied social return on investment methodology to calculate the social value of vehicle modifications for people with disabilities and funders.

  • A wide range of outcomes were identified including independence, greater access to employment and educational opportunity, improved mental health and wellbeing, enactment of family roles and improved social participation.

  • A significant co‐investment (consumer and funder) social return was identified of between $17.32 and $2.78 for each $1 invested depending on the level of investment in modifications.

CONFLICT OF INTEREST

The authors report no declarations of interest.

AUTHOR CONTRIBUTIONS

The study was conceptualised by AB, JR, SGH and SG. CH conducted the qualitative data collection and analysis. JC conducted the literature searches and quantitative data collection. CH lead the SROI analysis. CH prepared the initial draft of the manuscript and circulated it to all authors for critical review. All authors approved the submitted version of the paper.

FUNDING INFORMATION

This research was funded by the Lifetime Support Authority, State Government of South Australia (Grant Number: GA 00040).

Supporting information

 

Click here for additional data file. (36.1KB, docx)

Hutchinson C, Berndt A, Cleland J, Gilbert‐Hunt S, George S, Ratcliffe J. Using social return on investment analysis to calculate the social impact of modified vehicles for people with disability. Aust Occup Ther J. 2020;67:250–259. 10.1111/1440-1630.12648

REFERENCES

  1. Arvidson, M. , Battye, F. , & Salisbury, D. (2014). The social return on investment in community befriending. International Journal of Public Sector Management, 27(3), 225–240. 10.1108/IJPSM-03-2013-0045 [DOI] [Google Scholar]
  2. Arvidson, M. , & Lyon, F. (2014). Social impact measurement and non‐profit organisations: Compliance, resistance, and promotion. VOLUNTAS: International Journal of Voluntary and Nonprofit Organizations, 25(4), 869–886. 10.1007/s11266-013-9373-6 [DOI] [Google Scholar]
  3. Arvidson, M. , Lyon, F. , McKay, S. , & Moro, D. (2010). The ambitions and challenges of SROI. (Working Paper No. 49), Birmingham: Third Sector Research Centre; Retrieved from http://eprints.mdx.ac.uk/7104/ [Google Scholar]
  4. Australian Bureau of Statistics . (2015). Disability and Labour Force Participation, 2012 (Cat. No. 4433.0.55.006), Canberra: ABS. [Google Scholar]
  5. Australian Bureau of Statistics . (2016). Disability, ageing and carers, Australia, 2015 (Cat. No. 4430.0), Canberra: ABS. [Google Scholar]
  6. Australian Bureau of Statistics . (2017a). Motor vehicle census (Cat. No.9309.0). Canberra: ABS. [Google Scholar]
  7. Australian Bureau of Statistics . (2017b). Characteristics of employment (Cat No. 6333.0). Canberra: ABS. [Google Scholar]
  8. Australian Bureau of Statistics . (2018). Consumer price index (Cat No. 6401.0). Canberra: ABS. [Google Scholar]
  9. Australian National Audit Office . (2017). Efficiency of the Australia Council's Administration of Grants (Auditor‐General Report No. 7 of 2017‐2018), Canberra: Australian Government; Retrieved from https://www.anao.gov.au/work/performance-audit/efficiency-australia-councils-admin-grants [Google Scholar]
  10. Banke‐Thomas, A. O. , Madaj, B. , Charles, A. , & van den Broek, N. (2015). Social Return on Investment (SROI) methodology to account for value for money of public health interventions: A systematic review. BMC Public Health, 15, 582 10.1186/s12889-015-1935-7 [DOI] [PMC free article] [PubMed] [Google Scholar]
  11. Carpenter, C. , Forwell, S. J. , Jongbloed, L. E. , & Backman, C. L. (2007). Community participation after spinal cord injury. Archives of Physical Medicine and Rehabilitation, 88(4), 427–433. 10.1016/j.apmr.2006.12.043 [DOI] [PubMed] [Google Scholar]
  12. Carsguide . (2018). Carsguide: Pricing and specifications. Retrieved from https://www.carsguide.com.au/ [Google Scholar]
  13. Darcy, S. , & Burke, P. F. (2018). On the road again: The barriers and benefits of automobility for people with disability. Transportation Research Part A, 107, 229–245. 10.1016/j.tra.2017.11.002 [DOI] [Google Scholar]
  14. Di Stefano, M. , Stuckey, R. , MacDonald, W. , & Lavender, K. (2015). Vehicle modifications for drivers with disabilities: Developing the evidence base to support prescription guidelines, improve user safety and enhance participation (Research report # 071–0515‐R01), Melbourne, Victoria: Institute for Safety, Compensation and Recovery Research; Retrieved from https://www.tac.vic.gov.au/__data/assets/pdf_file/0011/267275/Vehicle-modifications-for-drivers-with-disabilities.pdf [Google Scholar]
  15. Eley, T. (2016). Disability‐related vehicle modifications. Precedent (Sydney, NSW), 38. Retrieved from https://search.informit.com.au/documentSummary;dn=066592683502404;res=IELAPA> [Google Scholar]
  16. Henriksson, P. , & Peters, B. (2004). Safety and mobility of people with disabilities driving adapted cars. Scandinavian Journal of Occupational Therapy, 11(2), 54–61. 10.1080/11038120410020511 [DOI] [Google Scholar]
  17. Housing Associations' Charitable Trust (HACT) . Social Value Bank. Retrieved from https://www.hact.org.uk/social-value-bank
  18. Kiyono, Y. , Hashizume, C. , Matsui, N. , Ohtsuka, K. , & Takaoka, K. (2001). Car‐driving abilities of people with tetraplegia. Archives of Physical Medicine and Rehabilitation, 82(10), 1389–1392. 10.1053/apmr.2001.26089 [DOI] [PubMed] [Google Scholar]
  19. Krlev, G. , Münscher, R. , & Mülbert, K. (2013). Social Return on Investment (SROI): State‐of‐the‐art and perspectives‐a meta‐analysis of practice in Social Return on Investment (SROI) studies published 2002–2012. Heidelberg, Germany: Centre for Social Investment; Retrieved from https://archiv.ub.uni-heidelberg.de/volltextserver/18758/1/CSI_SROI_Meta_Analysis_2013.pdf [Google Scholar]
  20. Lee, R. C. H. , Hasnan, N. , & Engkasan, J. P. (2018). Characteristics of persons with spinal cord injury who drive in Malaysia and its barriers: A cross sectional study. Spinal Cord, 56(4), 341–346. 10.1038/s41393-017-0034-2 [DOI] [PubMed] [Google Scholar]
  21. McGrath, R. M. , & Stevens, K. S. (2019). Forecasting the social return on investment associated with children's participation in circus‐arts training on their mental health and well‐being. International Journal of the Sociology of Leisure, 2(1), 163–193. 10.1007/s41978-019-00036-0 [DOI] [Google Scholar]
  22. Millar, R. , & Hall, K. (2013). Social return on investment (SROI) and performance measurement: The opportunities and barriers for social enterprises in health and social care. Public Management Review, 15(6), 923–941. 10.1080/14719037.2012.698857 [DOI] [Google Scholar]
  23. Moody, M. , Littlepage, L. , & Paydar, N. (2015). Measuring Social Return on Investment: Lessons from organisational implementation of SROI in the Netherlands and the United States. Nonprofit Management and Leadership, 26(1), 19–37. 10.1002/nml.21145 [DOI] [Google Scholar]
  24. Muyambi, K. , Gurd, B. , Martinez, L. , Walker‐Jeffreys, M. , Vallury, K. , Beach, P. , & Dennis, S. (2017). Issues in using social return on investment as an evaluation tool. Evaluation Journal of Australasia, 17(3), 32–39. 10.1177/1035719X1701700305 [DOI] [Google Scholar]
  25. National Disability Insurance Scheme . (2015). Assistive technology strategy. Canberra: NDIS; Retrieved from https://www.ndis.gov.au/about-us/strategies/assistive-technology-strategy [Google Scholar]
  26. National Disability Insurance Scheme . (2017). NDIS Price Guide: Australian Capital Territory, Northern Territory, South Australia, Western Australia. Canberra: NDIS; Retrieved from https://www.ndis.gov.au/medias/bin/../201718-ACT-NT-SA-WA-Price-Guide.docx [Google Scholar]
  27. National Disability Insurance Scheme . (2018). Operational Guidelines: 7. Vehicle Modifications, Canberra: NDIS; Retrieved from https://www.ndis.gov.au/Operational-Guideline/including-7 [Google Scholar]
  28. Nicholls, J. , Lawlor, E. , Neitzert, E. , & Goodspeed, T. (2012). A guide to social return on investment. London: Office of the Third Sector, The Cabinet Office; Retrieved from http://www.socialvalueuk.org/app/uploads/2016/03/The%20Guide%20to%20Social%20Return%20on%20Investment%202015.pdf [Google Scholar]
  29. Norweg, A. , Jette, A. M. , Houlihan, B. , Ni, P. , & Boninger, M. L. (2011). Patterns, predictors, and associated benefits of driving a modified vehicle after spinal cord injury: Findings from the National Spinal Cord Injury Model Systems. Archives of Physical Medicine and Rehabilitation, 92(3), 477–483. 10.1016/j.apmr.2010.07.234 [DOI] [PubMed] [Google Scholar]
  30. Pathak, P. , & Dattani, P. (2014). Social return on investment: Three technical challenges. Social Enterprise Journal, 10(2), 91–104. 10.1108/SEJ-06-2012-0019 [DOI] [Google Scholar]
  31. Ramakrishnan, K. , Chung, T. , Hasnan, N. , & Abdullah, S. (2011). Return to work after spinal cord injury in Malaysia. Spinal Cord, 49(7), 812–816. 10.1038/sc.2010.186 [DOI] [PubMed] [Google Scholar]
  32. Tsai, I. , Graves, D. , & Lai, C. (2014). The association of assistive mobility devices and social participation in people with spinal cord injuries. Spinal Cord, 52(3), 209–215. 10.1038/sc.2013.178. [DOI] [PubMed] [Google Scholar]
  33. Walker, J. (2018). The self‐driving car timeline – Predictions from the top 11 global automakers, Emerj; 21st December, Retrieved from https://emerj.com/ai-adoption-timelines/self-driving-car-timeline-themselves-top-11-automakers/ [Google Scholar]
  34. Willis, E. , Semple, A. C. , & de Waal, H. (2016). Quantifying the benefits of peer support for people with dementia: A Social Return on Investment (SROI) study. Dementia, 17(3), 266–278. 10.1177/1471301216640184 [DOI] [PubMed] [Google Scholar]

Associated Data

This section collects any data citations, data availability statements, or supplementary materials included in this article.

Supplementary Materials

 

Click here for additional data file. (36.1KB, docx)

Articles from Australian Occupational Therapy Journal are provided here courtesy of Wiley

RESOURCES