Table 1.
Analysis of variance of potential success factors across categories of success
| Factor | Average answer across all categories* | Distribution of factor is the same across categories of expansion (proven value) | Distribution of factor is the same across categories of expansion (all) | Distribution of factor is the same across categories of realised value | Interpretation and comment on secondary analysis‡ | |
| Significance† | Significance† | Significance† | ||||
| 1 | The initiative was designed to end once a set outcome had been achieved. | ↘ −0.40 | 0.274 | 0.172 | 0.317 | |
| 2 | The initiative was designed to end after defined period of time. | ↘ −0.58 | 0.011 | 0.011 | 0.288 | Unsurprisingly, even projects with high-realised value finish if they are time bound. |
| 3 | The initiative was designed to address an important healthcare need. | ↑ 1.63 | 0.765 | 0.712 | 0.135 | |
| 4 | There was public/political recognition and concern for the problem that the initiative was designed to address. | ↑ 1.23 | 0.939 | 0.66 | 0.201 | |
| 5 | The initiative was based on a strong evidence base, and it was credible that the stated benefits could be achieved through the project plan. | ↑ 1.35 | 0.696 | 0.355 | 0.299 | |
| 6 | The project was sufficiently funded. | ↗ 1.17 | 0.941 | 0.454 | 0.126 | |
| 7 | The project had sufficient infrastructure, such as buildings, office space, materials or supplies. | ↑ 1.52 | 0.952 | 0.842 | 0.613 | |
| 8 | There were sufficient members of staff with the right skills to meet the requirements of the initiative. | ↑ 1.42 | 0.003 | 0.002 | 0.013 | Skilled workforce is a critical success factor across all definitions of success. |
| 9 | Members of staff had sufficient energy and time to dedicate to the initiative. | ↑ 1.21 | 0.033 | 0.085 | 0.362 | Time and energy are critical to whether proven innovations expand. |
| 10 | There was sufficient administrative support to deliver and maintain the initiative. | ↗ 0.85 | 0.013 | 0.019 | 0.142 | Administrative support is critical to whether an innovation expands. |
| 11 | There was sufficient technical support to deliver and maintain the initiative. | ↗ 1.04 | 0.113 | 0.187 | 0.657 | |
| 12 | There was sufficient educational support to deliver and maintain the initiative. | ↑ 1.30 | 0.023 | 0.012 | 0.089 | Educational support is critical to whether an innovation expands. |
| 13 | External political or societal factors impacted negatively on the delivery of the initiative. | ↘ −0.91 | 0.191 | 0.141 | 0.005 | External political or societal factors appear critical to whether an innovation is able to realise its intended value (inconsistent exposure/response). |
| 14 | It was necessary to adapt the project so that it aligned more closely with external political or societal priorities. | ↘ −0.72 | 0.541 | 0.252 | 0.064 | |
| 15 | We had opportunities to demonstrate the benefits of this innovation within our organisation and/or to other organisations. | ↑ 1.59 | 0.237 | 0.053 | 0.02 | Unsurprisingly, innovations that were able to realise their intended value were more likely to be able to demonstrate the benefits of their innovation. |
| 16 | Steps were taken to raise the profile of the initiative, for example, through media, marketing, community engagement or publications. | ↗ 0.85 | 0.108 | 0.059 | 0.306 | |
| 17 | There are plans to replicate this innovation at other sites or spread it to other parts of the organisation. | ↗ 0.58 | 0.024 | 0.012 | 0.228 | Unsurprisingly, innovations that have become scaled up were more likely to say there were plans to spread their innovation. |
| 18 | The initiative integrated well into existing organisational structures, programmes or policies. | ↑ 1.26 | 0.012 | 0.002 | 0.059 | The ability of an innovation to integrate into existing organisational structures may be critical to whether it becomes scaled up. |
| 19 | It was necessary to adapt the initiative so that it achieved a good fit with existing organisational structures, programmes or policies. | → −0.09 | 0.053 | 0.035 | 0.115 | For innovations to scale up, they may need to adapt so that they fit within existing organisational structures. |
| 20 | The host organisation was ready and able to undertake the initiative. | ↑ 1.55 | 0.262 | 0.168 | 0.721 | |
| 21 | The initiative was hampered by opposition from within the host organisation. | ↓ −1.50 | 0.037 | 0.027 | 0.398 | However valuable an innovation is, it appears unlikely to survive if it is opposed within the host organisation. |
| 22 | The host organisation lacked the necessary values/culture to support and sustain the initiative. | ↓ −1.17 | 0.265 | 0.247 | 0.888 | |
| 23 | I was released from other duties so that I could implement this initiative. | ↘ −0.43 | 0.732 | 0.893 | 0.789 | |
| 24 | I had a supportive peer network that I could discuss any issues or problems with. | ↑ 1.32 | 0.385 | 0.562 | 0.79 | |
| 25 | I was internally motivated to implement this initiative. | ↑ 1.81 | 0.425 | 0.129 | 0.034 | Innovations appear more likely to realise their value if the innovator is internally motivated. |
| 26 | I found working on the initiative personally rewarding. | ↑ 1.81 | 0.147 | 0.067 | 0.022 | Unsurprisingly, there is a correlation between an innovation realising its value, and the innovator finding it rewarding. |
| 27 | I feel I had the right skills/experience/training to implement and sustain the initiative. | ↑ 1.62 | 0.033 | 0.008 | 0.023 | The skills of the innovator appear to be a critical success factor across all definitions of success. |
| 28 | I had sufficient energy and time to dedicate to the initiative. | ↗ 1.06 | 0.209 | 0.268 | 0.498 | |
| 29 | The project had sufficient input from experts with the necessary knowledge and experience. | ↑ 1.66 | 0.134 | 0.018 | 0.021 | Expert input appears critical to both realisation of value, and to whether it expands. |
| 30 | The outcomes and impact of the project were measured or assessed. | ↑ 1.37 | 0.060 | 0.125 | 0.119 | |
| 31 | We were able to demonstrate the effectiveness of the project. | ↑ 1.38 | 0.463 | 0.185 | 0.015 | Unsurprisingly, innovations that were able to realise their intended value were more likely to be able to demonstrate the effectiveness of their innovation. |
| 32 | Performance data were gathered and reported on a regular basis. | ↗ 1.04 | 0.346 | 0.266 | 0.159 | |
| 33 | Steps were taken to systematically improve and adapt the project. | ↑ 1.44 | 0.293 | 0.153 | 0.377 | |
| 34 | There was ongoing orientation and training available, for example, to new staff or to build capacity. | ↗ 0.87 | 0.034 | 0.03 | 0.153 | The availability of ongoing training may be critical to whether successful innovations scale up. |
| 35 | Staff were given time/incentives to attend the necessary training. | → 0.47 | 0.096 | 0.159 | 0.178 | |
| 36 | Staff were required to attend the necessary training | → 0.11 | 0.348 | 0.271 | 0.767 | |
| 37 | The initiative was difficult or complex to deliver. | → −0.09 | 0.294 | 0.163 | 0.158 | |
| 38 | The initiative helped to make things easier or more efficient. | ↗ 0.76 | 0.86 | 0.953 | 0.182 | |
| 39 | The initiative did not require special or extra effort. | ↓ −1.09 | 0.979 | 0.869 | 0.597 | |
| 40 | I believe that the staff delivering the initiative found the work/tasks rewarding and satisfying. | ↑ 1.59 | 0.368 | 0.456 | 0.743 | |
| 41 | The project team worked well together. | ↑ 1.74 | 0.416 | 0.796 | 0.893 | |
| 42 | There were clear responsibilities for individuals the work was shared across the team. | ↑ 1.45 | 0.945 | 0.533 | 0.066 | |
| 43 | Project was overly dependent on a particular individual or individuals. | ↗ 0.57 | 0.708 | 0.355 | 0.29 | |
| 44 | I believe that the team understood what the project was trying to achieve and that it would lead to improved processes and outcomes. | ↑ 1.62 | 0.218 | 0.165 | 0.772 | |
| 45 | There were rewards or incentives that supported engagement with, and continued delivery of, the initiative. | → 0.07 | 0.45 | 0.638 | 0.228 | |
| 46 | The activities and roles of the initiative were incorporated into job descriptions. | ↘ −0.30 | 0.29 | 0.243 | 0.141 | |
| 47 | Staff had time within their working hours to complete the tasks of the initiative. | ↗ 0.59 | 0.251 | 0.138 | 0.328 | |
| 48 | The initiative had leadership and/or champions who were committed and capable. | ↑ 1.62 | 0.003 | 0.001 | 0.006 | Leadership appears to be a highly significant success factor across all definitions of success. |
| 49 | There was an appropriate balance of power between those involved with the initiative. | ↗ 1.15 | 0.697 | 0.775 | 0.929 | |
| 50 | Team members were able to express their opinions, and their opinions were valued. | ↑ 1.91 | 1 (no variance) | 0.049 | 0.026 | Distributed decision-making may be a critical success factor across all definitions of success. It was common to all innovations of value that scaled up (hence no variance). |
| 51 | There was a sense of ownership and commitment by those involved with the initiative | ↑ 1.79 | 0.284 | 0.177 | 0.102 | |
| 52 | Staff who were responsible for delivering the initiative were involved as partners, and were able to shape the initiative. | ↑ 1.74 | 0.306 | 0.176 | 0.031 | Participatory processes with staff may be critical to the ability of a project to realise its intended value. |
| 53 | The beneficiaries (patients/service users) were involved as partners, and were able to shape the initiative. | ↗ 0.83 | 0.45 | 0.139 | 0.027 | Participatory processes with patients/service users may be critical to the ability of a project to realise its intended value. |
| 54 | The community in which it was situated was involved as partners, and was able to shape the initiative. | ↗ 0.96 | 0.177 | 0.034 | 0.023 | Participatory processes on a community level may be critical to both the ability of a project to realise its value and its scalability. |
| 55 | There was a collaborative network of people/organisations that helped to support and sustain the initiative. | ↑ 1.30 | 0.008 | 0.007 | 0.003 | The support of a collaborative network of people/organisation may be highly significant to both value creation and scalability. |
| 56 | It felt as though the initiative was imposed on us and there was little sense of ownership or commitment to the project. | ↓ −1.64 | 0.684 | 0.488 | 0.326 |
Analysis of variance of potential success factors across categories of success.
*Respondents on average ↑=agree, ↗=somewhat agree, →=neither agree nor disagree, ↘=somewhat disagree, ↓=disagree.
†Asymptotic significances are displayed. The significance level is 0.05. The darker the shading, the safer it is to reject the null hypothesis. Significance <0.05 indicates >95% certainty that the difference between categories is not random.
‡Secondary analysis examined the direction of the association and the strength of effect across categories of success.