| Evaluation thinking |
When doing a digital health project at PHEc, evaluation thinking, skills, and tools should be integrated into the project from the start. Evaluation needs to be a central part of the design process and iterative delivery of any PHE service or product.
|
As a Director of Public Health When commissioning services I need evaluation to be aligned closely with service delivery So that it is formative and not a separate piece of work
|
“You have to get them at the beginning. And it’s really hard when you don’t get them at the beginning, ‘cause then you’ve got to try and do a retrospective evaluation. You never have the right data. It’s never as good of quality.” |
| Evaluation canvas |
The evaluation canvas is PHE’s validated and accepted portfolio of metrics, tools, and methods for evaluating DHIsd. This canvas is the first step to creating a knowledge base on the effectiveness of digital health in meeting health outcomes and will support decisions on policy, practice, and research.
|
As an evaluator When designing or setting up an evaluation I need access to a range of evaluation tools and paradigms So that I can choose the one fit for the problem
|
“You have a set of tools and paradigms that are going to be suitable for different types of problems. When you’re investigating different types of problems, for particular types of stakeholders, which particular types of objectives, or even particular types of technology, or particular stages of innovation, you have to be careful about finding the right thing and not trying to squeeze a round peg into a square hole.” |
| Contract assistant |
A way for PHE teams to create strategic relationships with suppliers, supported by forward-thinking contracts. A core requirement when working for PHE will be embedding evaluation into the design and development of the DHI and allowing for flexibility in contracted deliverables as the DHI progresses.
|
As a public health professional When setting up evaluations I need to get the data-sharing agreements in place as simply as possible So that I can collect, collate, and evaluate the data
|
“It’s setting a level of expectation...as part of their bids, they need to articulate how they’re going to capture data and how they’re going to evaluate. And we needed some commonality so that if we have three sites, we can compare across the three sites... aggregate all our data together.” |
| Testing toolkit |
Simple tools and methods to enable PHE teams delivering a DHI to test all aspects of the service or product throughout the development journey. The toolkit could include a guide for face-to-face research, approaches to and models for planning and prototyping, functionality for randomizing users, and digital solutions for validating propositions in the market and/or against existing services.
|
As a digital professional When deciding what to design and how to design it I need to validate service propositions by testing assumptions So that I can be confident in committing resources to develop them
|
“It’s fundamental to my role...I’m the one who manages [metric company name] and...also the research we conduct with users to define and validate services prior to committing resources developing them. But also to maximize them later, so to use informal or design-led evaluation means to validate, to research, to prove assumptions prior to designing things.” |
| Development history |
A tool for PHE to record the full development history of their DHI project. This will support decision-making, facilitate process evaluation, and enable successful assurance in line with the Government Digital Service pipeline approach. This record would include user needs, decisions and rationale, testing methods and results, information about the underlying technology, and stakeholder mapping.
|
|
“You need a really good understanding of how and why these things are working and what you're changing...which is often much more complex than people gather.” |
| Data hub |
PHE’s data hub provides access to high-quality, accessible, and understandable public health data for providers, academia, and the wider market. Similar to Transport for London’s Open Data, PHE will encourage developers to use PHE’s data feed to solve public health problems in innovative ways. By setting the standard for the metrics in the data hub and promoting collaboration, this data hub may, in the future, allow for longitudinal analysis of DHIs.
|
As an evaluator When evaluating a health product or service I need access to clean, accessible and linked data from across the health system So that I can do my evaluation
|
“A lot of their datasets are in silos, they're maybe not using data the most effectively.” |
| Publish health outcomes |
A place for stakeholders to publish and share how DHIs have met or not met desired health outcomes. This promotes collaboration among teams working in similar areas and enables sharing of best practices. Collaboration among PHE, public health professionals, academia, and suppliers working in digital public health aligns with the Government’s Industrial Strategy and the NHSe Innovation Strategy.
|
As a digital professional When working with PHE I need to understand clearly what clinical data is required So that I can be clear what success or impact looks like and can provide clinical impact
|
“We moved away from a lot of input and process measures, and balanced it with output and outcome measures, so we know now what impact they're having with individuals that they're working on, particularly their mental well-being, the escalation of demand into adult social care, and how they're embedding into a neighbourhood context more than they used to before.” |
