Table 4.

Suggestions for those considering a collaborative demonstration project in genomics

Suggestion	Illustration from the demonstration project
Collaboration and agreement
Secure support from the participating institutions leadership	The high level, strategic view of what was to be achieved was determined by the members’ executive leader. Financial contribution (AUD$250,000 per annum per institution) ensured that organisations were committed to succeeding.
Build trust by choosing a host institution that is perceived by all members neutral and enabling.	The host institution does not provide clinical care or conduct diagnostic testing, but does have procurement processes that supported rapid progress.
Consider which governance structure bests supports both implementation within the participating organisations and future sustainability, e.g a collaboration agreement or company structure	A collaboration agreement was chosen in order to better retain the members’ and stakeholders’ sense of ownership of the activities and minimise the administration required.
Appoint an independent chairperson for the project steering committee.	The independent chair actively fostered a collaborative culture in this committee and was a powerful voice externally.
Engage with representative users, from multiple institutions, across the entire investigation cycle	The clinicians and medical scientists from the collecting laboratories and the labs performing the tests, were involved in determining the workflows and were interviewed for the process evaluation.
Where feasible, ensure all organisations are contributing to the activities they wish to be involved in and don’t force anyone to participate	One laboratory decided to only sequence patients for the demonstration project, another to only curate and report results and a third provided end-to-end testing.
Design and project conduct
To ensure early successes and build momentum, consider using agile development approaches	The bioinformatic pipeline, for instance, was selected and operational within 6 months
Conduct activities in the context in which genomics will take place in the future and as close to the expected practice or protocol care as possible	Testing was performed by accredited laboratories, not research laboratories.
	Testing was undertaken in batches as they arrived at the laboratory, not as a cohort or grouped by clinical indication.
Engage users with varying levels of knowledge and expertise in genomic medicine step-by-step when planning an implementation. A system designed around only the most expert users may not work well in the real world.	The initial result report template was developed at a workshop which included geneticists, other medical disciplines and a community representative. The detail advocated by specialist geneticists was initially overwhelming for clinicians without experience in genomics.
Construct the project management team to include both experienced project managers and subject matter experts –	The project management team were largely subject matter experts, who understood the technical task at hand, but insufficient experienced project managers.
Consider competitive processes to determine which clinical indications will be tested.	We used a consensus approach due to time constraints. Subsequently we find a competitive process results in greater trust in the process and motivation by the participating clinicians.
Be prepared for varying levels of information technology sophistication between differing health services.	Research infrastructure provided an environment that allowed rapid deployment and nimble testing of proof of concept bioinformatic analysis, variant curation and research access tools. Distinction needed to be drawn between research drivers (which require novel, cutting edge approaches and flexibility) and the requirements of clinical systesms (accuracy, reliability and reproducibility),
Establish good natured competition between clinical groups to accelerate recruitment.	The number of patient tests available for each clinical indication was contingent on a half-way review of progress. Recruitment progress was circulated fortnightly.
Allow more time for every activity than you expect it will need, as it will be more complex than you expect.	A 1 year programme took 2 years to complete, with the delays largely due to recruitment, testing turn around time, and availability of data for evaluation (e.g cost data)
Outcomes
Measure benefits at two levels: (1) the benefits arising from genomic sequencing as evidence for future value and (2) the benefits arising from conduct of the demonstration project to determine the impact of the investment made by the funders.	(1) Evidence for the use of genomics: Cost effectiveness of exomes in comparison to usual care
	(2) Benefits from the demonstration project: funding from the State Government.
Support a clinician to conduct the evaluation of the impact of genomic sequencing.	The evaluations were most thorough when a clinician was highly motivated (e.g undertaking the work as part of a PhD) as we did not fund clinicians. Clinicians are now funded to coordinate activity for each clinical indication and conduct the evaluation.
Think broadly about the potential issues in implementation in the laboratory and clinic at the outset.	Consideration should have been given to how the design of the demonstration project could provide data for the accreditation process.
When evaluating process, focus on those that are relevant to practice	When interviewed, many stakeholders identified issues that related to the research study (e.g. research consent requirements) and not clinical care.