Skip to main content
. 2022 Dec 14;1(12):e0000164. doi: 10.1371/journal.pdig.0000164

Table 3. Common government processes utilized across our three cases of study.

Governance Processes Description Quotes
Contracts - Setting expectations between parties
- Mitigates risks
- Holds parties accountable for certain deliverables, on certain timelines
- Examples include master service agreements, end user license agreements, etc.
“I think every decision should be as evidence as informed as possible. Usually in crisis, you, you feel that that has to go…so don’t sign a ten year contract… put it on a, you know, six month contract with the option of renewal (Case 1-Interview-Chief Medical Information Officer)
“We have a master service agreement with [tech company C CEO] and he agreed in our contract … we could extend it to anyone in our Ontario Health Team, and they named all the partners … But then what we did, though, was he did sign separate contracts with each of those groups that were actually coming on to our instance” (Case 2-Interview–Chief Information Officer)
Documentation - Provides records of key decisions/events for parties to look back onto
- Supports building institutional knowledge
“We’re using SharePoint now … everything is on SharePoint and we you know we work on communal documents that way as well. But definitely you know documenting methods and having them in an easy to find spot for everyone to access is, is important. . .” (Case 3-Interview-Data Scientist)
Key Performance Indicators (KPIs) and Metrics - Varying processes done to measure progress towards a specific goal
- Measuring the success or lack of towards an organization’s objectives in a partnership
“In terms of developing KPIs I think it’s important to look at what the strategic plan of the, you know, hospital or the service or the long-term care sector … It’s important to look at the strategic plan. And what usually within the strategic plan, you’re able to see you know what is important … When you’re able to actually see what the [quality improvement and patient safety goals] are and how those line up with the project … it actually also gets different key stakeholders on the same page.” (Case 1-Interview-Clinical Champion)
“There’s going to be stakeholders [who] will… think you can help with reducing burnout and it’s not completely defined because how do you define burnout right so it’s like, I think you can reduce the amount of time that people are taking to reach other people which can reduce burnout but there’s not a set of concrete, measures…” (Case 2 –Interview -CEO)
Meetings - Many different styles and variations; ranges from formal presentations to stakeholder daily operational huddles
- Tool to ensure transparency and alignment throughout the partnership
- Provide updates between parties
- Informally, the “glue” that holds partnership together
“… some of its out of my hand. . . That being said, we’ll always develop as much as possible close relationship to my main contact with a client and we’ll have actually separate meetings just one on one, that are very transparent. Usually, especially if I have a good relationship they can be very blunt, and they’ll sit there and outline, maybe the priorities at the time or the issues at the time. And really what they need from us to do, to deal with it. And that’s exactly what we’ll do in terms of codifying it.” (Case 3-Interview-Client Success)
Privacy Assessment - Formal process done to determine how a program/service could affect the privacy of an individual
- Helps avoid the potential negative effects of privacy breach
“Once we got a few hospitals, that’s when you can connect the Privacy Officer at one hospital to another one and be like, oh yeah let’s share notes. Until then, it was a privacy document that we created with our external privacy firm and the credibility was incomplete there … you can always get the hospital do all a privacy impact assessment on you, but if you make them do the entire work, you can probably assume that’s a lot of resources so your sale is probably not gonna go through” (Case 1 –Interview–CEO)
“From a modeling point of view. I think for the most part you always want to have, as disaggregated data as possible because it allows you to do a lot more analysis at that level. Obviously, complete disaggregation comes with huge privacy and ethical concerns so that’s… always kind of taken off the table” (Case 3-Interview-Senior Epidemiologist)
Procurement - Formal process of finding and agreeing to terms and acquiring services from external sources
- Done through a tendering or competitive bidding process
- In Ontario, broader public sector guidelines exist when buying solutions over $100,000
“The fragmented purchasing that we see in Ontario means it’s very very difficult for us to grow here in Canada. . . I can only get procurement from one hospital at a time.. . .It’s a very, very long procurement process. And for me to get to meaningful scale here Ontario I’m gonna have to get a lot of hospitals to buy my products, it’s very hard for me to be a successful company here in Ontario … So, the way that we procure in Ontario certainly drive some of our emerging technology superstars in the healthcare space to go south of the border or to other countries versus to try to operate in our market." (Case 1-Interview-CEO)
“I appreciate why there is a broader public sector procurement rules in place, because often, you may miss out on maybe something that actually would achieve your needs better because you’re not actually considering all options, so that’s why there’s this forced procurement which allows for more open bidding and you may actually get better responses right… now again pros and cons to it…” (Case 2-Interview-Chief Information Officer)
“Well, I think the key thing in a time of crisis is like time is everything right. . . an RFP process is equitable and it will open it up but there is a period of creating the RFP there’s a period of putting it out there and giving people time to apply going through a review process, etc. So, I mean these are the trade-offs that you got to decide like do you have that time, and like what is the cost of creating a very systematic process when you’re in the middle of a crisis. . ." (Case 3-Interview-CEO)
Product Roadmap - Living document that maps out the strategy, deliverables, and goals for a product overtime
- Essential tool for prioritizing development of new features
- Aid in achieving alignment between stakeholders in partnership
“I worked with them to kind of say, “Okay, this is a product roadmap that that we think is important for you” to develop a product to meet our needs and I think they saw that that would be generally applicable to other customers in their in their market and so it was again a symbiotic relationship…” (Case 2 –Interview–Chief Medical Information Officer)
“There’s like tactical things like keeping [clients] aware of advancements to the product so like any new features that might be coming out to communicating more long-term vision … And also relaying that information back to us, if there is you know specific requests and so on.” (Case 3 –Interview–Product Manager)
Project Charters - Formal document that clearly outlines the project objective
- Also includes a project’s scope and responsibilities
- The importance of such a document has been met with mixed opinions
“…I believe it’s imperative… it has been my experience that you have to get into some of those details and you have to write them down and you have to look at them a few times and go over them and read them out loud together in order to really make sure that you’re teasing out any assumptions people may be making about what we’re going to achieve together.” (Case 1-Interview-Vice President)
“They’re almost traditional artifacts that brought peace of mind to the traditional old guard on how things are done. …you can get by without an official charter as long as you almost have like a, like a charter cheat sheet … like a one pager or some sort of living like mission statement or principles that you’re guiding the implementation … I still think charters are important and they’re ideal if you have the time to build a robust one … but by no means do I think that they’re, they’re mandatory.” (Case 1-Interview-Sales)
Risk Register - Formal document used as a risk management tool
- Done to fulfil regulatory compliance for all risks identified
- Includes information about each risk, the nature of the risk, and mitigation measures
“… the kind of the risk log or risk register something that’s just regularly reviewed and updated based on how the project is going so you can get ahead of things or mitigate the things as best as you can.” (Case 1-Interview-Sales)
Security Assessment - Formal process to test an organization’s security preparedness
- Checks for vulnerabilities within technology systems and business processes
- Can include recommendations to lower the risk of future attacks
"I’m talking about cybersecurity. Typically when you do these things you like to do threat risk assessments you like to do penetration tests and things like that. I’m given limited time. You may not always have the luxury of doing the things that you would like to do to ensure you know that level of due diligence. . . so what controls are in place to ensure that, for example, anyone who needs access to the system is access is granted on, you know, least, least privileged required to do their job. . . . ensuring that security has been implemented to align with, you know, hospital policy as an example. . . things like you know password policies is a simple example." (Case 1-Interview-Vice President)
Stakeholder Engagement - A process done to identify the needs of key stakeholders
- Includes the identification, analysis, planning, and implementation of actions in which key groups are present
“…community health groups or community groups, organizations they already exist… you don’t have to make up a new body. They already exist. They are already doing work. Go talk to them and it could be as simple as just having a phone call with them. You don’t need to set up like a complicated consultation body and process…you do need to make sure that those voices are embedded throughout which obviously demands some kind of governance… rethinking of those mechanisms, but yeah, they already exist—so use them.” (Case 3 –Interview- Policy Analyst)
Standards and Certifications - Certifying products to meet standards (e.g., ISO)
- Helps signal buyers and competitors the adherence to best practice
- Determined by independent third-party
“[Tech Company A software] is now listed as validated … for virtual care… and meets all of the provincial privacy, security, interoperability, and technical requirements…” (Case 1-Document)
“Security is … evaluated based on security standards … there’s not a healthcare regulation on security. So people [are] evaluated on like different, different ones so one of standards ISO 27001, [ISO] 27002 so evaluated based on that … so we hired a privacy firm and we also hired a security firm… to expedite our sales process.” (Case 2-Interview-CEO)