Clinical Team Functioning and IT Innovation: A Study of the Diffusion of a Point-of-care Online Evidence System

Abstract

Objectives: To investigate the association between clinical team functioning and diffusion (awareness, use, and impact) of a 24-hour online evidence retrieval system. To examine the relationships between clinical team characteristics and the adoption of the online evidence system.

Design: 18 clinical teams, consisting of 180 clinicians from three Australian hospitals, were identified and studied. Teams were categorized as small (≤ 15 members) or large (> 15).

Measurements: Clinical team functioning was assessed using the Team Climate Inventory (TCI). Awareness, use, and impact of an online evidence retrieval system were measured using a self-administered questionnaire. The relationships between TCI scores and awareness, use, and impact were examined using t-tests and one-way ANOVAs. Chi square analyses were used to examine differences between small and large teams. Results were interpreted within a diffusion of innovations framework.

Results: Clinical team functioning was not related to awareness or use of the online evidence retrieval system. However, clinical team functioning was significantly associated with the impact of online evidence in terms of reported experience of improved patient care following system use. Clinicians in small teams (≤ 15 members) had higher levels of system awareness compared to large (> 15) teams.

Conclusions: Team functioning had the greatest impact on the fourth stage of innovation diffusion, the effective use of online evidence for clinical care. This supports Rogers’ diffusion of innovation theory, to the effect that different types of communication about an innovation are important at different stages in the diffusion process. Members of small teams were more aware of the system than members of large teams. Team functioning is amenable to improvement through interventions. The findings suggest that the role of team climate in the diffusion of information systems is a promising area for future research.

Background

A growing body of empirical evidence demonstrates that team functioning impacts positively on healthcare processes and outcomes.¹ For example, primary health care teams that function well have been shown to reduce hospitalization rates,^2–4 and palliative care teams can improve outcomes for cancer patients (reduced hospitalizations, reduced costs).⁵ In the United Kingdom, patient mortality in 61 hospitals was found to be negatively associated with the proportion of hospital staff working in teams.⁶ In addition, good team functioning has a positive impact on team members. Clinicians working in teams report better mental health compared with those not working in teams, and individual mental health increases as team functioning improves.¹ Such findings have led to research aimed at identifying the attributes of good clinical teams and understanding aspects of team functioning that result in improved performance.

Factors shown to contribute to effective teamwork include team composition, size, and organizational factors. For instance, systematic reviews have concluded that multidisciplinary teams can deliver improvements in patient care compared with standard care in a range of settings and patient groups.^5,7,8 Optimal team size for effective communication has been reported to be 11–15 members; larger teams tend to fragment into sub-teams.⁹

Team Climate for Innovation

One of the hypothesized reasons for improved performance of some teams is their ability to be innovative. Innovative teams are characterized by high levels of support and challenge; team members feel able to share and implement new ideas and have a clear mutual perception of the tasks and objectives of the team.¹⁰ After studying 148 teams from a range of health care and industrial settings, Anderson and West^11,12 developed a measure of team functioning, the Team Climate Inventory (TCI). This tool measures the four factors that they found were related to team effectiveness and innovation:

• Participative safety: how participative the team is in its decision-making procedures and how psychologically safe team members feel it is to propose new and improved ways of doing things

• Support for innovation: the degree of practical support for innovation attempts contrasted against the rhetoric of professed support by senior management

• Vision: how clearly defined, shared, attainable and valued are the team’s objectives and vision

• Task orientation: the commitment of the team to achieve the highest possible standards of task performance, including the use of constructive progress monitoring procedures

Several studies have subsequently demonstrated the utility of the TCI as a way of examining health care teams’ support for innovation.^{1,10,11,13–15} Results have shown that effective teams, as judged by external raters, have higher TCI scores than poorly functioning teams. TCI scores are also related to an increased level of innovative practice and improved quality of care.^1,16 In a study of health service management teams, innovative practice was measured through tape recordings of team meetings over a 12-month period, from which a heterogeneous list of 125 innovations was derived. Experts rated these innovations using Damanpour’s¹⁷ typology of technical and administrative innovations on six dimensions of magnitude: radicalness, novelty, benefit to patient care, benefit to patients, benefits to staff well-being and benefits to administrative efficiency. The TCI was shown to have predictive validity, accounting for 45% of overall innovativeness and 42% of the variation in innovation magnitude.¹²

One of the most significant innovations to health care practice in recent years is the introduction of sophisticated clinical information and decision support systems provided at the point of clinical care. Examples include physician order entry systems and online clinical evidence retrieval systems. The relationship between clinical team functioning and the adoption and diffusion of such IT innovations has not been previously examined.

Diffusion of Innovation

The conceptualization of diffusion of innovation has developed over many years, with empirical research encompassing a diverse range of technologies, products, and processes. Diffusion of innovation can be seen as an orderly process, progressing through predictable stages, or as an unpredictable and chaotic process.^18,19 Rogers’ seminal work describes the “innovation-decision process” in five stages: knowledge, persuasion, decision, implementation, and confirmation. Van der Ven et al.¹⁸ describe the “innovation journey” as having three main periods: initiation, development, and implementation/termination.

Common to both models is considerable support for the importance of social networks in endorsing and enabling the diffusion of innovation.^18,19 Integral to this approach is the view that knowledge does not flow vertically or in structured ways; instead, it flows back and forth within complex networks and relationships. In health care, the importance of social networks in the diffusion of innovations has been a consistent finding.^20,21 Ash²⁰ posits that greater diffusion will occur in organizations that encourage communication. This suggests that clinical team functioning plays an important role in IT uptake and use because within health care organizations the clinical team is one of the most important organizational levels at which information is communicated and work is negotiated. This layer of the organizational structure is rarely examined explicitly as a possible diffusion vehicle, with most studies concentrating on either individual or organizational characteristics. Other psychological theories of change such as social learning theory²² or the stages of change model,²³ also concentrate only on how social interactions and cognitive processes influence individual change. Although the links between innovative practice and team processes are extensively discussed in West’s work,²⁴ the links to diffusion of innovation theory are not clearly delineated.

We sought to investigate the relationship between clinical team functioning as measured by the TCI and diffusion of a specific IT innovation, a 24-hour, point-of-care, publicly funded, online evidence retrieval system. The State Health Department in New South Wales^* (NSW), Australia implemented this online evidence system, called the Clinical Information Access Program (CIAP), in all public hospitals in 1997.²⁵ The online address is <http://www.ciap.health.nsw.gov.au>. The aim of CIAP is to improve the access of clinicians (doctors, nurses and allied health staff) to up-to-date evidence to support patient care decisions. The CIAP website provides access to a wide range of bibliographic and other resource databases (Table 1▶). Using a password, clinicians can access CIAP at work, with multiple terminals typically available on wards and in clinician offices, and at home. CIAP is available to approximately 55,000 clinicians across the state, yet there is considerable variation in the rates of use by individual hospitals.²⁶ Past studies have shown low rates of use of information resources, with physicians reporting more use than other clinical groups.^27–29 Technical factors such as access to, and speed of, computers explain only a proportion of the variation in CIAP uptake.³⁰

Table 1 .

Resources Available on CIAP

Bibliographic Databases	Other Resources
Medline	Harrison’s Online (textbook)
Psycinfo	Cochrane database
CINAHL	Micromedex
EMBASE	Interactive ECG tutorials
Healthstar	Full Therapeutic Guidelines
38 full text journals (including JAMA, BMJ, NEJM)	MIMS (pharmaceutical resource) + CMI (consumer medicines information)
	Australian Medicines Handbook

The aim of the current study was to determine at what point in the diffusion process team functioning influences clinicians’ awareness and use of the online evidence system. Specifically, we sought to examine the relationships between clinical team functioning and (1) clinicians’ awareness of the innovation, (2) clinicians’ use of the innovation, and (3) the effective use of the innovation to improve health care.

Methods

Sample

The TCI was administered to 18 teams from three hospitals. Two hospitals (bed numbers = 188 and 112) were from the same geographical rural area health service. The third was a specialist metropolitan hospital (bed number = 140). Teams were both single (n = 14 teams) and multidisciplinary teams (n = 4 teams), composed variously of nurses, doctors and allied health staff. One hundred and eighty clinicians completed the survey. Team size ranged from 4 to 40 clinicians. The proportion of team members completing the TCI ranged from 27% to 88%, with an average response rate of 61%. There was no association between mean TCI score and response rate (Pearson r < 0.2, p > 0.05, mean TCI score = 3.4).

Measures

The Team Climate Inventory

The TCI¹² was given to all team members. It contains 44-items. Some items are presented as statements, such as “The team is open and responsive to change.” Subjects respond using a 5-point Likert scale from “strongly agree” to “strongly disagree.” Other items are presented as questions, such as, “How clear are you about what your team objectives are?” The associated 5-point scale ranges from “not at all” to “completely.”

The TCI has four subscales corresponding to its four-factor structure. In addition, a social desirability sub-scale of six items checks for excessive impression management and faking by respondents. Scores on this subscale, for example, assist in detecting respondents who answer the inventory in order to portray their team favorably. Such social desirability scales are frequently included in validated survey tools and enable this factor to be controlled for during analysis. The subscales, described in the introduction, are as follows:

• Participative safety (12 items)

• Innovation (8 items)

• Vision (11 items)

• Task orientation (7 items)

Extensive reliability and validity tests of the TCI have been conducted using data from healthcare teams in Britain.¹¹ These tests demonstrated that the TCI has good reliability (sub-scale reliability correlations range from 0.73 to 0.95). Construct validity showed the four-factor structure was robust. Criterion validity was measured by comparing independently rated recordings of team meetings with TCI scores. Correlations between these measures were positively significant statistically. Predictive validity was measured by collating lists of innovations implemented over a 12-month period by a subsample of teams.

The Innovation and Online Evidence Survey

Team use of CIAP, a 24-hour, point-of-care, online evidence retrieval system, was measured using a self-administered questionnaire distributed to team members. This survey included items related to awareness of the system, individual use, and frequency of use (defined in three categories: not used in previous month, used less than once a week, used once a week or more). In addition, as a measure of the system’s impact on clinical practice, clinicians were asked whether they had had direct experience of system use resulting in improvements in patient care.

Procedure

Clinical teams were identified in conjunction with senior managers and team leaders in each hospital. Both multidisciplinary teams (e.g., brain injury team) and single disciplinary (e.g., nursing) teams were included. The TCI and online evidence surveys were distributed to all staff members at a team meeting or given to the team leader to distribute and collect in sealed envelopes, which were returned to the researchers.

Analysis

TCI scores were calculated for members of all teams for each of the four subscales. Items within each subscale were added to calculate the sub-scale score. For members of the same team, the within-group inter-rater agreement was calculated using the James, Dermaree, and Wolf³¹ r_wg statistic (as cited in West et al.¹⁶). One team fell just below the recommended 0.70 level of within group agreement on three subscales (scores ranged from 0.62 to 0.69), indicating a lower level of shared perception of team climate. Four other teams fell in the range at 0.64 to 0.69 on one of the four subscales. The TCI data were subjected to a factor analysis for comparison with the original published TCI data.¹² Since the factor structure was comparable and satisfactory, we proceeded to examine aggregated sub-scale scores for each team (i.e. team members’ mean scores).

As team size has been shown to influence team function, teams were classified as small (15 members or less) or large (> 15 members). Chi-square analyses, t-tests and one-way ANOVAs were used to compare small and large team members’ awareness and use of the online evidence system.

Results

Team Functioning and Awareness of the Innovation

Seventy percent (n = 126) of clinical staff in the study was aware of the online evidence system. Overall, no association was found between TCI score and awareness of the online evidence system (t = 0.02, p > 0.05). This result was consistent for both small and large teams. However, clinicians in small teams were significantly more likely to report being aware of the innovation compared with clinicians in large teams (χ² = 11.9, df = 1, p < 0.001) (Table 2▶).

Table 2 .

Number and Percentage of Staff Aware of the Online Evidence System (n = 180)

	Teams > 15 Members (n = 128	Teams ≤ 15 Members (n = 52
Aware of system (n = 126)	80 (63.5%)	46 (88.5%)
Not aware of system (n = 54)	48 (37.5%)	6 (11.5%)

Team Functioning and Use of the Innovation

Clinicians who were aware of the online evidence system were asked to indicate whether they had ever used the system, and frequency of use in the past month using a three-point scale. Seventy-nine percent (n = 99) of clinicians who were aware of the system had used it at least once, and 54% (n = 75) had used the system in the past month. Use was not significantly related to TCI score for the total sample (t=1.4, df=123, p>0.05), or for small or large teams (for teams ≤ 15 members, t= –0.4, df = 44, p > 0.05; for teams > 15 members, t = 1.3, df = 76, p > 0.05).

Among clinicians who had used the system, the frequency of use in the previous month was not significantly related to TCI total score (F = 1.4, df = 95, p > 0.05). However, as Table 3 ▶shows, there was a positive trend between TCI scores and frequency of online evidence use, with scores for all subscales increasing with increased frequency of system use. There was no difference in the frequency of use between members of small and large teams (χ² = 0.72, df = 2, p > 0.05).

Table 3 .

Results of ANOVAs Comparing TCI Subscale Scores by Frequency of Online System Use in the Past Month among Clinicians Aware of the System (n = 98)

		Mean TCI Factor Scores (SD)
Frequency of Use in the Past Month	Number of Respondents	TCI mean	Participatory Safety	Innovation	Vision	Task Orientation
Never in last month	22	3.3	3.52 (0.67)	3.39 (0.75)	3.27 (0.74)	3.35 (0.73)
1–3 times/month	45	3.5	3.65 (0.62)	3.44 (0.68)	3.43 (0.54)	3.49 (0.56)
> 3 times/month	30	3.5	3.78 (0.52)	3.50 (0.47)	3.59 (0.54)	3.50 (0.49)
F (df 2, 96)		1.4	1.1	0.21	1.9	0.5
P value		0.25	0.32	0.81	0.16	0.58

Team Functioning and Effective Use of the Innovation to Improve Patient Care

Respondents who were aware of the online evidence system were asked whether they had had direct experience of system use resulting in an improvement in patient care. Forty percent (n = 39) of clinician users reported they had such experience.

A significant positive association was found between TCI score and direct experience of improved patient care as a result of using the evidence system (t = 2.3, p < 0.02). As Table 4 ▶shows, the clinicians with direct experience of improved patient care had significantly higher TCI scores for the Participative Safety and Vision subscales compared with teams with no direct experience of improved care.

Table 4 .

Mean TCI Scores and Number of Clinicians with Direct Experience of Improved Patient Care following Use of the Online Evidence System (n = 98)

		Mean TCI Factor Scores (SD)
Direct Experience	Number of Respondents	TCI mean	Participatory Safety	Innovation	Vision	Task Orientation
Yes	39	3.59	3.82 (0.52)	3.57 (0.51)	3.64 (0.55)	3.55 (0.58)
No	59	3.36	3.56 (0.63)	3.37 (0.69)	3.32 (0.60)	3.39 (0.57)
t-test		2.3	2.1	1.6	2.6	1.3
df		95	96	97	97	97
P value		0.02*	0.04*	0.12	0.01†	0.16

*Significant at p < 0.05.

†Significant at p < 0.01.

The relationship between experience of improved care and TCI score did not reach statistical significance for small teams (Table 5▶). In large teams, Vision scores were significantly higher for clinicians who had experienced improvements in patient care (t = 2.4, df = 58, p < 0.02) (Table 6▶).

Table 5 .

Mean TCI Scores and Number of Clinicians in Small Teams (≤ 15 Members) with Direct Experience of Improved Patient Care Following Use of the Online Evidence System (n = 39)

		Mean TCI Factor Scores (SD)
Direct Experience	Number of Respondents	TCI mean	Participatory Safety	Innovation	Vision	Task Orientation
Yes	19	3.69	3.99 (0.47)	3.70 (0.44)	3.69 (0.44)	3.64 (0.49)
No	20	3.61	3.89 (0.61)	3.71 (0.61)	3.59 (0.58)	3.48 (0.64)
t-test		0.53	0.54	-0.08	0.65	0.87
df		37	37	37	37	37
P value		0.59	0.59	0.93	0.52	0.39

Table 6 .

Mean TCI Scores and Number of Clinicians in Large Teams (>15 members) with Direct Experience of Improved Patient Care Following Use of the Online Evidence System (n = 59)

		Mean TCI Factor Scores (SD)
Direct Experience	Number of Respondents	TCI mean	Participatory Safety	Innovation	Vision	Task Orientation
Yes	20	3.48	3.67 (0.53)	3.45 (0.55)	3.59 (0.64)	3.47 (0.65)
No	39	3.23	3.39 (0.58)	3.20 (0.67)	3.20 (0.58)	3.37 (0.53)
t-test		2.01	1.8	1.4	2.4	0.75
df		57	57	58	58	58
P value		0.05*	0.09	0.16	0.02*	0.46

*Significant at p < 0.05.

†Significant at p < 0.01.

Discussion

We found no relationship between clinical team functioning and team members’ awareness or use of the online evidence system. We did, however, find a significant association between team functioning and the effective use of the online evidence system in terms of reported experience of improved patient care following system use. Team members with experience of improved care reported higher total TCI scores and higher Participatory Safety and Vision subscale scores than those without such experiences.

Participatory Safety scores provide an indication of the level at which members of a team feel able to express new ideas, communicate effectively, and believe that they are part of the decision-making process. It is likely that within such environments clinicians will feel able to, and supported in, raising clinical questions, seeking out relevant evidence, and then using that information in clinical decision-making processes.

Vision scores reflect the extent to which team members believe that they have clear, defined, shared, attainable, and valued objectives and vision of what the team is trying to achieve. Thus we suggest that the significant relationship found between high Vision scores and direct experience of evidence retrieval resulting in improved patient care reflect the teams’ focus on a shared vision for using evidence to improve patient care.

Task orientation encompasses a diverse range of team functioning attributes, including critical appraisal, idea generation, and pursuit of excellence. These attributes seem to have less impact on the decision to use the evidence system than do other aspects of team functioning. West and Anderson¹⁶ commented that in their study of hospital management teams, task orientation only predicted the administrative effectiveness of innovations introduced, indicating that this sub-scale is capturing different aspects of team processes not applicable to the type of innovation being examined here.

Although our data demonstrated a trend for greater use of the online evidence system among teams with higher TCI scores this result was not statistically significant. Previous UK studies¹¹ have found team functioning to be significantly related to the adoption of innovative practices. These studies were different from the current research in that they examined a wide range of practices that were labeled as innovative. In the current study we focused on a single innovative practice, the use of an online evidence system. Defining our innovation in terms of a set of evidence-based medicine activities (e.g., use of online evidence, application of critical appraisal processes, team meetings to discuss available evidence related to a specific patient’s care) may have produced a different result. Thus, we hypothesize that the TCI may not be sensitive enough to discriminate innovative practice as represented by use of an IT system. However, it can be argued that it is not sensible to separate a clinical information system from the clinical processes that it is aimed to support (in our case, the use of an evidence-based approach to decision-making). For example, use of an online evidence system is not of itself innovative. The innovation encompasses the appraisal and application of the retrieved evidence to inform direct patient care decisions. Thus, we suggest that future research in this area use a wider definition of innovation that includes use of clinical information systems as an integral part of the clinical practices that they aim to innovate.

Members of small teams reported higher levels of awareness of the evidence system than members of large teams. This finding may be related to the effectiveness of their communication channels in disseminating information about new innovations. Communica-tion in larger teams may be more fragmented. High levels of interruptions, working practices such as shifts, staff shortages and use of temporary staff may impact on the integration and application of a new tool into clinical work. In the current study we had no data about the clinical loads of the participating teams. However, no evidence suggests that clinical load is a confounder in the relationship between TCI score and online evidence use. Imbalances in knowledge and technical skill, implicit professional hierarchies, and the consequent impact on collaborative work have been found to be detrimental to the effective use of technology.³⁴ Our findings support the view that higher levels of collaboration and team working result in more effective use of the technology. We did not have sufficient numbers of single and multidisciplinary teams to test for differences between these groups. This issue warrants attention in future studies.

Links with Diffusion Theory

Team functioning had the greatest impact on Rogers’ fourth stage of innovation diffusion, the implementation phase, which in our case was the effective use of online evidence for clinical care. This finding concurs with diffusion of innovation theory. Different types of communication about an innovation are important at different stages in the diffusion process. In the initial stages (becoming aware of an innovation) global, general information is most effective, whereas at the implementation and effective use stages, local information and encouragement are more effective. The clinical team could therefore be conceptualized as a localized diffusion channel,¹⁹ and the climate of the team may be a factor that determines the effectiveness of the localized diffusion.

Other theorists discuss the differential effects of additional factors at various stages of innovation diffusion. ^35,36The extent of awareness and initial uptake may be affected by differences in marketing, advertising, and availability of the system. Our previous studies have demonstrated that hospitals use different marketing strategies.³⁷ Clinicians report differences in the way and extent to which the online evidence has been promoted and accessibility to the system, and they describe various mechanisms via which they heard about the online evidence system.^37,38 Thus, it is likely that global information dissemination strategies have the greatest influence on levels of clinician awareness of the online system, explaining the absence of a relationship between team functioning and system awareness.

Implications for the Introduction of Information Systems into Health Care Organizations

Team functioning is amenable to improvement through interventions. Information systems are often introduced to clinical teams without examining the teams’ readiness to use the system. By examining team functioning, one potentially important aspect of readiness can be understood and interventions to improve the teams’ openness to innovation can be considered. Results from the TCI can be used to highlight specific areas for intervention.³⁹

Increasing our understanding of where the influence of the team lies is beneficial in designing plans for the introduction of new tools or practices. The successful application of a tool—in this case, the effective use of an online evidence system to improve patient care—was associated with team functioning. We believe that the findings suggest a promising area for future research—most notably, the role of the clinical team as a potentially important vehicle for innovation diffusion and effective use within the healthcare setting.

Conclusions

Team functioning is an area of research that has received little attention in relation to the introduction and effective use of IT innovations in the health care system. Our study showed that good clinical team functioning was associated with effective use of an online evidence system and thus suggests a new focus for those seeking to successfully implement clinical IT systems. Implemen-tation strategies aimed at clinical teams may be more effective than the standard organizational or professionally based approaches to IT implementation.

This study highlights the value of drawing on research findings from other disciplines in seeking to understand and evaluate how clinical information systems are diffused and applied successfully to improve health care. Organizational, psychological, and sociological research within the health care system has consistently identified the importance of factors such as hierarchy, professional subcultures, local communication networks, and clinical team functioning. Yet empirical research examining the role of these elements in relation to the adoption and effective use of IT has been relatively sparse.

We have pointed to several areas where further research on the role of team functioning in the diffusion and use of clinical information systems is warranted, including examination of differences between small and large teams and single versus multidisciplinary teams.