The I-DECIDED clinical decision-making tool for peripheral intravenous catheter assessment and safe removal: a clinimetric evaluation

Abstract

Objective

To describe the clinimetric validation of the I-DECIDED tool for peripheral intravenous catheter assessment and decision-making.

Design and setting

I-DECIDED is an eight-step tool derived from international vascular access guidelines into a structured mnemonic for device assessment and decision-making. The clinimetric evaluation process was conducted in three distinct phases.

Methods

Initial face validity was confirmed with a vascular access working group. Next, content validity testing was conducted via online survey with vascular access experts and clinicians from Australia, the UK, the USA and Canada. Finally, inter-rater reliability was conducted between 34 pairs of assessors for a total of 68 peripheral intravenous catheter (PIVC) assessments. Assessments were timed to ensure feasibility, and the second rater was blinded to the first’s findings. Content validity index (CVI), mean item-level CVI (I-CVI), internal consistency, mean proportion of agreement, observed and expected inter-rater agreements, and prevalence-adjusted bias-adjusted kappas (PABAK) were calculated. Ethics approvals were obtained from university and hospital ethics committees.

Results

The I-DECIDED tool demonstrated strong content validity among international vascular access experts (n=7; mean I-CVI=0.91; mean proportion of agreement=0.91) and clinicians (n=11; mean I-CVI=0.93; mean proportion of agreement=0.94), and high inter-rater reliability in seven adult medical-surgical wards of three Australian hospitals. Overall, inter-rater reliability was 87.13%, with PABAK for each principle ranging from 0.5882 (‘patient education’) to 1.0000 (‘document the decision’). Time to complete assessments averaged 2 min, and nurse-reported acceptability was high.

Conclusion

This is the first comprehensive, evidence-based, valid and reliable PIVC assessment and decision tool. We recommend studies to evaluate the outcome of implementing this tool in clinical practice.

Trial registration number

12617000067370

Strengths and limitations of this study.

• This is the first validation study of a comprehensive peripheral intravenous catheter assessment and decision tool.

• The I-DECIDED tool demonstrated strong content validity among a group of international vascular access experts and clinicians.

• The I-DECIDED tool demonstrated high inter-rater reliability in adult medical-surgical wards of three Australian hospitals.

• Studies to evaluate the outcome of implementation of this tool in clinical practice are warranted.

Introduction

With 70% of hospital patients needing a vascular access device (VAD) for medical treatment,¹ inadequate assessments may contribute to current poor outcomes, where up to 69% of peripheral intravenous catheters (PIVCs) have painful complications or stop working before treatment is finished, due to occlusion, dislodgement, infiltration or phlebitis.² Equally concerning, clinical audits reveal 25%–50% of PIVCs remain in situ for no reason.^3–5

Improved assessment could prompt removal of idle catheters and early detection of complications.⁶ To date, efforts to improve PIVC outcomes using phlebitis tools, care plans, maintenance bundles, electronic records and journey boards have achieved varied results.^{7 8} Supporting evidence for phlebitis tools is not robust, as they fail to consider complications such as dislodgement, occlusion or infiltration, and do not prompt assessment of device need, function, dressing integrity, securement and infection prevention strategies.^{7 9} With these items already included in best practice guidelines,^10–15 the reported high rates of idle catheters, device failure, and complications indicate the need for a fresh approach to PIVC assessment and management.

The I-DECIDED tool was developed to address the high prevalence of idle PIVCs and common shortfalls with assessment and documentation.¹⁶ This is the first comprehensive, evidence-based, point-of-care tool for PIVC assessment and decision-making. The tool guides clinicians to perform a structured assessment and make a decision, based on that assessment. Simple prompts accompany each category (see figure 1). This paper reports on the clinimetric properties (reliability, validity, acceptability and feasibility) of this tool.

Figure 1.

I-DECIDED IV assessment and decision tool.

Methods

Instrument

International guidelines were reviewed,^10–15 with core aspects assembled into the mnemonic, I-DECIDED, a structured priority matrix for assessment and decision-making. The name (I-DECIDED) conveys accountability for decisions based on the assessment and it has been translated into Latin-based languages while preserving the meaning to enable broader translation into practice.

Study design and setting

Face and content validity assessments were undertaken prior to an interrupted time-series (ITS) study to examine the effect of implementing the tool in three hospitals in Queensland, Australia.¹⁶ Inter-rater reliability was assessed at pre-specified time-points (Baseline; Implementation; Evaluation). Ethical approval was obtained from Griffith University (Ref No. 2017/152), Queensland Health (HREC/17/QPCH/47) and St Vincent’s Health and Aged Care Human Research and Ethics Committee (Ref No. 17/28). All participants provided informed consent prior to participation, and the study was conducted in accordance with the Australian Government National Statement on Ethical Conduct in Human Research.¹⁷ The results are reported in accordance with the Guidelines for Reporting Reliability and Agreement Studies.¹⁸

Sample size and data analysis

Face validity, a subjective assessment that the tool measures what it is designed to measure,¹⁹ was assessed in December 2015 by emailing a draft of the tool to eight members of a vascular access working group, all experienced Australian nurse researchers with solid knowledge of current evidence and guidelines. Reviewers independently assessed each item and the tool as a whole and provided recommendations. Following discussions between the lead author and reviewers, some item wording was revised.

Content validity, the degree to which the content of an instrument is an adequate reflection of the construct to be measured,¹⁹ of each principle and corresponding items was undertaken with international experts (vascular access researchers and infection control professionals who had contributed to the most recent evidence-based vascular access guidelines) and experienced clinicians (nurses with weekly PIVC experience) to determine if the tool covered the essentials of PIVC assessment and decision-making. We deliberately targeted experts and clinicians separately to identify any differences between perspectives. During June–July 2017, the content validity surveys were emailed to male and female respondents with diverse expertise and skills, from a range of English-speaking countries. In all, 22 experts and 25 clinicians from adult and paediatric specialties in the authors’ clinical networks were informed of the study by the lead author by email and invited to complete the content validity questionnaire via online survey (research electronic data capture)²⁰ or paper form and return email (See online supplementary appendix 1). Survey completion was accepted as consent and identifying details of respondents were not collected.

Respondents rated each item in terms of its relevance to the underlying construct on a 4-point ordinal scale (1=not relevant, 2=somewhat relevant, 3=quite relevant, 4=highly relevant).²¹ The item-level content validity index (I-CVI) was calculated for each principle and item (number of respondents giving a rating of either three or 4, divided by the total number of respondents).²² Content validity index (CVI) for each item and overall mean I-CVI were calculated for both expert and clinician groups. Proportions of agreement for each participant, each item, and overall mean were calculated. Respondents were asked to review, comment and suggest changes on wording and structure of each section of the tool, and the tool as a whole. Respondents could participate in a Skype or telephone call with the lead author to provide further feedback, if desired. All written and verbal feedback was analysed, and minor wording revisions were made to produce the final tool.

Reliability is the proportion of total variance in the measurements that are due to ‘true’ differences between subjects.¹⁹ Inter-rater reliability is the ratio of variability between subjects to the total variability of all measurements in the sample.¹⁸ Inter-rater reliability was evaluated in three phases. In August 2017 (phase 1—Baseline), the lead author provided education on the tool to a research nurse at each hospital (registered nurses with ≥10 years’ clinical experience). The lead author and research nurses undertook 10 paired PIVC assessments to assess inter-rater reliability; this ensured the research nurses thoroughly understood the tool prior to collecting baseline data for the ITS study. Four months later, in phase 2 (Implementation), the tool and new VAD form (available in the protocol paper¹⁶ were rolled out across the participating wards. In February 2018, the lead author and research nurses undertook a further nine paired PIVC assessments to confirm continued consistency when using the tool. In April 2018 (phase 3—Evaluation), after hospital nurses had used the tool for 2 months, inter-rater reliability was evaluated between the research nurses and a convenience sample of three to six staff nurses (male and female, aged 25–60) at each hospital for a further 15 paired PIVC assessments. The number of participants available for each inter-rater reliability assessment depended on how many nurses had patients with a PIVC in situ at the time of the assessment. Each staff nurse only participated in one inter-rater reliability assessment. All patients and staff nurses provided verbal consent to participate in the assessments. In all, 34 paired assessments were undertaken for a total of 68 assessments. For each assessment, two assessors independently assessed the PIVC 5 min apart using the tool, ranking each item as a categorical binary response (yes/no). The second rater was blinded to the first’s findings, and the order of subjects varied between assessors to prevent systematic bias. Staff nurses were unaware that their judgement would be compared with other raters, to remove the possibility of a Hawthorne effect.¹⁸ Cronbach’s coefficient α was used to calculate the internal consistency of the items in the tool. To assess inter-rater variation, observed and expected agreements for each part of the tool, prevalence-adjusted bias-adjusted kappa (PABAK) and overall proportion of agreement were calculated.²³ When prevalence of a given response is very high or low, the kappa value may not be reliable, even when the observed proportion of agreement is quite high; therefore, we calculated the PABAK to more fully characterise the extent of inter-rater reliability between two raters.²³ Standard errors of measurement and Z scores were also calculated.

To assess Principles 1 (Identify presence of device) and 2 (Does patient need the device), raters checked for the presence of a PIVC and checked the patient’s chart for current orders; if none were present, the observers asked the patient’s nurse if any procedures were planned. For Principle 3 (Effective function), raters asked the patient if an infusion or flush had been administered in the past 12 hours, and if so, had there been any concerns. To assess Principle 4 (Complications), raters asked the patient about pain or tenderness and inspected the PIVC insertion site for signs and symptoms. With Principle 5 (Infection prevention), raters asked the patient if they had observed the nurse perform hand hygiene before touching the PIVC and scrub the needleless connector hub before administering IV medications or fluids. To assess Principle 6 (Dressing and securement), raters assessed the PIVC dressing for cleanliness and integrity and securement of the PIVC or administration set. For Principle 7 (Evaluate and Educate), raters asked the patient if they had questions and if the nurse had provided any education about the PIVC. To assess Principle 8 (Document), raters checked the patient chart for documentation of PIVC assessment in the past 12 hours. To assess Principle 9 (Decision), raters asked the patient if they knew of any plans for the PIVC that day and checked the patient’s chart for evidence of plans to remove or continue the PIVC.

Feasibility was assessed by timing inter-rater reliability assessments and by asking staff about the clarity of items and ease of completion of the tool. Acceptability of introducing the tool into practice was assessed with 30 registered nurses who participated in round table discussions at each hospital prior to the study. During these sessions, nurses discussed the terminology of the tool and provided feedback on the proposed VAD form. Suggestions were taken into consideration and minor sections of the care plan (shading, location of comments section) were modified prior to roll-out. Focus groups with staff nurses regarding PIVC assessment were undertaken prior to the roll out of the tool and at the end of the trial (Results of the focus groups are reported elsewhere).

Patient and public involvement

The I-DECIDED tool incorporates a prompt to evaluate patients’ (and family, if appropriate) knowledge and concerns about their PIVC and to provide education, as needed. This prompt was included after recent research revealed consumers wanted to be included in conversations about the management of their VADs.^{24 25} Specific patient advisers were not consulted for this study.

Results

Content validity

Complete responses for the content validity questionnaire were available for 7 (32%) experts and 11 (44%) clinicians from Australia, UK, the USA and Canada. Two experts (the UK and the USA) and one clinician (the USA) (all women) participated in a 30 min, one-to-one call with the lead author. These discussions focused on clarifying the recommended frequency of assessment, in particular with different nursing shift lengths, and discussions about nursing responsibility for vascular access decisions, which vary between hospitals and countries.

For vascular access experts, the mean CVI for the principles of the tool was 0.87 (range 0.29–1.00), and the mean I-CVI for all items of the tool was 0.91 (range 0.57–1.00). The mean proportion of agreement was 0.91 (range 0.83–0.98) (see table 1)

Table 1.

Ratings on a 48-item scale by seven vascular access experts: items rated three or four on a four-point relevance scale

Item	Description	E1	E2	E3	E4	E5	E6	E7	Number in agreement	Item CVI
1	Key principle: the presence of an IV device should be assessed each shift	✓	✓	✓	✓	✓	✓	✓	7	1.00
2	Does the patient have an IV device? (Inspect the patient and ask the patient if unsure)	✓	✓	✓	✓	✓	✓	✓	7		1.00
3	Has the patient had an IV device removed in the past 48 hours? (Ask the patient)	✓	✓	✓	✓	✓	✓	✓	7		1.00
4	If the patient has had an IV device removed in the past 48 hours, observe site for complications (post-infusion phlebitis and purulence).	✓	✓	✓	✓	✓	✓	✓	7		1.00
5	Key principle: the need for the IV device should be assessed each shift	✓	✓	✓	✓	✓	✓	✓	7	1.00
6	Has the IV device been used in the past 24 hours, or is it likely to be used in the next 24 hours?	✓	✓	✓	✓	✓	✓	✓	7		1.00
7	Can the patient switch to oral medications? Discuss with pharmacist and treating team.	✓	✓	✓	✓	✓	✓	✓	7		1.00
8	When no longer needed, the IV device should be removed.	✓	✓	✓	✓	✓	✓	✓	7		1.00
9	Key principle: effective flow and flush of the IV device should be assessed each shift	✓	✓	✓	✓	✓	–	✓	6	0.86
10	Does the IV device flow well?	✓	✓	✓	–	✓	✓	✓	6		0.86
11	Does the IV device flush well?	✓	✓	✓	–	✓	–	✓	5		0.71
12	If the IV device does not flow and flush, it should be removed.	✓	✓	✓	✓	✓	–	✓	6		0.86
13	Key principle: the IV site should be assessed for complications or concerns each shift	✓	✓	✓	✓	✓	✓	✓	7	1.00
14	Patient-reported pain ≥2 out of 10?	✓	✓	✓	✓	✓	✓	✓	7		1.00
15	Redness >1 cm from insertion site	✓	✓	✓	✓	✓	✓	✓	7		1.00
16	Swelling >1 cm from insertion site	✓	✓	✓	✓	✓	✓	✓	7		1.00
17	Any discharge at site	✓	✓	✓	✓	✓	✓	✓	7		1.00
18	Infiltration (IV fluid in surrounding tissues)	✓	✓	✓	✓	✓	✓	✓	7		1.00
19	Hardness (induration) of insertion site	✓	✓	✓	✓	✓	✓	✓	7		1.00
20	Palpable cord	✓	✓	✓	✓	✓	✓	✓	7		1.00
21	Other concerns? (itch, rash, blistering, etc)	✓	✓	✓	✓	✓	✓	✓	7		1.00
22	If complications occur, the IV device should be removed, after consultation with the treating team. Insert new IV device if needed	✓	✓	✓	✓	✓	✓	✓	7		1.00
23	Key principle: infection prevention and control practices should be performed each shift	✓	–	–	✓	✓	✓	✓	5	0.71
24	Use aseptic non-touch technique	✓	✓	✓	✓	✓	✓	✓	7		1.00
25	Hand hygiene	✓	✓	✓	✓	✓	✓	✓	7		1.00
26	Scrub the hub as per protocol and allow to dry before accessing IV device	✓	✓	✓	✓	✓	✓	✓	7		1.00
27	Any fever of unknown origin?	✓	✓	✓	–	✓	✓	–	5		0.71
28	Elevated white blood cell count?	✓	✓	✓	–	✓	✓	–	5		0.71
29	If the patient has a fever and/or elevated white blood cell count, with no obvious source of infection, the IV device should be removed and the IV site cultured as a possible source of bloodstream infection	✓	✓	✓	✓	✓	✓	–	6		0.86
30	Purulent discharge at the insertion site?	–	✓	✓	✓	✓	✓	✓	6		0.86
31	If the IV site has purulent discharge, the IV device should be removed and the IV site cultured as a possible source of bloodstream infection	✓	✓	✓	✓	✓	✓	✓	7		1.00
32	Key principle: dressing and securement practice should be assessed each shift	✓	✓	✓	✓	✓	✓	✓	7	1.00
33	Is the IV dressing clean, dry and intact?	✓	✓	✓	✓	✓	✓	✓	7		1.00
34	If the IV dressing is moist, visibly soiled or has loose/lifting edges, it should be changed	✓	✓	✓	✓	✓	✓	✓	7		1.00
35	Is the IV device and infusion tubing secured?	✓	✓	✓	✓	✓	✓	✓	7		1.00
36	Secure well with securement device, tape, net or bandage	✓	✓	✓	–	✓	✓	✓	6		0.86
37	Key principle: the patient/family’s knowledge and education needs should be assessed each shift, if possible	–	–	✓	–	–	–	✓	2	0.29
38	Evaluate patient/family understanding of reason for IV and plan for removal, if possible.	✓	–	✓	–	✓	–	✓	4		0.57
39	Educate patient/family as needed, if possible.	✓	–	✓	–	✓	✓	✓	5		0.71
40	Key principle: the IV assessment and actions taken should be documented each shift	✓	✓	✓	✓	✓	✓	✓	7	1.00
41	Insertion date and time	✓	✓	✓	✓	✓	–	✓	6		0.86
42	I-DECIDED assessment and relevant action taken	✓	✓	✓	✓	✓	–	✓	6		0.86
43	Removal date and time	✓	✓	✓	✓	✓	✓	✓	7		1.00
44	Key principle: the decision to continue or remove the IV device should be based on assessment and consultation with the treating team and the patient	✓	✓	✓	✓	✓	✓	✓	7	1.00
45	Decision 1. IV device should remain in place. No other change	✓	✓	✓	✓	✓	✓	–	6		0.86
46	Decision 2. IV device should remain in place, but dressing change done. IV and infusion tubing well secured	✓	✓	✓	✓	✓	✓	–	6		0.86
47	Decision 3. IV device removed and not replaced, in consultation with the treating team	✓	✓	✓	✓	✓	✓	–	6		0.86
48	Decision 4. IV device removed and replaced. Consulted with patient and team about best device and site	✓	✓	✓	✓	✓	✓	–	6		0.86
Proportion relevant		0.96	0.92	0.98	0.83	0.98	0.85	0.85		0.87 (mean)	0 .91 (mean)
									Mean expert proportion=0.91

CVI, content validity index; E, vascular access expert; IV, intravenous.

For experienced clinicians, the mean CVI for the principles of the tool was 0.96 (range 0.82–1.00), and the mean I-CVI for all items of the tool was 0.93 (range 0.55–1.00). The mean proportion of agreement was 0.94 (range 0.65–1.00) (see table 2). The content validity questionnaire elicited comments, which are summarised here. The complete list of responses is provided in online supplementary appendix 2.

Table 2.

Ratings on a 48-item scale by 11 experienced clinicians: items rated three or four on a four-point relevance scale

Item	Description	C1	C2	C3	C4	C5	C6	C7	C8	C9	C10	C11	Number in agreement	Item CVI
1	Key principle: the presence of an IV device should be assessed each shift	✓	✓	✓	✓	✓	✓	✓	✓	✓	✓	✓	11	1.00
2	Does the patient have an IV device? (Inspect the patient and ask the patient if unsure)	✓	✓	✓	✓	✓	✓	✓	✓	✓	✓	✓	11		1.00
3	Has the patient had an IV device removed in the past 48 hours? (Ask the patient)	✓	✓	✓	✓	✓	–	✓	✓	✓	✓	–	9		0.82
4	If the patient has had an IV device removed in the past 48 hours, observe site for complications (post-infusion phlebitis and purulence)	✓	✓	✓	✓	✓	–	✓	✓	✓	✓	✓	10		0.91
5	Key principle: the need for the IV device should be assessed each shift	✓	✓	✓	✓	✓	✓	✓	✓	✓	✓	–	10	0.91
6	Has the IV device been used in the past 24 hours, or is it likely to be used in the next 24 hours?	✓	✓	✓	✓	✓	✓	✓	✓	✓	✓	✓	11		1.00
7	Can the patient switch to oral medications? Discuss with pharmacist and treating team	✓	✓	✓	✓	✓	✓	✓	✓	✓	✓	–	10		0.91
8	When no longer needed, the IV device should be removed	✓	✓	✓	✓	✓	✓	✓	✓	✓	✓	✓	11		1.00
9	Key principle: effective flow and flush of the IV device should be assessed each shift	✓	✓	✓	✓	✓	✓	✓	✓	✓	✓	✓	11	1.00
10	Does the IV device flow well?	✓	✓	✓	✓	✓	✓	–	✓	✓	✓	–	9		0.82
11	Does the IV device flush well?	✓	✓	✓	✓	✓	✓	✓	✓	✓	✓	–	10		0.91
12	If the IV device does not flow and flush, it should be removed	–	✓	✓	✓	✓	✓	✓	✓	✓	✓	–	9		0.82
13	Key principle: the IV site should be assessed for complications or concerns each shift	✓	✓	✓	✓	✓	✓	✓	✓	✓	✓	✓	11	1.00
14	Patient-reported pain ≥2 out of 10?	✓	✓	✓	✓	✓	✓	✓	✓	✓	✓	–	10		0.91
15	Redness >1 cm from insertion site	✓	✓	✓	✓	✓	✓	✓	✓	✓	✓	✓	11		1.00
16	Swelling >1 cm from insertion site	✓	✓	✓	✓	✓	✓	✓	✓	✓	✓	✓	11		1.00
17	Any discharge at site	✓	✓	✓	✓	✓	✓	✓	✓	✓	✓	✓	11		1.00
18	Infiltration (IV fluid in surrounding tissues)	✓	✓	✓	✓	✓	✓	✓	✓	✓	✓	✓	11		1.00
19	Hardness (induration) of insertion site	✓	✓	✓	✓	✓	✓	✓	✓	✓	✓	✓	11		1.00
20	Palpable cord	✓	✓	✓	✓	✓	✓	✓	✓	✓	✓	–	10		0.91
21	Other concerns? (itch, rash, blistering, etc.)	✓	✓	✓	✓	✓	✓	✓	✓	✓	✓	✓	11		1.00
22	If complications occur, the IV device should be removed, after consultation with the treating team. Insert new IV device if needed	✓	✓	✓	✓	✓	✓	✓	✓	✓	✓	✓	11		1.00
23	Key principle: infection prevention and control practices should be performed each shift	✓	✓	✓	✓	✓	✓	✓	✓	✓	✓	✓	11	1.00
24	Use aseptic non-touch technique	✓	✓	✓	✓	✓	✓	✓	✓	✓	✓	✓	11		1.00
25	Hand hygiene	✓	✓	✓	✓	✓	✓	✓	✓	✓	✓	✓	11		1.00
26	Scrub the hub as per protocol and allow to dry before accessing IV device	✓	✓	✓	✓	✓	✓	✓	✓	✓	✓	✓	11		1.00
27	Any fever of unknown origin?	–	✓	✓	✓	✓	✓	✓	✓	–	✓	–	8		0.73
28	Elevated white blood cell count?	–	✓	✓	✓	✓	–	–	✓	–	✓	–	6		0.55
29	If the patient has a fever and/or elevated white blood cell count, with no obvious source of infection, the IV device should be removed and the IV site cultured as a possible source of bloodstream infection	–	✓	✓	✓	✓	✓	✓	✓	✓	✓	✓	10		0.91
30	Purulent discharge at the insertion site?	P	✓	✓	✓	✓	✓	✓	✓	✓	✓	–	10		0.91
31	If the IV site has purulent discharge, the IV device should be removed and the IV site cultured as a possible source of bloodstream infection	–	✓	–	✓	✓	✓	–	✓	✓	✓	–	7		0.64
32	Key principle: dressing and securement practice should be assessed each shift	P	✓	✓	✓	✓	✓	✓	✓	✓	✓	✓	11	1.00
33	Is the IV dressing clean, dry, and intact?	✓	✓	✓	✓	✓	✓	✓	✓	✓	✓	✓	11		1.00
34	If the IV dressing is moist, visibly soiled or has loose/lifting edges, it should be changed	✓	✓	✓	✓	✓	✓	✓	✓	✓	✓	✓	11		1.00
35	Is the IV device and infusion tubing secured?	✓	✓	✓	✓	✓	✓	✓	✓	✓	✓	✓	11		1.00
36	Secure well with securement device, tape net or bandage.	✓	✓	✓	✓	✓	✓	✓	✓	✓	✓	–	10		0.91
37	Key principle: the patient/family’s knowledge and education needs should be assessed each shift, if possible	✓	✓	✓	✓	✓	✓	–	✓	✓	✓	–	9	0.82
38	Evaluate patient/family understanding of reason for IV and plan for removal, if possible	✓	✓	✓	✓	✓	✓	–	✓	–	✓	–	8		0.73
39	Educate patient/family as needed, if possible	✓	✓	✓	✓	✓	✓	✓	✓	✓	✓	–	10		0.91
40	Key principle: the IV assessment and actions taken should be documented each shift	✓	✓	✓	✓	✓	✓	✓	✓	✓	✓	✓	11	1.00
41	Insertion date and time	✓	✓	✓	✓	✓	✓	✓	✓	✓	✓	✓	11		1.00
42	I-DECIDED assessment and relevant action taken	✓	✓	✓	✓	✓	✓	✓	✓	✓	✓	✓	11		1.00
43	Removal date and time	✓	✓	✓	✓	✓	✓	✓	✓	✓	✓	✓	11		1.00
44	Key principle: the decision to continue or remove the IV device should be based on assessment and consultation with the treating team and the patient	✓	✓	✓	✓	✓	✓	✓	✓	✓	✓	–	10	0.91
45	Decision 1. IV device should remain in place. No other change	✓	✓	–	✓	✓	✓	✓	✓	✓	✓	✓	10		0.91
46	Decision 2. IV device should remain in place, but dressing change done. IV and infusion tubing well secured	✓	✓	✓	✓	✓	✓	✓	✓	✓	✓	✓	11		1.00
47	Decision 3. IV device removed and not replaced, in consultation with the treating team	✓	✓	✓	✓	✓	✓	✓	✓	✓	✓	✓	11		1.00
48	Decision 4. IV device removed and replaced. Consulted with patient and team about best device and site	✓	✓	✓	✓	✓	✓	✓	✓	✓	✓	✓	11		1.00
Proportion relevant		0.90	1.00	0.96	1.00	1.00	0.94	0.90	1.00	0.94	1.00	0.65		0.96 (mean)	0.93 (mean)
													Mean clinician proportion 0.94

C, clinician; CVI, content validity index; IV, intravenous.

Principle 1: the presence of an IV device should be assessed each shift

All 18 respondents agreed. The prompt to assess for post-infusion phlebitis invoked five comments, with most respondents agreeing that assessing for post-infusion phlebitis is important but can be difficult if patients have communication difficulties (eg, stroke, capacity to understand, or capacity to give feedback) and is not possible after patient discharge.

Principle 2: the need for the IV device should be assessed each shift

Seventeen respondents agreed; however, one respondent commented that assessing PIVC need each shift was unrealistic and discussing changing to oral medications with the pharmacist and treating team raised workload concerns. Two respondents debated frequency of PIVC assessment, remarking that ‘each shift’ was unclear because shift length can vary according to the unit. One respondent noted that the Infusion Nurses Society Standards of Practice¹¹ call for daily assessment of need, rather than each shift.

Principle 3: effective flow and flush of the IV device should be assessed each shift

Seventeen respondents agreed, and 11 respondents offered diverse questions and opinions. Several argued that ‘flow and flush’ were subjective assessments and insufficient to determine PIVC function without first checking for obstruction. Flushing frequency was debated, and two respondents recommended adding ‘aspiration for blood return’. In response to this feedback, the wording was changed to ‘Effective function’.

Principle 4: the IV site should be assessed for complications or concerns each shift

All 18 respondents agreed with prompts to assess pain, redness, swelling, discharge, infiltration, extravasation, hardness or purulence. One respondent stated that palpable cord should not be included. Another said that this prompt contained too many signs and symptoms, many of which could be too subjective or difficult for the nurse to remember. Respondents’ comments varied regarding determining pain scores at the PIVC site. One respondent said a pain score of 1 with associated redness and swelling would be a valid reason to remove the PIVC; another respondent stated pain would not be addressed unless the pain score was greater than 5; yet, another recommended the question should prompt the nurse to identify the cause of the pain, rather than rely on a numerical score.

Principle 5: infection prevention and control practices should be performed each shift

In all, 16 respondents concurred; two experts disagreed with the principle but agreed with all the supporting prompts. Five respondents argued the inclusion of fever and elevated white cell count was inappropriate, as neither would prompt PIVC removal in most cases; one respondent argued that diagnosis of infection would be a team responsibility rather than nursing. A Skype respondent expressed concern that a nurse might identify the PIVC as a possible source of infection, which could lead to financial penalties in some health services. One respondent stated ‘purulent drainage’ fit better with the principle ‘complications’ and the infection section should focus on identifying signs of sepsis. Two respondents felt aseptic non-touch technique should be removed because it was not taught at every hospital.

Principle 6: dressing and securement practice should be assessed each shift

All 18 respondents agreed. Four respondents noted this prompt could be made clearer by requiring that the PIVC site remain visible for ease of inspection; however, the wording of this section was not changed because the guidelines accept either transparent or sterile gauze and tape dressings.¹³ Four respondents requested the prompts should specify exactly what should be secured (PIVC or administration set or both).

Principle 7: the patient/family’s knowledge and education needs should be assessed each shift, if possible

In all, 11 respondents supported this principle. Nine clinicians agreed that patient concerns about the PIVC were important to assess each shift, but only two experts felt this was relevant to include in the tool; five experts expressed concern that assessing patient knowledge needs each shift would be too frequent. Six respondents did not agree it was relevant to evaluate the patient’s and/or family’s understanding of the reason for the PIVC and plans for its removal.

Principle 8: the IV assessment and actions taken should be documented each shift

All 18 respondents agreed. One respondent stated that the documentation should include more details (eg, exact site of insertion, gauge size). Another commented that the tool would need to include more frequent prompts for paediatric PIVC assessment. A further suggestion was to include a prompt to replace PIVCs inserted in an emergency where asepsis could have been compromised.

Principle 9: the decision to continue or remove the IV device should be based on assessment and consultation with the treating team and the patient

In all, 17 respondents agreed: however, one respondent noted PIVC removal must comply with local institutional policy, rather than a nurse’s decision. Two respondents stated it would not be necessary to consult with the treating team before removing the PIVC if the nurse identified complications, as PIVC assessment is a nursing responsibility and nurses have the necessary skills and knowledge to make their own informed decisions in this area. This point was also raised in the Skype/telephone calls. Following this feedback, a clause was added: ‘Always consider local policy and consult with team and patient as required’.

Reliability

From 34 paired assessments, item-level proportion of inter-rater agreement ranged from 79.41% (patient education) to 100% (documentation of the decision) (See table 3). Overall Cronbach’s alpha was 0.746 and proportion of inter-rater agreement was 87.13%. Using the Landis and Koch²⁶ categorisation, the kappa values for each item of the tool were all in the substantial (0.61–0.80) range, except for ‘Identify if patient has a PIVC’ and ‘Document your decision’, which both scored almost perfect (0.81–1.00) and ‘Evaluate and Educate’, which scored in the moderate (0.41–0.60) range.

Table 3.

Inter-Rater reliability of I-DECIDED tool

	Observed agreement (%)*	PABAK	Cronbach’s alpha if item deleted	SE	Z	Prob>Z
Identify if patient has PIVC	97.06	0.9412	0.742	0.1712	5.50	0.0000
Does patient need PIVC	88.24	0.7647	0.673	0.1715	4.46	0.0000
Effective function of PIVC	85.29	0.7059	0.775	0.1712	4.12	0.0000
Complications at PIVC site	82.35	0.6471	0.699	0.1715	3.77	0.0001
Infection prevention	82.35	0.6471	0.716	0.1715	3.77	0.0001
Dressing and securement	82.35	0.6471	0.656	0.1715	3.77	0.0001
Evaluate and educate	79.41	0.5882	0.718	0.1712	3.44	0.0003
Document your decision	100.0	1.0000	–	0.1715	5.83	0.0000
Overall	87.13		0.746

*Expected agreement 50% for all items.

PABAK, prevalence-adjusted bias-adjusted kappa; PIVC, peripheral intravenous catheter.

Feasibility

During inter-rater reliability testing, the time to conduct each assessment ranged from 1 to 10 min (average 2 min). Longer assessments occurred when patients had questions about their PIVC or if troubleshooting the PIVC was required.

Acceptability

Although 25 education sessions were attended by 180 staff over three hospitals in phase 2, it was not possible to provide education to all staff at each site. Education was provided to all nurse unit managers, nurse educators and clinical facilitators, as well as many registered and enrolled nurses, physicians and administrative staff. Posters were displayed in staff tearooms and nurses’ stations, and lanyard cards were provided for all staff. During phase 3 focus groups, the lead author asked attendees if they had received instructions how to use the tool. There was no discernible difference in feedback between staff who had or had not received education. Consensus was that the tool was easy to follow and particularly useful for newly registered nurses and nursing students. The structured format for PIVC assessment was popular, but many disliked the added paperwork. Following the inter-rater assessments, the lead author asked nurses if they had attended an education session, and if not, how did they learn to use the tool. Approximately half of the nurses who participated in the inter-rater assessments had not received any formal education about the tool; they reported that they had either asked a colleague about it or that it was self-evident.

Discussion

This paper describes the clinimetric properties of the I-DECIDED tool for PIVC assessment in an inpatient population. The tool demonstrated strong content validity for adults and paediatrics among vascular experts and clinicians, and high inter-rater reliability, feasibility and acceptability in the adult medical-surgical wards of three Australian hospitals. As this is the first comprehensive, evidence-based tool for PIVC assessment and decision-making, the authors expect this will interest clinicians across inpatient settings.

The strength of this study was that content validity of the tool was confirmed by 18 vascular access experts and clinicians from a range of English-speaking countries. Lynn²⁷ advocated item-level CVI should be around 0.80 when there are six or more experts. The mean CVI and proportion of agreement for the principles and the individual items of the tool scored very highly for both experts (I-CVI 0.91; mean proportion of agreement 0.91) and experienced clinicians (I-CVI 0.93; mean proportion of agreement 0.94), confirming that this tool comprises the essentials of PIVC assessment and decision-making.

Feedback from content validity survey and verbal conversations revealed that some respondents did not think it appropriate to assess all items each ‘shift’, particularly as nursing shifts can vary in length up to 12 hours. Some respondents commented that daily assessment would be sufficient for items such as ‘need for the PIVC’ and ‘patient education’, while others remarked that daily assessment would not be frequent enough for some patient populations, such as paediatrics, where guidelines recommend hourly assessment for continuous infusions. While current guidelines¹¹ recommend daily assessment of PIVC need, we believe this assessment is warranted more regularly, particularly if the nurse knows that an administered medication is the final dose and removal is planned in the next few hours. The suggestion to consult the treating team prior to removing the PIVC was criticised by several respondents, who argued nurses possess the skills and knowledge to make their own informed decisions. While this is true for experienced nurses, it cannot be presumed that novice nurses and students will have confidence in their decision to remove or resite a PIVC.

Patient and family concerns about the PIVC and their education needs are often under-valued by healthcare workers,²⁸ and this was reflected in our findings that only 11 out of 18 survey respondents agreed with this principle. Surprisingly, only two of seven experts felt regular patient education should be included in the tool. In an Irish study, patients who did not know the reason for their PIVC were seven times more likely not to need the device.²⁹ In an Australian study of consumer experiences, patients and caregivers expressed the need for improved communication about PIVC insertion and care.²⁴ A recent survey of eight US hospitals reported that one-third of patients with concerns about their care did not feel empowered to speak up, and patients less likely to speak up included older, sicker, non-English-speaking, or patients with mental health issues.³⁰ While more hospitals are implementing mechanisms for patients and families to verbalise critical safety concerns, more needs to be done to change hospital culture to encourage patient collaboration in daily care decisions, particularly those that impact on infection management and prevention.^31–33 Including a prompt for clinicians to ask the patient about the PIVC has merit.

Testing inter-rater reliability among a variety of clinicians was another strength of this study. Paired assessments, performed immediately after each other, eliminated the likelihood of altered assessment findings resulting from medication or fluid administration, or time for symptoms to change. Blinding of the second assessor to the first assessor’s results and blinding the registered nurses to the research nurses’ results also strengthened the findings. While the overall proportion of inter-rater agreement was high for most items, the category of patient education demonstrated the lowest scores. This is not surprising, as the stability of patient-reported variables between assessments can be a confounder of inter-rater reliability testing.³⁴ For instance, if the first rater asked about pain or tenderness of the PIVC site, and received a negative response, this could have suggested concerns to the patient who then answered in the affirmative to the second assessor. Asking patients if their nurse had assessed the PIVC that shift or performed hand hygiene before touching the PIVC, or whether they had received any education about the PIVC, also elicited contradictory answers in some assessments. Some patients answered negatively in the first instance, but when asked the same question by the second rater, they answered in the affirmative. This was possibly due to suggestibility or an unwillingness to implicate the nurse, but we had no way to confirm or refute the findings.

Decision-making is a subjective process based on assessment, but the assessment itself should be a standardised process to ensure care is evidence based and comprehensive. PIVC decisions are often based on clinicians’ education and experience, and not all clinicians are conversant with current guidelines.^35–38 The I-DECIDED tool prompts clinicians to perform a structured PIVC assessment and document their decision based on that assessment. It is not a prescriptive tool designed to overrule local policies, although we do believe that decisions to continue or remove a PIVC should be based on comprehensive clinical assessment, and not simply dwell time or absence of phlebitis symptoms.⁶

Limitations. Construct validity could not be evaluated as PIVC assessment is highly subjective, and no gold standard exists for PIVC assessment and decision-making. Criterion validity could not be evaluated because there are no other comprehensive PIVC assessment tools in the literature. While multiple phlebitis tools exist, evaluation of their measurement properties is rare, and validity and reliability data are limited or absent. Inter-rater reliability assessments of the tool were completed by different sets of coders for different subjects, which can lead to a higher level of systematic bias or make it difficult to detect bias.³⁹ We tried to control for this by alternating the order of assessments and blinding each assessor to the other’s findings. Finally, inter-rater reliability was tested in seven medical-surgical wards in three hospitals. Each assessor only assessed each PIVC on one occasion; therefore, it was not possible to evaluate intra-rater reliability. Testing the tool’s reliability in other settings is strongly recommended. Feasibility and acceptability of the tool were reported as generally positive in this study, but further research is recommended to evaluate the strain on nursing workload of introducing this tool.

Conclusion

The I-DECIDED tool demonstrated strong content validity and high inter-rater reliability, feasibility and acceptability in medical-surgical wards of three hospitals. Implementation of this tool could prompt clinicians to provide comprehensive care and remove PIVCs when no longer needed or as soon as complications arise. Early detection and action could prevent painful PIVC complications, reduce the risk of bloodstream infection, and result in cost savings for healthcare services. Studies to evaluate the outcome of implementing this tool in clinical practice are recommended.