Point-of-care diagnostic lung ultrasound is highly applicable to the practice of medicine in Saudi Arabia but the current skills gap limits its use

Abstract

CONTEXT:

Coronavirus disease 2019 (COVID-19) has put a spotlight on point-of-care diagnostic lung ultrasound (POCDLUS). However, the spectra of respiratory disease and resources available for investigation vary internationally. The applicability of POCDLUS to internal medicine (IM) practice in Saudi Arabia and the current use by Saudi physicians are unknown.

AIMS:

The aim of the present study was to determine the applicability of POCDLUS to IM practice in Saudi Arabia and quantify the residents' current skills, accreditation, and use of POCDLUS.

METHODS:

A questionnaire was distributed to the IM residents at our institution to assess their knowledge, use of POCDLUS, and their perceptions of its applicability in IM.

STATISTICAL ANALYSIS:

Standard descriptive statistical techniques were used. Categorical data, presented as frequency, were compared using the Chi-squared test. The Likert scale responses, presented as mean ± standard deviation, were compared with a Student's t-test.

RESULTS:

In total, 100 residents participated (response rate 92.6%) and reported that POCDLUS was applicable to their practice. Identifying pleural effusions was most applicable. A small proportion (n = 7) had received training, nine used POCDLUS regularly, none were accredited and the overall self-reported level of knowledge was poor.

CONCLUSIONS:

Whilst POCDLUS is applicable to IM practice in Saudi Arabia, the significant skills gap preclude the provision of a POCDLUS service. As COVID-19 can cause an interstitial syndrome, our pandemic preparation response should include POCDLUS training. The current study is supported by a similar Canadian study and the international standardisation of POCDLUS training may be feasible. The findings of the current study may facilitate the development of POCDLUS training programs for internists throughout Saudi Arabia.

Due to the Coronavirus Disease 2019 (COVID-19) pandemic, health-care services have been inundated by patients presenting with acute respiratory symptoms.[1] Diagnostic point-of-care ultrasound (POCUS) of the lung (POCDLUS) is a fast, portable, noninvasive, and diagnostic tool.[2] Substantial evidence supports the clinical value of lung ultrasound (LUS).[1,2,3,4,5,6,7,8] The diagnostic accuracy of POCDLUS for many causes of acute respiratory conditions (e.g., pleural effusion, pneumothorax, pneumonia, and interstitial syndromes) is excellent.[1,2,3,4,5,6,7,8]

However, as POCDLUS is a relatively new technology, most physicians have little or no experience of its use. The cost of ultrasound (US) machines ranges from $2000 to $100,000 depending on the size and quality. POCDLUS is highly operator dependent, and to be effective, POCDLUS must be performed by competent practitioners.[1,2] Training to provide the required knowledge and skills is required to ensure safe, competent, and effective use of LUS.[9,10,11]

The implementation of POCDLUS requires a significant initial investment in terms of cost and provider training. Although international evidence-based recommendations for POCDLUS are available,[8] there is no consensus regarding training, competencies, or accreditation.[2] The Saudi Commission for Health Specialties does not, as yet, have a syllabus for training internal medicine (IM) residents or pulmonology fellows in POCDLUS.

Several countries, including the United Kingdom (UK) and Canada, have established POCUS curricula for physicians, which include POCDLUS.[12,13,14] Intuitively, it would seem appropriate to simply replicate one of these programs in Saudi Arabia. However, as the spectra of respiratory diseases in the Middle East differs from that in more temperate regions,[15] Western POCDLUS curricula may not be applicable to Saudi Arabia. Justification for the high initial cost of developing a POCDLUS service requires confirmation that POCDLUS is applicable to the current practice of IM in Saudi Arabia.

A successful training program must recognize and acknowledge adult learners' perceptions and needs.[10,16] Currently, the perceptions of Saudi IM residents regarding POCDLUS is unknown, in terms of the relevance to their practice, and the current level of skills. A needs assessment is required and the current use of POCDLUS should be quantified before a POCDLUS training program can be developed for Saudi Arabia.[10,16]

The aim of this study was to determine IM residents' perceptions regarding the applicability of POCDLUS and to identify the skills gap by quantifying their self-reported knowledge, training, accreditation, and use of POCDLUS in a Medical City in Saudi Arabia.

Methods

Ethical approval

The Institutional Review Board of King Abdullah International Medical Research Center approved this study.

Study design

This cross-sectional survey of IM residents was conducted at a Medical City in Saudi Arabia.

Participants

The setting of the study is a 1500-bed tertiary referral center, which hosts the largest IM residency training program in Saudi Arabia. All IM residents (postgraduate year [PGY] 1–4, n = 108) at our institution during the academic year October 2018–October 2019 were invited to participate. Informed consent was obtained before participation in the survey.

Survey development

A validated questionnaire investigating POCDLUS was adapted[10,17] with input from publications describing the applications of LUS[1,2,3,4,5,6,7] and the competencies required for POCUS.[12,13,14] From this base, an initial questionnaire was developed, with input from two researchers with expertise in POCDLUS and questionnaire design (main survey and RR) in May 2019. The questionnaire had four sections, including demographic information (gender, age range, and PGY of training), applicability of four diagnostic applications of LUS (i.e., a needs assessment), knowledge of 12 items relevant to POCDLUS and experience (training, accreditation, and use of POCDLUS). For each diagnostic application, two questions were asked initially: (1) How applicable is the indication for POCDLUS use to your practice of IM? and (2) What is your current skill in that area? For the items related to knowledge, a self-assessed level of knowledge was required. The questions related to experience were restricted to Yes/No responses.

After obtaining Institutional Review Board approval, the questionnaire was piloted with four non-IM residents to obtain input related to questionnaire length, content, and clarity. It was reported that the survey was too long, and it was revised using the feedback from the participants in the pretest.

We reflected on the best method to measure the skills gap in the sample. We believed that the residents' perceptions regarding the applicability of POCDLUS to their practice, interpreted in the context of their self-reported knowledge and experience (i.e., training, accreditation, and use of POCDLUS), would provide more useful information regarding the lack of skills and the interventions required than self-reported skill levels for each application of POCDLUS. Consequently, the questions related to the current level of skills for each of the diagnostic applications of LUS were discarded, to shorten the questionnaire. The relevance of the 12 knowledge items was re-reviewed. Only two questions were retained. One related to “basic US knowledge and use” and the second related to the ability to interpret POCDLUS findings, as the questions would provide a summative overview of the respondents' self-assessed skill in POCDLUS.

The final POCDLUS questionnaire included four diagnostic applications, and two knowledge items, in addition to the questions related to the baseline demographic information and experience described above [Appendix 1]. Using a paper-based questionnaire, the final questionnaire was distributed to the IM residents at various departmental teaching activities from September and October 2019. As this study was unfunded, no incentives were used.

Study outcomes

The perceived applicability of indications for POCDLUS in the practice of IM in Saudi Arabia was assessed using a 5-point Likert scale (1 very poor, 2 poor, 3 fair, 4 good, and 5 very good). Self-reported knowledge of basic US principles and lung US were also assessed on the same 5-point Likert scale. The training and accreditation in POCDLUS were determined using the closed questions (i.e., Y/N). The use of POCDLUS was assessed using an incremental scale (never, once a month, once a week, daily, more than once daily).

Statistical analysis

The data were analyzed using the standard descriptive statistical techniques. To facilitate the comparison of data, interval data, described as a 5-point Likert scale, were presented as both frequency and mean ± standard deviation (SD), as described by Watson et al.[10] The data were compared using the Student's t-tests or analysis of variance (ANOVA) as appropriate. Categorical variables were compared using a Chi-squared test. All analyses were performed using Excel version 2016 (Microsoft, USA).

Results

Demographic information, year of postgraduate training, and response rates

The sample's demographic information (i.e., age range, gender, and PGY of training) and the response rates are displayed in Table 1. A total of 100 (26 females and 74 males) of 108 (31 females and 77 males) residents participated in the study. The response rate was high (92.6%). The participation and response rates for each PGY group were as follows: PGY1 31/33, 94%, PGY2 25/28, 93%, PGY3, 23/28, 89%, PGY4 19/19, 100%. Although the response rate of the female participants (84%) was significantly lower than the male participants (96%, Chi-squared 4.82, P = 0.028), there were no statistically significant differences between the responses of male and females. All the responses were included in the final analysis.

Table 1.

Sample demographic information and response rates

Variables	n (RR % gender)n (RR % PGY)	Gender n (RR % gender)		Age (years) Range n (% PGY)
		Male	Female	21-25	26-30	31-35
Male	74 (96.1)	-	-	16 (21.6)	45 (60.8)	13 (17.6)
Female	26 (83.9)	-	-	9 (34.6)	14 (53.8)	3 (11.5)
PGY 1	31 (93.9)	22 (100)	9 (81.8)	20 (64.5)	11 (35.5)	0
PGY 2	25 (89.3)	17 (100)	8 (72.7)	5 (20)	20 (80)	0
PGY 3	25 (89.3)	23 (92.0)	2 (66.7)	0	19 (76)	6 (24)
PGY 4	19 (100)	12 (100)	7 (100)	0	9 (47.4)	10 (52.6)
Total	100 (92.6)	74 (96.1)	26 (83.9)	25	59	16

Sample demographic information and response rates. The table presents the sample demographics and response rates. Age ranges and response rates are stratified by PGY of training and gender. Data are presented as frequency and percentage of strata totals. n: Number of responses, PSY: Postgraduate year, RR: Response Rate

Applicability of point-of-care diagnostic lung ultrasound to internal medicine practice in Saudi Arabia

The applicability of the four uses of diagnostic POCDLUS to the practice of IM in Saudi Arabia is shown in Tables 2 and 3. Combining the perceptions related to all four indications for POCDLUS, applicability to their practice was fair to good (mean 3.4 ± SD 1.4, 73% of the responses were fair, good or very good, 51% of the responses were good or very good). However, there was a statistically significant difference between the groups with the one-way ANOVA (F (3,396) 21.8, P < 0.0001). The applicability of POCDLUS to identify pleural effusion was rated the highest, 81 residents rated this as either good or very good (mean applicability score 4.3 ± SD 0.9). The participants considered the identification of pneumothorax (mean 3.4 ± SD 1.4) and consolidation (mean 3.1 ± SD 1.5) to be less relevant to their practice. The identification of interstitial syndromes was considered the least relevant to the practice of IM in Saudi Arabia. However, 33 residents rated the applicability of this indication for POCDLUS as good or very good (mean 2.9 ± SD 1.5).

Table 2.

Internal medicine residents’ perceptions regarding the applicability of lung US to their clinical practice and self-reported knowledge of lung ultrasound

Response (Likert scale)	Indication for diagnostic lung ultrasound (n)				Knowledge (N)
	Effusion	Pneumothorax	Consolidation	Interstitial Syndrome	Basic US	Lung US
Very poor	1	13	23	27	34	66
Poor	2	13	10	18	28	16
Fair	16	22	28	22	27	10
Good	28	22	13	7	8	6
Very good	53	30	26	26	3	2
Total	100	100	100	100	100	100

Internal medicine residents’ perceptions regarding the applicability of lung US to their clinical practice and self-reported knowledge of lung US. This table presents internal medicine residents’ perceptions on applicability of diagnostic lung US to their clinical practice and their self-reported knowledge of lung US. Data are presented as frequency, N: Number of responses, PSY: Postgraduate year, US: Ultrasound

Table 3.

Internal medicine residents’ mean applicability scores for the relevance of lung ultrasound to their clinical practice and mean self-reported knowledge scores for lung ultrasound

Variables	Indication for diagnostic lung ultrasound (Mean±SD)				Knowledge (mean±SD)
	Effusion	Pneumothorax	Consolidation	Interstitial syndrome	Basic US	Lung US
PGY 1	4.3±1.0	3.2±1.3	3.0±1.5	2.5±1.4	2.2±1.1	1.7±1
PGY 2	4.4±0.8	3.3±1.4	2.6±1.4	2.5±1.6	2.5±1.2	1.6±1.2
PGY 3	4.3±0.8	3.5±1.5	3.1±1.4	3.0±1.5	2.1±0.9	1.7±1.0
PGY 4	4.2±1.0	3.9±1.2	3.8±1.4	3.8±1.4	1.9±0.9	1.4±0.8
Male	4.2±1.0	3.6±1.3	3.2±1.5	3.0±1.5	2.1±1.1	1.6±1.0
Female	4.3±1.1	2.9±1.3	2.9±1.5	2.6±1.5	2.4±1.1	1.8±1.0
Overall	4.3±0.9	3.4±1.4	3.1±1.5	2.9±1.5	2.2±1.1	1.6±1

Internal medicine residents’ mean applicability scores for relevance of lung US to their clinical practice and the mean self-reported knowledge scores for lung US. This table presents internal medicine residents’ opinions on applicability of lung US and self-reported knowledge of lung US. Applicability and knowledge are rated on a 5-point Likert Scale (1, Very Poor; 2, Poor; 3, Fair; 4, Good, and 5, Very Good). Data are stratified by PSY of training (PGY) and gender and presented as mean±SD. SD standard deviation, PSY: Postgraduate year, US: Ultrasound

Residents' training in, knowledge of, and skills gap related to point-of-care diagnostic lung ultrasound

The self-reported training in POCDLUS is shown in Table 4. Only four participants (4%, F 2, M 2) received any formal training in the US during medical school and three (3%, M 3) received some formal training during their residency.

Table 4.

Internal medicine residents’ training, accreditation and use of lung ultrasound in clinical practice

Grade/gender	Undergraduate training (n, %; male n)	Postgraduate training (n, %; male n)	Accreditation (n)	Use of lung US (n, %; male n)
PGY 1	3 (9.7; male 2)	0 (0)	0	0 (0)
PGY 2	0 (0)	1 (4; male 1)	0	4 (16; male 2)
PGY 3	1 (4; male 1)	2 (8; male 2)	0	3 (12; male 3)
PGY 4	0 (0)	0 (0)	0	2 (11; male 2)
Male	3 (4.1)	3 (4.1)	0	7 (9.5)
Female	1 (3.8)	0	0	2 (7.7)
Total	4 (4)	3 (3; male 3)	0	9 (9; male 7)

Internal medicine residents’ training, accreditation and use of lung US in clinical practice. This table presents internal medicine residents’ training, accreditation and use of lung US in clinical practice. Responses are stratified by PSY of training (PGY) and gender. n: Number of respondents, US: Ultrasound

The self-reported knowledge of basic US use and ability to interpret POCDLUS are displayed in Tables 2 and 3. The sample generally self-reported a poor level of knowledge of basic US use (mean 2.2 ± SD 1.1), and the ability to interpret POCDLUS findings (mean 1.6 ± SD 1.0). The self-reported ability to interpret POCDLUS findings was significantly lower than the sample's overall opinion of the applicability of all diagnostic uses for POCDLUS (mean 3.4 ± SD 1.4, P < 0.0001), suggesting a lack of skills.

Accreditation and use of lung ultrasound in clinical practice

The sample's accreditation in POCDLUS and self-reported use of POCDLUS in clinical practice are shown in Table 4. The vast majority (91, 91%) never use POCDLUS. Although none had accreditation, nine (9%, female 2, male 7) reported regular use of lung US in clinical practice. There was no statistically significant difference in the regular use of POCDLUS between the male and female groups (Chi-squared 0.12, P = 0.73). Of the nine residents who reported the regular use of POCDLUS, only one had any formal training. Of the four residents (4%) who received some training at medical school, none used lung US in clinical practice, and of the three (3%) who received some training during their residency, only one used POCUS in clinical practice.

Discussion

As POCDLUS can greatly facilitate the management of COVID-19 globally, the use of POCDLUS by internists and pulmonologists has increased exponentially in the past few months.[1] However, the spectra of respiratory diseases in the Middle East varies from that in other regions.[15] A justification for the significant start-up costs of a POCDLUS service requires confirmation that this technology is applicable to the practice of medicine in Saudi Arabia. This study, therefore, describes IM residents' perceptions of the applicability of four indications for diagnostic POCDLUS to the practice of IM at a Medical City in Saudi Arabia.

The residents' perceptions of the applicability of lung ultrasound and self-reported knowledge of lung ultrasound

The Saudi IM residents reported that POCDLUS would be very applicable to their practice [Table 2], with the identification of pleural effusion the most applicable indication. However, the sample reported a poor level of self-reported knowledge of POCDLUS.

The residents' current level of training, accreditation, and use of diagnostic point-of-care diagnostic lung ultrasound

The current study describes the residents' training, accreditation, and experience in POCDLUS [Table 2]. As the survey was conducted toward the end of the academic year, PGY1 residents had almost completed 1 year of training and PGY4 were at the end of their residency. In this context, the observations and recommendations are expected to be relevant to pulmonology fellows, at the start of their fellowships.

A similar study in the UK collated IM residents' accreditation and experience in POCDLUS. The use of POCDLUS by internists in the UK is high. Smallwood et al. 2015[17] reported that 85.3% of the UK IM trainees use POCUS and 20.3% were accredited in POCDLUS. In comparison, a small proportion (9%) of the IM residents in Saudi Arabia use POCDLUS and none are accredited (0%). The assessment of the skills gap can guide educational interventions to resolve this discrepancy.

Evaluation of the skills gap

A pragmatic measure of a skill gap is the difference between self-reported ability to perform a skill and the perceived usefulness of that skill.[10] The sample's self-reported ability to interpret POCDLUS findings (mean 1.6 ± SD 1.0, [Table 2]) was significantly lower than their overall opinion of the applicability of all uses for POCDLUS (mean 3.4 ± SD 1.4, [Table 2]).

The majority (81) agreed that the applicability of POCDLUS for the identification of pleural effusion was good or very good, but only nine (9%) use POCDLUS. Of the group who reported regular use of US, only one reported prior formal training. The findings suggest a significant skills gap in POCDLUS in the residency program. The unsupervised performance of POCDLUS by untrained residents also raises governance issues and patient safety concerns. Correcting these issues must be accomplished by a training program and the development and implementation of formal processes for supervision, governance and accreditation.

Ability to provide a diagnostic lung ultrasound service

In the UK, a preexisting pool of skilled operators supported the effective use of POCDLUS during the COVID-19 pandemic.[1] It is disturbing, particularly in the context of the COVID-19 pandemic, that the ability of IM residents to provide POCDLUS at our institution is virtually nonexistent. None of the respondents had any accreditation in POCDLUS and 91% had never used POCDLUS. The COVID-19 pandemic heightened the urgency to develop POCDLUS at our institution, which requires the development of a curriculum and a training program. However, the pandemic preparation plan of the department of medicine included training regarding the use of POCDLUS to diagnose interstitial syndromes and COVID-19.

Relevance of existing diagnostic lung ultrasound training programs to Saudi Arabia

The residents' perceptions regarding the applicability of POCDLUS and the skills gap [Table 2] are similar to what is reported by Canadian IM residency programs.[10] This may be because POCDLUS findings, whilst useful, are relatively nonspecific. Although there are regional differences in the differential diagnosis of pleural effusion,[15] for example, the use of POCDLUS to detect effusions is universally applicable to the practice of IM. This observation suggests that the international standardization of basic POCDLUS training may be possible and curricula developed in other countries may be relevant to internists in Saudi Arabia. However, any new syllabus for training Saudi IM residents in POCDLUS should fulfil the regulatory requirements of the Saudi Commission for Health Specialties to provide nationally recognised accreditation.

Strengths and limitations

The study has some limitations. The knowledge-related data were self-reported and the accuracy can be questioned.[18] However, the sample generally reported little knowledge of basic US and a poor ability to interpret POCDLUS. This is consistent with our personal observations and is probably valid.

The availability of trainers, cost, time constraints, and the local patient case mix must all be considered when designing curricula. This is of paramount importance when face-to-face training of practical skills on real patients is required. These issues are best understood by clinician educators. Our residents may not have considered these multi-dimensional issues whilst responding to the survey. To develop a curriculum for POCDLUS, the perspectives regarding the feasibility and patient safety,[10,14] as well as the trainers' perceptions must be combined with the residents' views.

The participants in the current study considered the identification of interstitial syndromes[8] as the least useful application of POCDLUS to their IM practice. However, the survey was conducted just before the COVID-19 pandemic, and the sample's perceptions were based on their prepandemic case-mix. Viral pneumonias, such as COVID-19 cause an interstitial syndrome that can be identified by LUS.[1] Recently, COVID-19 has supplanted indigenous patterns of respiratory diseases, and POCDLUS are used extensively to detect interstitial syndromes and exclude other causes of acute respiratory symptoms globally.[1]

In view of their low self-reported ability to interpret POCDLUS findings, the residents were probably not familiar with the term “interstitial syndrome.” A survey of Canadian IM residents supports this finding.[10] Despite the residents' perceptions of poor applicability, we strongly recommend that IM residents globally are trained to identify interstitial syndromes with POCDLUS. This case in point highlights the critical importance of obtaining clinician educators' perceptions of the educational needs of trainees during curriculum development.

In addition, the survey was administered to IM residents in only one residency program in Riyadh, Saudi Arabia, limiting the generalizability of the findings. It should be noted that the survey had a very high response rate and our institution hosts the largest IM residency program in Saudi Arabia. The views of the residents are likely to be representative of other residents training in IM throughout Saudi Arabia, and other countries with well-developed healthcare systems. Our observations and their perceptions related to POCDLUS should be taken into account when developing training programs to safely and effectively integrate POCDLUS into the practice of IM.

Contribution to the existing literature

The current study provides evidence related to the applicability of POCDLUS to the practice of IM in Saudi Arabia. It also revealed that the largest IM residency program in Saudi Arabia has a significant skills gap in POCDLUS. These observations can guide the development of a POCDLUS curriculum for IM. Such a curriculum must be aligned with the regulatory framework of the Saudi Commission for Health Specialties. However, training programs for POCDLUS should focus on the knowledge, applications, and procedures that are perceived to be most relevant, and where the lack of skills are the highest.

The survey demonstrated that the IM residency program at our institution cannot provide a POCDLUS service during the COVID-19 pandemic. As our IM residency program is the largest in Saudi Arabia, this may reflect the current state of affairs throughout the Kingdom. The pandemic preparation action plan of the department of medicine at our institution included an accelerated training program for POCDLUS, focusing primarily on the identification of interstitial syndromes and the diagnosis of viral pneumonias, such as COVID-19.

Conclusions

The current study suggests that POCDLUS is highly applicable to the practice of IM in Saudi Arabia. However, a significant skills gap and lack of training, experience, and accreditation currently preclude the provision of a POCDLUS service to IM patients at our institution. This information supported the coordination of the Department of Medicine's preparation for the COVID-19 pandemic and highlighted the urgent need to develop a national curriculum to train internists in POCDLUS. Despite regional differences in respiratory diseases, the residents' responses regarding the applicability of POCDLUS were very similar to IM residents in Canada. POCDLUS training programs developed in Canada and other countries may be relevant to the practice of IM in Saudi Arabia and the international standardisation of POCDLUS training may be possible. Institutions considering POCDLUS training programs for IM residents, should take cognizance of the current findings when developing curricula.

Financial support and sponsorship

Nil.

Conflicts of interest

There are no conflicts of interest.

Appendix

Appendix 1: Supplemental Material 1. Survey on diagnostic lung ultrasound

DEMOGRAPHICS

What stage of your career are you in?

PGY-1 PGY-2 PGY-3 PGY-4 Gender? Male Female

Age 16-20 21-25 26-30 31-35 36-40 41-45

APPLICABILITY OF LUNG US TO PATIENT CARE

Please rate the applicability of the following uses of Lung US to your clinical practice on this 5 point scale

Very poor Poor Fair Good Very Good

Identifying pleural effusion

Identifying pneumothorax

Identifying interstitial syndrome

Identifying lung consolidation

The following section seeks your general knowledge and Lung US skills. How would you rate your own knowledge/skills on the following?

How would you rate your current level of knowledge or skills in the following domains?

Basic Ultrasound Knowledge and Use Very Poor Poor Fair Good Very Good

Ability to interpret US findings - pulmonary system

Very Poor Poor Fair Good Very Good

ULTRASOUND TRAINING AND USAGE

Did you receive formal training in Lung US during medical school?

No YES

Did you receive/are you receiving formal training in Lung US during post-graduate/residency training

No YES

Do you have any formal accreditation in Lung US?

No YES

How often do you use Lung US in clinical practice?

Never

Once per month Once per week 3-4x per week Daily > Once/day

Do you have any other comments or concerns?