Peer review of third trimester abdominal circumference measurements

Abstract

Background

Third trimester growth scans represent a significant proportion of the workload in obstetric ultrasound departments. The objective of these serial growth scans is to improve the antenatal detection of babies with fetal growth restriction. The aim of this paper is to describe a method of peer review for third trimester abdominal circumference measurements which is realistic within busy obstetric ultrasound departments in the UK.

Method

Twenty-two, third trimester, measured abdominal circumference images were randomly selected. Images were assessed subjectively by 12 sonographers using the image Criteria Achieved Score. For quantitative assessment, termed the Inter-operator Variability Score, three of the abdominal circumference (AC) images were blindly remeasured. Following this, a questionnaire was used to ascertain which image criteria sonographers considered most important and to reach an agreement on correct caliper placement.

Results

The least frequently met image criteria with the lowest Criteria Achieved Score related to an oblique abdominal circumference section. These included fetal kidney present (Criteria Achieved Score 24.6%), multiple oblique ribs (Criteria Achieved Score 39.4%) and oblique spine (Criteria Achieved Score 37.5%). Caliper placement was also identified as inconsistent.

Discussion

This study demonstrates that the perfect AC section is not always possible and sonographers use their professional judgement to determine whether an image is acceptable. Seventy-three percent of the images reviewed were of an acceptable standard. There can be inconsistencies in sonographer opinion regarding what is an acceptable third trimester abdominal circumference image. These differences need to be addressed to maximise the effectiveness of the third trimester ultrasound examination.

Conclusion

Peer review can be used to monitor scan quality and identify areas of inconsistency.

Background

Third trimester growth scans represent a significant proportion of the workload of the routine obstetric ultrasound department. In 2015, Mothers and Babies: Reducing Risk through Audits and Confidential Enquiries across the UK (MBRRACE-UK) identified a failure to detect poor growth antenatally as a contributing factor to poor obstetric outcome.¹ As a result, the Saving Babies Lives’ care bundle was published in 2016,² with the updated version 2 published in 2019.³ A risk assessment tool was published as part of the care bundle. This tool aims to identify women at risk of fetal growth restriction (FGR) and recommends serial growth scans for high-risk women from 26 to 28 weeks of gestation. The objective of these serial growth scans is to improve the antenatal detection of small for gestational age (SGA) babies and those who may have FGR. A Delphi consensus of experts⁴ defined both early (less than [<]34 weeks) and late (34 weeks or greater than [>] 34 weeks) FGR as either:

• Estimated fetal weight (EFW) or abdominal circumference (AC) < 3rd centile

or

• • EFW or AC <10th in combination with an abnormal pulsatility index (PI) of either:

•  a) cerebroplacental ratio, defined as a PI < 5th centile.

•  b) umbilical artery, defined as a PI > 95th centile.

Nationally, there is currently no recommended EFW chart. The sensitivity and specificity for the detection of adverse pregnancy outcome varies between charts with a recent paper reporting sensitivity between 15% and 32%, and specificity between 94% and 77% using EFW < 10th percentile.⁵ The implementation of the Saving Babies Lives care bundle means that accurate fetal biometry is key to the identification of SGA babies. The importance of high-quality ultrasound has been recognised by both the British Medical Ultrasound Society (BMUS)⁶ and the Royal College of Radiologists (RCR)⁷ through their respective published guidance recommending peer review. The peer review of combined screening scans is well established within the United Kingdom (UK).⁸ Limited data describing the qualitative assessment of the 20-week anomaly scan have been published, from London⁹ and the Netherlands,¹⁰ but as yet obstetric image peer review is not common practice in the UK.

The Intergrowth-21 consortium has described a robust system of continual quality control and image standardisation including both qualitative and quantitative components for growth scans.^11,12 The results of the Intergrowth-21 quality control programme demonstrated high levels of both intra-operator and inter-operator variability, described how inconsistent scanning was identified and what additional training was provided to operators when necessary. This system of quality control involves different image criteria for head circumference (HC) to those currently used by the Fetal Anomaly Screening Programme (FASP), although those used for AC are interchangeable between the two groups. In addition, the Intergrowth-21 method of review is labour intensive and therefore potentially unrealistic within busy departments that deliver a routine obstetric screening services as part of their imaging workload.

Given that AC measurements are arguably the most important fetal measurement taken during the third trimester growth scan, effective peer review would be best focused on the accuracy of AC measurements performed during these scans. The reason for this is that AC measurement alone can be used to identify small babies but also makes a greater contribution to the EFW than HC and femur length (FL) measurements.¹³ Dudley suggested that large differences in intra- and inter-operator variability in fetal biometry compromised the accuracy of EFW.¹⁴ Contrasting with these data is a more recent systematic review which suggested that the accuracy of EFW has improved over the last decade.¹⁵ Dudley has subsequently further contributed to this debate by addressing the implications of using incorrect measurement techniques for selected size charts.¹⁶

A further issue for consideration is the lack of equivalence between the normal range of AC size measurements and that of the EFW at the same gestation. Thus, for example, an AC measurement that lies on the 3rd or 10th centile on the Chitty AC size chart does not necessarily equate to an EFW on the 3rd or 10th centile. This difference creates difficulties in potential interpretation of a SGA flow chart that requires further action if the AC, EFW or both are on the 3rd or 10th centile.

The aim of this paper is to describe a method of peer review for third trimester AC measurements which is practical for implementation within busy obstetric ultrasound departments in the UK.

Method

Images of 22 measured ACs from singleton pregnancies obtained by 12 different sonographers (each sonographer performed between 1 and 3 scans each) were selected randomly between January and June 2017 from the departmental electronic image storage system. Demographic details collected from the 22 women were body mass index (BMI) at the 8 to 10 weeks midwife booking visit together with gestational age and amniotic fluid index (AFI) as assessed and reported at the third trimester growth scan. A PowerPoint presentation of the 22 measured AC images was prepared. All scoring and subsequent measurements were performed independently.

1. The AC Image Score (ACIS). Twelve sonographers were each given the PowerPoint presentation and, using their professional opinion, asked to rate subjectively the measured AC section demonstrated in each of the 22 images as ‘good’, ‘acceptable’ or ‘poor’. These images are available online in Appendix 1.

2. The Criterion Achieved Score (CAS). The sonographers used the eight image criteria for AC derived from the FASP recommended AC section¹⁷ to assess the 22 AC images. Each criterion was either ‘met’ or ‘not met’.

3. The Inter-operator Variability Score (IVS). The AC ellipse caliper trace and resulting measurement data were removed from three of the 22 measured AC images as shown in Figure 1. Eight sonographers remeasured each of the three images. Multiple remeasurements of an image were permitted, with the sonographer’s preferred measurement recorded.

Figure 1.

The three AC images (a–c), used for assessment of the inter-operator variability score (IVS) with caliper placement as per the original reports. The calipers in each image were removed prior to the IVS. The images are images 1, 4 and 5 from online Appendix 1.

The 12 participating sonographers were made aware of the results of the first three components the study prior to undertaking the two final parts of the study.

• 4. Ranking of AC criteria. The same 12 sonographers were asked to rank the eight criteria in the order they considered necessary to produce an acceptable measured AC section.

• 5. Correct caliper placement. Finally, each sonographer was given four images of the same AC section, each showing a different caliper placement and therefore a difference resulting AC measurement (Figure 2(a) to (d)). They were asked to select which of the four identical images they considered showed the correct caliper placement and therefore the correct AC measurement.

Figure 2.

Correct caliper placement study. The same AC image showing four different caliper placement options: (a) AC = 301.5 mm; ( b) AC = 283.9 mm; (c) AC = 269.8 mm; (d) AC = 302.4 mm. AC: abdominal circumference.

Results

The 22 women had a mean gestational age of 34 + 6weeks (26 + 3 to 39 + 3weeks) and mean AFI of 15.6 cm (10.8 to 23.9 cm). The BMI at booking was available in 16 cases and gave a mean BMI of 26.05 (18.73 to 42.31).

The AC Image Score

The sonographer rating of good, acceptable or poor for each of the 22 images is shown in Figure 3 and Table 1. Consensus was defined as agreement between eight or more of the 12 sonographers (≥66%). An image was determined to be clinically safe if the consensus score was ‘good’ or ‘acceptable’. A total of 16 (73%) of the 22 images were assessed as either acceptable or good by ≥ 8 (66%) of sonographers. The remaining six (27%) images were assessed as poor by ≥5 (42%) of sonographers. The images that were assessed as poor correspond to cases 2, 4, 7, 8, 16 and 20 in Figure 3, Table 1 and online Appendix 1.

Figure 3.

Bar graph showing the percentage of 12 sonographers who subjectively and independently rated each of 22 measured third trimester AC images as good/acceptable or poor. AC: abdominal circumference.

Table 1.

Abdominal circumference image score (ACIS).

	Image rating
Image	Good	Acceptable	Poor
1	4	8	0
2	0	1	11
3	6	5	1
4	2	3	7
5	4	7	1
6	1	8	3
7	2	5	5
8	1	5	6
9	5	5	2
10	5	5	2
11	3	6	3
12	2	6	4
13	5	6	1
14	5	3	4
15	7	3	2
16	0	5	7
17	7	2	3
18	9	3	0
19	7	5	0
20	1	5	6
21	8	3	1
22	4	4	4

Independent and subjective rating by 12 sonographers of 22 measured third trimester AC images as ‘good’, ‘acceptable’ or ‘poor’. AC: abdominal circumference.

The Criterion Achieved Score

Each of the eight criteria for AC was either ‘met’ (score = 1) or ‘not met’ (score = 0), with the maximum achievable CAS for each criterion being 264 (22 images scored by 12 sonographers = 264). The image criteria and their relative scores are shown in Table 2. Column B shows the total CAS for each criterion. Columns C, D and E show the CAS tabulated against the ACIS ratings of ‘good’, ‘acceptable’ or ‘poor’.

Table 2.

Criterion Achieved Score (CAS).

	A Image Criterion	B Number (%) meeting the criterion (n = 264)	C Number (%) meeting criterion with good image rating (n = 264)	D Number (%) meeting criterion with acceptable image rating (n= 264)	E Number (%) meeting criterion with poor image rating (n = 264)
1	Magnified to fill 50% of image	244 (92.4%)	86 (32.6%)	93 (35.2%)	65 (24.6%)
2	Visible stomach bubble	233 (88.3%)	89 (33.7%)	90 (34.1%)	54 (20.4%)
3	Round section	229 (86.7%)	89 (33.7%)	97 (36.7%)	43 (16.3%)
4	No more than one-third the umbilical vein present	208 (78.8%)	84 (31.2%)	81 (30.7%)	43 (16.3%)
5	Correctly placed calipers/Ellipse placed correctly on the outer edge of the AC	185 (70.1%)	86 (32.6%)	74 (28.0%)	25 (9.5%)
6	Spine seen as three dots/Spine seen in cross section (three dots)	165 (62.5%) (8.7%)	77 (29.2%)	61 (23.1%)	27 (10.2%)
7	Unbroken and short rib on one side	160 (60.6%)	76 (28.8%)	58 (22.0%)	26 (9.8%)
8	No kidney seen in the image	65 (24.6%)	22 (8.3%)	23 (8.7%)	20 (7.6%)

Table of results showing the number of image/sonographer scores for meeting each of the eight image criteria required for measurement of the AC, tabulated according to overall ACIS image rating of ‘good’, ‘acceptable’ or ‘poor’. Correct magnification was achieved in 92.4% of cases, with a visible stomach bubble, a round section and no more than one-third of the umbilical vein being present in 88.3%, 86.7% and 78.8% of cases, respectively. Correct caliper placement was only achieved in 70.1% of cases. The spine being seen in cross section, as three dots, was achieved in 62.5% of cases and an unbroken and short rib on one side in 60.6% of cases. Seeing no kidney in the image was the least frequently met criterion, achieving a score of 24.6%. AC: abdominal circumference.

The Inter-operator Variability Score

Eight of the 12 sonographers remeasured images 1 and 5 and six sonographers remeasured image 4 as shown in Table 3. The subjective CAS score for caliper placement in images 1 and 4 was rated as ‘good’ or ‘acceptable’ by 10/12 sonographers and by 11/12 in image 5. The average remeasured AC in case 1, of 331.4 mm (range 320.3–354.0 mm) was 2.8 mm less than the original measurement of 334.2 mm. A smaller remeasurement was also obtained from image 4 where the average remeasured AC of 272.8 mm (range 253.8–288.2 mm) was 9.4 mm less than the original measurement of 282.2 mm. A larger average remeasurement difference, of 1.0 mm, was obtained in image 5 where the original measurement was 323.9 mm compared to the average remeasurement of 324.9 mm (range 311.8 mm–328.7 mm). The percentage difference between the original and average remeasured AC for image 1 was −0.8%, −3.3% for image 4 and +0.3% for image 5. There was no significant difference between the original and remeasured AC in any of the three images. Image 4 had the greatest measurement difference and an ACIS score of ‘poor’ by 7/12 sonographers. Images 1 and 5 achieved smaller measurement differences and had ACIS scores of ‘good’ or ‘acceptable’ by 11/12 and 12/12 sonographers.

Table 3.

Inter-operator Variability Score (IVS).

Image	Corresponding image from Figure 1	Original AC measurement (mm)	Number (%) of 12 sonographers considering the original AC caliper placement correct	Range of remeasured AC measurements (mm)	Maximum difference between remeasured AC measurements (mm)	Average remeasured AC measurement (mm)	Difference (%) between average remeasured measurement and the original AC measurement (mm)
1	1a	334.2	10 (83%)	320.3–354.0	33.7	331.4	−2.8 (−0.8%)
4	1b	282.2	10 (83%)	253.8–288.2	34.4	272.8	−9.4 (−3.3%)
5	1c	323.9	11 (92%)	311.8–328.7	16.9	324.9	+1.0 (0.3%)

The original measurement and range of measurements, their maximum differences and average remeasured measurement of three third-trimester AC images shown in Figure 1, following independent remeasuring. Eight sonographers remeasured cases 1 and 5 and six sonographers remeasured case 4. AC: abdominal circumference.

Ranking of AC criteria

The AC criteria were ranked in order of importance by 12 sonographers, with 1 being the most important and 8 being the least important, as shown in Table 4. Agreement amongst all 12 sonographers that one criterion was the most important would therefore achieve a score of 12 and that one criterion was the least important would therefore achieve a score of 96.

Table 4.

Ranking of AC criteria.

	FASP image criterion	Combined rank scores
2	Visible stomach bubble	26
5	Correctly placed calipers/Ellipse correctly placed on the outer edge of the AC	26
4	No more than one-third of the umbilical vein present	45
1	Magnified to fill 50% of image	55
6	Spine seen as three dots/Spine seen in cross section (three dots)	65
7	Unbroken and short rib on one side	65
3	Round section	72
8	No kidney seen in the image	78

The combined rank scores from 12 sonographers for each of the eight image criteria for the measured AC. Score of 1 = most important; score of 8 = least important. Full agreement between the 12 sonographers for the most important criterion would therefore score 12 and for the least important criterion would score 96. AC: abdominal circumference; FASP: Fetal Anomaly Screening Programme.

A visible stomach and the ellipse correctly placed on the outer edge of the AC ranked equal in importance (score = 26), followed by no more than one-third of the umbilical vein being present (score = 45) and correct magnification (score = 55). A score of 65 was achieved for both the spine being seen in cross section, as three dots, and an unbroken, short rib on one side. A round section was ranked seventh, achieving a score of 72. Seeing no kidney in the image was ranked as the least important criterion, with a score of 78.

Correct caliper placement

The greatest consensus regarding caliper placement was met with Figure 2(b); 10 of the 12 (83%) sonographers selected this image as demonstrating acceptable and correct caliper placement. Figure 2(c) and (d) were each chosen by one sonographer.

Discussion

For EFW to be as accurate as possible, measurement of the AC must also be as accurate and as reproducible as possible. This requires both intra- and inter-operator variability to be as small as possible in terms of the individual operator and the departmental service, respectively. Intergrowth-21 has established a gold standard for peer review of growth scans. However, it is a labour-intensive approach which is unrealistic within the average UK obstetric ultrasound department. The present study suggests a realistic and effective way of AC ultrasound measurements peer view, using image criteria. The qualitative and quantitative assessments of the images provide useful information about the quality/accuracy of the scan and ensure the peer review process is clinically relevant.

The present study has established image criteria and ranked their importance in achieving accurate third trimester AC measurements. Scoring of the 22 images gave an indication, firstly, of the acceptability of the image – and therefore its AC measurement – based on the sonographers’ qualitative rating. In total, 16 (73%) of the 22 images were rated as ‘acceptable’ or ‘good’. This means 73% of the images presented, each of which was obtained, measured, and reported by a member of the group, was of a standard that could be considered as ‘safe’. The 22 scans from which these images were selected represent approximately 10% of the singleton third trimester growth scans performed monthly within our department. Table 2 shows a correlation between both overall (ACIS) and the specific (CAS) image review suggesting that sonographers were able to use both types of image review simultaneously to distinguish between good, acceptable and poor images. In addition, the IVS showed smaller differences in measurement for the two images rated as ‘good’ or ‘acceptable’ by ACIS score. This supports the idea that meeting the image criteria enables more reproducible measurements between sonographers.

A peer review process for image scoring to be capable of successful implementation requires strong consensus amongst the users as to which criteria are important. In this study, these are derived from the FASP recommended section for measurement of the AC. Our results indicate that a visible stomach and correct caliper placement are the most important criteria, both receiving the lowest score of 26 (Table 4). It is noteworthy that the ‘traditional’ criterion of a round section was of limited importance to the group, receiving the second highest score of 72. This suggests a realistic attitude to the challenges of measuring a third trimester AC which is frequently not circular, compared to those of the second trimester fetus which usually is. This also implies an element of image self-assessment by sonographers at the time of the scan to determine whether an image is acceptable for measurement. The effect of amniotic fluid volume and maternal BMI were not considered within this study and are aspects of the scan which deserve further exploration.

Despite identifying correct caliper placement as an important image criterion (Table 4), caliper placement only achieved a CAS of 185 (70.1%) from the initial peer review (Table 2) and was the second least frequently met ACIS criterion. This raises issues both of individual technique and consensus as to where the calipers should be placed in order to measure a third trimester AC ‘accurately’. There was however a consensus regarding caliper placement in the sonographer questionnaire with 10/12 (83%) of sonographers selecting Figure 2 b as demonstrating an acceptable caliper placement and best caliper placement. It should be noted that this was the only image of the four in which the caliper placement included an equal skin thickness at the top and the bottom of the AC section. The results showed that there is a discrepancy between the perceived importance of caliper placement by sonographers, ranked as the second most important criterion, and the ability for the criterion to be met in the initial peer review. However, reassuringly, further analysis revealed that correct caliper placement was one of the least frequently met image criteria in images rated poor overall. It was therefore unexpected that there was such a strong consensus between the sonographers regarding which image in the questionnaire had the best caliper placement. One explanation for this is that the questionnaire was carried out after the initial peer review was discussed with staff which may have influenced their choice. Alternatively, this supports a possible discord between theoretical knowledge and clinical practice that is at variance with providing a standardized and robust service.

Establishment of a consensus between sonographers with regards to what constitutes an acceptable AC section is key to successful acceptance and implementation of a departmental approach to reproducible AC measurements. The stronger the consensus within the group as to these two factors, the more likely the peer review process will be implemented successfully. If the consensus is weak then the process is less likely to succeed. There are two potential causes to this problem. The first is the lack of knowledge and/or understanding, for example, of not having been taught or being unaware that kidney should not be present in the correct section. This shortfall is relatively easy to address given correctly focused education. The second cause is a difference in professional opinion, for example, where the skin line along the upper border of the AC outline runs compared to that along the lower border. Resolving this issue is perhaps more challenging and requires discussion, agreement, consensus and inevitable change of practice for some operators. Resolving these issues is vital if a safe departmental standard and minimal inter-operator variability are to be achieved. Thus, although there was consensus for the caliper placement in Figure 2 b, those sonographers who preferred the caliper placement in another of the images would be required to change their practice of caliper placement if agreement for caliper placement, and therefore reproducibility of measurement, is to be obtained.

The observation that there was a tendency to over measure the AC reflects the tendency to overestimate EFW compared with actual birth weight, as reported in the literature.¹⁵ This is of relevance as the Hadlock B formula that combines HC, AC and FL and is the one of the most commonly used EFW formula, is heavily influenced by the AC measurement. The tendency to over measure is likely to be due to a conscious or subconscious desire for the fetal growth to ‘match’ the gestational age.¹⁸ The tendency towards overestimation is also thought to be more pronounced within SGA fetuses.¹⁹ This is the group of fetuses in which a small difference in AC measurement can influence whether or not a fetus is identified as also being growth restricted according to the Delphi consensus, with all the known implications for extra clinical management and extra use of hospital resources. However, removal of the gestational age equivalent read-out for AC from ultrasound machine displays so that it is not possible to ‘match’ measurements to the gestational age has been shown to be ineffective and does not support the assumption that sonographers ‘measure to fit’.²⁰ It could, however, be argued that in order to improve measurement accuracy both the measurement and gestational age need to be removed from the ultrasound machine display.¹⁸ Also, it should be acknowledged that poor technique will lead to inaccuracies not only in AC section selection and measurement but also in HC, FL and resulting EFW reporting. This study did not compare the actual birth weight with the EFW, but this is a topic which deserves further exploration.

This study included very experienced sonographers (>10 years’ experience), experienced sonographers and newly qualified sonographers. During the group feedback given as part of the present study, the sonographers described how the peer review had made them more aware of their own practice and potential bias. Peer review is a concept supported by the RCR and identified when the BMUS audit tool was first implemented.^6,7 In this peer review, it was felt unfair to identify individual sonographers whose scores, or ratings, differed consistently from the majority of the group. However, this could be considered in future as part of the individual appraisal process to provide individual sonographers with more personalised feedback on their performance.

This study has identified the image criteria which are most difficult to meet when measuring the third trimester AC. Sonographers were able to consistently produce AC sections which were round, appropriately magnified, and included the stomach bubble and one-third of the umbilical vein. To improve their third trimester image section further sonographers should try to avoid an oblique AC section. The authors would recommend that sonographers do this by aiming to have no kidney in the image, including an unbroken short rib and ensure that the spine is represented in cross section as three dots. Reaching a consensus on caliper placement within a department is also important. Following this study, the sonographers in the authors’ department aim to include equal skin thickness both anteriorly and posteriorly in their measurements as shown in Figure 2(b).

Conclusion

We should accept that the perfect AC section is not always possible in the third trimester and that sonographers use their professional judgement to determine whether an image is acceptable. Accurate fetal biometry is important as small variations in measurement can affect how women are managed. This study has shown that there are inconsistencies in opinion between the members of a departmental team as to what constitutes an acceptable third trimester AC image and how it is measured. These differences need to be recognised and addressed if the effectiveness of the third trimester clinical ultrasound service is to be maximized. To improve their third trimester image section further, sonographers should try to avoid an oblique AC section and aim for more consistent caliper placement. Peer review can be used to do this by monitoring scan quality to identify areas of inconsistency. Where performed regularly, peer review has the potential to minimize both intra- and inter-operator variability in AC measurements.