Three-dimensional view of out-of-plane artifacts in photoacoustic imaging using a laser-integrated linear-transducer-array probe

Abstract

Research on photoacoustic imaging (PAI) using a handheld integrated photoacoustic probe has been a recent focus of clinical translation of this imaging technique. One of the remaining challenges is the occurrence of out-of-plane artifacts (OPAs) in such a probe. Previously, we proposed a method to identify and remove OPAs by axially displacing the transducer array. Here we show that besides the benefit of removing OPAs from the imaging plane, the proposed method can provide a three-dimensional (3D) view of the OPAs. In this work, we present a 3D reconstruction method using axial transducer array displacement. By axially displacing the transducer array, out-of-plane absorbers can be three-dimensionally visualized at an elevation distance of up to the acquired imaging depth. Additionally, OPAs in the in-plane image are significantly reduced. We experimentally demonstrate the method with phantom and in vivo experiments using an integrated PAI probe. We also compare the method with elevational transducer array displacement and take into account the sensitivity of the transducer array in the 3D reconstruction.

1. Introduction

Photoacoustic imaging (PAI) is a promising medical imaging technique [[1], [2], [3], [4], [5]] thanks to its advantages over conventional techniques. PAI provides images with clinically sufficient imaging depth, high resolution, and optical contrast. It uses short pulsed laser light to illuminate the sample and detects the ultrasound (US) waves generated by optically absorbing substances. Photoacoustic (PA) images are then formed from the detected signals. The reconstructed images provide localized information of the presence of substances like oxy- and deoxyhemoglobin, melanin and lipids, also depending on the optical wavelength used. In clinical applications, such information may help to diagnose various diseases in early stages [4,[6], [7], [8]].

A conventional PAI system is mechanically bulky and relatively expensive. Reducing the size and cost is necessary for this technique to find a place in clinics. Integrating a low cost laser source, such as laser diodes, into a handheld US probe is an outstanding alternative [[9], [10], [11], [12], [13], [14]]. However, one of the main drawbacks of such a system is the occurrence of out-of-plane artifacts (OPAs), also called clutter [[15], [16], [17], [18]]. In in vivo imaging, OPAs appear as spurious objects in the imaging plane [18] which might lead to wrong conclusions. Correcting OPAs is therefore of importance.

Recently, we proposed a method of using transducer array displacement to identify and remove OPAs [18,19]. In this method, the transducer array is displaced axially while the sample and light source are fixated. Consequently, OPAs move along the displacement while in-plane image features remain at the same position. OPAs therefore can be identified and removed.

Here we demonstrate that though the imaging plane remains the same during the axial displacement, this method still can form a volumetric image providing a three-dimensional (3D) view of the OPAs, even with an axial displacement of a few millimeters only. This is a new advantage of our proposed method [18]. In fact, 3D imaging has been a focus in PAI research by using a 2D transducer array [20,21] or scanning a 1D transducer array [22,23]. Using a 2D transducer array requires considerable effort and experience of the user for image acquisition and interpretation. [24]. In addition, a system with a 2D transducer array is complex making it clinically unaffordable [24]. Using a 1D transducer array might overcome those limitations, however, scanning the array in the elevation direction is needed to form a 3D image. As a consequence, the field of view is then limited within the scanning distance.

In the previous work [18], image segmentation is used in order to identify moving features along the displacement as OPAs. The OPAs are then removed by setting their corresponding image pixels to 0. In this way, detected OPAs are completely removed out of the image however, background information behind the OPAs is also lost [18]. In our 3D reconstruction method presented here, OPAs are reduced in the in-plane image while the background information still remains.

To demonstrate the method, we used a handheld PAI probe showing potential benefit of this method to clinical use. We then compared this method, in terms of resolutions of in-plane and out-of-plane absorbers and signal-to-noise (SNR) ratio, with 3D imaging with elevational displacement of the transducer array. We also demonstrated it with in vivo experiments. Additionally, we improved the reconstruction with sensitivity compensation of the transducer array.

2. Method

We use x, y, and z to represent the elevation, lateral, and axial direction, respectively.

2.1. 3D reconstruction based on direct back-projection

Normally, data acquired by a 1D transducer array provides a 2D image representing the imaging plane of the transducer array. However, out-of-plane absorbers might appear in that reconstructed image [[17], [18], [19]]. A 1D transducer array, therefore, actually provides additional information outside of the imaging plane.

Fig. 1 presents a configuration of imaging an out-of-plane absorber and steps of forming a 3D projection. Fig. 1(a) shows a schematic drawing of a configuration in which a black nylon absorber is placed elevationally outside the imaging plane but still inside the light beam. Particularly, it is positioned ∼ 3.5 mm axially and ∼ 6.5 mm elevationally away from the transducer array. Demi-water is used as a coupling medium. The transducer array is in the direction perpendicular to the drawing plane. The photoacoustic imaging system is described in [18]. Fig. 1(b) is a reconstructed pressure distribution image (before the envelope detection) in the yz plane, using a Fourier transform based reconstruction algorithm [25]. An OPA is clearly visible in this image. Fig. 1(c) depicts the axial profile of the dashed red line in Fig. 1(b). This axial line is then back-projected in its own xz plane, seen in Fig. 1(d). A 3D projection is formed by back-projecting in the xz plane for all lines in the reconstructed pressure distribution image.

Fig. 1.

(a) Configuration for imaging an out-of-plane absorber. (b) Pressure distribution image. (c) Axial profile of the dashed red line in (b). (d) xz projection.

The probe is then axially displaced to a new position. A new 3D projection in the same ROI as the original one is formed at the new position, in the same way as described above. Fig. 2(a)–(b) present 2 axial lines of the original and new projection respectively. The signal from the out-of-plane absorber clearly moves up. These 2 3D projections are summed up together. Fig. 2(c) shows an xz image of the sum in which back-projected signals intersect at the position in agreement with the location of the out-of-plane absorber. Fig. 2(d) shows an image, in the same xz plane, of the sum of 6 3D projections. In the end, envelope detection is performed using the Hilbert transform providing a reconstructed 3D image. Fig. 2(e) shows a Hilbert transformed xz image of the reconstructed 3D image.

Fig. 2.

3D reconstruction based on direct back-projection. Axial profile at the original position (a) and a new position (b) of the probe. xz image of the sum of 2 (c) and 6 (d) 3D projections. (e) Hilbert transformed image.

A 3D projection can also be formed by back-projecting the RF signal of all transducer elements volumetrically. This approach gives the same result as the approach of back-projecting a 2D image described above. However, it is more time consuming, therefore the latter is preferred in this work.

It is worth noting that the elevation length of the reconstructed volume is equal to the acquired imaging depth and is not limited by the displacement distance. Thus out-of-plane absorbers can be visualized in an elevation distance of up to the acquired imaging depth as long as their signals are detected by the transducer array.

2.2. 3D reconstruction based on back-projection and multiplication

The number of displacement positions might be limited resulting in strong streak-like reconstruction artifacts as seen in Fig. 2(e). To reduce these artifacts, we use a reconstruction algorithm similar to the method proposed in [26,27]. Here we call it back-projection and multiplication (BPM). In principle, all projections, before being summed up together, are combinatorially coupled and multiplied.

Let $b_i$ denote a 3D projection at position $i$ of the transducer array. $b_i$ is a 3D matrix. In direct back-projection (BP), reconstructed pressure distribution, $p_{BP}$ is the sum of all projections:

$$p_{BP}=\sum _{i=1}^M{b}_i,$$ (1)

where M is the total number of projections, and in this case, it is the total number of displacement positions of the transducer array. In BPM, 2 projections $b_i$ and $b_j$ are coupled using signed geometric mean as [26]:

$${\hat{b}}_{ij}=sign(b_i{b}_j)\sqrt{\left|b_i{b}_j\right|},$$ (2)

where $b_i{b}_j$ is the element by element product of the two matrices $b_i$ and $b_j$. The reconstructed pressure distribution, $p_{BPM}$, in BPM is computed as [26]:

$$p_{BPM}=\sum _{i=1}^{M-1}\sum _{j=i+1}^M{\hat{b}}_{ij}.$$ (3)

$p_{BPM}$ is then band-pass filtered to remove the DC and high frequency components produced by multiplying the RF. In the end, $p_{BPM}$ is Hilbert transformed providing a 3D image.

Fig. 3 shows images in an xz plane using direct BP (Fig. 3(a)) and BPM (Fig. 3(b)). Streak-type reconstruction artifacts are observably reduced in BPM compared to BP. To provide more insight into reconstruction artifact reduction, we select signal, noise, and artifact windows as solid back, dotted white, and dashed white rectangles respectively, see in Fig. 3(a). The bigger artifact window covers a large area of reconstruction artifacts while the smaller one is on a specific reconstruction artifact. The yellow framed image at the bottom right corner of Fig. 3(a) is an enlargement of the absorber for a better visualization of the signal window.

Fig. 3.

Direct back-projection (a) vs back-projection and multiplication (b). Solid black, dotted white, and dashed white rectangles in (a) are signal, noise, and artifact windows respectively.

Let S, N, and A denote the mean value of all pixel values in the signal, noise, and artifact windows respectively. We then define signal-to-noise ratio (SNR) as S/N and artifact-to-noise ratio (ANR) as A/N. Table 1 shows a comparison of SNR and ANR between direct BP and BPM. While SNR is approximately the same, ANR is significantly reduced in BPM. This is one of significant advantages of BPM over direct BP presented in [26,27].

Table 1.

Direct back-projection vs back-projection and multiplication.

	BP	BPM
SNR	68.53	68.04
ANR (large window)	5.96	3.79
ANR (small window)	9.06	2.74

We have just presented a 3D reconstruction algorithm based on BPM for axial displacement above. For elevation displacement, the reconstruction is in principle the same. At each elevation displacement position, a 3D projection is created. BPM is then used to reconstruct a 3D image.

3. Experimental setup

Experiments were carried out using a handheld PAI probe, shown in Fig. 4. The probe is connected to a commercial US scanner MyLabOne (Esaote Europe BV, The Netherlands) which can acquire data at a sampling frequency of up to 50 MHz with 12 bit digitization. It was used in research mode to acquire data for off-line processing in an external PC (Matlab 2019b, Intel Core i7 3.41 GHz, 8 GB of RAM).

Fig. 4.

Integrated PAI probe.

In the probe, a US transducer array and a light source are integrated. The US transducer array comprises 128 elements with a pitch of 0.245 mm. It has a center frequency of 7.5 MHz with a bandwidth of 100 %. The light source, integrated into the probe, consists of two stacks of diode lasers bars (10 bars per stack). It can emit light at 2 different wavelengths (760 and 940 nm) at a repetition rate of up to 10 kHz. Table 2 presents specifications of the diode lasers working at a repetition rate of 0.5 kHz. As the 940 nm light has much higher pulse energy, we use light at this wavelength in our experiments.

Table 2.

Lasers specifications at a repetition rate of 0.5 kHz.

wavelength [nm]	pulse energy at the output [mJ]	pulse width [ns]	Fluence at the output (1/e2) [mJ/cm2]
760	0.58	63.6	0.74
940	1.66	53.4	2.01

The probe is mounted on 2 motorized translation stages (MTS50A-Z8, Thorlabs, Germany). These stages are used to displace the probe axially and elevationally.

4. Results

A total of 11 pressure distribution images at 11 displacement positions were acquired: 1 at the original position, 5 for axial displacements and 5 for elevation displacements. The displacement step size was 0.525 mm resulting in total 2.625 mm displacement axially and elevationally. Acquired raw data was applied a 5th-order high-pass Butterworth filter with a cut-off frequency of 0.5 MHz and averaged over 100 pulses. We then reconstructed pressure distribution images using a Fourier transform based reconstruction algorithm [25].

We first determined axial, lateral, and elevation resolutions of 3D images and compared the resolutions between axial and elevation displacement. We then demonstrated the reconstruction algorithm with in vivo experiments. We also measured the sensitivity of the transducer array which afterwards we used to improve our 3D images.

In all supplementary videos presenting 3D reconstruction, we added an orange frame into the 3D images to represent the imaging plane.

4.1. Resolutions of an in-plane image feature

Fig. 5(a) shows the probe with an associated coordinate system. Fig. 5(b) illustrates the experiment configuration from a side view of the probe. An absorber cut out from a black suture (USP 5/0, Vetsuture Nylon, The Netherlands) with a thickness of ∼ 140 $\mu m$ (smaller than the imaging resolutions of this probe) was used in this experiment as an acoustically point-like absorber. This absorber was embedded in agarose (2%) in a petri dish (Greiner Bio-One GmbH, Germany) and placed underneath the transducer array inside the imaging plane, see Fig. 5(b)). The coupling medium was demi-water.

Fig. 5.

In-plane absorber configuration. (a) PAI probe with an associated coordinate system. (b) Schematic elevation view of the experiment configuration.

Fig. 6 shows results of 3D reconstruction using axial displacement (Fig. 6(a)–(b)) (see also Appendix A, Video 1) and elevation displacement (Fig. 6(c)–(d)) (see also Video 2). Columns from left to right are in-plane images and in an xz plane crossing the absorber, respectively. Images in the same row share the same color bar.

Fig. 6.

Reconstructed images using axial ((a)-(b)) and elevation ((c)-(d)) displacement. (a) and (c) In-plane images. (b) and (d) Images in an xz plane crossing the absorber.

The SNR in the reconstruction using elevation displacement is considerably lower than using axial displacement. Consequently, electrical noise at the depth of ∼ 2 mm is more pronounced in the reconstructed image. The reason is that in elevation displacement, in-plane absorbers relatively move away from the imaging plane and the sensitivity of the transducer decreases further outside of the imaging plane.

3D reconstruction using axial displacement does not provide elevation information of in-plane absorbers. The reason is that the imaging plane does not change elevationally. It can be seen in Fig. 6(c) that the point-like source is reconstructed as a curve elevationally. However, this is not the case in using elevation displacement despite of strong effect of streak-type reconstruction artifacts.

Fig. 7 shows elevation, lateral, and axial profiles (top to bottom) of the reconstructed image feature using axial, and elevation displacement (left to right). All figures share the same y label shown on the left. The profiles are fitted into Gaussian curves. Resolutions are then determined as the FWHM of the fitted Gaussian curves, shown in Table 3.

Fig. 7.

Profiles of the reconstructed image features using axial (left column) and elevation (right column) displacement. From top to bottom: elevation, lateral, and axial profiles.

Table 3.

Resolutions of an in-plane image feature.

	Axial displacement [mm]	Elevation displacement [mm]
relevation	2.43	1.38
rlateral	0.36	0.38
raxial	0.30	0.29

The lateral and axial resolutions are approximately the same in the reconstruction using axial and elevation displacement while the elevation resolution is better (almost 2 times) with using elevation displacement.

4.2. Resolutions of an out-of-plane artifact

The same experimental configuration as described in section 4.1 was used. The black absorber in this case was placed ∼ 4 mm axially and ∼ 4.5 mm elevationally away from the transducer array, see Fig. 8(a). Fig. 8(b)-(e) are results of 3D reconstruction using axial ((b)-(c)) (see also Video 3) and elevation ((d)-(e)) (see also Video 4) displacement. Images in the same row share the same color bar on the right. Fig. 8(b) and (d) and (c) and (e) show images in an yz and an xz plane crossing the absorber respectively.

Fig. 8.

Out-of-plane absorber imaging. (a) Schematic elevation view of the experiment configuration. (b)-(c) and (d)-(e) are reconstructed images using axial and elevation displacement respectively. (b) and (d) Images in an yz plane crossing the absorber. (c) and (e) Images in an xz plane crossing the absorber.

Table 4 presents the resolutions of an out-of-plane artifact in 3D reconstruction using axial and elevation displacement. The axial and lateral resolutions are slightly better with using elevation displacement. This is because in elevation displacement, out-of-plane absorbers relatively move towards the imaging plane. The elevation resolution, on the other hand, is approximately the same.

Table 4.

Resolutions of an out-of-plane artifact.

	Axial displacement [mm]	Elevation displacement [mm]
relevation	0.53	0.54
rlateral	0.39	0.36
raxial	0.40	0.32

It is worth noting that the resolutions also depend on the location of the absorber. However, this study of in-plane and out-of-plane absorber imaging provides some insight into comparison between reconstruction using axial and elevation displacement.

4.3. In vivo experiment

We also demonstrated the reconstruction using axial displacement with in vivo experiments. We compared the results with the reconstruction using elevation displacement. Fig. 9 shows the configuration of the in vivo experiments. We imaged a volunteer’s finger with a black ink mark on the skin mimicking a strongly absorbing epidermis or objects like birth marks as an out-of-plane absorber. The finger was positioned in such a way that the ink mark was a few millimeters away from the transducer array elevationally but still inside the light beam, see Fig. 9(a). Demi-water was used as a coupling medium. Fig. 9(b) shows an acquired PA image. Features (dashed white circle) move up along with the axial displacement (seen in Video 5) revealing them as OPAs. They appear at the position that should be expected from the elevational location of the ink mark. Probably, these are OPAs of the ink mark.

Fig. 9.

In vivo experiment. (a) Experiment configuration. (b) Acquired PA image. Dashed circle indicates OPAs.

At a certain distance of the axial displacement and beyond, OPAs disappear, seen in Video 1. The reason is that at that distance, the ink mark was outside of the light beam. Furthermore, few features move down during the displacement. These are probably reflections caused by signals generated due to the absorption by the transducer array and reflected back off the skin and bone layers. These reflections move down since the distance of the probe to the reflectors increases during the displacement.

Fig. 10 shows volume renderings of 3D reconstruction using axial (Fig. 10(a)–(c)) (see also Video 6) and elevation (Fig. 10(d)–(f)) (see also Video 7) displacement. For a better visualization, 3D images are thresholded one sixth of each image’s maximum. Images in the same row share the same color bar on the right. The 3D images are presented in different view angles (in different columns). The solid black arrow in all images indicates the ink mark. It is ∼ 4 mm away from the imaging plane.

Fig. 10.

In vivo 3D reconstruction using axial displacement ((a)-(c)) and elevation displacement ((d)-(f)). Solid black arrows indicate the ink mark. Dashed black arrow indicates some blood vessels.

Only some blood vessels can be visualized after thresholding in the case of using elevation displacement. This is due to the SNR of the in-plane image being lower in the reconstruction using elevation displacement than using axial displacement, as pointed out in Section 4.1. As also mentioned in Section 4.1, reconstruction using axial displacement does not provide elevation information of in-plane absorbers. This can be seen again in Fig. 10(b) in which skin and blood vessels appear as curves elevationally. In contrast, the blood vessels appear as thread structures elevationally (dashed arrow, Fig. 10(e)) in reconstruction using elevation displacement.

4.4. Out-of-plane artifact reduction

In addition to providing a 3D view of out-of-plane absorbers, the 3D reconstruction using axial transducer array displacement also significantly reduces OPAs in the in-plane images. In this way, segmentation is not needed as in the method used in the previous work [18]. In this section, we will further analyze the experimental results presented in Sections 4.2 and 4.3 to give more insight into OPA reduction, and compare with the previous method using segmentation.

Fig. 11 shows images of the out-of-plane absorber phantom (upper row) and in vivo experiment (lower row) presented in Sections 4.2 and 4.3 respectively. All images in the same row share the same color bar shown at the end of each row. For a better visualization, images of the out-of-plane absorber phantom are linearly scaled to have the same dynamic range while images of the in vivo experiment are in their own dynamic range. Images in different columns in order from left to right are acquired images, in-plane images of the 3D reconstruction using axial transducer array displacement, and OPA removed images using the previous method [18] (axial transducer array displacement with segmentation). Solid and dashed rectangles represent OPA and noise windows respectively.

Fig. 11.

OPA reduction. Upper row: images of the out-of-plane absorber phantom presented in Section 4.2. Lower row: images of the in vivo experiment presented in Section 4.3. Left column: original acquired images. Middle column: in-plane images of the 3D reconstruction using axial displacement. Right column: OPA removed images using the previous method [18]. Solid and dashed rectangles represent OPA and noise windows respectively. Arrows indicate removed image features.

As OPAs move up during the axial displacement, they are averaged out in the in-plane image of the 3D reconstruction, seen in Fig. 11(b) and (e), whereas they are removed by setting the corresponding pixel value to 0 using the previous method, seen in Fig. 11(c) and (f). The white arrows in Fig. 11(f) show in-plane image features that are partly removed.

Table 5 presents an ANR comparison between acquired PA images and in-plane images of the 3D reconstruction using axial transducer array displacement. As the pixel value of the OPAs is set to 0 in the images of using the previous method, so they are not included in this ANR analysis.

Table 5.

ANR comparison between acquired PA images and in-plane images of the 3D reconstruction using axial transducer array displacement.

	Acquired image	3D reconstruction
Phantom	3.23	1.60
In vivo	6.35	4.15

It can be seen that OPAs are clearly reduced in the image of the 3D reconstruction using axial transducer array displacement. The reflections of the transducer array noted in Section 4.3 are also averaged out as they move down during the axial displacement, seen in Fig. 11(e). These artifacts (the OPAs and the reflections) are completely removed in the images of using the previous method. Some image features (marked with white arrows) are also removed due to some slight movement during the experiment, seen in Fig. 11(f).

4.5. xz sensitivity compensation

Compensating the sensitivity of the transducer array might improve the reliability of the reconstructed image. The sensitivity compensation can be achieved by weighting the back-projected signals [28]. In this work, the out-of-plane sensitivity of the transducer array is much lower than the in-plane sensitivity. Consequently, the reconstructed initial pressure of out-of-plane absorbers is much lower than its real initial pressure. We experimentally measure the sensitivity of the transducer array and take it into account in the 3D reconstruction.

Fig. 12(a) illustrates the configuration of measuring the sensitivity of the transducer array. A US probe which consists of only a transducer array was used. The transducer array is identical to the one in the PAI probe. The US probe was mounted on 2 translation stages which could translate axially and elevationally. The sample was a black absorber embedded in agarose as described in Section 4.1. The light source (at a wavelength of 790 nm), described in [18], and sample were fixated. The coupling medium was a suspension of 1% Intralipid 20 % in demi-water.

Fig. 12.

Measuring the spatial sensitivity distribution of the transducer array. (a) Schematic experiment setup. (b) xz sensitivity.

We translated the probe as a raster scan in the xz plane. At each position of the probe, one image was acquired with averaging of 100 laser pulses. In such a way, we could neglect the laser fluctuation and assume that the initial pressure from the absorber was a constant. The maximum pixel value of the absorber in each image was extracted as a sensitivity value at the corresponding position of the probe. Fig. 12(b) shows a sensitivity map extracted from the whole scan. The 0 mm on the x axis presents the position when the absorber was exactly underneath the probe.

The step size of the raster scan was larger than the spatial size of a pixel in 3D reconstruction. We use interpolation to match the spatial pixel size of the sensitivity map with reconstructed images.

We assume that all transducer elements have the same xz sensitivity map. Therefore, we apply the xz sensitivity compensation map equally laterally to the reconstructed 3D image. This point will be discussed further in the Discussion section 5. Additionally, the sensitivity of the transducer array also depends on the acoustic wavelength of the source. This, however, was not considered in the sensitivity compensation. We will also discuss further this point in the Discussion section 5.

In 3D reconstruction using axial or elevation displacement, the sensitivity of the transducer array is also shifted corresponding to the displacement. We thus shift the sensitivity map to match the displacement and expand it laterally to a 3D sensitivity map. We then divide (element by element division) the corresponding 3D projection by the 3D sensitivity map. All sensitivity compensated 3D projections are then used for reconstruction using BPM.

4.5.1. Phantom validation

We first validate the sensitivity compensation with a phantom experiment. The experiment configuration is shown in Fig. 13(a). It is similar to the configuration described in Section 4.1. There is one absorber placed inside the imaging plane and another ∼ 3 mm outside of the imaging plane. Fig. 13(b) and (d) are volume renderings of 3D reconstruction using axial and elevation displacement (see also Video 8 and Video 9), respectively. These 3D images are thresholded in such a way to visualize the two absorbers. The arrow in Fig. 13(b) indicates a reconstruction artifact while most of the reconstruction artifacts are thresholded out in the reconstruction using elevation displacement. Sensitivity compensation is then applied to these images resulting in images on the right (Fig. 13(c) and (e)) (see also Video 10 and Video 11). All images are displayed in their own dynamic range with a color bar on the top.

Fig. 13.

In-plane and out-of-plane absorber imaging. (a) Experiment configuration. 3D images using axial displacement without (b) and with (c) sensitivity compensation respectively. 3D images using elevation displacement without (d) and with (e) sensitivity compensation. Arrows indicate a strongly enhanced reconstruction artifact.

With sensitivity compensation, the signal of the out-of-plane absorber is considerably enhanced. This is reasonable since the out-of-plane absorber was placed closer to the light output and to the center of the laser beam compared to the in-plane absorber.

Reconstruction artifacts that are elevationally far away from the imaging plane are strongly enhanced with the sensitivity compensation. As a result, they are even more pronounced than real image features. A reconstruction artifact marked with an arrow in Fig. 13(b) is an example. This artifact is significantly enhanced with the transducer sensitivity taken into account, seen in Fig. 13(c).

4.5.2. In vivo validation

Fig. 14 shows results of the in vivo experiment with sensitivity compensation. From top to bottom are images without and with sensitivity compensation alternatively row by row. The first 2 rows and the last 2 rows are results of reconstruction using axial displacement and elevation displacement respectively. Images in the same row have the same dynamic range and each row has its own dynamic range. Thus the color bar has a dynamic maximum. The solid arrow in all images indicates the ink mark.

Fig. 14.

In vivo 3D reconstruction with sensitivity compensation. 3D images using axial displacement without ((a)–(c)) and with ((d)–(f)) (see also Video 12) sensitivity compensation. 3D images using elevation displacement without ((g)–(i)) and with ((j)–(l)) (see also Video 13) sensitivity compensation. Solid arrows indicate the ink mark, dashed arrows indicate strongly enhanced reconstruction artifacts.

In the images with sensitivity compensation, the signal of the ink mark is considerably stronger than the signal from the skin and blood vessels. This is reasonable since the ink mark was positioned closer to the light output resulting in higher local light fluence. In addition, the ink mark is more absorbing than the skin and blood vessels.

The sensitivity map was measured from ∼ 2.5 mm on the z axis and up to ∼ 5.3 mm on the x axis. This is the reason why sensitivity compensated images appear as cropped at those boundaries.

The dashed arrow in Fig. 14(b) indicates a reconstruction artifact which is significantly enhanced with the transducer sensitivity taken into account, seen in Fig. 14(e). Similarly, Fig. 14(h) and (k) present another example in the case of using elevation displacement.

5. Discussion

The occurrence of OPAs negatively influences the quality and reliability of the images in PAI. Correcting OPAs thus benefits the translating of PAI to clinics. In addition to the capability of identifying and removing OPAs, our proposed method provides a 3D view of out-of-plane absorbers. Moreover, it offers significant advantages over existing 3D imaging methods in terms of visualizing out-of-plane absorbers. First, for a given measurement time, the proposed method enhances the quality of the in-plane images since the probe remains in the imaging planes and adds up all in-plane images while this is not the case with elevation displacement. Secondly, the elevation visualization is not limited by the displacing distance as in the case of 3D reconstruction by scanning a linear transducer array [22,23]. The proposed method can spatially visualize in a larger extent than the displacing distance, up to the acquired imaging depth. Thirdly, the method does not require a 2D transducer array for the 3D reconstruction as in [20,21]. 2D transducer arrays require complex associated systems resulting in unaffordable cost for many clinical applications. Lastly, we demonstrated the method using a linear-transducer-array probe with integrated lasers showing its promising potential for clinical use.

In the proposed 3D reconstruction method, OPAs and reflections of the transducer array are averaged out in the in-plane image as shown in Section 4.3. In this way, background information behind the OPAs still remains whereas it is removed along with OPAs in our previous work [18]. Additionally, reducing OPAs with the proposed method does not rely on segmentation, and is more robust to slight movement of the sample than in the case of the previous method.

Though the 3D reconstruction using axial displacement can three-dimensionally visualize out-of-plane absorbers, it cannot show in-plane absorbers in an elevationally correct manner. All in-plane absorbers appear in a curve shape elevationally. This is the main drawback of this method. However, our main scope is to correct in-plane images and visualize out-of-plane absorbers.

Unlike in our previous work [18], the light source is also displaced in this work since it is integrated in the probe, resulting in changes of the illumination during the displacement. However, since the displacement was just a few millimeters, we did not observe significant influence of the illumination changes on our reconstructed image. On the other hand, validating our proposed method with a compact and low-cost PAI probe shows its potential for clinical application.

A limited number of displacement positions might lead to strong streak-type reconstruction artifacts. In this work, we used back-projection and multiplication (BPM) which in principle is similar to delay-multiply-and-sum [26,27] to improve the reconstruction. Applying methods to further reduce the reconstruction artifacts, such as [29,30], will be included in our future work.

BPM improves the reconstruction at a cost of longer processing time due to a greater amount of calculation [26]. The amount of calculation is proportional to M (number of projections) in direct BP, and M(M-1)/2 in BPM. In this work, for a ROI of 18 $\times$ 16 $\times$ 18 mm³ (x $\times$ y $\times$ z) it took ∼ 5 s for each reconstruction run. In this amount of processing time, creating 6 3D projections took ∼ 0.48 (6 $\times$ 0.08) seconds while BPM took ∼ 4.5 s. Using a faster reconstruction algorithm [31] would significantly reduce the reconstruction time.

In our previous work [18,19], it took 2−3 min for each displacement experiment due to low repetition rate of the laser (20 Hz) and slow translation stages. In this work, the experiment time was reduced to a few tens of seconds thanks to the high repetition rate of the integrated laser. However, this is still far from real-time imaging. The main part the experiment time was due to the low speed of the translation stages. In addition, we did not use the highest repetition rate of the laser, 10 kHz (1 kHz was used). Using faster translation stages and a higher repetition rate of the laser might significantly reduce further the acquisition time.

As pointed out in [18], the principle of identifying and removing OPAs also holds for lateral out-of-plane absorbers. OPAs of lateral out-of-plane absorbers still move along the axial displacement of the probe. Thus visualizing lateral out-of-plane absorbers is also feasible.

As shown in Sections 4.1 and 4.2, elevationally displacing the probe can also form a 3D view of out-of-plane absorbers with more elevation information and a better elevation resolution of in-plane image features compared to the proposed method. However as a cost, for a given total measurement time the quality of the in-plane image is significantly worse, shown in Sections 4.1 and 4.3. Therefore, the proposed method is more favorable in scope of correcting OPAs in the in-plane image.

In sensitivity compensated reconstruction, we used a sensitivity map measured on an acoustically point-like source for other samples. This is not an ideal way for compensating the sensitivity since the sensitivity of the transducer also depends on the acoustic wavelength of the source. In PAI, especially in in vivo imaging, the acoustic wavelength of the signal might be unknown leading to erroneous compensation for the sensitivity. On the other hand, we also applied a measured sensitivity map to all transducer elements assuming they are all the same. This might be incorrect due to interferences between transducer elements, hence we did not include the influence of the sensitivity compensation on artifact reduction. Applying a correct sensitivity map of the transducer array will make the sensitivity compensation more proper.

6. Conclusion

We have demonstrated a three-dimensional (3D) reconstruction approach by using axial transducer array displacement. This is a new advantage of our recently proposed method [18,19] for identifying and removing out-of-plane artifacts (OPAs). By displacing the transducer array axially for a few millimeters only, we could three-dimensionally visualize out-of-plane absorbers within an elevation distance of up to the acquired imaging depth and also reduce OPAs in the in-plane image. We also compared the results with the 3D reconstruction using elevation displacement. We then improved the reconstruction by taking the sensitivity of the transducer array into account. In vivo demonstrations using a compact photoacoustic imaging system show promising clinical applicability of our proposed method.

Funding

European Union’s Horizon 2020 research and innovation programme (CVENT, H2020-731771)

Declaration of Competing Interest

The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.

Biographies

Ho Nhu Y. Nguyen (M.Sc.) received a M.Sc. degree in Optics and Photonics from the Karlsruhe Institute of Technology, Germany and Aix-Marseille University, France in 2016 as a fellow of the Erasmus Mundus programme. He joined the Institute for Biological and Medical Imaging at the Helmholtz Zentrum München, Germany for his master thesis on photoacoustic microscopy. He is currently a Ph.D. candidate at the Biomedical Photonic Imaging group, University of Twente, The Netherlands and his current research focuses on the development of a compact, low cost photoacoustic imaging system and clinical translation of this imaging technique.

Wiendelt Steenbergen obtained a Master degree in Aerospace Engineering at the Delft University of Technology (1988), a PhD degree in fluid dynamics at the Eindhoven University of Technology (1995) and joined the University of Twente, Enschede (the Netherlands). In 2000 he was appointed assistant professor in biomedical optics and broadened his scope to low-coherence interferometry and photoacoustic and acousto-optic imaging. In 2010 he became full professor and group leader of the Biomedical Photonic Imaging group of the University of Twente. His current research interests include speckle based perfusion imaging, combined photoacoustic and ultrasound imaging applied to rheumatology, and quantification of photoacoustic imaging using acousto-optics.

Appendix A. Supplementary data

The following are Supplementary data to this article: