Reducing radiation dose for NN-based COVID-19 detection in helical chest CT using real-time monitored reconstruction

Abstract

Computed tomography is a powerful tool for medical examination, which plays a particularly important role in the investigation of acute diseases, such as COVID-19. A growing concern in relation to CT scans is the radiation to which the patients are exposed, and a lot of research is dedicated to methods and approaches to how to reduce the radiation dose in X-ray CT studies. In this paper, we propose a novel scanning protocol based on real-time monitored reconstruction for a helical chest CT using a pre-trained neural network model for COVID-19 detection as an expert. In a simulated study, for the first time, we proposed using per-slice stopping rules based on the COVID-19 detection neural network output to reduce the frequency of projection acquisition for portions of the scanning process. The proposed method allows reducing the total number of X-ray projections necessary for COVID-19 detection, and thus reducing the radiation dose, without a significant decrease in the prediction accuracy. The proposed protocol was evaluated on 163 patients from the COVID-CTset dataset, providing a mean dose reduction of 15.1% while the mean decrease in prediction accuracy amounted to only 1.9% achieving a Pareto improvement over a fixed protocol.

1. Introduction

Computed tomography (CT) today has moved to the front line of medical diagnostic imaging (Hsieh and Flohr, 2021, Scott and McCann, 2018), and the development of the methods of X-ray computed tomography continues to evolve, along with the methods of automatic medical image analysis (Wang et al., 2020, Zhang, Xie, et al., 2021, Zhou et al., 2019). The introduction of helical computed tomography in the early 1990s (Kalender et al., 1990) provided the conditions necessary for quick scanning of complete organs such as the lungs or body sections (White, 1996). In the helical CT, the patient is moved slowly through the gantry during continuous rotation of the X-ray tube, and thus the X-ray tube covers a helical trajectory around the patient. The corresponding issues with the scanning quality associated with the movement of the table feed have been carefully studied (Polacin et al., 1992), and multiple tomographic reconstruction algorithms have been proposed to work with this protocol (Bruder et al., 2000, Schöndube et al., 2010).

Helical chest CT is often used as a methodology for a medical examination for the purpose of diagnostics of such diseases as COVID-19 (Brodeur et al., 2021). In the context of a global pandemic, however, it requires the time of several radiology specialists to manually inspect each scanning result (Hani et al., 2020). Due to shortages of medical staff, automatized methods associated with COVID-19 screening began to develop. Some works (Amyar et al., 2020, Hu et al., 2020, Oh et al., 2020) presented deep learning frameworks which are capable of detection and localization of lesions in CT scans. A deep learning-based model for automatic COVID-19 detection in chest CT was developed (Zheng et al., 2020). The algorithm achieved sensitivity and specificity values larger than 0.9, which is clinically applicable. In Rahimzadeh et al. (2021) a high-speed system was presented for accurate and fully-automated COVID-19 detection from the chest CT scan images. This system is based on the combination of ResNet50V2 and FPN (Feature Pyramid Network) architectures and is able to achieve 98.49% accuracy in per-slice image classification task on 7996 test images.

A significant issue associated with CT scans is the radiation exposure (Wiest et al., 2002) since there is evidence that the radiation dose values administered during modern CT scans are enough to possibly contribute to the cancer occurrence (Smith-Bindman, 2015). The total radiation dose is proportional to the number of projections collected, radiation intensity and exposition time and a lot of efforts are being made to achieve the reduction of each of these contributions. However, such dose reduction inevitably leads to the lowering of image quality. To compensate for the quality loss associated with dose reduction, specifically designed approaches are applied during the reconstruction phase (Matenine et al., 2020, Villarraga-Gómez and Smith, 2020), including deep learning-based methods for regularization (Zhang, Liu, et al., 2021), prediction of the missing projections (Anirudh et al., 2018) or even performing reconstruction from two orthogonal X-rays using the generative adversarial network framework (Ying et al., 2019). The task of dose reduction is often approached from the hardware side of CT, with one of the most recent examples being photon-counting detectors (Becker et al., 2023). More so, whole new acquisition protocols and geometry setups are proposed, such as cycloidal CT (Hagen et al., 2020), together with the neural network (NN) frameworks aimed to compensate for the quality loss introduced by these new dose reduction protocols (Pelt et al., 2022). Despite a variety of approaches to address the problem of low-dose CT, these methods represent fixed scanning protocols.

A serious disadvantage of the fixed protocol dose reduction methods is that the quality of reconstruction is not known until the very end of the acquisition process, when the entire dose has already been administered. This means that if the reconstruction of medically unacceptable quality was produced (i.e. the dose reduction was too extreme) the examination process will have to be repeated and the dose reduction method could actually result in dose amplification.

This issue is understood in the CT community and various attempts have been made to approach it. Thus “on-the-fly” reconstruction is already implemented for micro-CT (Schwartz et al., 2022) in order to perform visual quality evaluation and control during the acquisition process. Another method utilizes a real-time reconstruction to estimate the optimal angle and position for acquiring of the subsequent projections (Wang et al., 2022). In spite of significance of these approaches, demonstrating the necessity of real-time reconstruction, this concept is applied to micro-CT setups, where a single experiment may take hours or even days. In medical setups the time scale is much shorter, and the quality estimation must be automatic rather than visual and operator based.

An alternative approach of monitored tomographic reconstruction (MTR) was introduced (Bulatov et al., 2020) and investigated (Bulatov, Chukalina, et al., 2021, Bulatov, Mukovozov, and Arlazarov, 2021) which focuses on real-time estimation of whether the information obtained during the scanning is enough for a good reconstruction. Essentially, MTR regards the process of tomographic reconstruction as an anytime algorithm (Dean & Boddy, 1988) – a process which could be interrupted at any time, and which yields results with increasing quality while obtaining either new observations or consuming additional computational resources.

MTR can be understood as an advanced method of control over the acquisition protocol, which optimizes the acquisition for a pre-chosen reconstruction algorithm and its parameters within the chosen measurement task. The projection data collected during the MTR can still be used to perform reconstruction with different algorithm or parameters, but MTR would not provide the assurance that the goal (which may, in example, be formulated as achieving some pre-set average image quality) in this case will also be achieved, while it does provide such assurance in the case of original algorithm with condition that the goal can be achieved for reconstruction from full-dose data. MTR is generally consist from three main parts: (1) reconstruction from partial, currently available data, i.e. partial reconstruction; (2) estimation of the sufficiency of collected data to solve the task experiment is designed for; (3) decision making of whether to proceed or stop the acquisition process based on reconstructions history by applying some stopping rule. In the scope of this method, the scanning process may stop at different times for different objects, with an increase of mean effectiveness in terms of the tradeoff between the reconstruction quality and administered dose. The mean reduction of the dose with retaining the same mean accuracy compared to the fixed protocol was demonstrated using the data obtained with a laboratory microtomography setup (Bulatov et al., 2020) and a nano X-ray computed tomography setup (Bulatov, Chukalina, et al., 2021) in case of 2D sections reconstruction with a random protocol of acquiring X-ray projections.

The goal of this paper is to apply monitored reconstruction approach to helical geometry CT in the framework of COVID-19 detection in chest CT scans for achieving a dose reduction. To make it easier to navigate the paper we would like to summarize our main contributions in advance and then briefly describe the content of the different sections. We consider our main contributions as:

1. Creating helical CT acquisition protocol suitable for the MTR task.

2. Creating the MTR framework for dose reduction, reasonably independent of the “expert” used for decision making.

3. Implementing the algorithm for COVID-19 detection task on the base of introduced MTR framework and NN-based decision making, with an extensive validation of the algorithm in numerical experiments.

4. Demonstrating that MTR leads to a Pareto improvement of the achieved dose reduction and image quality over a fixed protocol in similar conditions.

Implementing of MTR is associated with several difficulties, each of one will be dealt individually. First, we are required to construct such an acquisition protocol which will allow us to apply MTR within itself in the first place. This is done by introducing a by-sector acquisition in Section 3.1. Second, the method to estimate the sufficiency of collected data should be chosen. For this purpose, considering the task of COVID-19 detection, we propose to use a pre-trained COVID-19 detection NN model (Rahimzadeh et al., 2021) as an expert (Section 3.2). Next, a stopping rule for this protocol should be developed. To achieve this, we study the behaviour of NN predictions on the single slice basis for reconstructions from partial projection data (Section 3.3). Finally, we construct a full working MTR protocol from these parts (Section 4) and validate it (Section 5) on the public dataset (Rahimzadeh et al., 2021), which was also used to train the NN model (Rahimzadeh et al., 2021). Finally, the results are discussed in Section 6, and the conclusion together with potential future work are given in Section 7.

2. The problem statement and terminology

In this section we formally introduce the main problem to solve together with the relevant terminology. We start with the problem formulation, leading with basic definitions appearing in helical CT and then move to the dose reduction problem discussion from the point of view of MTR. We aim to apply MTR to the task of automatic COVID-19 detection on CT scans for achieving dose reduction. Since medical CT setups are dominantly work with helical geometry, we would need to apply MTR to a helical CT which, as we will see, is a challenging task in itself.

To better understand the issues associated with implementing MTR in helical CT, let us briefly consider the differences between circular and helical geometries. In circular CT the object is being radiographed from different angles, with each measurement resulting in a single projection of the whole object registered by the detector. In this sense, each projection is determined by a single angle $\left(b\right)$. The classical acquisition process is not limited by the consecutive angle increment, although it is often performed that way. The helical CT is different in a major way. The source of X-ray and detector are rotating while the object is moving at a constant speed and each projection in helical CT only registers some part of the object. This can be understood as the source and detector going around the object in a helical path, and each projection is determined by a single point on this path $(b,z)$. It is very important for helical CT to maintain rotation and object feed speeds at a constant level, which means that the acquisition has to be incremental. After the acquisition is finished, the 3D image is reconstructed. The 3D image consists of 2D images, i.e. layers or slices, which are located within a fixed distance between them (i.e. slice thickness).

Helical CT is dominant for medical implications and the important issue within it is the X-ray dose administered to the patient. To simplify our explanation, we can think that each registered projection roughly corresponds to some average dose level $c$, and a total dose can be calculated as $N_p\times c$, where $N_p$ is the total projections number. One way to reduce the full dose is to reduce $c$, which depends on X-ray intensity and spectrum, exposition time, etc. One of the most successful examples of such dose reduction protocols applied in real CT setups is Automatic Exposure Control (AEC), which is based on X-ray intensity modulation calibrated by prescan data (Söderberg & Gunnarsson, 2010). The other way is to reduce $N_p$, which is the main dose reduction mechanism in MTR. Although MTR should not compete with AEC, since these protocols do not share the same mechanism of dose reduction and it is expected that it would be possible to effectively combine them together.

As was pointed out in the introduction, MTR is generally consist of three main parts: reconstruction from partial, currently available data, i.e. partial reconstruction; estimation of the sufficiency of collected data to solve the task, the experiment is designed for; and finally making a decision whether to proceed or to stop the acquisition process based on partial reconstructions history by applying constructed and calibrated stopping rule.

For the first part of MTR, an essential property of acquisition protocol is the ability to stop at different points of acquisition to perform a reconstruction, which is not usually provided by a helical scan protocols. The second part of estimation of data sufficiency is, to some extent, external to MTR protocol in a way that we need to have some “expert” which will decide if the currently produced reconstruction is enough to solve such tasks as COVID-19 detection, and more importantly the expert should provide some “metric” of its confidence in the result. When the task to apply MTR and the metric to be evaluated are chosen, the stopping rules should be adopted and calibrated to achieve a reasonable trade-off between the dose reduction and the confidence loss measured by the provided metrics.

Since MTR was introduced in Bulatov et al. (2020) it was only studied in the circular type geometry, and there is a good reason for this. The implication of MTR in circular geometry is quite straight forward once the general framework is formulated. In these conditions, each consecutive projection adds an equal amount of information for the whole volume and the stopping may happen at any time. The task for MTR was generally formulated as achieving dose reduction by maintaining the reconstruction quality, measured against full dose reconstruction with using $\mathcal{L}2$ norm, on some predefined level. This metric depends almost monotonously on the total projection count and allows one to effectively estimate the change in image quality without knowing the reference full dose image.

On the contrary, implementation of MTR to the task of automatic COVID-19 detection in the helical CT framework, looks quite challenging from the very start. First of all, we need to construct such a helical acquisition protocol allowing for non-uniform projection count reduction. Second, some metric adequate to the COVID-19 detection task should be chosen and the stopping criteria should be constructed, which should be able to be applied to multiple parts of volume independently. We further aim to consecutively solve these problems.

3. Materials and methods

3.1. Acquisition protocol

We start with introducing a helical acquisition protocol, suitable for MTR framework. For this task, the certain restrictions imposed by the application of MTR to a helical geometry on acquisition protocol should be taken into account. Namely, the thickness of slices should be predefined, since the stopping criteria will have to be applied to each slice individually. This leads to 3D image being divided into $K$ slices, where $K$ can be defined by experiment settings and pre-scan data. In general, slice thickness is usually chosen after the acquisition process is finished and may be varied in quite a wide range, although some manufacturers provide the automatic optimization of scan parameters for the desired z-axis resolution (Ulzheimer & Freund, 2012). Further, once some stopping criterion is met, one should be able to stop the acquisition for the current slice, and move to another. The last requirement seems to be quite hard to met since modern CT setups utilize multi-row detectors with 16 to 256 rows, which correspond to a single projection taking part in reconstruction of several slices. Regardless, below we will demonstrate that it is indeed possible to construct a protocol meeting these requirements.

Let us consider a single projection and the volume of reconstruction. We define a slice of volume to be affected by projection, when there is at least one ray from a source to a detector cell passing through this slice (Fig. 1). This way, for a fixed slice number $k$ there is always a fixed range of projections affecting this slice $[B\left(k\right),U\left(k\right)]$, of a total $N\left(k\right)=U\left(k\right)-B\left(k\right)+1$ projections count. This range constitutes the lifespan of a slice. The same way for each projection there is a range of slices affected by it. The simplest “trivial” stopping criterion possible is stopping by reaching a fixed number of projections per slice. In this case, if a fixed projection count affected the slice $k^{\prime }$ at the projection number $R\left(k^{\prime }\right)$, the acquisition proceeds from the first projection that does not affects the current slice, which is $U\left(k^{\prime }\right)+1$, and the range $r\left(k^{\prime }\right)=U\left(k^{\prime }\right)+1-R\left(k^{\prime }\right)$ of projections is skipped. Note, that for all the following slices which are also affected by the skipped projections, the total possible projection count per slice will change by not more than $r\left(k^{\prime }\right)$. This means that such a “trivial” stopping rule will result in a constant number of projections affecting each individual slice, although the acquired projections will not be uniformly distributed across the scanning path. Obviously, this mechanism of dose reduction is not very optimal, since it is much easier to just reduce the projection count uniformly, but this protocol does allow to apply MTR by replacing the stopping criteria based on reconstructions from partial projection data.

Fig. 1.

The illustration for the notation used in a framework description.

Here one major concern still remains, since each projection affects multiple slices, it is very likely that the same range of projections $r\left(k^{\prime }\right)$ affects the next slice with $k^{\prime }+1$ index, and would be missing from the data necessary to reconstruct this slice. This is only relevant for the case when the next slice needs more information than the present one, and when the majority of skipped projections are located at the end of $[B\left(k\right),U\left(k\right)]$ interval. Although the described situation will indeed lead to “blind” reduction of the amount of data for the next slice, neighbouring slices are more likely to be morphologically similar and require the similar amount of projections to effectively solve the same task. This problem would also require some hard limit on the projections being skipped, since if the necessary stopping criteria are met too early, it may produce too much damage to the slices that follow.

It should be noted, it is not physically possible to make a reconstruction after each single projection is added to the data. Therefore, we introduce the concept of a sector, which is just a batch of $N_s$ sequential projections. This way, within the MTR framework, a reconstruction is performed after a single sector is added to the projection data and the decision is made whether to proceed the acquisition for the consecutive sector or to skip the remaining sectors affecting the current slice. We will further operate with the sectors rather than a single projections, this way the notation for the range of sectors affecting the slice $k$ will be similarly denoted as $[B\left(k\right),U\left(k\right)]$. The sector is considered affecting the slice $k$ if there is at least one projection in this sector affecting the slice $k$.

There is also another point to be made. Excluding a range of consecutive sectors from projection data would inevitably lead to severe damage to the reconstructed image, in case a common reconstruction algorithms are used. We propose compensating for this by not completely skipping the sector, but rather significantly reducing the frequency of acquired projections within the “skipped” sector. Although this is still not enough to use the classical reconstruction algorithms (i.e. helical FBP), it does open up several possibilities, i.e. applying algebraic algorithms (SART, SIRT, etc.) or specific methods designed to provide missing projections compensation (Hayes et al., 2021, Xiao et al., 2022) to produce a reconstructions without the corresponding artifacts. We will further denote the acquisition with reduced frequency of projections collection as “reduced”, opposed to “full” regime, corresponding to no skipped projections.

3.2. NN model and dataset

The next important part of MTR framework is the estimation of the data sufficiency in the context of solving the necessary task. Considering the COVID-19 classification task, it is quite an interesting opportunity to utilize neural network based automatic diagnosis and decision support systems. These systems are ideal candidates for the role of “experts” in such an automated system, since they not only predict the result, but also produce the important confidence level in their decision. Out of a variety of such systems we chose a publicity available pre-trained classification neural network called “Fully Automated Deep Learning-based Network For Detecting COVID-19” based on a combination of ResNet50V2 with FPN architectures (Rahimzadeh et al., 2021). While being openly accessible and trained on a large publicly available COVID-CTset dataset, this NN model is designed to diagnose the possibility of COVID-19 presence on a single slices of reconstructed CT image, resulting in a single number prediction $\theta$ per slice, which is easy to analyze and utilize in stopping rules construction. This prediction lies in range $\theta \in [0,1]$, where values close to $0$ means that the slice does not present COVID-19 features and values close to $1$ means that COVID-19 features are detected on the slice by the NN. Predictions with $\theta >0.5$ are considered by the authors as positive (Rahimzadeh et al., 2021). Once a number of positively detected slices exceed 10% the whole patient is also considered to be positive.

The COVID-CTset dataset (Rahimzadeh et al., 2021), utilized for training and validation of neural network as well as for our experiments, contains $63849$ images from $377$ patients including $95$ COVID-19 patients and $282$ healthy persons. Patient data here represent 3D lung CT images, each patient having up to $3$ reconstructions with different slice thickness (1.5–$8\mathrm{mm}$) which have been produced from the same projections data. This can be understood as a natural augmentation process since it is usual for medical studies to create several reconstructions with distinct single axis resolutions. It is worth noting that the resolution along the other two axes remains constant within a single patient and is significantly higher, corresponding to a pixel size of 0.5–$0.85\mathrm{mm}$ with a fixed image size of 512 × 512 pixels.

It is worth noticing that the COVID-CTset dataset is significantly “unbalanced”, in the sense that non-covid patients amount to roughly 75% with only 25% of covid patients data. In the training phase the authors (Rahimzadeh et al., 2021) dealt with it by choosing a “balanced” subset of patients for the training. This procedure was conducted 5 times on 5 different training sets or folds, resulting in 5 different NN models marked from Fold1 to Fold5. For the training phase the transfer learning from the ImageNet (Deng et al., 2009) was used together with Nadam optimizer and the Categorical Cross-entropy loss function. The model utilizes ResNet50V2 as a backbone to create a set of features for a feature pyramid network. The FPN architecture used was similar to the original version of FPN (Lin et al., 2017) with the difference that adding layers are replaced with concatenation layers.

All the numerical experiments and validations that conducted in the present paper and will appear in the later sections are produced on the same COVID-CTset dataset described above. As an “expert” the pretrained Fold1 model of ResNet50V2 with FPN architecture was used.

3.3. Stopping rules

The last piece of the MTR framework is the stopping rule, i.e. the rule based on partial reconstructions history, aimed to decide in which regime the acquisition is going to proceed for the current slice.

Let us analyze the problem of stopping the scanning process on the level of a single slice, in a way similar to Bulatov et al. (2020). Each slice in the dataset may either contain the COVID-19 features, or may not, thus the per-slice ground truth is a single number $\theta \in \{0,1\}$, where $0$ means that the slice does not present COVID-19 features and $1$ means that COVID-19 features should be detected on the slice by the NN. During the acquisition process, we observe a series of projection batches from consequent sectors, either “full” or “reduced”, such that after each batch of projections is acquired we can produce a partial reconstruction of the slice from currently available data. Consecutive adding of new information should result in increasing of image quality and decreasing of amount and intensity of reconstruction artifacts, with the “full” sectors supplying more information and a higher dose than the “reduced” sectors. Each partial reconstruction is passed as an input to the classification network, allowing to obtain a sequence $C_1,C_2,\dots ,C_n,\dots$ of network responses, where $C_i\in [0,1]$ – the value of the membership estimation to the “covid” class.

Within the simplest “anydose” model (Bulatov et al., 2020), if after taking $n$ observations the process is stopped, the total loss is calculated as a sum of the classification error and the total cost of the obtained projections. Assuming that the total number of projections after adding $n$ sectors is denoted as $p\left(n\right)$ and each projection carrying a fixed cost $\gamma$, the loss can be expressed as:

$$Los{s}_n={\Vert C_n-\theta \Vert }_2+\gamma \cdot p\left(n\right).$$ (1)

The task of constructing a stopping rule is the task of determining the stopping time $N$ that minimize the expected total loss $\mathrm{E}\left(Los{s}_N\right)$, based on the previously obtained observations. Given that this problem does not seem to comply with the monotone stopping problem criteria, the method discussed in Bulatov et al. (2020) could not be directly applied.

The Fig. 2 illustrates the modelled partial reconstructions with $10$, $20$, $30$, and $50$ sectors affecting the slice, for a slice with COVID-19 features.

Fig. 2.

Example partial reconstructions of a slice with $10$, $20$, $30$, and $50$ sectors affecting the slice (patient $82$, SR $2$, slice $16$).

Let us perform a rough simulation of the process on the slice level in order to analyze the dynamics of the NN predictions $C_1,C_2,\dots ,C_n$. We chose a random subset of slices with an equal thickness $6\mathrm{mm}$ (the most common thickness across the whole dataset) from the COVID-CTset dataset (Rahimzadeh et al., 2021). Then, for each original reconstruction, the projection data was synthetically produced by simulation. Each slice has a limited number of projections which could affect its reconstruction, the lifespan of a slice, which could be directly calculated from the geometry of the experiment, and in our case constituted from $\sim 500$ projections. Having chosen the size of a sector as $10$ projections, the lifespan amounted to $50$ sectors per slice. We performed consecutive exclusion of random sectors from a set of sectors comprising the lifespan of a chosen slice, with subsequent reconstructions of the slice. At last, the COVID-19 classifier network model (Rahimzadeh et al., 2021) was used on all images to obtain a potential “history” of predictions $C_1,C_2,\dots ,C_n,\dots$ for each slice. This experiment was performed for $80$ slices, distributed equally between negative and positive final predictions of the neural network.

The Fig. 2 illustrates the modelled partial reconstructions with $10$, $20$, $30$, and $50$ sectors affecting the reconstruction, for a slice with COVID-19 features.

The Fig. 3 shows the dynamics of the per-slice network predictions against the scale of the increasing number of sectors affecting each slice, separately for the slices with no COVID-19 features (Fig. 3(a)) and for the slices for which the NN detects COVID-19 given a full reconstruction (Fig. 3(b)). The figures reproduce example dynamics for some of the slices (denoted in colour) and the mean response for a subset of slices from the dataset (denoted with a black line). Given this dynamics, we can observe the following pattern: when no reconstruction information is available (i.e. when only a few sectors affects the slice) the NN output corresponds to “no covid” label, however with the increase of the number of sectors affecting the slice, the output quickly jumps to the “covid” label, presumably due to the specific response of the network on the artifacts present in a partial reconstruction. The further dynamics differ between the negative and positive cases: for the negative cases (no COVID-19 features) normally the response will return back to the “no covid” label as the influence of the artifacts subsides, whereas for the positive cases the response will continue to correspond to the “covid” label up until the end of the process.

Fig. 3.

Dynamics of the NN responses for the negative (a) and positive (b) slices with the increasing number of sectors affecting each slice.

Given the observed pattern, let us evaluate two very simple approaches to the per-slice stopping strategy within the proposed dose reduction mechanism, with the major difference between them being that the first does not require any monitoring, and the second relies on the partial reconstruction results:

• 1.
  “Trivial” stopping rule. Given a stopping threshold $L$, the slice analysis is stopped when the projections from $L$ sectors which affect the slice have been collected:

  $$N_{\mathrm{Trivial}}\left(L\right)=L.$$ (2)
• 2.
  “Monitored” stopping rule. Given a cut-off threshold $L$, a prediction threshold $P$, and a stopping threshold $T$, firstly it is checked whether the NN prediction of the “covid” label surpassed the prediction threshold $P$ during the first $L$ steps of the process. If it does not, the process is stopped at the stage $L$. If during the first $L$ stages of the process the NN response spiked higher than $P$, then the process is stopped at the $T$th stage after the stage $n_s\left(P\right)$ on which the first such spike occurred, but not earlier than at the stage $L$:

  $$n_s\left(P\right)=\left\{\begin{array}{cc}\hfill & -\infty ,\text{if}\underset{n\le L}{max}\left\{C_n\right\}\le P;\hfill \\ \hfill & min\{n:C_n>P\},\text{otherwise.}\hfill \end{array}\right)$$ (3)

  $$N_{\mathrm{Monitored}}(L,P,T)=max\{L,T+n_s\left(P\right)\}.$$ (4)

  This stopping rule, while still being simple, requires the monitored reconstruction of the object slices, in order to generate the sequence of predictions $C_1,C_2,\dots ,C_n$.

Cut-off threshold $L$ in these two rules has close but not exactly the same physical meaning. As was shown in “acquisition protocol” section in the case of “Trivial” rule, threshold $L$ results in exactly $L$ sectors affecting each slice. In case of “Monitored” rule the parameter $L$ has a meaning of hard threshold, i.e each slice would be affected by not less than $L$ sectors. This leads to a simple conclusion that “Monitored” rule will have greater or at best the same dose as “Trivial” rule with the same $L$, and the dose improvement for “Monitored” rule should be expected only over the “Trivial” rules with smaller values of $L$.

The parameter $P$ in the “Monitored” rule have a simple meaning of “spike” detection, and since this features appear as very sharp transition from $0$ to $1$, the results are almost independent of its value if it is reasonably high. Here we choose it to be $P=0.8$. The hard threshold $L$ was also fixed to reduce the degrees of freedom for the pilot study. The value $L=30$ was chosen to be close to a half of maximum sectors count, since the reconstructions at this point of data acquisition start to resemble the full dose image (Fig. 2), while there is still some room for the monitored dose reduction. We here only vary the stopping threshold $T$ to obtain the dependencies of average prediction quality from average administered dose. For the “Trivial” rule a single parameter $L$ exists, which was also varied to produce the dependencies, demonstrated at Fig. 4.

Fig. 4.

Performance profiles for the “Trivial” and “Monitored” per-slice stopping rules. Values of relevant thresholds are printed near profile points. Lower is better.

Given the obtained histories of the partial reconstruction of the slices for a subset of a dataset, let us construct and analyze the performance profiles for these stopping rules: the plots of how the mean achieved prediction error at stopping time depends on the mean number of sectors affecting each slice per history of a slice. The lower position of the curve indicates a higher efficiency of the stopping rule, i.e. either the lower mean achieved error given the same mean number of sectors, or the lesser mean number of sectors given the same mean error. The constructed profiles are presented in Fig. 4. It can be observed that the monitored stopping rules demonstrate the gain over trivial rule on the level of slices in a significant range of varied parameters. This gain appears as the lower error level for the same mean dose. However, such improvement may not directly transfer to the full CT process after implementation of this per-slice stopping rules into the MTR protocol due to the different (and overall correlated) order of sectors in the history of predictions. Thus, after summarizing the framework in the next section, we will perform the modelling of the full helical CT reconstruction process according to the proposed protocol, and compare the per-slice stopping rules in its context.

4. Full MTR framework

In the previous sections we described all the main parts of MTR framework for the COVID-19 classification task in helical CT geometry. We now ready to introduce the full MTR framework, formalizing the acquisition protocol and binding it up together with monitoring process and stopping rules.

Let us start with summarizing the acquisition protocol for helical chest CT described in Section 3.1. The target object is separated into $K$ slices, where the number $K$ and the thickness of each slice is determined by pre-scan, the hardware setup, the parameters of the scanning process, and the reconstruction method. During the scanning process, the rotating X-ray source covers a helical trajectory relative to the target object, which is separated into $S$ sectors with equal angular length, each sector consisting from a total $N_s$ projections. The scanning process satisfies the following conditions:

• 1.While the X-ray source is moving inside the sector $s\in \{1,2,\dots ,S\}$, a fixed number of projections is obtained and added to the total set of X-ray projections which are used to perform the reconstruction.
• 2.Each slice $k$ is affected by only a consecutive subset of sectors which form the range of $[B\left(k\right),U\left(k\right)]$, where $1\le B\left(k\right)\le U\left(k\right)\le S$. The indices of sectors which will affect the slice $k$ (including $B\left(k\right)$ and $U\left(k\right)$) may be calculated from experimental geometry and thus are known in advance.
• 3.During the scanning process, we can dynamically change the frequency of projections acquisition on the level of each sector, before its start. Particularly, it is possible to have two modes of projection acquiring: the “full” mode, with the maximal number of projections per sector, set in advance with the scanning protocol, and the “reduced” mode, where during the sector only a portion of the projections are acquired. The collection of the projections of a sector in a “reduced” mode reduces the dose administered to the target object, and this would be a primary way of dose reduction in the constructed method.
• 4.For each slice $k$ we can at any time obtain the result of reconstruction from partial, currently available projection data, which influence this slice, and for each such reconstruction result, we can obtain the classification result $C\left(k\right)\in [0,1]$, which will be interpreted as a membership estimation for the “covid” class.

For the setup which complies with the requirements listed above, we propose the following dose reduction protocol for a scanning process with automatic per-slice classification (see Fig. 5):

Fig. 5.

The principal scheme of the dose reduction method using per-slice stopping rules. The marker $m$ denotes the projection acquiring mode, and $n$ denotes the index of the next sector which should be processed in the “full” mode. Indexes $s$ and $k$ correspond to current sector and slice respectively.

• 1.The projection acquisition process starts from the first sector $s=1$ in the “full” mode and the monitoring process starts from the slice $k=1$.
• 2.For each sector, we take either full or reduced set of projections, depending on the current acquisition mode.
• 3.If the sector was processed in the “full” projection acquisition mode, we perform the partial reconstruction of all the slices affected by the sector $s$, i.e. $[k,U^{-1}\left(s\right)]$, to collect the history of predictions. Here $U^{-1}\left(s\right)$ denotes the number of the last slice affected by the sector $s$.
• 4.For the currently monitored slice $k$ check its stopping criterion by the predictions history. If the stopping criterion is met, we set the current projection acquisition mode to “reduced” and the index of the next sector for which the mode should be returned to “full” as $U\left(k\right)$.
• 5.If we took the projections in the “reduced” mode, we proceed to the next sector, i.e. after the stopping criterion is met all the subsequent sectors which influence the slice $k$ will be taken in the “reduced” mode.
• 6.If $U\left(k\right)$ is reached, increment $k$ by setting $k=k+1$.

The dose reduction is achieved due to the reduction of projection acquisition frequency in some of the sectors, which in turn is initiated due to the stopping criteria for each partially reconstructed slice. If the stopping criterion for the slice does not depend on the partial reconstruction results (e.g. the stopping is performed by setting a threshold of the number of acquired projections passing through the slice), then such process may not be considered as “monitored” and still corresponds to a fixed protocol, with the dose reduction which could presumably be achieved (or even surpassed) by other fixed protocols, e.g. by reducing projections acquisition frequency (uniformly opposed to proposed protocol). However, an interesting question is whether it is possible to increase the effectiveness of the dose reduction, without a significant decrease in the classification accuracy, by using a “monitored” approach, where the per-slice stopping rules depend on the partial reconstruction results in some way or another.

Thereby we introduced a new MTR protocol for dose reduction in COVID-19 classification task in helical CT setups, which will further be evaluated in numerical experiments.

5. Experiment description and results

In order to model and evaluate the framework described above and presented at Fig. 5 we need to obtain the source projection data to perform the reconstruction with a variable number of projections. This kind of data is not available in the dataset (Rahimzadeh et al., 2021) (and is not generally available at all within open data repositories). Thus, for the purposes of this work, we had to produce the synthetic projections data from the original reconstructions. All the information necessary to reproduce the acquisition process was kindly provided within COVID-CTset including the parameters of the setup geometry, patient metadata, scanning protocol details and the model of the medical CT scanner “Siemens SOMATOM” which was used to collect the dataset (Rahimzadeh et al., 2021). Unfortunately, there is no information in the dataset for the number of projections per rotation. Therefore, this value was chosen to be half of the maximum available magnitude for this CT scanner ($600$ out of $1200$ projections per rotation), which was empirically derived to be close to the minimal value at which reconstructions can be performed without serious loss of quality. Collection of the projection data and the subsequent reconstruction was performed using Astra-Toolbox v1.9.9dev (van Aarle et al., 2015), with FDK algorithm as a starting reconstruction point for iterative SIRT algorithm with 500 iterations (Gilbert, 1972). The modelled images are slightly differ from the original ones due to the use of an alternative reconstruction algorithm (Flohr et al., 2003). This difference may be summarized as a mild blurring of an image, and weak streak artifacts, localized in the outer-lungs areas. This difference is more closely studied in Section 6.

The adequacy of the described modelling process and the sustainability of the pre-trained COVID-19 classifier network to this process was validated by simulating the whole dataset and reproducing the results of Rahimzadeh et al. (2021) on the simulated data. The outcome of this validation is summarized in Table 1. As we can see from Table 1, while the modelling process yields reconstructed slice images with slightly lower quality, the NN response differs only for the $200$ slices, which corresponds to 0.48% of the full COVID-CTset dataset.

Table 1.

Validation of the per-slice COVID-19 predictions after modelling the scanning process with respect to the original published results on the COVID-CTset (Rahimzadeh et al., 2021).

Folder	Correct slices		Incorrect slices		Diff. (covid)	Diff. (normal)	Diff. total
	Original	Modelled	Original	Modelled
1	8214	8134	108	188	13 (0.15%)	67 (0.81%)	80 (0.96%)
2	8215	8147	128	196	31 (0.37%)	37 (0.44%)	68 (0.81%)
3	8143	8152	186	177	13 (0.15%)	22 (0.26%)	35 (0.42%)
4	8205	8184	110	131	3 (0.003%)	18 (0.21%)	21 (0.25%)
5	8141	8129	94	106	28 (0.34%)	12 (0.14%)	40 (0.48%)
Total	40918	40746	626	798	88 (0.22%)	112 (0.26%)	200 (0.48%)

To quantitatively evaluate the proposed dynamical protocol, we chose all the patients from the COVID-CTset dataset which had a reconstruction with a slice thickness of 6 mm (the largest and the most common thickness across the dataset). This restriction is intentional, since it provides an opportunity to have a fixed number of projections per reconstructed layer across the subset, which simplifies the interpretation of the results. The resulting subset represents 163 patients, with 62 COVID-19 positive and 101 negative ones, for a total of 7286 slices. To support this choice a comparison of distributions of patients by sex and age is provided in Table 2. For the chosen subset, the entire modelling process was reproduced for both types of stopping rules within a range of control parameters. During the experiment, the sectors were consequently included in the projection data, based on the framework described in Section 4. For each added sector all slices that had any information available were reconstructed and classified using the COVID-19 classifier network (Rahimzadeh et al., 2021). Before the next sector is included, the decision is made to add a “full” or “reduced” sector based on the rules described in Section 3.3. Here the “reduced” acquisition mode corresponded to taking 20% of projections per sector with the “full” sectors consisting of $10$ projections.

Table 2.

Patients distribution by sex and age for the full dataset and for the subset of patients chosen for validation.

Patients	Sex	Age
		10–19	20–29	30–39	40–49	50–59	60–69	70-80
Full dataset (pos)	F	0	2	6	12	5	9	5
	M	2	1	10	14	16	8	5
Full dataset (neg)	F	6	19	57	38	20	4	2
	M	6	22	48	38	14	7	1
Subset (pos)	F	0	2	4	7	3	6	5
	M	0	0	6	14	10	1	4
Subset (neg)	F	4	7	30	18	10	2	0
	M	2	1	13	10	4	0	0

The control parameters for the experiments were chosen taking into consideration the properties of the experimental geometry. As earlier, the lifespan of a slice consisted of $\sim 50$ sectors. This meant that the stopping threshold $L$ of the trivial stopping rule Eq. (2) with a value more than $50$ would effectively result in always taking a full set of projections from all sectors. As described in Section 3.3 for the “Monitored” rule a cut-off threshold $L$ was chosen to be $30$, the prediction threshold $P$ with the value of 0.8. The stopping threshold $T$ was selected from the value range $[10,30]$.

After modelling the process for each patient we measured the per-slice prediction accuracy, as well as the reduction of the dose (expressed in a relative number of performed projections), in relation to a fixed protocol where all sectors were added to the projection data with a full dose. The results of the modelling process are shown in Fig. 6 and are summarized in Table 3. The measurements presented in Fig. 6 and Table 3 represent the mean accuracy (i.e. the mean ratio of the per-slice predictions which coincide with the predictions obtained with a full dose) and the mean dose calculated as a mean ratio of the acquired projections to the number of projections in a full protocol (see Table 4).

Fig. 6.

Performance profiles for the full process modelling. The horizontal axis represents the mean dose as a ratio between the number of obtained projections and the number of projections in a full protocol. The vertical axis represents the mean per-slice prediction accuracy in relation to the predictions obtained with a full protocol. Higher is better.

Table 3.

Achieved mean per-slice prediction accuracy and dose relative to the full protocol.

Stopping rule	Parameters	Acc. (%)	Dose (%)
Eq. (2)	$L=30$	83.2	67.4
	$L=35$	91.7	74.9
	$L=40$	95.5	81.9
	$L=45$	97.2	88.3
Monitored Eq. (4) $L=30$ $P=0.8$	$T=10$	87.0	68.1
	$T=15$	91.7	$\mathbf{73.0}$
	$T=20$	$\mathbf{96.0}$	$\mathbf{79.3}$
	$T=30$	$\mathbf{98.1}$	$\mathbf{84.9}$

Table 4.

Summary from Table A.1, demonstrating number of patients for which a Pareto improvement was achieved while using monitored rules compared to trivial rules.

Patient type	Comparison 1			Comparison 2
	Trivial	Monitored	None	Trivial	Monitored	None
Full set	6	98	59	8	111	44
Positive subset	5	40	17	6	34	22
Negative subset	6	34	22	2	77	22

Fig. 6(a) demonstrates the “Monitored” rule exhibits a Pareto improvement over the baseline “Trivial” rule: the measurements corresponding to the stopping threshold values $T=15$, $T=20$, and $T=30$ achieve higher mean per-slice prediction accuracy and lower mean administered dose with respect to a “Trivial” rule with corresponding stopping thresholds $L=35$, $L=40$, and $L=45$ respectively. The points at which the monitored stopping rule Eq. (4) achieve a Pareto improvement also highlighted bold in Table 3. The point $T=10$ does not demonstrate such a payout relative to trivial rule with $L=30$ since its hard thresholds coincide and the monitored rule would always stop later than this particular trivial rule. Although it is worth noticing that this point is still lying above the trivial rule curve, and therefore there should probably exist a point $L$ where such a payoff would be achieved.

From the practical point of view the most important cases are the ones that lay closer to the end of the curve at Fig. 6(a), since the values of dose reduction there are quite realistic and the image quality loss is not yet critical. Further we will make a close consideration of two pairs of points at which a Pareto payoff is achieved, highlighted at Fig. 6(a) as “Comparison 1” and “Comparison 2”.

It is important to validate that the Pareto improvement in dose reduction and prediction quality is achieved not only in average but also on a single patient level. Table A.1 lists the decrease of per-slice prediction accuracy for each of the $163$ patients for which the process modelling was performed. It can be noted, that for the “Trivial” stopping rule, corresponding to a fixed protocol, the dose reduction stays virtually the same for all patients, whereas the “Monitored” stopping rule results in a significantly different dose reduction for different patients. It can also be noted that in most cases one of the “Monitored” stopping rules achieve a Pareto improvement over a fixed protocol rules, although it is not always the case. Trivial rules achieve Pareto achievement in very limited cases restricted to a small final accuracy change. In most of these cases monitored rules give the same accuracy or miss by one slice prediction.

Table A.1.

Dose reduction and per-slice prediction accuracy decrease for individual patients in the experiments with “Trivial” and “Monitored” stopping rules within the investigated range of parameters.

Patient	Comparison 1				Comparison 2
	Trivial rule, Eq. (2), $L=40$		Monitored rule, Eq. (4), $L=30$, $P=0.8$, $T=20$		Trivial rule, Eq. (2), $L=45$		Monitored rule, Eq. (4), $L=30$, $P=0.8$, $T=30$
	% Acc. $\downarrow$	% Dose $\downarrow$	% Acc. $\downarrow$	% Dose $\downarrow$	% Acc. $\downarrow$	% Dose $\downarrow$	% Acc. $\downarrow$	% Dose $\downarrow$
1 SR 3 (pos)	$-5.7$	−17.9	$-2.9$	−17.5	$-5.7$	−11.4	$\mathbf{-\; 2.9}$	$\mathbf{-\; 13.9}$
5 SR 2 (pos)	$-4.1$	−17.5	$-0.0$	−16.9	$-4.1$	−11.1	$\mathbf{-\; 0.0}$	$\mathbf{-\; 11.8}$
9 SR 2 (pos)	$-4.2$	−17.6	$-12.5$	−22.6	$-2.1$	−11.4	$-10.4$	−18.6
10 SR 3 (pos)	$-8.6$	−17.9	$\mathbf{-\; 5.7}$	$\mathbf{-\; 22.6}$	$-8.6$	−11.4	$\mathbf{-\; 5.7}$	$\mathbf{-\; 13.2}$
15 SR 3 (pos)	$\mathbf{-\; 2.9}$	$\mathbf{-\; 17.9}$	$-2.9$	−17.4	$-2.9$	−11.4	$\mathbf{-\; 2.9}$	$\mathbf{-\; 13.1}$
16 SR 3 (pos)	$-5.7$	−17.9	$\mathbf{-\; 2.9}$	$\mathbf{-\; 18.2}$	$-2.9$	−11.4	$\mathbf{-\; 0.0}$	$\mathbf{-\; 15.7}$
17 SR 2 (pos)	$-0.0$	−17.5	$\mathbf{-\; 0.0}$	$\mathbf{-\; 20.9}$	$-0.0$	−11.1	$\mathbf{-\; 0.0}$	$\mathbf{-\; 16.0}$
18 SR 3 (pos)	$-8.6$	−17.9	$-5.7$	−12.7	$\mathbf{-\; 8.6}$	$\mathbf{-\; 11.4}$	$-8.6$	−6.7
20 SR 2 (pos)	$-2.1$	−17.7	$\mathbf{-\; 2.1}$	$\mathbf{-\; 20.4}$	$-2.1$	−11.3	$-4.3$	−14.5
20 SR 4 (pos)	$-2.9$	−17.9	$\mathbf{-\; 0.0}$	$\mathbf{-\; 20.9}$	$-2.9$	−11.4	$\mathbf{-\; 0.0}$	$\mathbf{-\; 16.9}$
21 SR 2 (pos)	$-6.5$	−17.7	$\mathbf{-\; 4.3}$	$\mathbf{-\; 22.7}$	$-2.2$	−11.2	$\mathbf{-\; 2.2}$	$\mathbf{-\; 15.7}$
21 SR 4 (pos)	$-5.7$	−17.9	$-8.6$	−22.1	$-0.0$	−11.4	$-5.7$	−16.0
24 SR 2 (pos)	$-2.3$	−17.7	$\mathbf{-\; 2.3}$	$\mathbf{-\; 18.7}$	$-2.3$	−11.5	$\mathbf{-\; 2.3}$	$\mathbf{-\; 13.8}$
26 SR 3 (pos)	$\mathbf{-\; 0.0}$	$\mathbf{-\; 17.9}$	$-2.9$	−16.2	$-2.9$	−11.4	$\mathbf{-\; 2.9}$	$\mathbf{-\; 13.6}$
28 SR 2 (pos)	$-4.2$	−17.6	$\mathbf{-\; 0.0}$	$\mathbf{-\; 17.7}$	$-2.1$	−11.4	$-4.2$	−15.3
29 SR 2 (pos)	$-2.1$	−17.7	$\mathbf{-\; 2.1}$	$\mathbf{-\; 21.5}$	$-0.0$	−11.3	$-4.3$	−15.2
32 SR 2 (pos)	$-2.0$	−17.5	$-4.1$	−20.8	$-2.0$	−11.1	$-6.1$	−14.6
32 SR 4 (pos)	$-14.3$	−17.9	$-8.6$	−15.2	$\mathbf{-\; 2.9}$	$\mathbf{-\; 11.4}$	$-5.7$	−9.7
34 SR 2 (pos)	$-6.7$	−17.6	$-8.9$	−19.9	$-4.4$	−11.1	$\mathbf{-\; 2.2}$	$\mathbf{-\; 14.5}$
35 SR 3 (pos)	$-17.1$	−17.9	$-5.7$	−15.1	$-11.4$	−11.4	$-2.9$	−9.7
35 SR 4 (pos)	$-2.1$	−17.6	$\mathbf{-\; 2.1}$	$\mathbf{-\; 18.6}$	$-0.0$	−11.4	$\mathbf{-\; 0.0}$	$\mathbf{-\; 15.7}$
40 SR 3 (pos)	$-11.4$	−17.9	$-5.7$	−16.4	$-5.7$	−11.4	$-2.9$	−9.7
40 SR 4 (pos)	$-8.5$	−17.7	$\mathbf{-\; 6.4}$	$\mathbf{-\; 18.9}$	$-6.4$	−11.3	$\mathbf{-\; 2.1}$	$\mathbf{-\; 13.9}$
41 SR 4 (pos)	$-4.4$	−17.6	$\mathbf{-\; 4.4}$	$\mathbf{-\; 22.7}$	$-2.2$	−11.1	$\mathbf{-\; 2.2}$	$\mathbf{-\; 19.1}$
42 SR 4 (pos)	$\mathbf{-\; 12.5}$	$\mathbf{-\; 17.7}$	$-12.5$	−16.3	$-10.0$	−11.2	$-5.0$	−8.6
49 SR 2 (pos)	$-2.1$	−17.7	$\mathbf{-\; 0.0}$	$\mathbf{-\; 18.2}$	$-0.0$	−11.3	$\mathbf{-\; 0.0}$	$\mathbf{-\; 13.2}$
50 SR 2 (pos)	$-13.6$	−17.6	$\mathbf{-\; 9.1}$	$\mathbf{-\; 19.9}$	$-11.4$	−11.4	$\mathbf{-\; 6.8}$	$\mathbf{-\; 13.8}$
51 SR 2 (pos)	$-2.1$	−17.6	$\mathbf{-\; 0.0}$	$\mathbf{-\; 18.1}$	$-2.1$	−11.4	$\mathbf{-\; 0.0}$	$\mathbf{-\; 13.2}$
57 SR 2 (pos)	$-2.1$	−17.7	$\mathbf{-\; 2.1}$	$\mathbf{-\; 19.5}$	$-2.1$	−11.3	$\mathbf{-\; 2.1}$	$\mathbf{-\; 14.5}$
58 SR 2 (pos)	$-0.0$	−18.0	$-2.6$	−20.7	$-2.6$	−11.8	$\mathbf{-\; 2.6}$	$\mathbf{-\; 16.7}$
59 SR 4 (pos)	$-2.5$	−17.7	$\mathbf{-\; 2.5}$	$\mathbf{-\; 20.7}$	$\mathbf{-\; 2.5}$	$\mathbf{-\; 11.2}$	$-5.0$	−10.9
65 SR 3 (pos)	$-15.0$	−17.7	$\mathbf{-\; 15.0}$	$\mathbf{-\; 18.8}$	$-10.0$	−11.2	$-2.5$	−8.6
65 SR 4 (pos)	$-13.0$	−17.7	$\mathbf{-\; 13.0}$	$\mathbf{-\; 17.9}$	$-6.5$	−11.2	$\mathbf{-\; 4.3}$	$\mathbf{-\; 13.8}$
74 SR 4 (pos)	$-8.6$	−17.9	$-11.4$	−22.8	$-8.6$	−11.4	$\mathbf{-\; 2.9}$	$\mathbf{-\; 16.2}$
75 SR 3 (pos)	$-20.0$	−17.6	$\mathbf{-\; 11.1}$	$\mathbf{-\; 17.6}$	$-13.3$	−11.1	$\mathbf{-\; 11.1}$	$\mathbf{-\; 11.4}$
76 SR 3 (pos)	$-0.0$	−17.7	$\mathbf{-\; 0.0}$	$\mathbf{-\; 19.1}$	$\mathbf{-\; 0.0}$	$\mathbf{-\; 11.2}$	$-0.0$	−11.1
77 SR 4 (pos)	$-8.6$	−17.9	$\mathbf{-\; 8.6}$	$\mathbf{-\; 18.6}$	$-5.7$	−11.4	$-2.9$	−10.0
78 SR 3 (pos)	$-2.0$	−23.0	$\mathbf{-\; 2.0}$	$\mathbf{-\; 25.2}$	$-2.0$	−16.5	$\mathbf{-\; 0.0}$	$\mathbf{-\; 19.3}$
82 SR 2 (pos)	$-2.3$	−17.7	$\mathbf{-\; 2.3}$	$\mathbf{-\; 19.2}$	$-2.3$	−11.5	$\mathbf{-\; 0.0}$	$\mathbf{-\; 11.5}$
84 SR 2 (pos)	$-4.2$	−17.6	$\mathbf{-\; 2.1}$	$\mathbf{-\; 20.6}$	$-0.0$	−11.4	$-2.1$	−15.4
86 SR 3 (pos)	$-2.9$	−17.9	$\mathbf{-\; 2.9}$	$\mathbf{-\; 18.4}$	$-2.9$	−11.4	$-5.7$	−13.7
91 SR 2 (pos)	$-2.0$	−17.5	$\mathbf{-\; 0.0}$	$\mathbf{-\; 20.5}$	$-2.0$	−11.1	$\mathbf{-\; 0.0}$	$\mathbf{-\; 15.3}$
96 SR 3 (pos)	$-14.3$	−17.9	$\mathbf{-\; 11.4}$	$\mathbf{-\; 18.5}$	$-14.3$	−11.4	$\mathbf{-\; 11.4}$	$\mathbf{-\; 17.0}$
99 SR 3 (pos)	$-0.0$	−17.9	$\mathbf{-\; 0.0}$	$\mathbf{-\; 18.9}$	$-0.0$	−11.4	$-2.9$	−13.2
101 SR 2 (pos)	$-2.3$	−17.6	$\mathbf{-\; 2.3}$	$\mathbf{-\; 21.4}$	$-2.3$	−11.4	$-4.5$	−15.5
103 SR 3 (pos)	$-5.0$	−17.7	$\mathbf{-\; 5.0}$	$\mathbf{-\; 20.6}$	$-2.5$	−11.2	$-10.0$	−20.4
105 SR 3 (pos)	$-8.6$	−17.9	$\mathbf{-\; 8.6}$	$\mathbf{-\; 20.6}$	$-8.6$	−11.4	$\mathbf{-\; 5.7}$	$\mathbf{-\; 13.2}$
115 SR 4 (pos)	$\mathbf{-\; 2.9}$	$\mathbf{-\; 17.9}$	$-2.9$	−16.7	$\mathbf{-\; 2.9}$	$\mathbf{-\; 11.4}$	$-2.9$	−9.7
117 SR 2 (pos)	$-2.3$	−17.7	$-4.7$	−18.8	$-0.0$	−11.5	$-4.7$	−15.1
119 SR 2 (pos)	$-4.7$	−17.7	$\mathbf{-\; 2.3}$	$\mathbf{-\; 19.1}$	$-2.3$	−11.5	$\mathbf{-\; 0.0}$	$\mathbf{-\; 11.9}$
121 SR 3 (pos)	$-2.9$	−17.9	$\mathbf{-\; 0.0}$	$\mathbf{-\; 18.5}$	$-0.0$	−11.4	$\mathbf{-\; 0.0}$	$\mathbf{-\; 13.2}$
122 SR 2 (pos)	$-8.5$	−17.7	$\mathbf{-\; 4.3}$	$\mathbf{-\; 21.3}$	$-4.3$	−11.3	$\mathbf{-\; 2.1}$	$\mathbf{-\; 13.6}$
125 SR 2 (pos)	$-0.0$	−17.7	$\mathbf{-\; 0.0}$	$\mathbf{-\; 20.0}$	$-0.0$	−11.2	$\mathbf{-\; 0.0}$	$\mathbf{-\; 14.2}$
127 SR 3 (pos)	$-2.9$	−17.9	$\mathbf{-\; 2.9}$	$\mathbf{-\; 21.2}$	$-2.9$	−11.4	$\mathbf{-\; 0.0}$	$\mathbf{-\; 19.5}$
130 SR 2 (pos)	$-10.3$	−17.7	$\mathbf{-\; 10.3}$	$\mathbf{-\; 19.1}$	$-7.7$	−11.5	$-10.3$	−15.4
131 SR 2 (pos)	$-14.9$	−17.7	$\mathbf{-\; 6.4}$	$\mathbf{-\; 18.1}$	$-10.6$	−11.3	$-4.3$	−10.5
131 SR 3 (pos)	$-15.6$	−17.6	$-4.4$	−14.3	$-8.9$	−11.1	$-0.0$	−7.9
133 SR 4 (pos)	$\mathbf{-\; 0.0}$	$\mathbf{-\; 17.7}$	$-0.0$	−15.2	$\mathbf{-\; 0.0}$	$\mathbf{-\; 11.2}$	$-0.0$	−7.7
136 SR 4 (pos)	$-7.5$	−17.7	$-10.0$	−25.4	$-5.0$	−11.2	$\mathbf{-\; 5.0}$	$\mathbf{-\; 18.8}$
140 SR 2 (pos)	$-4.3$	−17.7	$\mathbf{-\; 2.2}$	$\mathbf{-\; 22.1}$	$-2.2$	−11.2	$\mathbf{-\; 2.2}$	$\mathbf{-\; 17.2}$
145 SR 2 (pos)	$-4.1$	−19.2	$-0.0$	−18.5	$-2.0$	−12.7	$-0.0$	−12.5
145 SR 4 (pos)	$-11.1$	−19.4	$-2.2$	−16.4	$-8.9$	−12.8	$-2.2$	−11.3
146 SR 2 (neg)	$-4.7$	−17.7	$\mathbf{-\; 2.3}$	$\mathbf{-\; 18.1}$	$-2.3$	−11.5	$\mathbf{-\; 2.3}$	$\mathbf{-\; 14.5}$
147 SR 2 (neg)	$-17.4$	−17.7	$\mathbf{-\; 17.4}$	$\mathbf{-\; 20.3}$	$-15.2$	−11.2	$\mathbf{-\; 10.9}$	$\mathbf{-\; 13.8}$
148 SR 2 (neg)	$-4.2$	−17.6	$\mathbf{-\; 4.2}$	$\mathbf{-\; 22.1}$	$-0.0$	−11.4	$\mathbf{-\; 0.0}$	$\mathbf{-\; 15.4}$
152 SR 2 (neg)	$-0.0$	−17.6	$-2.1$	−22.1	$-0.0$	−11.4	$\mathbf{-\; 0.0}$	$\mathbf{-\; 19.6}$
153 SR 2 (neg)	$-8.2$	−17.5	$-4.1$	−16.0	$-4.1$	−11.1	$-0.0$	−10.1
155 SR 2 (neg)	$-0.0$	−17.5	$-2.0$	−25.8	$-0.0$	−11.1	$\mathbf{-\; 0.0}$	$\mathbf{-\; 20.1}$
156 SR 2 (neg)	$-4.8$	−17.8	$\mathbf{-\; 2.4}$	$\mathbf{-\; 19.9}$	$-2.4$	−11.4	$\mathbf{-\; 0.0}$	$\mathbf{-\; 12.7}$
161 SR 2 (neg)	$-9.1$	−17.6	$\mathbf{-\; 9.1}$	$\mathbf{-\; 21.7}$	$-11.4$	−11.4	$\mathbf{-\; 4.5}$	$\mathbf{-\; 17.1}$
163 SR 2 (neg)	$-8.1$	−18.0	$-13.5$	−19.2	$-2.7$	−11.6	$-5.4$	−12.9
165 SR 2 (neg)	$-4.3$	−17.7	$-6.4$	−20.0	$-0.0$	−11.3	$\mathbf{-\; 0.0}$	$\mathbf{-\; 13.0}$
166 SR 2 (neg)	$-0.0$	−17.7	$-4.3$	−24.6	$-0.0$	−11.3	$-2.1$	−25.7
174 SR 2 (neg)	$-4.2$	−19.2	$\mathbf{-\; 0.0}$	$\mathbf{-\; 22.4}$	$-2.1$	−12.8	$\mathbf{-\; 0.0}$	$\mathbf{-\; 16.8}$
176 SR 2 (neg)	$-2.1$	−19.2	$\mathbf{-\; 0.0}$	$\mathbf{-\; 22.8}$	$-2.1$	−12.8	$\mathbf{-\; 0.0}$	$\mathbf{-\; 18.5}$
183 SR 2 (neg)	$-0.0$	−19.2	$\mathbf{-\; 0.0}$	$\mathbf{-\; 19.8}$	$-0.0$	−12.8	$\mathbf{-\; 0.0}$	$\mathbf{-\; 15.8}$
184 SR 2 (neg)	$-2.0$	−19.2	$-4.1$	−22.5	$-0.0$	−12.7	$-4.1$	−18.5
188 SR 2 (neg)	$-0.0$	−17.5	$-2.0$	−20.9	$-0.0$	−11.1	$\mathbf{-\; 0.0}$	$\mathbf{-\; 12.3}$
191 SR 2 (neg)	$-0.0$	−19.2	$-6.2$	−23.0	$-0.0$	−12.8	$\mathbf{-\; 0.0}$	$\mathbf{-\; 15.8}$
192 SR 2 (neg)	$-2.2$	−19.3	$\mathbf{-\; 2.2}$	$\mathbf{-\; 25.2}$	$-0.0$	−12.8	$\mathbf{-\; 0.0}$	$\mathbf{-\; 18.6}$
196 SR 2 (neg)	$-2.1$	−17.6	$-6.2$	−22.1	$-0.0$	−11.4	$-2.1$	−12.7
205 SR 2 (neg)	$-4.4$	−17.6	$\mathbf{-\; 4.4}$	$\mathbf{-\; 19.7}$	$-4.4$	−11.1	$\mathbf{-\; 0.0}$	$\mathbf{-\; 12.4}$
207 SR 2 (neg)	$-0.0$	−17.6	$-6.2$	−24.3	$-0.0$	−11.4	$-6.2$	−19.5
209 SR 2 (neg)	$-4.1$	−17.5	$\mathbf{-\; 4.1}$	$\mathbf{-\; 18.7}$	$-0.0$	−11.1	$-2.0$	−13.9
214 SR 2 (neg)	$-0.0$	−19.2	$-2.1$	−23.5	$-0.0$	−12.7	$-2.1$	−16.0
215 SR 2 (neg)	$-0.0$	−19.2	$-4.1$	−22.6	$-0.0$	−12.7	$\mathbf{-\; 0.0}$	$\mathbf{-\; 15.6}$
216 SR 2 (neg)	$-8.7$	−19.3	$\mathbf{-\; 4.3}$	$\mathbf{-\; 21.5}$	$-4.3$	−12.8	$\mathbf{-\; 2.2}$	$\mathbf{-\; 15.3}$
221 SR 2 (neg)	$-0.0$	−19.2	$\mathbf{-\; 0.0}$	$\mathbf{-\; 22.4}$	$-0.0$	−12.7	$\mathbf{-\; 0.0}$	$\mathbf{-\; 18.5}$
223 SR 2 (neg)	$-0.0$	−17.8	$-2.4$	−28.2	$-0.0$	−11.3	$-2.4$	−23.1
227 SR 2 (neg)	$-0.0$	−17.6	$-2.1$	−23.7	$-0.0$	−11.4	$\mathbf{-\; 0.0}$	$\mathbf{-\; 16.7}$
228 SR 2 (neg)	$-6.7$	−17.6	$\mathbf{-\; 0.0}$	$\mathbf{-\; 21.2}$	$-4.4$	−11.1	$\mathbf{-\; 0.0}$	$\mathbf{-\; 16.7}$
230 SR 2 (neg)	$-2.0$	−17.5	$-4.1$	−25.2	$-0.0$	−11.1	$\mathbf{-\; 0.0}$	$\mathbf{-\; 22.4}$
232 SR 2 (neg)	$-0.0$	−19.2	$\mathbf{-\; 0.0}$	$\mathbf{-\; 22.2}$	$-0.0$	−12.7	$\mathbf{-\; 0.0}$	$\mathbf{-\; 17.8}$
233 SR 2 (neg)	$-0.0$	−17.7	$-2.1$	−20.9	$-0.0$	−11.3	$\mathbf{-\; 0.0}$	$\mathbf{-\; 18.8}$
234 SR 2 (neg)	$-2.1$	−19.2	$-6.4$	−23.9	$-0.0$	−12.7	$\mathbf{-\; 0.0}$	$\mathbf{-\; 16.5}$
237 SR 2 (neg)	$-2.0$	−17.5	$-6.1$	−22.9	$-2.0$	−11.1	$\mathbf{-\; 0.0}$	$\mathbf{-\; 17.2}$
240 SR 2 (neg)	$-2.2$	−19.4	$\mathbf{-\; 2.2}$	$\mathbf{-\; 22.8}$	$-0.0$	−12.8	$\mathbf{-\; 0.0}$	$\mathbf{-\; 18.4}$
241 SR 2 (neg)	$-0.0$	−17.6	$-2.1$	−21.3	$-0.0$	−11.4	$-2.1$	−16.1
245 SR 2 (neg)	$-0.0$	−19.2	$-4.3$	−22.4	$-0.0$	−12.7	$-4.3$	−16.1
248 SR 2 (neg)	$-0.0$	−19.2	$\mathbf{-\; 0.0}$	$\mathbf{-\; 22.0}$	$-0.0$	−12.7	$\mathbf{-\; 0.0}$	$\mathbf{-\; 17.8}$
253 SR 2 (neg)	$-0.0$	−17.7	$\mathbf{-\; 0.0}$	$\mathbf{-\; 20.5}$	$-0.0$	−11.3	$\mathbf{-\; 0.0}$	$\mathbf{-\; 16.5}$
254 SR 2 (neg)	$-0.0$	−17.6	$-4.2$	−22.0	$-0.0$	−11.4	$-2.1$	−20.8
258 SR 2 (neg)	$-6.1$	−17.5	$\mathbf{-\; 4.1}$	$\mathbf{-\; 22.1}$	$-4.1$	−11.1	$\mathbf{-\; 0.0}$	$\mathbf{-\; 14.7}$
259 SR 2 (neg)	$-2.1$	−19.2	$\mathbf{-\; 0.0}$	$\mathbf{-\; 20.3}$	$-0.0$	−12.8	$\mathbf{-\; 0.0}$	$\mathbf{-\; 15.8}$
260 SR 2 (neg)	$-2.0$	−19.2	$\mathbf{-\; 0.0}$	$\mathbf{-\; 21.9}$	$-0.0$	−12.7	$\mathbf{-\; 0.0}$	$\mathbf{-\; 18.5}$
262 SR 2 (neg)	$-2.3$	−17.7	$\mathbf{-\; 2.3}$	$\mathbf{-\; 20.7}$	$-2.3$	−11.5	$\mathbf{-\; 0.0}$	$\mathbf{-\; 12.9}$
264 SR 2 (neg)	$-6.8$	−17.6	$\mathbf{-\; 0.0}$	$\mathbf{-\; 19.8}$	$-2.3$	−11.4	$\mathbf{-\; 0.0}$	$\mathbf{-\; 14.8}$
269 SR 2 (neg)	$-4.1$	−17.5	$\mathbf{-\; 2.0}$	$\mathbf{-\; 18.9}$	$-0.0$	−11.1	$-2.0$	−17.4
273 SR 2 (neg)	$-0.0$	−17.7	$\mathbf{-\; 0.0}$	$\mathbf{-\; 24.9}$	$-0.0$	−11.5	$\mathbf{-\; 0.0}$	$\mathbf{-\; 18.7}$
274 SR 2 (neg)	$-6.1$	−17.5	$\mathbf{-\; 4.1}$	$\mathbf{-\; 21.3}$	$-2.0$	−11.1	$\mathbf{-\; 0.0}$	$\mathbf{-\; 15.2}$
276 SR 2 (neg)	$-10.4$	−17.6	$\mathbf{-\; 4.2}$	$\mathbf{-\; 21.4}$	$-4.2$	−11.4	$\mathbf{-\; 0.0}$	$\mathbf{-\; 15.1}$
280 SR 2 (neg)	$-0.0$	−17.6	$-2.3$	−19.0	$-0.0$	−11.4	$\mathbf{-\; 0.0}$	$\mathbf{-\; 13.7}$
283 SR 2 (neg)	$-7.0$	−17.7	$\mathbf{-\; 0.0}$	$\mathbf{-\; 20.9}$	$-4.7$	−11.5	$\mathbf{-\; 0.0}$	$\mathbf{-\; 14.5}$
284 SR 2 (neg)	$-4.5$	−17.6	$\mathbf{-\; 0.0}$	$\mathbf{-\; 19.3}$	$\mathbf{-\; 0.0}$	$\mathbf{-\; 11.4}$	$-0.0$	−11.2
285 SR 2 (neg)	$-0.0$	−17.6	$-2.3$	−19.2	$-0.0$	−11.4	$\mathbf{-\; 0.0}$	$\mathbf{-\; 13.7}$
287 SR 2 (neg)	$-2.4$	−17.8	$-0.0$	−16.8	$\mathbf{-\; 0.0}$	$\mathbf{-\; 11.4}$	$-0.0$	−8.9
290 SR 2 (neg)	$-6.4$	−17.7	$\mathbf{-\; 4.3}$	$\mathbf{-\; 20.6}$	$-4.3$	−11.3	$\mathbf{-\; 0.0}$	$\mathbf{-\; 15.7}$
292 SR 2 (neg)	$-2.3$	−17.7	$-4.7$	−18.9	$-2.3$	−11.5	$\mathbf{-\; 0.0}$	$\mathbf{-\; 11.5}$
296 SR 2 (neg)	$-13.6$	−17.6	$\mathbf{-\; 11.4}$	$\mathbf{-\; 19.2}$	$-13.6$	−11.4	$\mathbf{-\; 9.1}$	$\mathbf{-\; 14.6}$
299 SR 2 (neg)	$-0.0$	−17.8	$-2.4$	−19.5	$-2.4$	−11.4	$\mathbf{-\; 0.0}$	$\mathbf{-\; 13.3}$
300 SR 2 (neg)	$-10.6$	−17.7	$\mathbf{-\; 6.4}$	$\mathbf{-\; 20.6}$	$-10.6$	−11.3	$\mathbf{-\; 4.3}$	$\mathbf{-\; 12.7}$
302 SR 2 (neg)	$-12.5$	−17.6	$\mathbf{-\; 8.3}$	$\mathbf{-\; 19.3}$	$-12.5$	−11.4	$\mathbf{-\; 4.2}$	$\mathbf{-\; 13.0}$
303 SR 2 (neg)	$\mathbf{-\; 2.2}$	$\mathbf{-\; 17.6}$	$-4.4$	−16.1	$-2.2$	−11.1	$\mathbf{-\; 2.2}$	$\mathbf{-\; 12.0}$
308 SR 2 (neg)	$-8.5$	−17.7	$\mathbf{-\; 8.5}$	$\mathbf{-\; 19.9}$	$-4.3$	−11.3	$\mathbf{-\; 0.0}$	$\mathbf{-\; 13.3}$
309 SR 2 (neg)	$-18.8$	−17.6	$\mathbf{-\; 10.4}$	$\mathbf{-\; 19.7}$	$-8.3$	−11.4	$\mathbf{-\; 4.2}$	$\mathbf{-\; 15.4}$
311 SR 2 (neg)	$-10.4$	−17.6	$\mathbf{-\; 8.3}$	$\mathbf{-\; 20.2}$	$-6.2$	−11.4	$\mathbf{-\; 0.0}$	$\mathbf{-\; 13.0}$
312 SR 2 (neg)	$-2.0$	−17.5	$-4.1$	−20.0	$-2.0$	−11.1	$\mathbf{-\; 0.0}$	$\mathbf{-\; 13.4}$
313 SR 2 (neg)	$-2.1$	−17.7	$-6.4$	−20.7	$-2.1$	−11.3	$\mathbf{-\; 0.0}$	$\mathbf{-\; 12.7}$
314 SR 2 (neg)	$-10.9$	−17.7	$\mathbf{-\; 8.7}$	$\mathbf{-\; 19.0}$	$-6.5$	−11.2	$\mathbf{-\; 2.2}$	$\mathbf{-\; 11.2}$
316 SR 2 (neg)	$-13.0$	−17.7	$\mathbf{-\; 13.0}$	$\mathbf{-\; 21.4}$	$-6.5$	−11.2	$\mathbf{-\; 0.0}$	$\mathbf{-\; 14.8}$
319 SR 2 (neg)	$-2.0$	−19.2	$-8.2$	−20.6	$-2.0$	−12.7	$\mathbf{-\; 0.0}$	$\mathbf{-\; 13.3}$
320 SR 2 (neg)	$-0.0$	−17.5	$-4.1$	−20.6	$-2.0$	−11.1	$\mathbf{-\; 0.0}$	$\mathbf{-\; 14.9}$
328 SR 2 (neg)	$-2.2$	−17.6	$-6.7$	−21.5	$-2.2$	−11.1	$\mathbf{-\; 0.0}$	$\mathbf{-\; 14.5}$
331 SR 2 (neg)	$-0.0$	−17.6	$\mathbf{-\; 0.0}$	$\mathbf{-\; 19.8}$	$-0.0$	−11.1	$\mathbf{-\; 0.0}$	$\mathbf{-\; 16.8}$
332 SR 2 (neg)	$-2.3$	−17.6	$\mathbf{-\; 0.0}$	$\mathbf{-\; 18.4}$	$-0.0$	−11.4	$\mathbf{-\; 0.0}$	$\mathbf{-\; 11.9}$
333 SR 2 (neg)	$-2.3$	−17.7	$\mathbf{-\; 0.0}$	$\mathbf{-\; 18.4}$	$-0.0$	−11.5	$\mathbf{-\; 0.0}$	$\mathbf{-\; 16.7}$
334 SR 2 (neg)	$-4.3$	−17.7	$\mathbf{-\; 4.3}$	$\mathbf{-\; 20.5}$	$-0.0$	−11.2	$-4.3$	−16.2
335 SR 2 (neg)	$-4.7$	−19.4	$\mathbf{-\; 2.3}$	$\mathbf{-\; 24.5}$	$-0.0$	−13.0	$\mathbf{-\; 0.0}$	$\mathbf{-\; 19.4}$
338 SR 2 (neg)	$-4.1$	−17.5	$\mathbf{-\; 2.0}$	$\mathbf{-\; 20.6}$	$-2.0$	−11.1	$\mathbf{-\; 0.0}$	$\mathbf{-\; 17.7}$
348 SR 2 (neg)	$-12.2$	−17.5	$\mathbf{-\; 6.1}$	$\mathbf{-\; 19.6}$	$-6.1$	−11.1	$\mathbf{-\; 2.0}$	$\mathbf{-\; 12.6}$
349 SR 2 (neg)	$-2.1$	−17.6	$\mathbf{-\; 0.0}$	$\mathbf{-\; 19.1}$	$-0.0$	−11.4	$\mathbf{-\; 0.0}$	$\mathbf{-\; 12.5}$
350 SR 2 (neg)	$-6.1$	−17.5	$\mathbf{-\; 0.0}$	$\mathbf{-\; 22.4}$	$-0.0$	−11.1	$-4.1$	−17.4
360 SR 2 (neg)	$-0.0$	−17.7	$-2.3$	−22.0	$-0.0$	−11.5	$-0.0$	−11.5
384 SR 2 (neg)	$-4.5$	−17.6	$\mathbf{-\; 2.3}$	$\mathbf{-\; 22.6}$	$-4.5$	−11.4	$\mathbf{-\; 0.0}$	$\mathbf{-\; 17.5}$
395 SR 2 (neg)	$-6.4$	−19.2	$\mathbf{-\; 4.3}$	$\mathbf{-\; 21.0}$	$-2.1$	−12.7	$-4.3$	−18.7
397 SR 2 (neg)	$-2.0$	−19.2	$\mathbf{-\; 2.0}$	$\mathbf{-\; 22.5}$	$-2.0$	−12.7	$\mathbf{-\; 0.0}$	$\mathbf{-\; 17.8}$
399 SR 2 (neg)	$-0.0$	−19.2	$\mathbf{-\; 0.0}$	$\mathbf{-\; 23.7}$	$-0.0$	−12.8	$\mathbf{-\; 0.0}$	$\mathbf{-\; 18.8}$
402 SR 2 (neg)	$-2.0$	−19.2	$-4.1$	−24.4	$-2.0$	−12.7	$\mathbf{-\; 2.0}$	$\mathbf{-\; 17.0}$
403 SR 2 (neg)	$-0.0$	−19.3	$-2.2$	−25.5	$-0.0$	−12.8	$\mathbf{-\; 0.0}$	$\mathbf{-\; 19.0}$
404 SR 2 (neg)	$-2.2$	−19.3	$\mathbf{-\; 0.0}$	$\mathbf{-\; 21.5}$	$-0.0$	−12.8	$\mathbf{-\; 0.0}$	$\mathbf{-\; 18.1}$
405 SR 2 (neg)	$-0.0$	−19.4	$\mathbf{-\; 0.0}$	$\mathbf{-\; 23.8}$	$-0.0$	−12.8	$\mathbf{-\; 0.0}$	$\mathbf{-\; 19.7}$
408 SR 2 (neg)	$-6.4$	−17.7	$\mathbf{-\; 6.4}$	$\mathbf{-\; 19.5}$	$-6.4$	−11.3	$\mathbf{-\; 0.0}$	$\mathbf{-\; 14.5}$
411 SR 2 (neg)	$-12.8$	−17.7	$\mathbf{-\; 8.5}$	$\mathbf{-\; 20.5}$	$-6.4$	−11.3	$\mathbf{-\; 2.1}$	$\mathbf{-\; 13.2}$
413 SR 2 (neg)	$-0.0$	−17.6	$-4.2$	−21.2	$-0.0$	−11.4	$-2.1$	−20.9
415 SR 2 (neg)	$-0.0$	−19.4	$-4.7$	−20.8	$-0.0$	−13.0	$-0.0$	−13.0
417 SR 2 (neg)	$-0.0$	−19.4	$-2.2$	−23.2	$-0.0$	−12.8	$\mathbf{-\; 0.0}$	$\mathbf{-\; 16.7}$
418 SR 2 (neg)	$-0.0$	−19.3	$-4.3$	−24.6	$-0.0$	−12.8	$-2.2$	−17.7
421 SR 2 (neg)	$-0.0$	−17.6	$-2.1$	−21.1	$-0.0$	−11.4	$-6.2$	−18.2
423 SR 2 (neg)	$-2.1$	−17.7	$\mathbf{-\; 2.1}$	$\mathbf{-\; 18.5}$	$-2.1$	−11.3	$\mathbf{-\; 0.0}$	$\mathbf{-\; 13.9}$
430 SR 2 (neg)	$-4.3$	−17.7	$-6.4$	−19.6	$-0.0$	−11.3	$\mathbf{-\; 0.0}$	$\mathbf{-\; 14.5}$
432 SR 2 (neg)	$-2.2$	−17.7	$-17.4$	−22.9	$-2.2$	−11.2	$\mathbf{-\; 0.0}$	$\mathbf{-\; 13.5}$
433 SR 2 (neg)	$-2.2$	−17.7	$\mathbf{-\; 0.0}$	$\mathbf{-\; 20.0}$	$-2.2$	−11.2	$\mathbf{-\; 0.0}$	$\mathbf{-\; 14.2}$
434 SR 2 (neg)	$-4.1$	−17.5	$\mathbf{-\; 2.0}$	$\mathbf{-\; 23.0}$	$-2.0$	−11.1	$-4.1$	$-18.$

The full code for all of our experiments is available in reproducible capsule on the “Code Ocean” platform https://codeocean.com/capsule/0949104/tree.

6. Discussion

We will start by formulating the main idea in less strict and more simple words. The case of a fixed protocol basically means that the operator acts almost blindfolded, and once the experiment parameters are set, nothing can interfere with the acquisition process, meaning a full predefined dose will be administered to a patient. In MTR we offer to use the projections already available at the moment to reduce the dose in cost of additional computation time. This is done by repeated reconstructions from available data with automatic evaluation of images by the chosen “expert”. Since this task is not even considered in general CT research, a lot of problems should be resolved before an actual setup could use the MTR protocol to its fullest. This work can be considered as a pilot attempt to partially solve all of these problems to the extent of constructing a working MTR protocol. This attempt is important since it demonstrates the fundamental possibility of such way of dose reduction and, moreover, allows us to estimate a potential gain of its use.

While there are available open datasets of chest CT scans, with and without pathologies, the original projection data which is required for the research of new methods of CT reconstruction and custom scanning protocols is not generally available. For this reason, the present study was performed using a modelled process, where the synthetic data had to be produced from the original reconstructed images. Given that the data obtained in such a way does not fully correspond to the actual data which could be collected from a helical CT scanner, and the reconstruction algorithm which was utilized does not fully correspond to the one implemented in the original protocol, the performed experiments could be considered as a pilot study. The results of the study show promise, and can serve as the first step for further examination with the data obtained directly from a helical CT setup.

The most significant effect which could be seen from the performed experiments is that even a simple real-time monitoring of the reconstruction process, with the calculation of the earliest time where the reconstruction analysis NN changes the output (see Eq. (4)) with a following step-based thresholding, outperforms the trivial step-based thresholding rule, which corresponds to a fixed protocol with similar mechanism of dose reduction. The payoff is higher for patients without COVID-19, presumably due to the fact that on some slices without COVID-19 features the NN classifier does not change its output at all, in which case the rule Eq. (4) stops at a significantly lower stage, which leads to a larger dose saving. It should be noted that such slices can occur in both COVID-19 and non-COVID-19 patients, and the probability of occurrence in the latter is higher.

We should now discuss the results at a single patient level. An unwanted behaviour here would be to raise the error on some patients, significantly lowering the dose, since it still could lead to improvement on average, while sacrificing prediction accuracy on some part of patients. Table A.1 demonstrates that indeed no such an unwanted behaviour is occurring in the conducted experiments. We will further discuss only Comparison 2 results, since in our opinion it corresponds to the most practically significant case without major prediction quality reduction. In most cases monitored rule demonstrates a straight Pareto improvement over the trivial one. We will now concentrate on the other cases for which monitored rule is not able to achieve Pareto improvement, in order to demonstrate that these cases are not significant enough to affect the average values of dose nor prediction quality. Considering all the cases for which trivial rule wins over monitored rule, only in two cases (Patients 32, 59) the prediction accuracy is decreased, missing by a single slice prediction. In all the other cases trivial rule wins by slightly lowering the dose, while producing the same prediction quality. Cases in which no rule is able to achieve a Pareto improvement over one another may be divided into two groups: (1) monitored rule having higher dose and higher prediction quality, (2) monitored rule having lower dose and lower prediction quality. Of these two groups, the second one is more worrying since it potentially could produce the described unwanted behaviour. There are 25 patients in this group, of which only three cases (Patients 9, 21, 103) with more than 5% accuracy reduction (i.e. missing more than by a single prediction) from the trivial rule.

From the statistical point of view, these losses seem to be reasonable considering the overall improvement in the major part of the dataset, although for the tasks different from NN classification of slices it is up to the specialists to decide if the image quality loss is acceptable. For this reason, we would like to provide some example reconstructions choosing patients and layers by the following criteria: (a) dose reduction values are the highest and NN prediction is changed from full dose, (b) dose reduction values are the highest and the prediction quality loss is zero, meaning the NN is not that sensitive to the dose reduction. We will compare the resulting reconstructions for the full dose and for the monitored stopping rules modelling. Applying the first criteria to both positive and negative patients we produced 2 sets of images (Fig. 7(a), Fig. 7, Fig. 7, 7(e)). For this set the predictions on reconstructions from full dose are correct, and from the reduced dose are not. The second criterion is more relevant to a negative patients, since they should be more sensitive to the artifacts in the lung area (Fig. 7(c), 7(f)). In the second case both predictions are correct.

Fig. 7.

Example slice images reconstruction after the modelling. Top row represent with full range of projections, Bottom row with monitored rule.

Having all the data, it is also quite straightforward to reproduce the diagnostic experiments from Rahimzadeh et al. (2021). As a simple diagnosis procedure, it was proposed to mark patients positive at 10% of positive slices from a total slices count. The results of this experiment are provided in Table 5. There we only keep the patients for which the cases of misdiagnosis happened in either of the columns. All such cases are highlighted by a bold font. It is interesting that there is only two cases of a false negative prediction and it is not correlated with the dose reduction. There is also only one case of misdiagnosis for monitored rule, in which for full dose the correct diagnosis was produced, while for a trivial rule there are five of such cases. While this experiment is not particularly illustrative of the advantages of using the monitored rules, we still find it important to demonstrate another validation of the stability of a chosen modelling procedure.

Table 5.

Reproducing of by patient testing of a diagnosis procedure from Rahimzadeh et al. (2021) on modelled data. Only the patients for which misdiagnosis happened are represented out of total 163. Misdiagnosed cases are highlighted by bold font.

Patient	Total slices count	Full dose			Comparison 2
					Trivial rule, Eq. (2), $L=45$			Monitored rule, Eq. (4), $L=30$, $P=0.8$, $T=30$
		n positive	$\text{\%}$ positive	diagnosis	n positive	$\text{\%}$ positive	diagnosis	n positive	$\text{\%}$ positive	diagnosis
5 SR 2 (pos)	49	0	0.0	neg	2	4.1	neg	0	0.0	neg
121 SR 3 (pos)	35	0	0.0	neg	0	0.0	neg	0	0.0	neg
147 SR 2 (neg)	46	0	0.0	neg	7	15.2	pos	5	10.9	pos
161 SR 2 (neg)	44	6	13.6	pos	9	20.5	pos	6	13.6	pos
163 SR 2 (neg)	37	3	8.1	neg	4	10.8	pos	3	8.1	neg
216 SR 2 (neg)	46	9	19.6	pos	11	23.9	pos	8	17.4	pos
296 SR 2 (neg)	44	13	29.5	pos	15	34.1	pos	13	29.5	pos
300 SR 2 (neg)	47	0	0.0	neg	5	10.6	pos	2	4.3	neg
302 SR 2 (neg)	48	0	0.0	neg	6	12.5	pos	2	4.2	neg
309 SR 2 (neg)	48	1	2.1	neg	5	10.4	pos	3	6.2	neg
319 SR 2 (neg)	49	15	30.6	pos	16	32.7	pos	15	30.6	pos

It should be emphasized that in the present study we aimed to create a framework, reasonably independent from the NN method used in an “expert” role for a quality estimation. Here we purposefully used a pre-trained per-slice COVID-19 classifier network (Rahimzadeh et al., 2021) to see if it is possible to apply the variable-dose protocol with real-time monitored reconstruction without having to retrain the classification networks. The obtained results show that the NN-based methods of tomographic images analysis allow to construct new and effective scanning protocols with decreased radiation dose, by using them as “experts” which could judge whether there is enough diagnostic information in an incomplete reconstruction. This shows another potential application of computer vision methods, even if the final diagnostic decision is made by a medical professional. From the point of a practical application, however, the proposed protocol could benefit from custom trained per-slice classifiers which could be robust against the artifacts which appear due to incomplete projections, and more sophisticated stopping rules could be designed to further increase the amount of dose savings.

The protocol which involves the described and evaluated monitoring implies a lot of additional computation, which has to be performed in real-time during the scanning process. Thus, for practical implementation of this approach, the speed of currently developed helical CT reconstruction algorithms becomes a critical bottleneck. Up to now, the speed of integral algorithms is considered to be enough for most of the tasks, but the development of monitoring tomography approach may force us to reconsider these requirements, demanding even more efficient algorithms.

7. Conclusion

In this paper, we proposed a novel real-time monitored reconstruction scanning protocol in a helical CT geometry, which allows to reduce the total radiation dose administered to the patient by analyzing the intermediate reconstructed images using a pre-trained NN-based classifier. Within this protocol, for the first time the stopping rules of the monitored reconstruction were applied in the helical CT scheme and with a directly measurable accuracy metric. The proposed method was evaluated on a subset of 163 patients. The average dose reduction achieved using the proposed protocol with monitored reconstruction achieved 15.1% of dose reduction while producing average prediction quality loss of 1.9% (see Table 3). For a baseline method based on the same protocol dose reduction and quality loss amounted to 11.7% and 2.8% correspondingly. These results demonstrate that by monitoring the acquisition process a Pareto improvement of the radiation dose and prediction quality can be achieved, compared with a fixed protocol without real-time reconstruction. These results are valid not only in average but often even on a single-patient level. Maximum dose savings achieved by using monitored rule compared to the trivial rule are reaching $7\text{\%}$ while providing better prediction accuracy.

The neural network-based classifier which was used to perform the monitoring was not re-trained, as the goal of this paper was to evaluate the higher-level approach itself. We consider the obtained results to be quite promising, although there is still a wide prospect for a future work. Primarily it is necessary to design and train custom NN-based classifiers which would be robust against the partial reconstruction artifacts. This will allow to construct and utilize significantly more effective per-slice stopping rules, which would result in further reduction of the radiation dose with small to none impact on the prediction accuracy. It will also be important to validate the obtained results on the original raw projections, collected by the X-ray setup, since the experiments conducted in this study had to rely on simulated projection data, due to the required raw data is not generally available in public datasets.

In spite of the mentioned limitations, it is important to note, that the described approach demonstrates an ability to reduce the imparted radiation dose during the scanning process. Given the importance of this task, the authors believe that it can be highly useful and relevant for creating next-generation medical imaging setups, as well as become a tool for decision support systems and automated diagnostics.

CRediT authorship contribution statement

Konstantin B. Bulatov: Conceptualization, Writing. Anastasia S. Ingacheva: Modelling and experiments, Writing. Marat I. Gilmanov: Modelling and experiments, Writing. Marina V. Chukalina: Reviewing and editing. Dmitry P. Nikolaev: Conceptualization. Vladimir V. Arlazarov: Supervision.

Declaration of Competing Interest

The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.

Appendix. Tabled validation experiment data

Dose reduction and per-slice prediction accuracy decrease for individual patients in the experiments with “Trivial” and “Monitored” stopping rules within the investigated range of parameters. The cases where better quality is reached with less or equal dose are highlighted by bold font.

Data availability

Used data is from open dataset.