Severity Assessment of COVID-19 Based on Feature Extraction and V-Descriptors

Abstract

Digital image feature recognition is significant to industrial information applications, such as bioengineering, medical diagnosis, and machinery industry. In order to supply an effective and reasonable technology of the severity assessment mission of coronavirus disease (COVID-19), in this article, we propose a new method that identifies rich features of lung infections from a chest computed tomography (CT) image, and then assesses the severity of COVID-19 based on the extracted features. First, in a chest CT image, the lung contours are corrected for the segmentation of bilateral lungs. Then, the lung contours and areas are obtained from the lung regions. Next, the coarseness, contrast, roughness, and entropy texture features are extracted to confirm the COVID-19 infected regions, and then the lesion contours are extracted from the infected regions. Finally, the texture features and V-descriptors are fused as an assessment descriptor for the COVID-19 severity estimation. In the experiments, we show the feature extraction and lung lesion segmentation results based on some typical COVID-19 infected CT images. In the lesion contour reconstruction experiments, the performance of V-descriptors is compared with some different methods, and various feature scores indicate that the proposed assessment descriptor reflects the infected ratio and the density feature of the lesions well, which can estimate the severity of COVID-19 infection more accurately.

I. Introduction

In THE big data era, the automatic image feature recognition releases heavy labor and promotes the intelligent development of industrial applications, such as 3-D reconstruction [1], [2], vision sensing [3], [4], and action recognition [5]. We use the image processing techniques to screen the computed tomography (CT) images of coronavirus disease (COVID-19), which will play an important role in clinical diagnosis and treatment.

Since December 2019, many cases of unexplained pneumonia have appeared in Wuhan, China [6]. After a short while, this disease started pandemic spreading and outbreaking as an international public health emergency. Through the biomedical check, the pathogenic virus was found to be a new coronavirus, which was named as COVID-19 by the World Health Organization [7]. Nowadays, the prevention, diagnosis, and treatment of the COVID-19 are still big challenges in some countries and regions. Under the conditions that false positives consume plenty of treatment resources, false negatives increase the propensity for super spreaders, and hospitals have scarce medical resources, so the CT image screening based on machine has a profound impact for human beings. It is known that the golden standard for the COVID-19 diagnosis is the reverse transcription polymerase chain reaction (RT-PCR) test [8], but it has some problems, such as resource shortage, limited sampling methods, and high false negatives [9], [10]. Since most COVID-19 infected patients have salient radiologic characteristic with pneumonia, such characteristic can be used for the diagnosis and assessment of disease progress. The chest X-rays (CXR) and CT are two available chest imaging screening means. On account of the CXR has high misdiagnosis probabilities and obscure lung markings, and the CT scans show relatively clear isolated or multiple patchy ground-glass opacities (GGO) [11], [12], so the CT image feature is adopted in COVID-19 diagnosis. Based on the specific signs of the chest CT images, we design a new feature recognition method for COVID-19 diagnosis, and the lesion extraction results for different types of lung CT images are shown in Fig. 1. Nowadays, in some countries and regions, the prevention and treatment of COVID-19 still face many challenges: the false positive patients with fever consume a large amount of emergency treatment resources and bring about the shortage of medical supplies (medical workers, PPE, ventilators, beds needed, etc.); the false negative patients have not been isolated in time to become “super spreaders,” which accelerates the further spread of this infectious pandemic. Compared with manual screening, the CT screening method based on computer image processing technology has many advantages, as follows: it reduces the time for patient estimation, decreases the medical support staffs, and alleviates the pressure on medical staffs and materials of the hospital.

Fig. 1.

Chest CT images shown in the first column are from COVID-19 infected patients, wherein the first three CT images belong to mild type, the fourth and the fifth CT images belong to common type, and the sixth CT image belongs to serve type. The second column shows the extracted lung regions, and the last column shows the detected lung lesion regions (lung regions are denoted in blue color, and lesion regions are denoted in pink color).

At present, the identification and assessment methods of COVID-19 have been continuously proposed. Tang et al. [13] relied on a large number of accurately labeled training samples and it only recognizes the severe and nonsevere lesion types. The method in [14] requires precise laboratory quantitative indices in COVID-19 severity assessment. Freitas et al. [15] proposed a method that assesses the severity of the epidemic in different regions from the perspective of epidemiological principles. A semisupervized COVID-19 segmentation method based on a small number of annotated instances is proposed in [16]. Mohamed et al. [17] realized CXR image segmentation based on the slime mould algorithm with the whale optimization algorithm. Besides, there are other methods based on artificial intelligence [18]–[21]. However, these methods not only require a lot of time for training, but also need a lot of accurately labeled data for training. In fact, most expert first-line clinicians participate in heavy treatment works, and they do not have enough time and energy for labeling, which affects the accuracy of artificial intelligence models. Moreover, the pretrained model may not be suitable for all types of CT machines, and its deployment platform needs to have strong graphics processing capabilities. In order to overcome the difficulties in machine learning algorithms, we propose a new method to assess the severity of COVID-19 automatically based on graphics and image processing technology.

The image processing techniques have been widely used in the healthcare system and CT screening [22]–[25]. Most of the existing algorithms are based on a specific disease and may not be applicable to COVID-19. Adopting the effective feature extraction algorithms and utilizing the discontinuous orthogonal function systems, we design a new assessment descriptor, which integrates the morphological feature (i.e., V-descriptor), density (or grayscale) features (i.e., coarseness, contrast, roughness, and entropy), and the infected rate. Our motivation is to propose a nonmachine learning image processing technology that can be easily embedded in medical devices. It can provide accurate and rapid severity assessment of COVID-19 while overcoming the dependence on accurately labeled training data. It is easy to transplant, and requires low equipment platform and computing resources.

In this article, a new digital feature descriptor for COVID-19 diagnosis using the V-system is proposed. The V-system is a generation of the Haar basis function, which combines with the sequences of piecewise polynomials of different degrees [26]–[28]. V-descriptors designed from the V-system not only effectively decompose and represent multiple lesions quickly but also overcome the Gibbs phenomenon [29]–[32]. Moreover, V-descriptors have the invariance of translation, rotation, and zoom properties, which can provide accurate degree descriptors of CT scans for patients at different stages. In the proposed method, the chest CT images need to be preprocessed to correct the lung contours first. Then, the lung regions can be segmented based on their contours. Generally, the contours of the infected lungs are different with the normal lungs, which have various textures inside [22], [33], [34]. Next, the coarseness, contrast, roughness, and entropy textures are extracted based on the intensity feature of the lung regions, and they are fused to extract the infected area and the lesion contours. Finally, the texture or intensity features and V-descriptors of lesion and lung contours are fused to generate the appropriate degree descriptors, which can be used to assessment the disease severity. The new method can reflect the infection ratio, and the intensity of lesions [13], which can achieve more accurate severity assessment, and the new method can provide a degree score for clinical assessment and treatment. The contributions of this article are concluded as follows.

• 1)A novel method for COVID-19 CT image GGO feature extraction is designed. It is a fusion texture feature that includes coarseness, contrast, roughness, and entropy. Furthermore, this method can reconstruct the contours of the infected regions exactly.
• 2)We propose an effective chest CT image severity assessment descriptor based on image gray-scale feature and morphology. It can overcome the Gibbs phenomenon of discontinuous contours. This novel descriptor has an invariance of geometry transform and is able to calculate the lesion degree of the lung, and it can be used for severity assessment of COVID-19.
• 3)Through the massive (2000+) experimental result analysis, we summarize the severity assessment experience score. We divide the score value into three intervals that can help medical staff to quickly read and diagnose.

The rest of this article is organized as follows. The related work about image feature descriptors and pretreatment are introduced in Section II. The method of the new digital assessment descriptor for COVID-19 CT diagnosis is demonstrated in Section III. The experiments and estimation are stated in Section IV. Finally, Section V concludes this article.

II. Related Work

We introduce the preliminary knowledge in this section, to facilitate the understanding of our work. First, the Gibbs phenomenon that occurs in Fourier series can be overcome by the V-system, and the V-descriptors for CT image feature extraction are designed in this article. Second, the definition of the V-system is briefly presented [28]. Finally, the pretreatment of lung contours is also reviewed.

A. Gibbs Phenomenon in Contour Reconstruction

Bilateral lung involvement is commonly observed in chest CT scans from symptomatic patients infected with COVID-19. The bilateral lung and more than one lesions on the lung are regarded as multiobject graph in this article. The orthogonal function system is effective in morphological feature extraction of multiobject contours. In fact, traditional eigenfunction series, such as Fourier series and continuous wavelet transform, have undershoots and overshoots at strong discontinuities, which is called Gibbs phenomenon. Fig. 2 shows the Gibbs phenomenon with Fourier series. Therefore, the continuous orthogonal function systems cannot accurately represent separate multiobjects.

Fig. 2.

Gibbs phenomenon in orthogonal representation for the contours of the lungs and infected regions. (a)–(c) Reconstructed lung contours using Fourier series with 25 terms, 50 terms, and 100 terms, respectively. (d)–(f) Reconstructed lesion contours using Fourier series with 25 terms, 50 terms, and 100 terms, respectively.

B. Definition of V-System

The V-system can overcome the Gibbs phenomenon, which has been verified in [26]–[28]. Therefore, we adopt it to construct V-descriptors for the lung feature extraction. The V-system of degree 2 is used to extract the infection morphological features. The V-system functions can be generated by

where . We denote as the function in the class of the group in the V-system of degree . The V-system of degree is the Haar system that is a widely known wavelet.

When =2, the basis functions of the V-system are

III. Method

This section elaborates the gray GGO feature extraction method and the assessment descriptor of COVID-19 severity on lung CT scans, which will play a crucial part in the radiological examination of COVID-19 assessment.

A. Motivation

Currently, COVID-19 has not been effectively controlled in some countries and regions, which has brought considerable pressure to society and hospitals. The challenge of combating COVID-19 is mainly reflected in the fact that false positive patients occupy scarce treatment resources; false negative patients have not been isolated in time to become super spreaders, which has accelerated the spread of the virus; the intake of suspected patients has caused great panic to hospitals and medical workers. Considering the important role of CT imaging features in the prevention and treatment of COVID-19, we combine computer image processing technology and then propose an automatic CT screening algorithm.

The patients who infected with COVID-19 have typical CT imaging features [11], [12]. The CT screening can prevent the false negatives of RT-PCR tests to a certain extent, so that suspected patients can be isolated in time. However, traditional manual reading has some problems, such as inaccurate identification, slow examination speed, and limited number of professional doctors. The COVID-19 CT screening based on artificial intelligence has high requirements for the accuracy of labeled data, and the accuracy of sampling data cannot be guaranteed in the early stage of the outbreak. Therefore, this article proposes a new CT assessment algorithm based on traditional image processing technology, which uses image texture feature extraction algorithms and V-descriptors that play an advantage in discontinuous signal representation. The proposed algorithm can quickly obtain the CT severity estimate of patients with high accuracy, which plays a satisfactory effect in the clinics of the hospital.

B. Lung Infected Symptoms on COVID-19 CT

The diagnosis findings of COVID-19 for the infected cases possess typical CT imaging manifestation, which is a strong recommendation for this disease prevention and treatment. The multiple, patchy, and scattered GGO density shadows are mostly appeared in COVID-19 CT images. The GGO signs with the thickening of interlobular septa are classified as “paving stone-like” changes. The thinner CT scan layers can clearly display GGO and the thickening of interlobular septa. The X-ray has worse resolution than CT in GGO edges. In summary, the CT imaging features of lesion are sorted, i.e., dominant distribution, quantity, shape, density, and concomitant signs variable [11], [35].

Specifically, according to the period after the onset of clinical manifestations, the CT imaging usually demonstrates some stages based on different treatment plans, such as ultraearly stage, early stage, rapid progression stage, consolidation stage, and dissipation stage [11]. In this article, we focus on the CT imaging features to assessment the disease severity. Since the CT imaging manifests different characteristics, we use digital image processing techniques to automatically classify the chest CT images. In order to avoid clinical misdiagnosis and individual differences of patients, we roughly classify the CT severity assessment into the following three types: mild, common, and severe degrees. Generally, the patients of the ultraearly or early stage have mild CT manifestations: single or multiple scattered patchy or GGO, together with thickened of interlobular septa. The patients of progression stage normally have common CT manifestations: multiple patch GGO and pulmonary interstitial changes with air-bronchogram in lungs. The patients of consolidation and dissipation stage have severe CT manifestation: multiple patch consolidations with rare pleural effusion, and the range of GGO is smaller than that of common stage. Therefore, we adopt coarseness, contrast, roughness, and entropy to extract lesion region and texture features.

Algorithm 1: COVID-19 Diagnosis System.

• 1:Input: Raw CT Image;
• 2:Output: COVID-19 Infected Degree Descriptor;
• 3:set = ;
• 4: = Boundary Extraction();
• 5:[] = Lung Region Extraction();
• 6: is lungs area, is a binary image.
• 7: = Lung Contours Extraction();
• 8: is Lungs boundary contours of 2 vectors .
• 9: = Calculate Coarseness();
• 10: = Calculate Contrast();
• 11: = coarseness+contrast;
• 12: = Calculate Entropy();
• 13: = Boundary Extraction();
• 14: is a image of infected region.
• 15:[] = Lung Region Extraction();
• 16: is infected area, is a binary image.
• 17: = Lung Contours Extraction();
• 18: is infected boundary contours of 2 vectors .
• 19:set = length of , = length of ;
• 20: = V-system Construction();
• 21: is a V-system matrix of degree .
• 22: = Calculate V-Descriptor();
• 23: = ; is the infected ratio.
• 24: = ;
• 25: is the COVID-19 severity assessment descriptor.
• 26:The calculate methods of are following formula 12;

C. Pretreatment of Lung Region and Contours

At the beginning of chest CT feature extraction, the lung region needs to be determined, and then it can be used to extract the lung contours. The complete lung contours and their coordinates are required in the morphological characteristic of the lungs. Since a CT image is always a gray-scale image, the nearest neighbor pixel tracking after binarization is a commonly used method for external contour acquisition. Specifically, the original CT image is properly binarized at first, and then the lung contours can be extracted based on the edge extraction.

In order to ensure the rotation, translation, and scale invariance when performing V-system orthogonal transforms on the lung and lesion contours, the direction of the starting point and the tracking direction need to be specified. First, after binarizing the CT image, the image pixel position is mapped into a coordinate system, and the largest vertical coordinate is used as the starting point of the contour. The closest neighboring point of the starting point is defined as the nearest starting point in the horizontal direction. When the next neighboring point is searched, the diagonal direction of the corresponding upper neighboring direction is set as the starting point. When the acquired CT image is high-definition, it can be set as the adjacent point of radius as needed. When the CT image has low resolution, the nearest adjacent point can be searched by using the nine grid method. Until the ending point and the starting point is in coincidence, it is recorded as a complete contour of the double lungs. After acquiring the lung contours, it can be judged whether there is a lung lesion defect, and if there is, it should be replenished in manual. Afterward, the object area of the lungs is binarized, and the internal image of the lungs is extracted again. At this time, the complete message of the lungs (area and contour) can be obtained, which is conducive to analyze the lung's inner situation independently.

D. Coarseness, Contrast, Roughness, and Entropy Features of Lung Lesion

From medical diagnostic images and COVID-19 confirmed cases, it can be found that the density shadow of GGO is the primary definite imaging sign of COVID-19. Since this kind of sign is severely individualized with different persons, and its diversity of shapes puts forward relatively high requirements for graphic aid imaging. At the same time, only using V-descriptors to perform graphical description of the morphological sign of lesions is limited. The use of texture features to describe GGO is also very important. Based on the previous research, it can be found that the ground glass shadow of the lesion area mainly has the gray change and possesses similar characteristics like “stone road sign,” “gray snow,” and “tree fog.” We analyze a large number of lung's CT image features and plan to use the graphics and image features composed of coarseness, contrast, roughness, entropy, and V-system descriptors as the core-parameters to design the COVID-19 severity algorithm.

The size of the added moving window of the lung region is . The pattern of moving window and feature extraction process are described in Fig. 4. Coarseness, contrast, roughness, and entropy provide the gray-scale texture features of the window which are extracted in different ways. The analysis of the distribution and change of the feature value provides a good basis for extraction of the lesion area. In this article, coarseness, contrast, roughness, and entropy are denoted as , and , respectively. The detailed calculation formulas are described as follows.

Fig. 4.

Feature extraction process with moving window.

The average over the neighborhood with the size at the point is

where is the gray value of the pixel in a CT image. For each pixel in a CT image, the average differences for pairs of nonoverlapping adjacent regions in horizontal and vertical orientations are signed as and , respectively. When maximals in the either direction, we have max. At each point, the best size that gives the highest output value is picked. Then, the average value of is used as the coarseness estimation of the image, as follows:

The gray-scale image contrast is a useful tool for measuring the image characteristics. First, we should calculate the fourth moment of the moving window and the standard deviation of the moving window. We set , and the contrast is defined as . In addition, the roughness has the form . The ratio of gray-scale value for a local point in the lung region to the total gray-scale value of the neighborhood is represented as , which has the form

Then, is further used to define the local gray-scale information entropy. The local gray-scale information entropy can be calculated by

Since the size of a CT image is unknown and unfixed, we design a reasonable normalization method that can stabilize the core-parameter to avoid the influence of image size. The infected region, uninfected region, and lung region are represented as , , and , respectively. Then, the areas of , , and are set as , , and , respectively. In this way, the infected ratio of a lung system is defined as , and the proposed normalization method is calculated as

where represents core-parameter, e.g., when we use entropy to replace , formula (11) has the form

E. Assessment Descriptor of Lung Lesion

The lung and lesion contours with more than one independent line segments can be regarded as multiple objects. In multiobject detection, some properties are extremely valuable, such as invariance in translation, rotation, and scale. Hence, the reconstruction method using V-descriptors is good fitted. Let the multiobject lung and lesion contours possess vectors in the image , then for each contour point, the horizontal coordinate , and the vertical coordinate are defined as

Algorithm 2: Functions of Feature Extraction and V-System.

• 1:set .
• 2:function Moving Window
• 3:set = size of I.
• 4:for to do
• 5:for to do
• 6: = I( to to );
• 7: end for
• 8: end for
• 9:Return:
• 10: end Function
• 11:function Calculate Coarseness
• 12: = Moving Window()
• 13:;
• 14:;
• 15:;
• 16:
• 17:
• 18:;
• 19:Return:
• 20: end Function
• 21:function Calculate Contrast
• 22: = Moving Window()
• 23:;
• 24:;
• 25:;
• 26:;
• 27:;
• 28:Return:
• 29: end Function
• 30:function Calculate Entropy
• 31: = Moving Window()
• 32:;
• 33:;
• 34:Return:
• 35: end function
• 36:function V-system Construction
• 37:;
• 38: =
• 39:
• 40:Return:
• 41: end Function

The interval [0,1] is divided into subintervals . and are piecewise polynomials of degree over the subinterval, and are mapped to the subinterval, and we obtain

where . Since V-system possesses the reproducibility, and the polynomial of degree has dimensional freedom, then

where

is denoted as the descriptor of , which has the form

after that, we get

In addition, the contour energy of the lungs and lesions is defined as

The morphological energy is changeless in rotation for same contours. We can conduct recognition and classification on multilesion contours by using such descriptors. In the representation of multilesion contours, the normalization of V-descriptors is necessary. To ensure the transform invariance in translation, rotation, and scale of V-descriptors, the method of normalization can be written as

where . is called as the normalized descriptor of . Besides, since , is the direct current (dc) term. However, the dc term preserves the length energy of multilesion contours, so the dc term is discarded when the descriptors are used. Ultimately, the diagnosis descriptor kernel operator is based on V-descriptors of degree 2, i.e., the energy is defined as

where and are the and components of V-descriptor energy for lung contours, respectively. and are the and components of V-descriptor energy for infected contours, respectively.

From formulas (11) and (12), the severity assessment descriptor is defined as

Fig. 3 shows the flowchart of the proposed algorithm. In order to understand and rebuild the proposed algorithm conveniently, Algorithms 1 and 2 are provided, which describe the kernel algorithm of our system in detail. The new method has the following property: since the lung contours are discontinuous, and the V-system can overcome the Gibbs phenomenon of Fourier series in discontinuous signal representation, the lesion contour feature extraction based on V-descriptors is more accurate. V-descriptors as the feature representation methods have advantages at rotation invariance, translation invariance, and scaling invariance. Overall, the grayscale feature algorithm can fast and accurately extract the effective lesion information, and V-descriptors can represent morphology complexity of lung and lesion contours.

Fig. 3.

Flowchart of the proposed assessment algorithm. The key operations are denoted in violet rectangles, and the core parameters are denoted in blue rectangles. In the first row, the area and contours of bilateral lungs are obtained through the preprocessing of the CT image. In the second row, the lung infection area and its contours are obtained through the intensity feature (coarseness, contrast, roughness, and entropy) extraction. In the third row, multifeatures are fused to generate the final assessment descriptor, which is used to estimate the severity of COVID-19 lung infection.

IV. Experiments and Estimation

In this section, we perform some experiments to demonstrate the performance of the proposed method, and the main contents include the following: the sample CT images, feature extraction results, lesion contour reconstruction results, lung and lesion segmentation results, and classification estimation.

A. COVID-19 CT Dataset

In this article, the dataset UCSD-AI4H (2000 CT images of COVID-19 infected patients downloaded from¹ is used to test and evaluate the proposed algorithm. In addition, the selected 750 images (the similar images are abandoned) from the UCSD-AI4H dataset are used for classification estimation (see Section IV-E). Except for the six cases used in the testing of the new algorithm, all patients recorded in the UCSD-AI4H dataset are diagnosed with COVID-19. The severity assessment program is tested by using MATLAB R2018b in Windows 10 64-bit with Intel Core i7-9700 CPU at 3.00 GHz. When obtaining a severity assessment descriptor for an image with pixels, the proposed algorithm costs 8.62 s, and when the image size changes to pixels, the execution time changes to 116.25 s. Therefore, we suggest that the image size should adjust with downsampling for the suitable time-consuming in the real application. 14 (1.8%) CT images cannot be used to calculate the digital evaluation descriptors because of the gray or shooting mistake of the images. The rest CT images obtain digital assessment descriptors in experiments. In these results, 2 (0.2%) CT images are ignored of gray features because of the mild state of the disease, and 743 (99.07%) effective digital assessment descriptors are obtained. The results show that the new method supply scientific and effective support for the diagnosis of COVID-19.

B. Feature Extraction Experiment

The lesions on CT imaging mainly have morphology and density (or grayscale) features. For the morphological characteristic, we use the V-system to calculate the orthogonal descriptor of the overall multicontour lesions. For the density feature, we use the contrast and entropy to obtain the differences of the adjacent points and their own grayscale energy, and then the coarseness and roughness are combined with V-descriptors. Therefore, the improved V-descriptors can extract morphological characteristic and gray-scale feature at the same time.

The six cases in each row of Fig. 5 correspond to the first column of Fig. 1 in order. As shown in Fig. 5, the coarseness and roughness show the inner density of lungs, and they can indicate the detail texture. We can also find that the contrast on the grayscale difference of boundaries, and the entropy on the gray value change of regions. These four features are combined to calculate a parameter, which is used to describe the condition (or the grayscale difference) of the lungs.

Fig. 5.

Coarseness, contrast, roughness, and entropy feature maps of six selected cases that infected with COVID-19 disease. The original CT images are shown in Fig. 1. The first row corresponds to the extracted coarseness features. The second row shows the extracted contrast features. The third row shows the extracted roughness features. The last row shows the extracted entropy features.

C. Lesion Contour Reconstruction Experiment

The experimental results are given in Fig. 6. The infection contours extracted from three different CT images. Then, we use three testing algorithms, i.e., the Fourier series, Haar wavelet, and V-system, to reconstruct the infection contours. In order to verify the reconstruction performance, we adopt different terms (i.e., 30 and 75 decomposition coefficients) of three testing algorithms in the experiments. It can be found that the V-system has the best performance under the two situations compared with other orthogonal function systems. In fact, the V-system can overcome the Gibbs phenomenon of continuous orthogonal function systems, and it can also avoid the strong discontinuity of Haar wavelet. Therefore, we use V-descriptors to extract the CT features of COVID-19 infected patients.

Fig. 6.

Reconstruction results of infected contours using different orthogonal function systems. The CT images shown in the first column are the infected contours extracted from three original CT images. Other images are the reconstruction results using different methods (i.e., Fourier series, Haar wavelet, and V-system) with 30 and 75 terms.

In fact, the contours of the bilateral lungs and infected regions are important information for severity assessment. Therefore, V-descriptors are adopted in the proposed method to measure the infection ration of the whole lungs through the contour information.

D. Lung and Infection Segmentation Experiment

As an important step in the proposed algorithm, the segmentation method of the lung and infected regions affects the proposed method directly. The lung shape varies from person to person. It is difficult to find a stationary model or pattern to extract lung regions. If the infected regions affect the lung boundary (as shown in the first column of Fig. 1), it will greatly increase the difficulty of lung extraction. Thus, the human computer interaction is chosen to correct lung boundary. When lung region is determined, the infected region can be extracted by gray-characteristics, e.g., coarseness, contrast, roughness, and entropy. From Fig. 1, it can be found that the segmentation method provides an accurate result of lung and infected regions.

E. Classification Estimation

The pneumonia is the dominating symptom of the COVID-19 infected patients, and this kind of pneumonia causes bilateral lung infections. The CT images of most COVID-19 infected patients have different degrees of GGO for the lesions, so some researchers have found that the infection ratio of the whole lung and the volume of GGO regions can be used to indicate the severity of the disease. However, the textures and the gray values of lesions are still important for severity assessment, which cannot be displayed by the previous methods. The proposed assessment descriptor can embody not only the ratio of infections on bilateral lungs [13], but also the density of the lesions. There are 2000 CT images in the testing dataset, and 734 high-resolution images are selected in this experiment.

In Figs. 7–12, the color areas of green, yellow, and red represent the different severity in mild type, common type, and severe type, respectively. When the digital descriptor value is lower than 8.5 (fluctuation range 0.5), the patient is in the mild type of COVID-19. Since the digital descriptor value is lower than 50 (fluctuation range 5) and higher than 8.5 (fluctuation range 5), the patient is in the common type of COVID-19. When the value is higher than 50 (fluctuation range 5), the patient is in the emergency that may threaten the life of the patient. In addition, for clinicians, it is necessary to combine the epidemiological history and the RT-PCR testing result for further diagnosis. We perform the classification estimation among different methods to evaluate their abilities. The experimental results are given in Figs. 7–12. The infected rates on bilateral lungs are drawn in each figure, and the CT images are ordered based on the infected rate values. It can be found that there is a strong correlation between the two indices in Fig. 7, i.e., the infected rate and the assessment descriptor. In fact, the proposed assessment descriptor is fused by coarseness, roughness, contrast, entropy, and V-descriptors features. Specifically, the No. 431, No. 464, No. 585, and No. 641 samples with red circles of Fig. 7 are shown in Fig. 8. The four CT images are arranged in descending order of infected rate, but it can be seen that their severity is not consistent with the infected rate: the severity of No. 431 is greater than that of No. 464, and the severity of No. 585 is greater than that of No. 641. This is because the severity assessor not only includes the feature of the infected rate, but also contains the intensity feature of the lesion. Since the density and gray scale of lung consolidations in No. 431 and No. 585 are greater than those in No. 464 and No. 641, the severity assessment is inconsistent with the infected rate. Besides, the same assessment value of patients at different infected ratios means that the infection densities of them are various. If a patient with a small infection ratio has a larger infection density, it will have the same assessment value as a patient with a large infection ratio, but the severity interval will not change. The proposed descriptor can reflect the infected rate and the content of the lesions at the same time, which is very useful in severity assessment of COVID-19.

Fig. 7.

Changing trends of assessment descriptor and infected rate among 734 COVID-19 infected patients, and the correlation between the two indices is strong.

Fig. 12.

Changing trends of entropy and infected rate among 734 COVID-19 infected patients.

Fig. 8.

These COVID-19 CT images are No. 431, No. 464, No. 585, and No. 641 samples in Fig. 7 with red circles. Although the infected rate of (a) is smaller than that of (b), and the infected rate (c) is smaller than that of (d), the severities of them are reverse.

Fig. 10.

Changing trends of contrast and infected rate among 734 COVID-19 infected patients.

Fig. 11.

Changing trends of roughness and infected rate trends among 734 COVID-19 infected patients.

In the comparison experiment, the coarseness, contrast, roughness, and entropy feature values are drawn in Figs. 9–12, respectively. It can be found that the correlations among these features and the infected rate are weak. Therefore, the severity assessment descriptor has advantages than other methods.

Fig. 9.

Changing trends of coarseness and infected rate among 734 COVID-19.

V. Conclusion

During the high infection incidence of COVID-19 disease, a large amount of clinical experience shows that CT imaging scanning is a fast and an effective clinical diagnosis method. A new severity assessment of COVID-19 based on the CT images was proposed in this article. Compared with manual reading, this algorithm has fast speed and high accuracy properties. The development of the COVID-19 CT severity assessment is of great significance to the society and human beings. It can help frontline clinicians to diagnosis the CT images, effectively alleviate the false negative problem of RT-PCR test, and help hospitals and doctors relieve the pressure of receiving suspected patients. Specifically, the morphological feature (using V-descriptors) and the density features (using the coarseness, contrast, roughness, and entropy textures) are both extracted to construct the severity assessment descriptor. The experimental results demonstrate that the proposed strategy not only reflects the infected ratio on the bilateral lungs, but also manifests the density of the lesions. The new descriptor processes more information than most of the existing methods that only indicate the infection ratio, so the new descriptor is more appropriate to evaluate the severity assessment of COVID-19. In the future, we will apply it to the practical scenes and make improvement in other aspects.

Acknowledgment

The authors would like to thank the Associate Editor and the anonymous reviewers who provided very insightful comments and helpful suggestions for this article.

Biographies

Ben Ye received the B.S. degree in software engineering from Beijing Normal University, Zhuhai, China, in 2013, and the M.S. and Ph.D. degrees in space information technology from the Macau University of Science and Technology, Macau, China, in 2015 and 2019, respectively.

He is currently a Postdoctoral Fellow with the Faculty of Information Technology, Macau University of Science and Technology. His research interests include high-resolution image stitching, remote sensing data processing and analysis, image processing, and computer graphics.

Dr. Ye was a recipient of the Third Prize of the Macau Science and Technology Award-Technological Invention Award, Macau, in 2018.

Xixi Yuan received the B.S. degree in software engineering from Beijing Information Science and Technology University, Beijing, China, in 2016, and the M.S. degree in information technology in 2018 from the Macau University of Science and Technology, Macau, China, where she is currently working toward the Ph.D. degree in computer technology and application with the Faculty of Information Technology.

Her research interests include computer graphics and image processing.

Zhanchuan Cai (Senior Member, IEEE) received the Ph.D. degree in computer software and theory from Sun Yat-sen University, Guangzhou, China, in 2007.

From 2007 to 2008, he was a Visiting Scholar with the University of Nevada, Las Vegas, NV, USA. He is currently a Professor with the Faculty of Information Technology, Macau University of Science and Technology, Macau, China, where he is also with the State Key Laboratory of Lunar and Planetary Sciences, Macau University of Science and Technology. His research interests include image processing and computer graphics, intelligent information processing, multimedia information security, and remote sensing data processing and analysis.

Prof. Cai is a Member of the Association for Computing Machinery, the Chang’e-3 Scientific Data Research and Application Core Team, and the Asia Graphics Association. He is also a Distinguished Member of the China Computer Federation. He was a recipient of the Third prize of the Macau Science and Technology Award-Natural Science Award, in 2012, the BOC Excellent Research Award from the Macau University of Science and Technology, in 2016, the Third Prize of the Macau Science and Technology Award-Technological Invention Award, in 2018, and the Second Prize of the Teaching Achievement Award from the Macau University of Science and Technology, in 2020.

Ting Lan received the M.S. degree from the University of Macau, Macau, China, in 2014, and the Ph.D. degree from the Macau University of Science and Technology, Macau, China, in 2019.

He is currently a Postdoctoral Fellow with the Faculty Information Technology, Macau University of Science and Technology. His research interests include image processing, data processing and analysis, and computer graphics.

Dr. Lan was a recipient of the First Prize at the 14th China Postgraduate Mathematical Contest in Modeling, China Academic Degrees and Graduate Education Development Center and China Graduate Mathematical Contest in Modeling Committee, in 2017, and the Third Prize of the Macau Science and Technology Award-Technological Invention Award, in 2018.

Funding Statement

This work was supported in part by the Science and Technology Development Fund of Macau under Grant 0038/2020/A, Grant 0052/2020/AFJ, and Grant 0069/2018/A2 and in part by the Open Project Program of the State Key Laboratory of Virtual Reality Technology and Systems at Beihang University under Grant VRLAB2019C02.