Novel Multi-Modal Throat Inflammation and Chest Radiography based Early-Diagnosis and Mass-Screening of COVID-19

3.1. Depth-Wise Separable Convolution

Depth-Wise Separable Convolutions [14, 15] are a form of factorized convolutions. A standard convolution is factorized into a depth-wise convolution and 1 X 1 convolution known as point-wise convolution. In a standard convolution new set of outputs are generated by filtering and input-combining in a single step. The depth-wise separable convolution divides this process into two layers: filtering and combining. A single filter is applied to each input channel in depth-wise convolutions. The outputs of the depth-wise convolutions are then combined by the 1 X 1 point-wise convolutions. This technique significantly reduces computational complexity and model size. Consider the following notations:

F – Input Feature Map

G – Output Feature Map

Ĝ – Filtered Output Feature Map

– Depth-wise convolutional kernel

D_F – Spatial Width and Height of the square input feature map

M – Number of input channels (input depth)

D_G – Spatial Width and Height of a square output feature map

N – Number of output channel (output depth)

D_K – Spatial width and height of the square kernel

A standard convolution layer takes input as a feature map F of dimension D_F × D_F × M, and generates output as a feature map G of dimension D_F × D_F × N.A. A convolution kernel K of dimension D_K × D_K × M × N. Considering stride one and padding, the output feature map is computed as shown by Eq 1.

(1)

Thus, the computational cost for standard convolution can be computed, as shown by Eq 2.

(2)

From equation 2, it can be observed that the number of input channels, kernel size, number of output channels, and the size of the feature map controls the computational cost multiplicatively. The proposed architecture addresses these terms by implementing depth-wise separable convolutions to drop the interaction between the output channel's number and the kernel's size. For substantial computational cost reduction, the filtering and combination steps are split into two steps by utilizing factorized convolutions: depth-wise separable convolutions.

Depth-wise convolutions and point-wise convolutions are the constituents of depthwise separable convolution. Depth-wise convolutions apply a single filter per each input channel. 1 X 1 point-wise convolution then creates a linear combination of the depthwise layer's output. Batchnorm and ReLU non-linearities are utilized for both depth-wise and point-wise convolution layers. The standard and depth-wise separable convolution has been distinguished, as shown in Fig. (5). For one filter per input channel, the depth-wise convolution is as shown by equation 3, where is the depth-wise convolution kernel of size D_K × D_K × M, where the m_th filter in is applied to the channel of the filtered output feature map Ĝ .

(3)

Depth-wise convolution is significantly efficient relative to standard convolution. The computational cost of depth-wise convolution is as shown by Eq 4.

(4)

Depth-wise convolution only filters input channels; however, these are not combined to create new features. An additional layer to compute a linear combination of the output of depthwise convolution through 1 X 1 convolution is needed to generate these new features.

Thus, the combination of depth-wise convolution and point-wise convolution is called Depth-Wise Separable Convolution. Fig. (6) helps visualize the architectural difference between standard and depth-wise separable convolutions considering 3 X 3 convolutions. The sum of the depthwise and 1 X 1 point-wise convolutions gives us the Depth-Wise Separable Convolutions cost, as shown by Eq 5.

(5)

Hence, by expressing convolution as an amalgamation of separate filtering and combination steps, the computation cost can be significantly reduced, as shown by Eq 6.

(6)

Thus, the proposed method implements depth-wise separable convolution for efficient throat inflammation and chest x-ray based COVID-19 diagnosis.

3.2. Throat Inflammation based COVID-19 Early Prediction

Throat inflammation is one of the perceivable symptoms of COVID-19 that can be utilized for efficient mass screening of susceptible patients. This section formulates the deep learning architecture for throat inflammation detection, which can be deployed for accurate early prediction of Coronavirus patients. The utilization of fine-tuning and transfer learning techniques has been proposed considering the limited dataset acquired from web-scrapping and medical professionals' aid.

MobileNet consisting of 28 layers utilize 3 X 3 depthwise separable convolutions. The original MobileNet has been trained and evaluated over millions of images across 1000 classes of the ImageNet [16] dataset. Dense layers are added to the pre-trained model, and the entire model has been fine tuned. Using the pre-trained weights entirely is not practical since this research focuses on biomedical image-based classification for COVID-19 prediction. Since the initial layers extract low-level features, including edges and shape, preserving them becomes imperative. As shown in Fig. (7), dense layers have been added for classification, and all the layers except the last 21 layers have been frozen. Experiments on the fine-tuned model indicate promising results and validate the described idea.

3.3. Chest X-Ray based COVID-19 Diagnosis

Once the mass screening and early-prediction have been performed by utilizing throat inflammation-based analysis, accurate diagnosis can be performed by implementing the proposed architecture for Chest X-Ray based analysis since several COVID-19 patients have been diagnosed with pneumonia; hence radiological examination is considered beneficial.

Xception architecture consisting of a linear stack of 36 depth-wise separable convolutional layers with residual connections for feature extraction has been effectively fine-tuned. As shown in Fig. (8), specific dense layers have been added for accurate classification. The original XceptionNet has been trained and evaluated over millions of images across 1000 classes of the ImageNet dataset. As mentioned previously, initial layers extract fundamental features; subsequently, the first 26 layers are frozen. Experiments showcase commendable results and indicate the reliability of the method for confirming the patient's susceptibility.

3.4. Loss Function and Label Smoothing

The proposed architectures for Throat Inflammation and Chest X-Ray based COVID diagnosis deploy the Categorical Cross-Entropy Loss, also known as the Softmax Loss: Softmax Activation followed by a Cross-Entropy Loss.

The architecture is trained to output the probability distribution over the classes for each image. Considering the One-Hot Encoding of the classes, the loss can be elucidated as:

The derivative respect to positive and negative classes can be shown by Eqs. 7 and 8.

(7)

(8)

Label smoothing technique replaces one-hot encoded label with its mixture with uniform distribution. Label Smoothing is beneficial when the loss function is cross-entropy, and the model applies the softmax function to the penultimate layer to compute its output probabilities. The One-Hot encoded labels encourage the most extensive possible logit gaps to be fed into the softmax function.

Intuitively, large logit gaps combined with the bounded gradient make the models less adaptive and too confident about their predictions. The smoothed labels encourage small logit gaps and subsequently result in better model calibration and prevent overconfident predictions.

3.5. Optimizer

To ensure quick convergence, such that the loss function reaches the local minima in less number of epochs, a meticulously selected optimizer must be incorporated. Adam [17] optimizer ensures quick convergence, such that the loss function reaches the local minima in fewer number of epochs. Adam's learning rate is scaled by utilizing squared gradients, and the advantage of the Momentum is taken by applying the moving average of the gradient.

If the decaying hyperparameter determines how rapidly accumulated previous gradients decay is much larger than the learning rate, then the accumulated previous gradients will be dominant in the update rule; hence the gradient at the iteration will not change the current direction rapidly. On the other hand, if the decaying hyperparameter is much smaller than the learning rate, the accumulated gradients act as a smoothing factor for the gradient.

To overcome this, Nesterov Accelerated Gradient (NAG) [18] calculates the gradient w.r.t approximate future position of various parameters. NAG thus acts as the correction factor for the Momentum method. Moreover, NAG mitigates the issue of oscillations that arise in the Momentum method at large learning rates. Nesterov-accelerated Adaptive Moment Estimation (Nadam) [19, 20] combines Adam and NAG, such that Nadam can be interpreted as Adam with Nesterov momentum

Consider the following notations:

w_t – Weight at timestamp t

g_t – Gradient w.r.t w at timestamp t

α – Learning Rate

m – Estimate for the first moment of gradient

v – Estimate for the second moment of gradient

Such that:

(9)

(10)

(11)

(12)

Considering ADAM as an optimizer:

(13)

Considering NADAM as an optimizer:

(14)

The Nesterov acceleration over Adam promotes faster convergence of the loss function to the local minima as compared to Adam. Hence, the proposed architecture utilizes Nadam as an optimizer for the Throat Inflammation based mass-screening and early-prediction; and Chest X-Ray based COVID-19 diagnosis.

4.1. Data Preparation

For throat images, the data was collected from the web through extensive web scraping. Around 200 images were identified to be of interest, out of which 146 images had sufficient visual information for classification and were then segregated into infected and normal images by a medical professional [21]. Moreover, image augmentation steps like width shifting, height shifting, brightness range, zooming were performed for the deep learning-based method so that the learned model could be better at generalizing all conditions. Augmentation resulted in a total of 300 images which were further divided into training and validation.

An openly available public dataset was utilized for training the Chest X-ray model. To train the chest X-Ray model, two combined datasets, i.e., for COVID Positive images, 182 images of the posteroanterior (PA) view were collected from the IEEE Dataport [22], which is a growing collection of chest radiography images and images of normal patients were obtained from Kaggle's Chest X-Ray (Pneumonia) [23] dataset.

4.2. Training Details

Model training was achieved using the Tensorflow framework. The input image was resized accordingly to the input requirement of the models. The throat infection detection model was trained using the following hyperparameters:

• Number of epochs: 100
• Batch size: 32
• Optimizer: NADAM
• Momentum Parameters: β1 = 0.9 and β2 = 0.999
• Learning Rate: 3e-4

The model for Chest X-Ray analysis was trained using the following hyperparameters:

• Number of epochs: 10
• Batch size: 32
• Label Smoothing: 0.01
• Optimizer: NADAM
• Momentum Parameters: β1 = 0.9 and β2 = 0.999
• Learning Rate: 3e-4

The convergence of training and validation loss for the throat infection model is shown in Figure 9. The figure shows that the model correctly classifies whether the input image is throat infected or not.

As showcased in Fig. (9), training loss and validation loss are converging with an increase in epochs, and there is not any significant difference in training and validation loss; hence one can even conclude that model is not overfitting. The training accuracy can be observed in Fig. (10).

Further, the chest X-Ray classification model for COVID-19 architecture demonstrates promising results as represented (Figs. 11 and 12).

4.3. Evaluation

For evaluating medical models, choosing the right and sufficient evaluation metrics is necessary as these metrics play an important role in ascertaining the practical performance. In this section, various evaluation metrics are described and calculated.

The confusion matrix mentions the model's outcomes in terms of the true positive, the true negative, the false positive, and the false-negative samples. Tables 1 and 2 show the Confusion Matrix for the model's predictions w.r.t the ground truth.

These values can be used to calculate different parameters such as Positive Predicted Value (PPV), Negative Predicted Value (NPV), Sensitivity, Specificity, F1 Score, and Accuracy. All the metrics mentioned above have been calculated and summarized in Table 3.