Face Mask Detection using Transfer Learning of InceptionV3

In plain words

Wearing a mask in crowded places was a public-health rule during COVID-19, and watching every person by hand is hard. This paper asks whether a computer can tell, from a picture of a face, whether the person is wearing a mask.

The authors did not build a vision model from scratch. They took a network that had already learned to recognise everyday objects from a very large image collection, cut off its final output layer, and attached a small new head that answers only "mask" or "no mask". Retraining that head, and the top of the network, is much cheaper than training everything from zero.

The data is the other half of the problem. Their dataset has 1570 images, so they multiplied the training pictures with simple edits such as flips, rotations and blur, so the network sees more variety.

The reported results are very high, but they come from one simulated dataset and the authors' own tests. The card's Pitfalls section lists where those numbers need care.

TL;DR

• Transfer learning: an ImageNet-pretrained InceptionV3 with its last layer removed, plus a new head (average pool 5x5, flatten, dense 128 ReLU, dropout 0.5, dense 2 softmax) §4.2 fig. 4.
• Training images get eight augmentation operations (six named) and are converted to 224x224 single-channel grey §4.1. The Fig. 2 example shows the augmentation fig. 2.
• Trained 80 epochs of 42 steps each §4.2. Author-reported test accuracy 100% on SMFD, against 92.89% to 99.3% for the baselines in Fig. 7 tab. 1 fig. 7.
• Missing: optimizer, learning rate, batch size, loss function, augmentation magnitudes and which layers were frozen. Several counts do not match (see Pitfalls).
• Not code-verified: no official code was found.

Problem

Detecting people who are not wearing masks in public places, to help enforce mask rules during COVID-19 §1. The authors argue that few mask-detection papers existed and that existing face work targets identity rather than masks §2. Their stated constraint is the limited number of labelled images, since access to real faces is restricted for privacy reasons §4.

Method

The pipeline has two phases. Phase 1 augments the training images. Phase 2 trains a transfer-learning classifier. Evaluation uses the metrics in §5.

1. Dataset split. SMFD has 1570 images: 785 masked and 785 unmasked §3. The authors use 1099 images for training and 470 for testing §3. The validation split, class balance per split and the random seed are not stated.

2. Image augmentation §4.1 fig. 2. Training images are augmented with what the text calls eight distinct operations. Six are named: shearing, contrasting, horizontal flip, rotation, zoom and blur §4.1. The magnitudes, probabilities and whether augmentation was applied at random or as a fixed expansion are not stated. The augmented images are rescaled to 224 x 224 and converted to one grey channel §4.1. The augmented image count is not stated.

3. Backbone. InceptionV3, pretrained on ImageNet, is loaded as the base §4.2. Its last layer is removed §4.2. The paper does not say which backbone layers were frozen or fine-tuned.

4. New head, applied in this order §4.2 fig. 4: average pooling with a 5 x 5 window, a flatten layer, a dense layer with 128 ReLU units, dropout with rate 0.5, and a dense layer with 2 softmax units for the two classes.

5. Training. Trained for 80 epochs with 42 steps per epoch §4.2. Fig. 3 labels the training stage "model training via transfer learning" and the loop "retrain till minimal loss" fig. 3. The text fixes the epoch count and gives no stopping criterion beyond it.

6. Metrics §5. With TP, FP, TN and FN from the confusion matrix:

   Accuracy = \frac{TP + TN}{TP + TN + FP + FN} eq. 1

   Precision = \frac{TP}{TP + FP} eq. 2

   Sensitivity = \frac{TP}{TP + FN} eq. 3

   Specificity = \frac{TN}{TN + FP} eq. 4

   IoU = \frac{TP}{TP + FP + FN} eq. 5

   MCC = \frac{TP \times TN - FP \times FN}{\sqrt{(TP+FP)(TP+FN)(TN+FP)(TN+FN)}} eq. 6

   The paper also gives a "classification error" eq. 7, see Pitfalls:

   CE = \sum_{i=1}^{n} Y_i \log_2(p_i) eq. 7

The paper gives these steps in prose only. The pseudocode below is our reconstruction from §4.1 and §4.2, not a listing from the paper.

# Augment one training image  [§4.1] [fig. 2]
# Input:  face image (colour)
# Output: image, 224 x 224, one grey channel

def augment_training_image(image):
    # The six operations named in the paper; parameters not stated  [§4.1]
    image = shear(image)
    image = adjust_contrast(image)
    image = flip_horizontal(image)
    image = rotate(image)
    image = zoom(image)
    image = blur(image)
    # Resize, then convert to a single grey channel  [§4.1]
    image = to_grey(resize(image, height=224, width=224))
    return image

# Build the transfer-learning classifier  [§4.2] [fig. 4]
# Input:  pretrained InceptionV3 base (ImageNet weights)
# Output: model giving 2 class probabilities (mask, no mask)

def build_classifier(base):
    # Remove the base's last classifier layer  [§4.2]
    features = base.without_last_layer()
    # Added head, in order  [§4.2] [fig. 4]
    pooled = average_pool(features, size=(5, 5))
    flat = flatten(pooled)
    hidden = dense(flat, units=128, activation="relu")
    hidden = dropout(hidden, rate=0.5)
    return dense(hidden, units=2, activation="softmax")

# Train the classifier  [§4.2] [fig. 3]
# Optimizer, learning rate, batch size and loss are not stated
def train(model, train_images, train_labels):
    # 80 epochs of 42 steps each  [§4.2]
    for epoch in range(80):
        for step in range(42):
            # Batch size not stated; sample from augmented images
            batch = sample_batch(train_images, train_labels)
            loss = classification_loss(model, batch)
            update_weights(model, loss)
    return model

Hyperparameters

Results

Author-reported. Setup: SMFD, InceptionV3 transfer learning versus eight baselines trained "under the same environment" §5. Metric for Fig. 7 is not stated on the figure. The 470-image test split is stated in §3, but the confusion matrices in fig. 5 total 388 test images (see Pitfalls).

Confusion matrix counts, as printed in fig. 5:

Baseline comparison, values printed on the bars of fig. 7 (metric not labelled):

Fig. 7 includes ResNet50, which the text does not name among the compared models §5. The text names decision tree, SVM, MobileNet, MobileNetV2, Xception, VGG16 and VGG19 §5.

Qualitative output: the model draws a box around each face and labels it green (mask) or red (no mask) with a confidence score §5.1 fig. 8.

Paper vs code

No official code. Searched the paper and its arXiv page for a repository or project link (none; the arXiv page has no comments field), then GitHub repository search for the title words and for "face mask inceptionv3 transfer learning SMFD" (zero results both times). The only URL in the bibliography is the SMFD dataset page [bib 18], which is a dataset, not method code.

Pitfalls

• Counts do not match. The text gives 1099 training and 470 test images §3. The confusion matrices total 2630 training samples (1315 per class) and 388 test samples (194 per class) fig. 5. The paper does not say whether the training figure is the augmented set or why the test figure is smaller. Reproduce from the stated split and treat the matrix counts as unexplained.
• Augmentation is under-specified. "Eight distinct operations" are mentioned, but only six are named §4.1. No magnitudes, probabilities or output sizes are given. The text says training images are augmented; whether test images were augmented is not stated.
• Augmented copies and leakage. The paper does not say how the split was made or whether augmented versions of one source image can land in both training and test sets. The reported 100% test accuracy cannot be interpreted without that.
• Grey input. Images are converted to one grey channel §4.1, but the backbone is described as ImageNet-pretrained §4.2. The paper does not say how the first layer was adapted for one channel.
• Frozen versus trained layers. The text does not say which backbone layers are frozen. The result depends on it, and the card cannot infer it.
• Stopping rule. The text fixes 80 epochs and 42 steps §4.2. Fig. 3's "retrain till minimal loss" fig. 3 is not defined further.
• Loss and "classification error". Eq. 7 has no minus sign eq. 7. As written it is a log-likelihood, not a loss, so the Table 1 loss values cannot be checked against it. The text calls it "classification error" §5. Any reimplementation has to choose the loss itself; the paper does not state one.
• Headline numbers differ. The abstract gives 99.9% training accuracy [abstract]; Table 1 gives 99.92% tab. 1. The conclusion gives 99.92% and 99.9% (specificity) §6.
• Boxes come from an unstated detector. The method is described as a classifier of whether a person wears a mask §4.2. Fig. 8 draws boxes around faces fig. 8, but the paper does not say how the face boxes are produced.
• Baselines are not fully specified. The text says baselines were trained "under the same environment" §5. Their hyperparameters, epochs and metric are not stated. Fig. 7 has no metric label.
• Only one dataset. All results are on SMFD §3, a simulated dataset. The paper reports no evaluation on real-world photographs. The dataset's licence is not stated in the paper; its source is cited to a GitHub URL [bib 18].
• Costs. Compute is stated only as Google Colab §5. No timing, memory or GPU type is given.

Lineage

• Builds on: InceptionV3, "Rethinking the Inception Architecture for Computer Vision" (arXiv:1512.00567, bibliography entry [19]). The paper cites several pretrained models together §4.2 [19-24].
• Compared against: decision tree, SVM, MobileNet, MobileNetV2, Xception, ResNet50, VGG16 and VGG19 §5 fig. 7.
• Superseded by: not known from content in this wiki.