Missing data imputation

Overview

Some entries of a dataset are not recorded: pixels, sensor readings or survey answers. Two tasks follow. One is to train a generative model on the incomplete data, without discarding every example that has a gap. The other is to fill the gaps in a partial example. The paper addresses both and assumes that the missingness is completely random, so the mask does not depend on the data 1902.09599 §1 1902.09599 §2.

Each incomplete example is a pair of values and a binary mask, where a mask entry of 1 marks an observed value 1902.09599 §2. Missing entries are filled with a constant τ before they reach a critic 1902.09599 eq. 3. The paper's identifiability results show that the choice of τ does not change whether the true distribution can be uniquely recovered. Uniqueness depends only on the null space of the masks 1902.09599 §3.

Variants

Adversarial generation with a learned mask

The approach in MisGAN has three networks. A data generator and a mask generator each have a critic. The data critic judges real and generated examples after the same mask is applied, so the comparison is between like-for-like inputs 1902.09599 §2 1902.09599 eq. 5. The mask generator is trained on its own mask critic, and it also receives a weighted copy of the data loss 1902.09599 eq. 7. A separate imputer, trained with its own critic, fills the gaps in a partial example, and it keeps the observed entries unchanged 1902.09599 §4 [arxiv:1902.09599 eq. 12].

What it does not need is a known model of the missingness process. The paper contrasts this with AmbientGAN, which treats missingness as a corruption with a known, parameterised form 1902.09599 §1.

When to use which

The card supports only one variant, so the table compares this approach against the constraints it can meet, not against other methods.

Implementation sketch

The masking and imputation steps below follow the paper's equations. The training loop is in the card, which cites the release 1902.09599 §2. Shapes are (batch, dim).

# Fill missing entries with a constant  [arxiv:1902.09599 eq. 3]
def mask_fill(x, m, tau):
    return x * m + tau * (1 - m)

# Keep observed entries; fill the rest with imputer output
# [arxiv:1902.09599 eq. 12]
def impute(x, m, omega, g_hat):
    # noise only where entries are missing
    noisy = x * m + omega * (1 - m)
    # the imputer's output is used only at missing positions
    fill = g_hat(noisy) * (1 - m)
    return x * m + fill

The mask generator's output is soft, in (0, 1), while real masks are binary. The paper relaxes the mask with a sigmoid at temperature λ = 0.66 1902.09599 §2.

Contradictions

Only one card lists this concept, so there are no disagreements between papers to report. Within the paper, the release differs from the text in several places. These are not resolved here.

• The text says the imputer's loss term β trains the data generator through an L_i term that scores G_x(z) 1902.09599 §4 1902.09599 eq. 10. The card's reading of the release suggests that term does not reach the data generator as written there (see the card's Paper vs code).
• The text gives the MNIST imputer as three layers with 500 hidden units 1902.09599 §5.1. The default script uses four linear layers of width 784.
• The text gives 300 training epochs for MisGAN on every dataset 1902.09599 app. D. The scripts default to 500 for MNIST and 600 for CelebA.

Papers

• MisGAN: Learning from Incomplete Data with Generative Adversarial Networks: the only card for this concept. It gives a masked-critic GAN, a learned mask generator and an adversarial imputer for data missing completely at random.