MisGAN: Learning from Incomplete Data with Generative Adversarial Networks

In plain words

Many real datasets have holes: pixels that were never recorded, sensors that dropped out, survey questions left blank. A standard GAN learns only from complete examples, so holes are a problem.

MisGAN keeps two networks instead of one. One learns to draw complete examples. The other learns the pattern of holes. The pattern is learnable because, in the training data, we always know which entries are missing. The draw-the-examples network is judged only after the same holes are punched into its output, so the judge compares like with like.

Once trained, a third network can fill the holes in a partial example. It adds noise, so it can give a different plausible filling on each run.

TL;DR

MisGAN is a GAN framework for data with missing entries, under the assumption that the missingness is completely random. It trains a complete-data generator and a mask generator, each with its own critic, and a separate imputer. The authors report better FID than ConvAC, zero/mean imputation, matrix factorisation and GAIN on MNIST, CIFAR-10 and CelebA, and the gap widens at high missing rates. The MNIST numbers are in figures only, and the CIFAR-10 results have no code in the release.

Problem

Standard GAN training needs fully observed data §1. Missing data is common, and each example may have different missing entries §1. The paper assumes missing completely at random (MCAR): the mask does not depend on the data §1 §2. Its theory covers finite-alphabet data only §3.

Prior work, AmbientGAN, treats missingness as a corruption with a known, parameterised process. The paper argues that general missingness is not of that form §1.

Two theoretical points frame the design §3:

• The choice of fill constant τ does not change whether the true distribution can be uniquely recovered (Theorem 1 and Corollary 1). Uniqueness depends only on the masks' null space.
• Knowing where the missing entries are does not make recovery easier (Corollary 2).
• Two distributions give the same masked-data distribution if and only if their marginals over the observed entries agree for every mask with positive probability (Theorem 2). This justifies training on masked data.

Method

Representation. An incomplete example is a pair (x, m), where m is a binary mask and m_d = 1 marks observed entries. Missing values in x are ignored §2. The masking operator fills missing entries with a constant τ:

f_\tau(x, m) = x \odot m + \tau\,(1 - m) \qquad [\text{eq. 3}]

Two generator–critic pairs. MisGAN has a data pair (G_x, D_x) and a mask pair (G_m, D_m). Each generator takes its own noise §2. The mask generator output is relaxed from {0,1} to (0,1) with a sigmoid at temperature λ = 0.66 §2 mnist_generator.py mnist_generator.py.

Losses (Wasserstein form). The mask loss compares real masks with generated masks eq. 4. The data loss compares the real masked data with the generated data after the generated mask is applied eq. 5:

\mathcal{L}_m = \mathbb{E}_{m}[D_m(m)] - \mathbb{E}_{\varepsilon}[D_m(G_m(\varepsilon))] \qquad [\text{eq. 4}]

\mathcal{L}_x = \mathbb{E}_{(x,m)}[D_x(f_\tau(x,m))] - \mathbb{E}_{\varepsilon, z}[D_x(f_\tau(G_x(z), G_m(\varepsilon)))] \qquad [\text{eq. 5}]

Objectives. G_x minimises the data loss eq. 6. G_m minimises the mask loss plus α times the data loss eq. 7. The coefficient α is set to 0.2, which the authors found to help even though α = 0 is also possible §2. Both critics must be 1-Lipschitz; the implementation enforces this with a gradient penalty §2 utils.py.

\min_{G_x} \max_{D_x} \mathcal{L}_x \qquad [\text{eq. 6}]

\min_{G_m} \max_{D_m} \mathcal{L}_m + \alpha\,\mathcal{L}_x \qquad [\text{eq. 7}]

Discriminator input. D_x sees masked samples as if they were fully observed. It needs no knowledge of which entries are missing or of τ §2. Standard complete-data architectures can therefore be reused §2.

Training schedule. Discriminators are updated several times for each generator update §2. The paper does not state the number of critic steps. The code uses five misgan.py mnist_misgan.py.

The training loop below is our rendering of eqs. 3–7 and the released loop. The paper has no algorithm box.

# MisGAN training, MCAR case  [eq. 3-7] [code: src/misgan.py#L104-L160]
# Input:  dataset of (data, mask) pairs; tau; alpha; n_critic
# Output: complete-data generator G_x, mask generator G_m

def train_misgan(dataset, tau, alpha, n_critic):
    # Draw a real batch and mask it with tau  [eq. 3]
    # [code: src/misgan.py#L106-L115]
    real_data, real_mask = next_batch(dataset)
    masked_real = real_mask * real_data + (1 - real_mask) * tau

    # Fresh noise for both generators  [§2] [code: src/misgan.py#L118-L124]
    fake_data = G_x(sample_z())
    fake_mask = G_m(sample_eps())    # soft values in (0, 1)
    masked_fake = fake_mask * fake_data + (1 - fake_mask) * tau

    # WGAN-GP critic step on masked data and on masks  [eq. 4-5]
    # Gradient-penalty coefficient 10 in code  [code: src/utils.py#L17-L23]
    update_critic(D_x, masked_real, masked_fake)
    update_critic(D_m, real_mask, fake_mask)

    # Every n_critic critic steps, one generator step
    # [code: src/misgan.py#L134]
    if step % n_critic == 0:
        # Fresh noise for the generator step  [code: src/misgan.py#L147-L152]
        fake_data = G_x(sample_z())
        fake_mask = G_m(sample_eps())
        masked_fake = fake_mask * fake_data + (1 - fake_mask) * tau

        # G_x minimises its data loss  [eq. 6] [code: src/misgan.py#L154]
        data_loss = -D_x(masked_fake).mean()
        update(G_x, data_loss)

        # G_m minimises mask loss + alpha * data loss  [eq. 7]
        # Reuses data_loss from above  [code: src/misgan.py#L158-L160]
        mask_loss = -D_m(fake_mask).mean()
        update(G_m, mask_loss + alpha * data_loss)

Mask generator is not binary. The mask generator outputs sigmoid(x / λ) with λ = 0.66, so generated masks are soft. Real masks are binary §2 mnist_generator.py misgan.py.

Imputation. To impute, the authors add an imputer G_i with its own critic D_i §4. The imputer takes the incomplete example and noise ω, and keeps the observed entries. The imputed output is

G_i(x, m, \omega) = x \odot m + \widehat{G}_i(x \odot m + \omega \odot \bar{m}) \odot \bar{m} \qquad [\text{eq. 12}]

where Ĝ_i is any network with the same input and output shape §4. Noise is injected only where entries are missing, so the noise amount scales with the number of missing entries §4.

The imputer loss L_i contrasts generated complete samples G_x(z) with imputed real samples §4. The paper trains the joint objective with β = 0.1 for G_x eq. 10 §4. A stand-alone imputer can be trained with a pretrained G_x, without the incomplete data eq. 11 §4.

Hyperparameters

Results

Author-reported. Setup: MNIST (conv and FC variants), CIFAR-10, CelebA at 64×64. Baselines: ConvAC (for generation), zero/mean imputation, matrix factorisation and GAIN (for imputation). Missing rates from 10% to 90% in 10% steps §5.1. The text gives few numbers; most results are in figures.

The authors also argue that FID ranks methods differently from RMSE, and that MisGAN is worse on RMSE but better on FID at 90% block missingness app. C. They report that GAIN training was unstable on block missingness and that MisGAN trained more stably, without a number §5.2.

Paper vs code

Official code: github.com/steveli/misgan, MIT, copyright Steven Cheng-Xian Li (code at commit f30dd73e). The repository includes a notebook and both a PyTorch 1.x and a 1.6 variant; this card uses src/.

• Epochs. The paper says MisGAN trains for 300 epochs on every dataset app. D. The MNIST script defaults to 500 mnist_misgan.py and CelebA to 600 celeba_misgan.py. Which values produced the paper's figures is not stated.
• Imputer epochs. The paper gives 1000 for MNIST and CIFAR-10 and 600 for CelebA app. D. The CelebA imputer script defaults to 800 celeba_misgan_impute.py.
• Imputer architecture (MNIST). The paper says a three-layer network with 500 hidden units §5.1. The default is 784-784 with four linear layers mnist_imputer.py mnist_imputer.py. The imputation script's default imputer is the "comp" variant mnist_misgan_impute.py.
• Mask transform in imputation. The paper uses a sigmoid mask generator §2. The imputation scripts default to a fusion transform with a rescaled, clipped sigmoid mnist_misgan_impute.py mnist_misgan_impute.py. The main MNIST script defaults to plain sigmoid mnist_misgan.py.
• α for imputation and CelebA. The paper gives α = 0.2 §2 without distinguishing tasks. The MNIST imputation default is 0.1 mnist_misgan_impute.py and so is CelebA celeba_misgan.py.
• β term does not reach G_x. The paper trains G_x on L_x plus β·L_i eq. 10. The released update adds β times the imputer loss to the data-generator loss misgan_impute.py. But that imputer loss is computed from real data and the imputer alone misgan_impute.py, so as far as we can read, it gives G_x no gradient. The paper's L_i term scores G_x(z), which the release does not do here.
• Imputer output activation. The paper leaves the activation of Ĝ_i unstated. The code applies a sigmoid, keeping pixels in (0, 1) imputer.py.
• Imputer learning rate. The imputer and its critic use 2e-4; the MisGAN networks use 1e-4 misgan_impute.py. Not stated in the paper.
• γ. The code has an optional observed-entry fit term, default off misgan_impute.py. The paper does not describe it.
• CelebA preprocessing. The paper says the aligned images were cropped and resized to 64×64 §5. The code center-crops to 108 pixels, then resizes to 64 with bicubic interpolation masked_celeba.py.
• Variable-size MNIST masks. The paper says side lengths are drawn from 25–75% of the image §5.1. The code draws integers from 7 to 20 masked_mnist.py, because randint excludes its upper bound. The paper's 93.75% maximum missing rate matches a 7×7 observed block.
• CelebA block masks. The code uses a minimum side of 12 pixels, with an exclusive upper bound masked_celeba.py. The paper does not give CelebA mask sizes in the text.
• CIFAR-10. The paper reports CIFAR-10 results §5 app. E. The release has no CIFAR-10 script; only MNIST and CelebA are included.
• Gradient penalty and critics. The critic objective, gradient penalty (target norm 1) and critic/generator update ratio match eqs. 4–7 utils.py misgan.py.
• Mask fixed per example. Masks are generated once at dataset construction with seed 0 masked_mnist.py. Each training example keeps its mask across epochs. The paper does not say whether masks are resampled.
• Generator crop. The code crops an intermediate feature map to 7×7 before the last layers mnist_generator.py. The paper only says the generator follows DCGAN app. D.
• Follow-up. The README points to a follow-up repository by the same author, "partial-encoder-decoder", for faster and more stable training README.md. We have not reviewed it.

Pitfalls

• Soft masks. Generated masks lie in (0, 1); real masks are binary. The mask critic can tell them apart by how saturated they are, and the paper does not discuss this. The λ = 0.66 temperature is the paper's only control over saturation §2. The code's "hard" variants are not analysed in the paper.
• Fill value and images. With τ = 0 and pixels in [0, 1], missing pixels look like black pixels. The paper notes that for τ = 0 the masked output is the product x ⊙ m, so the data and mask roles can swap in the masked output alone. That is why the mask critic is needed §5.1.
• Missing data model. Only MCAR is implemented and analysed §2 §3. Under MAR or NMAR the theory and the mask-generator design do not apply.
• Baseline selection. For GAIN, the authors use the checkpoint with the best FID rather than a fixed epoch app. H. GAIN is therefore selected on the same metric used to compare it with MisGAN, which favours the baseline. MisGAN is trained for a fixed number of epochs.
• FID is a distribution metric. Its sample count equals the training-set size §5, and it can disagree with RMSE on imputation app. C. Pick the metric that matches the task.
• Cost. MisGAN trains two generators and two critics for generation, and adds an imputer and a third critic for imputation. The paper gives no wall-clock or memory figures.
• Unstated values. Learning rate, batch size, critic steps and gradient penalty weight are not in the paper. Take them from the code, and say so when you use them.
• Theory limits. Theorems 1 and 2 assume finite alphabets, while the experiments use continuous pixels §3. They describe identifiability of the masked distribution, not the GAN's convergence.

Lineage

• Builds on: AmbientGAN, which treats missingness as a known corruption; MisGAN drops the known-process assumption and adds a mask GAN §1. Wasserstein training with gradient penalty §2 app. D.
• Compared against: ConvAC, a tractable generative model §5.1; zero/mean imputation, matrix factorisation and GAIN for imputation §5.2.
• Superseded by: not stated in the paper. The README names a follow-up from the same author (see Paper vs code).