Nested case-control sampling

Overview

A survival likelihood compares each event with everyone still at risk at that moment. For large data sets that sum is expensive and does not batch 1907.00825 §3.2. The sampling idea keeps the comparison but draws a small set of controls from the risk set for each event, so the loss touches only a few individuals per event 1907.00825 §3.2. The invariant every variant keeps is that each event is paired with at least one control drawn from the risk set at the event time, and the pair loss is log(1 + exp(difference of risk scores)) 1907.00825 §3.2 eq. 9.

Variants

One sampled control per event

Draw one control j for each event i and average log(1 + exp[g(x_j) - g(x_i)]) over events 1907.00825 §3.2 eq. 8 eq. 9. Controls are drawn with replacement, and a new control is drawn in every iteration 1907.00825 §3.2. Because the loss has no unique minimiser for a non-linear g, the paper adds lambda times the sum of |g| over the sampled outputs 1907.00825 §3.3 eq. 10. The paper's own experiments use lambda in {0.1, 0.01, 0.001, 0} 1907.00825 app. A.1. Card: Time-to-Event Prediction with Neural Networks and Cox Regression.

On simulated data, one control gives a mean log-likelihood gap of about 0.1 % to the exact partial likelihood on the smallest sets 1907.00825 §5 1907.00825 fig. 3.

Sampled control with time as an input

The network receives time as a covariate. Each event i is paired with a control j drawn from its risk set, and both are evaluated at the event time T_i 1907.00825 §3.4 eq. 12. The sampled set has fixed size, so the cost per epoch is O(n) rather than O(n^2) for the full risk set 1907.00825 §3.4. Card: Time-to-Event Prediction with Neural Networks and Cox Regression.

Full risk set for the baseline

Sampling is used only for training. The Breslow increments at each distinct event time sum over the full risk set 1907.00825 §3.5 eq. 13. The paper allows a random subset of training rows to estimate the baseline, at the price of a coarser time grid and no error analysis 1907.00825 §3.5. Card: Time-to-Event Prediction with Neural Networks and Cox Regression.

Number of controls per event

The simulations compare 1, 2, 4 and 8 controls 1907.00825 §5. More controls give lower training error, but the convergence speed does not change and the computation grows with the number of controls 1907.00825 fig. 1 1907.00825 §5. Card: Time-to-Event Prediction with Neural Networks and Cox Regression.

When to use which

The number of controls is varied only in the simulations 1907.00825 §5. The real-data tables compare whole models, not control counts 1907.00825 tab. 2 1907.00825 tab. 7.

Implementation sketch

The common core is the pair loss over one sampled control per event. Pseudocode follows the paper's equations; the case-inclusion question is in Contradictions.

# Sampled-control loss for one mini-batch of events  [arxiv:1907.00825 §3.2]
# Input:  g (network on covariates); x_case (B, d); risk_sets per event
# Output: scalar loss

def sampled_control_loss(g, x_case, risk_sets, lam):
    # Draw one control per event from its risk set, with replacement  [§3.2]
    x_ctrl = sample_one_per_row(risk_sets)
    g_case = g(x_case)
    g_ctrl = g(x_ctrl)
    # Pair loss, averaged over events  [eq. 9] [eq. 8]
    loss = mean(log(1 + exp(g_ctrl - g_case)))
    # Penalty on sampled outputs, pulls them toward 0  [eq. 10]
    return loss + lam * (sum(abs(g_case)) + sum(abs(g_ctrl)))

For the time-as-input variant, evaluate both members of each pair at the event time T_i and feed the time to the network 1907.00825 §3.4 eq. 12. For the baseline, sum over the full risk set at each distinct event time 1907.00825 §3.5 eq. 13.

Contradictions

• The sampled set is said to include the event individual itself 1907.00825 §3.2, but eq. 9 draws the control from the risk set without that individual 1907.00825 eq. 9. The paper does not resolve this. Excluding the case is the cleaner reading of eq. 9, but the paper does not say which one its experiments used.

Papers

• Time-to-Event Prediction with Neural Networks and Cox Regression: introduces a sampled-control loss for a neural Cox model, and a time-as-input variant that drops proportional hazards.