---
kind: concept
slug: discrete-choice-modeling
title: Discrete choice modeling
summary: >-
  Predicting which option a chooser picks from an offered set. Parametric random-utility models fix the utility form; RUMnet learns it with a neural network and can approximate any random-utility model. Author-reported on two datasets: best accuracy, but a random forest had the lowest Swissmetro loss.
tags: [discrete-choice, random-utility, neural-network, choice-modeling]
papers: ["2207.12877"]
related: [multi-layer-perceptron, random-forest, universal-approximation, multi-label-classification]
status: draft
updated_at: 2026-10-19
---

## Overview

Given the attributes of an offered set (assortment) of products and of a
customer, predict the probability that each product is chosen
[arxiv:2207.12877 §1]. The shared core is the random utility maximization
(RUM) idea: each product and each customer has unobserved attributes, every
product gets a utility from them, and the chosen product is the one with the
highest utility. The choice probability is the chance that a product's utility
is the maximum [arxiv:2207.12877 §2.1]. Variants differ in how they model the
utility function and the unobserved attributes. The variants below all target
this same quantity.

## Variants

### Parametric RUM models (MNL, nested logit, latent-class MNL)

These fix the utility function and the distribution of the unobserved parts in
advance. The paper says the fit then depends on the modeller's choices
[arxiv:2207.12877 §1] [§2.1]. Multinomial logit (MNL) is the basic member.
Latent-class MNL allows customer segments [arxiv:2207.12877 §5.2] and is the
stronger of the two parametric baselines in the paper's tables
[arxiv:2207.12877 tab. 1]. No card yet for these; they appear only as
benchmarks in the RUMnet card below.

### RUMnet: a learned RUM approximator

RUMnet keeps the RUM structure but replaces the random fields with sample
averages. It draws K samples of the unobserved product attributes and K samples
of the unobserved customer attributes, each produced by a feed-forward network.
A utility network scores every product for each of the K^2 sample pairs, a
softmax turns those scores into probabilities, and the K^2 results are averaged
[arxiv:2207.12877 §2.2] [§2.3]. It is trained by minimising the negative
log-likelihood of the observed choices [arxiv:2207.12877 §4.1]. The paper shows
that the set of RUMnet models is dense among RUM models, so any RUM model can
be approximated as closely as wanted given enough samples and network size
[arxiv:2207.12877 §3]. Card: [Representing Random Utility Choice Models with Neural Networks](/papers/2207.12877).

### Neural special cases: TasteNet and DeepMNL

The paper treats these as special cases of RUMnet: with K = 0, or with a linear
utility [arxiv:2207.12877 §2.4]. On Expedia they come next to RUMnet on loss
(2.069 and 2.092) and accuracy (0.412 and 0.413) [arxiv:2207.12877 tab. 1].
No card yet.

### Model-free classifiers: random forest and vanilla neural network

These do not impose the RUM structure [arxiv:2207.12877 §1]. The paper tunes
the random forest over tree counts {50, 100, 200, 400} and depths {5, 10, 20}
on validation loss [arxiv:2207.12877 §6.2]. The random forest has the lowest
Swissmetro loss in the paper but the highest Expedia loss, and the paper ties
its Expedia overfitting to the larger input dimension [arxiv:2207.12877 tab. 1]
[§6.3] [§6.4]. Both are covered by the RUMnet card; neither has its own card.

## When to use which

The paper's results are author-reported, on two datasets (Swissmetro with about
10K observations and three alternatives; Expedia with about 398K search queries
and up to 39 alternatives) [arxiv:2207.12877 §6.2] [app. E]. Treat the rows
below as the conditions under which the paper's evidence applies.

| Situation | Use | Why |
|---|---|---|
| Choices may follow any random-utility model and the utility form is unknown | RUMnet | Approximation result for the RUM class [arxiv:2207.12877 §3]. The result is about approximation, not about the sample size needed |
| Small data (around 10^3 choice events), unobserved attributes independent | RUMnet over latent-class MNL | In the paper's synthetic setting 4, the log-likelihood gap was about 4% at T = 10^3 in RUMnet's favour; the two converged near T = 10^4 [arxiv:2207.12877 §5.2] [fig. 4] |
| Very small data (around 10^2 choice events) | Not established | In the same setting both models shrank to the MNL level [arxiv:2207.12877 §5.2] |
| Many alternatives and attributes (Expedia-like) | RUMnet | RUMnet had the best Expedia loss (2.018) and accuracy (0.425); the random forest overfit there [arxiv:2207.12877 tab. 1] [§6.3] |
| Score is Swissmetro log-likelihood only | Random forest | Lowest Swissmetro loss (0.523), against RUMnet 0.561 [arxiv:2207.12877 tab. 1] |
| Tight training budget | Random forest | The paper says the random forest trains about an order of magnitude faster than the neural models [arxiv:2207.12877 app. E] |
| Expect long training runs | Plan for hours | About 45 minutes for the largest RUMnet on Swissmetro and about 36 hours on Expedia, in the paper's runs [arxiv:2207.12877 app. E] |
| Unobserved product and customer attributes are correlated | Not established | The approximation and the synthetic gains assume independence of unobserved attributes [arxiv:2207.12877 §2.1] [§5.2] |
| Variable-size assortments | Either, with padding or masking | Neural models mask unavailable products in training; random forests need dummy products to fill fixed-size sets [arxiv:2207.12877 app. E] |
| Data may not come from a RUM model | Not established | The theory does not assume the data come from a RUM model [arxiv:2207.12877 §4.1]; this page cites no experiment on that case |

## Implementation sketch

Core of the RUMnet forward pass, one choice event. Each step is cited; the
paper's Gumbel-noise motivation for the softmax is not needed in code
[arxiv:2207.12877 §2.2].

```pseudocode
# Average softmax choice probabilities over K x K sample pairs  [§2.2] [§2.3]
# Input:  products (kappa, d_x), customer (d_z,), K product and customer nets
# Output: choice probabilities (kappa,), summing to 1

def rumnet_probs(products, customer, eps_nets, nu_nets, utility_net):
    # Start from zero probability for every alternative  [§2.3]
    total = zeros(products.shape[0])
    for eps_net in eps_nets:
        # Sampled unobserved product attributes  [§2.2]
        eps_sample = eps_net(products)
        for nu_net in nu_nets:
            # Sampled unobserved customer attributes  [§2.2]
            nu_sample = nu_net(customer)
            # Concatenate observed and sampled attributes per alternative  [§2.3]
            h = concat(products, eps_sample,
                       repeat(customer, products.shape[0]),
                       repeat(nu_sample, products.shape[0]))
            # Score each alternative under this sample pair  [§2.3]
            utilities = utility_net(h)
            # Softmax over alternatives replaces the argmax  [§2.2]
            total = total + softmax(utilities)
    # Average over the K^2 sample pairs  [§2.3]
    return total / (len(eps_nets) * len(nu_nets))
```

```pseudocode
# Training objective: mean negative log-likelihood of observed choices  [eq. 2]
# Input:  chosen indices (T,), per-event probabilities (T, kappa)
# Output: scalar loss

def choice_nll(chosen, probs):
    # Probability the model gave to each observed choice  [eq. 2]
    chosen_probs = probs[arange(T), chosen]
    # Average negative log  [eq. 2]
    return -mean(log(chosen_probs))
```

## Contradictions

- **Significance of the RUMnet vs DeepMNL gap.** The paper says the difference
  is statistically significant on both datasets [arxiv:2207.12877 §6.3]. Its
  footnote gives paired t-test p-values of 5.4e-2 on Swissmetro and 9.33e-10 on
  Expedia [arxiv:2207.12877 §6.3]. The Swissmetro value is above the common
  0.05 threshold. This page reports both as the paper gives them.
- **Training schedule: paper versus repository.** The paper's table gives 1,000
  epochs with patience 200 for Swissmetro and 100 epochs for Expedia
  [arxiv:2207.12877 app. E]. The authors' repository configs use different
  epoch counts on both datasets, and a different patience on Swissmetro. Which
  schedule produced the reported numbers is not stated.
- **Activation: theory versus experiments.** The theory uses ReLU networks
  [arxiv:2207.12877 §4.1]. The experiments use ELU [arxiv:2207.12877 §6.2]
  [app. E]. The approximation result is stated for ReLU, so the results on
  ELU are not covered by the same theorem.

## Papers

- [Representing Random Utility Choice Models with Neural Networks](/papers/2207.12877): RUMnet, a neural approximator of RUM choice models, with a universal-approximation result and benchmarks on Swissmetro and Expedia.