---
kind: concept
slug: multi-criteria-decision-making
title: Multi-criteria decision making
summary: >-
  Learn a user's ranking of mined patterns as a weighted sum of interestingness measures, with weights from the Analytic Hierarchy Process (AHP) over pairwise agreement gaps. Passive mode learns from rankings; active mode asks pairwise questions. Trade-off: much faster than an SVM ranker, but less accurate in passive mode. Author-reported.
tags: [pattern-mining, interactive-learning, preference-learning]
papers: ["2610.16511"]
related: [active-learning, bradley-terry-model, reward-modeling]
status: draft
updated_at: 2026-10-24
---

## Overview

A pattern miner can return far more rules than a person can read, and which
ones matter depends on the person. Each rule gets scores from generic
interestingness measures, and no single measure matches a given user. Learning
a personal scoring function from feedback usually needs many examples and long
fits [arxiv:2610.16511 §1].

The concept treats the choice of measure weights as a multi-criteria decision.
The score is a weighted sum of the measures [arxiv:2610.16511 §4.1 eq. 4]. The
user ranks a few patterns, each measure gets an agreement score against that
ranking, the measures are compared pairwise, and AHP turns those comparisons
into weights [arxiv:2610.16511 §4.1 alg. 1].

The invariant shared by the variants is that the final score is a weighted sum
of measures scaled to [0, 1], and the weights are the principal eigenvector of
a reciprocal pairwise matrix [arxiv:2610.16511 eq. 6, alg. 1]. What differs is
where the rankings come from: a fixed set given up front, or questions chosen
one at a time.

## Variants

### Passive AHP ranking

The user supplies one or more rankings over small sets of patterns. The
weights are relearned for each ranking and the last result is returned
[arxiv:2610.16511 alg. 2] [arxiv:2610.16511 §4.3]. The experiments use a single
large ranking as input [arxiv:2610.16511 §4.4]. Card:
[Boosting the Learning for Ranking Patterns](/papers/2610.16511).

Reported results (association rules, five FIMI datasets, three simulated
users): the mean Spearman rho is 0.94, 0.83 and 0.79 for Rand-Emu, Lex-Emu and
Chi-Emu, and the mean CPU time is 38 to 41 s [arxiv:2610.16511 tab. 5]. The
SVM baseline finished only on Hepatitis, in 851 to 979 s, and it is more
accurate there on every emulator [arxiv:2610.16511 tab. 5] [arxiv:2610.16511
§6.6 B]. Per-dataset AHP times range from 9 to 84 s [arxiv:2610.16511 tab. 5].

### Active AHP ranking with sensitivity queries

Active mode starts from equal weights. Each round it draws a random sample of
patterns (theta = 10^3 in the experiments), picks the adjacent pair with the
smallest sensitivity, asks the user to rank that pair, and relearns the weights
[arxiv:2610.16511 alg. 2] [arxiv:2610.16511 §4.3.1] [arxiv:2610.16511 eq. 7].
The stopping rule is a custom criterion or a fixed number T of queries; the
experiments use T = 20 [arxiv:2610.16511 §4.3] [arxiv:2610.16511 §6.7 A].

Reported results: at 10 queries AHP beats the SVM ranker on Hepatitis,
Mushroom and Retail, and both are close on T40. On Connect, AHP stays at rho
0.10 for every budget, below SVM at 50 and 100 queries [arxiv:2610.16511
tab. 7]. With simulated answer errors of 0 to 40 percent on Chi-Emu and 20 queries,
AHP's rho falls from about 77 percent to about 70 percent, and SVM's falls
from about 71 to 54 percent [arxiv:2610.16511 §6.8]. AHP
latency is at most 0.02 s against 6 to 12 s for SVM [arxiv:2610.16511 §6.8].

## When to use which

| Situation | Use | Why |
|---|---|---|
| Many rules, a user can rank a small set, runtime matters | Passive AHP | AHP finishes in 9 to 84 s on every dataset and emulator; SVM times out beyond Hepatitis [arxiv:2610.16511 tab. 5] |
| A user can answer pairwise questions, answers may be wrong | Active AHP | Stable under simulated answer errors and low latency [arxiv:2610.16511 §6.8] |
| Passive mode, and accuracy matters more than runtime, as on Hepatitis | SVM ranker (not covered here) | The paper reports SVM more accurate on Hepatitis in passive mode [arxiv:2610.16511 §6.6 B] |
| Datasets like Connect in the paper's active comparison | Check first | AHP rho stays at 0.10 across budgets there [arxiv:2610.16511 tab. 7] |
| More than nine measures, or non-rule patterns | Not established | Complexity claim assumes at most 9 measures [arxiv:2610.16511 §4.4]; other pattern types are untested [arxiv:2610.16511 §7] |
| Real users whose preferences are not a weighted sum | Not established | Stated assumption [arxiv:2610.16511 §4.1]; all users are simulated [arxiv:2610.16511 §6.2] |
| Need code to start from | Write your own | No official code is linked and the named repository could not be cloned [arxiv:2610.16511 §6.4.1] |

## Implementation sketch

The common core is the weight learner. Both variants call it once per ranking.
The paper's Kendall-W-style agreement is used as printed in eq. 5, and the
gap-to-scale rule is not stated.

```pseudocode
# Agreement of each measure with a ranking  [arxiv:2610.16511 §4.1] [eq. 5]
# Input:  measures M (m items); ranking S (n patterns, best first)
# Output: agreement K[i] in [0, 1] per measure
def agreement(measures, ranking):
    # Sum measure rank and user rank per pattern, as in eq. 5
    ...
    return K

# Weight learner  [arxiv:2610.16511 alg. 1]
def learn_weights(measures, ranking):
    K = agreement(measures, ranking)
    A = ones(m, m)                              # reciprocal AHP matrix
    for i < j:
        # Gap between two measures  [arxiv:2610.16511 §4.1]
        delta = K[i] - K[j]
        # Map |delta| to the AHP 1..9 scale; rule not stated  [eq. 8]
        s = scale_to_ahp(abs(delta))
        # Larger value goes on the side of the better measure
        A[j, i], A[i, j] = (s, 1 / s) if delta < 0 else (1 / s, s)
    # Principal eigenvector, positive, normalised to sum 1  [eq. 1]
    w = principal_eigenvector(A)
    return w / w.sum()                          # printed weights sum to 0.99

# Score a pattern  [arxiv:2610.16511 eq. 6]
def score(pattern, w, measures):
    return sum(w[i] * scale01(measures[i](pattern)) for i in range(m))
```

Notes for implementers:

- The paper gives an AHP check (consistency ratio below 0.1) but neither
  algorithm computes it, and the random-index table is not given
  [arxiv:2610.16511 §3.2].
- Active mode picks pairs by the sensitivity of the predicted score
  [arxiv:2610.16511 §4.3.1] [eq. 7]; the sort inside it is not counted in the
  stated complexity [arxiv:2610.16511 §4.4].

## Contradictions

- **Averaging the gaps.** The text says the pairwise gaps are averaged over
  the user rankings, [arxiv:2610.16511 §4.2], but the algorithm sets the gap
  from the current ranking only [arxiv:2610.16511 alg. 1]. Implementers should
  pick one and test it on the paper's running example [arxiv:2610.16511 §5].
- **Running-example accuracy.** The text reports 91 percent ranking accuracy
  for the running example [arxiv:2610.16511 §5]. The table in that example
  marks seven of ten patterns as correct, which is 70 percent
  [arxiv:2610.16511 tab. 1].
- **Mean virtual best measure.** The text reports a mean VBM of 91 percent
  [arxiv:2610.16511 §6.6 A]. The 15 VBM cells in the correlation table
  [arxiv:2610.16511 tab. 4] average 0.88.
- **Connect in the active comparison.** The text reports AHP as better overall
  and does not discuss Connect [arxiv:2610.16511 §6.7 B]. The table shows AHP
  at 0.10 on Connect for all budgets, below SVM at 50 and 100 queries
  [arxiv:2610.16511 tab. 7]. The constant 0.10 may be a copy error.

## Papers

- [Boosting the Learning for Ranking Patterns](/papers/2610.16511): AHP-based
  weight learning over interestingness measures, with passive and active
  modes and a sensitivity query heuristic.