Multi-criteria decision making

Overview

A pattern miner can return far more rules than a person can read, and which ones matter depends on the person. Each rule gets scores from generic interestingness measures, and no single measure matches a given user. Learning a personal scoring function from feedback usually needs many examples and long fits 2610.16511 §1.

The concept treats the choice of measure weights as a multi-criteria decision. The score is a weighted sum of the measures 2610.16511 §4.1 eq. 4. The user ranks a few patterns, each measure gets an agreement score against that ranking, the measures are compared pairwise, and AHP turns those comparisons into weights 2610.16511 §4.1 alg. 1.

The invariant shared by the variants is that the final score is a weighted sum of measures scaled to [0, 1], and the weights are the principal eigenvector of a reciprocal pairwise matrix 2610.16511 eq. 6, alg. 1. What differs is where the rankings come from: a fixed set given up front, or questions chosen one at a time.

Variants

Passive AHP ranking

The user supplies one or more rankings over small sets of patterns. The weights are relearned for each ranking and the last result is returned 2610.16511 alg. 2 2610.16511 §4.3. The experiments use a single large ranking as input 2610.16511 §4.4. Card: Boosting the Learning for Ranking Patterns.

Reported results (association rules, five FIMI datasets, three simulated users): the mean Spearman rho is 0.94, 0.83 and 0.79 for Rand-Emu, Lex-Emu and Chi-Emu, and the mean CPU time is 38 to 41 s 2610.16511 tab. 5. The SVM baseline finished only on Hepatitis, in 851 to 979 s, and it is more accurate there on every emulator 2610.16511 tab. 5 [arxiv:2610.16511 §6.6 B]. Per-dataset AHP times range from 9 to 84 s 2610.16511 tab. 5.

Active AHP ranking with sensitivity queries

Active mode starts from equal weights. Each round it draws a random sample of patterns (theta = 10^3 in the experiments), picks the adjacent pair with the smallest sensitivity, asks the user to rank that pair, and relearns the weights 2610.16511 alg. 2 2610.16511 §4.3.1 2610.16511 eq. 7. The stopping rule is a custom criterion or a fixed number T of queries; the experiments use T = 20 2610.16511 §4.3 2610.16511 §6.7 A.

Reported results: at 10 queries AHP beats the SVM ranker on Hepatitis, Mushroom and Retail, and both are close on T40. On Connect, AHP stays at rho 0.10 for every budget, below SVM at 50 and 100 queries [arxiv:2610.16511 tab. 7]. With simulated answer errors of 0 to 40 percent on Chi-Emu and 20 queries, AHP's rho falls from about 77 percent to about 70 percent, and SVM's falls from about 71 to 54 percent 2610.16511 §6.8. AHP latency is at most 0.02 s against 6 to 12 s for SVM 2610.16511 §6.8.

When to use which

Implementation sketch

The common core is the weight learner. Both variants call it once per ranking. The paper's Kendall-W-style agreement is used as printed in eq. 5, and the gap-to-scale rule is not stated.

# Agreement of each measure with a ranking  [arxiv:2610.16511 §4.1] [eq. 5]
# Input:  measures M (m items); ranking S (n patterns, best first)
# Output: agreement K[i] in [0, 1] per measure
def agreement(measures, ranking):
    # Sum measure rank and user rank per pattern, as in eq. 5
    ...
    return K

# Weight learner  [arxiv:2610.16511 alg. 1]
def learn_weights(measures, ranking):
    K = agreement(measures, ranking)
    A = ones(m, m)                              # reciprocal AHP matrix
    for i < j:
        # Gap between two measures  [arxiv:2610.16511 §4.1]
        delta = K[i] - K[j]
        # Map |delta| to the AHP 1..9 scale; rule not stated  [eq. 8]
        s = scale_to_ahp(abs(delta))
        # Larger value goes on the side of the better measure
        A[j, i], A[i, j] = (s, 1 / s) if delta < 0 else (1 / s, s)
    # Principal eigenvector, positive, normalised to sum 1  [eq. 1]
    w = principal_eigenvector(A)
    return w / w.sum()                          # printed weights sum to 0.99

# Score a pattern  [arxiv:2610.16511 eq. 6]
def score(pattern, w, measures):
    return sum(w[i] * scale01(measures[i](pattern)) for i in range(m))

Notes for implementers:

• The paper gives an AHP check (consistency ratio below 0.1) but neither algorithm computes it, and the random-index table is not given 2610.16511 §3.2.
• Active mode picks pairs by the sensitivity of the predicted score 2610.16511 §4.3.1 eq. 7; the sort inside it is not counted in the stated complexity 2610.16511 §4.4.

Contradictions

• Averaging the gaps. The text says the pairwise gaps are averaged over the user rankings, 2610.16511 §4.2, but the algorithm sets the gap from the current ranking only 2610.16511 alg. 1. Implementers should pick one and test it on the paper's running example 2610.16511 §5.
• Running-example accuracy. The text reports 91 percent ranking accuracy for the running example 2610.16511 §5. The table in that example marks seven of ten patterns as correct, which is 70 percent 2610.16511 tab. 1.
• Mean virtual best measure. The text reports a mean VBM of 91 percent 2610.16511 §6.6 A. The 15 VBM cells in the correlation table 2610.16511 tab. 4 average 0.88.
• Connect in the active comparison. The text reports AHP as better overall and does not discuss Connect 2610.16511 §6.7 B. The table shows AHP at 0.10 on Connect for all budgets, below SVM at 50 and 100 queries 2610.16511 tab. 7. The constant 0.10 may be a copy error.

Papers

• Boosting the Learning for Ranking Patterns: AHP-based weight learning over interestingness measures, with passive and active modes and a sensitivity query heuristic.