Local learning

Overview

Local learning gives each query its own model, fitted to the training samples near that query. It has three steps: select a few training samples in the neighbourhood of the test input, train a classifier on only those samples, and apply that classifier to the test input alone 1804.10892 §3.3.

The paper notes that k-nearest-neighbours fits this scheme as its simplest case, because its decision rule involves no fitting. It also notes that almost any other classifier can be used in the same place 1804.10892 §3.3. What every variant shares is that the fitted model answers one query and is then discarded. Because each query gets its own boundary, a linear base classifier produces a non-linear decision rule overall 1804.10892 §3.3.

The authors say that local learning had been used only with handcrafted features in their earlier work, and that their paper is the first to use it with deep features 1804.10892 §1. That earlier work is not in this wiki.

Variants

Only one card covers this concept, so the list has one entry.

Cosine neighbours with a per-query linear SVM

The paper fuses three CNN feature sets with a bag-of-visual-words vector, L2-normalises them, and selects 200 training neighbours by cosine similarity. It then trains a one-vs-all linear SVM on those neighbours and uses it for the one test image 1804.10892 §3.3 1804.10892 §4.2. Card: Local Learning with Deep and Handcrafted Features for Facial Expression Recognition.

Reported effect: on the same fused features, the local SVM beats the global SVM on 7 of the 8 accuracy columns in the paper's main table. The gaps range from 0.25 to 2.17 points. The one column where the global SVM wins is AffectNet 7-way without augmentation 1804.10892 tab. 1. The same authors report that the local SVM beats k-NN with the same 200 neighbours, 74.92% against 70.33% on FER 2013 without augmentation 1804.10892 §4.3.

When to use which

Implementation sketch

The generic form below takes the base learner as an argument. The paper's settings are given after it.

# Local learning at prediction time, generic form  [arxiv:1804.10892 §3.3]
# Inputs: train_x (N, D) features; train_y (N,) labels
#         query (D,); k neighbours; fit(): any base learner
def predict_local(train_x, train_y, query, k, fit):
    # Distance from the query to each training sample  [§3.3]
    dist = distance(train_x, query)          # (N,)
    # Keep the k samples nearest to the query  [§3.3]
    idx = argsort(dist)[:k]
    # Fit a fresh model on that subset only  [§3.3]
    model = fit(train_x[idx], train_y[idx])
    # The model answers this one query, then is discarded  [§3.3]
    return model.predict(query)

Paper's settings: cosine similarity, k = 200, one-vs-all linear SVM, LibSVM 1804.10892 §4.2. The paper sets C = 100 for combined models. The text does not say whether that value also covers the local SVM 1804.10892 §4.2. The paper tuned these values on the FER 2013 and FER+ validation sets and reused them on AffectNet without retuning 1804.10892 §4.2.

Contradictions

No cross-paper disagreement: one card so far. Within the paper, the description of feature normalisation differs. The text L2-normalises the deep and bag-of-words vectors separately before concatenation 1804.10892 §3.3. The caption of the pipeline figure places the L2 normalisation after concatenation 1804.10892 fig. 3. The two orders give different neighbour geometry, and the paper's code is not available to settle which one was used.

Papers

• Local Learning with Deep and Handcrafted Features for Facial Expression Recognition: fuses three CNN feature sets with bag-of-visual-words features, then trains a linear SVM per test image on its 200 nearest training neighbours.