Support vector machine

Overview

Ten of the sixteen cards train an SVM and use its output inside a larger system, or as the classifier itself. It gives a class label for a fixed-length vector 1604.03225 §5.3, a spam label for TF-IDF vectors 2405.11619 p. 8, or a weight per firewall rule 2406.13547 §3. It gives a decision label for an EHR order from ranked features 2605.04666 p. 3, and a label for a spiral drawing from image statistics 2609.07408 p. 11. In active learning it gives the decision values that pick the next pixels to label 2104.07784 §4.1 alg. 1. It gives the final label for a graph-based semi-supervised classifier 2604.26818 §3. It gives a grasp label from sensor statistics 2204.06343 §4.2.2 and a terrain label from IMU and force-torque statistics 2411.04700 §3.1. It also gives confidence scores that a later search combines 1608.01413 §4.1 §4.3 §4.4.

Two cards use the SVM as a point of comparison. The joint-angle card uses a support vector regressor as its only non-sequential baseline 2412.17991 §2.1.6. A benchmark treats the SVM as one of 14 models and asks how sensitive it is to feature scaling 2506.08274 §5.1. The other five cards do not run an SVM. A theory card treats it as a limit, not a component 1806.00468 §3. A survey lists it among text classifiers 1904.08067 §4.6 but runs no experiments 1904.08067 §1 §5. A review of class imbalance names SVM-based remedies and reports no SVM numbers 1305.1707 §2.2 §4. A review of disease diagnosis lists SVM among the most frequent algorithms in its 102 studies, and gives no SVM results of its own 2112.15538 §6 fig. 10. A review of remote sensing for mineral exploration describes SVMs for lithology, alteration and mineral mapping, and runs no experiments of its own 2103.07678 §5.2.2 §1.

The shared part of the trained cases is the training step: the features are fixed before the SVM sees them, and the SVM is fitted on labelled examples 1604.03225 §5.3 1608.01413 §4 2605.04666 p. 3 2204.06343 §4.2.2 2411.04700 §3.1. The active-learning card refits the SVM on the growing labelled set at each round [arxiv:2104.07784 alg. 1]. The graph-cut card is the exception on the label side: the SVM is fitted on labels that a graph assigns to unlabelled points, and only on those it is confident about 2604.26818 §3. What differs is how the SVM is set up. The facial expression card uses an RBF kernel from libsvm and tunes C and gamma by grid search 1604.03225 §5.3. The grasp card searches kernel, C and gamma over a grid, separately for each gripper, action and object set, and standardises the feature vector 2204.06343 §4.1 §4.2.2. The terrain card searches C, gamma and the kernel, and reports the best value per input set 2411.04700 §3.1 tab. 2 tab. 3 tab. 4. The arithmetic card uses the Illinois-SL package with its default settings and does not state the kernel, C or the regulariser 1608.01413 §4.4. The spam card uses a linear kernel and does not state C 2405.11619 p. 9. The firewall card fixes C at 0.5, chosen from five values, and compares an L1 with an L2 penalty 2406.13547 §4.1. The EHR card uses a linear SVM and does not state C, the library or the feature scaling 2605.04666 p. 3. The image card runs the SVM with library default settings and gives no parameter values 2609.07408 p. 11. The joint-angle card gives one fixed setting for its regressor, gamma = 0.01 and C = 1, and describes no search 2412.17991 §2.1.6.

The theory card is different in kind. It is a theory paper with no experiments. It does not use an SVM as a component. It describes which margin solution gradient descent reaches on a linear network, and for fully connected networks that solution is the hard-margin SVM direction 1806.00468 §3. The result rests on assumptions the paper states but does not prove 1806.00468 §3.

Seventeen cards are compiled so far, so this page covers the uses they describe and nothing more.

Variants

RBF SVM on boosted features

Card: Geometric Feature-Based Facial Expression Recognition in Image Sequences Using Multi-Class AdaBoost and Support Vector Machines. AdaBoost first selects features. For each selected feature the paper keeps the maximum displacement over the sequence (type one) or the maximum change in angle and distance (type two), so each video gives one vector with two values per feature 1604.03225 §5.3. An RBF SVM is trained on these vectors. With 400 selected features (800 dimensions) it reports 97.35% average accuracy under 5-fold cross-validation on CK+ 1604.03225 tab. 6. With 200 features it reports 95.50% and with 100 features 93.20%; the AdaBoost-DTW classifier itself reports 95.17% 1604.03225 tab. 3 tab. 4 tab. 5. So the SVM beats that classifier only at 200 and 400 features.

Binary and multiclass SVM scorers in constrained inference

Card: Solving General Arithmetic Word Problems. A binary SVM scores whether each quantity is needed for the solution. A multiclass SVM scores which operation sits at the lowest common ancestor of each quantity pair 1608.01413 §4.3 §4.4. A beam search of width 200 combines these scores under a positive-answer and, for "how many" questions, an integer-answer constraint 1608.01413 §4.1. The relevance SVM labels 94.7% of AI2 quantities correctly in the relaxed sense, and 89.1% of problems fully correctly 1608.01413 tab. 5. The authors report 72.0% on AI2 with all constraints and 78.0% with the positive constraint only [arxiv:1608.01413 tab. 6], so the constraint setting changes the headline number.

Linear SVC on TF-IDF text

Card: Novel Interpretable and Robust Web-based AI Platform for Phishing Email Detection. The paper merges six public spam collections into 82,486 emails and turns each email into TF-IDF weights 2405.11619 p. 7 eq. 1. A linear-kernel SVC is trained on an 80/20 split, with random state 42 2405.11619 p. 8 p. 9. Its chosen model reports accuracy 0.991 and F1 0.99 on the 16,498-email test split 2405.11619 tab. 2. The same table gives the SVC on Word2Vec features an F1 of 0.71 and 0.82 2405.11619 tab. 2, so the choice of features moved the score more than the choice of classifier did.

Two cautions come from the card. The 0.991 is the best of eleven rows, all scored on the same test split, and the paper does not say whether any choice was made on a validation split 2405.11619 tab. 2 p. 8. The labels are spam and ham, not phishing, so the score does not show phishing detection 2405.11619 p. 11.

Linear SVM on rule-firing features

Card: ModSec-Learn: Boosting ModSecurity with Machine Learning. Each SQL-injection rule of the OWASP Core Rule Set becomes one binary feature, which is 1 when the rule fires on a query 2406.13547 §3. A linear SVM with an L1 or an L2 penalty learns one weight per rule, and the learned weights replace the fixed severity points of ModSecurity 2406.13547 §3. The training set has 40,000 samples and the test set 10,000, balanced between classes 2406.13547 §4.1. Author-reported TPR at 1% FPR at paranoia level 4 is 99.02 for the SVM and 68.55 for vanilla ModSecurity 2406.13547 tab. 1.

The L1 and L2 SVM rows in that table are identical, so the penalty shows up only as sparsity in the weights. The 18 zero-weight rules are reported for logistic regression with L1, not for the SVM 2406.13547 §4.4. The paper states that the other hyperparameters keep their defaults 2406.13547 §4.1.

Linear SVM on top-ranked features

Card: Feature importance analysis for patient management decisions. Each candidate temporal feature gets a single-feature AUC, computed on 2,900 patients. The top 1, 3 or 30 features by that AUC train a linear SVM, which is tested on the remaining patients 2605.04666 p. 3 p. 5. The feature set has 9,223 candidates, but the SVM tables cover only 10 lab orders and 10 medications 2605.04666 p. 3 tab. 1 tab. 2. On those 20 decisions the held-out AUC is 69.6–87.7% with the top feature and 80.5–89.8% with the top 30 2605.04666 tab. 1 tab. 2. The paper reports no external baseline 2605.04666 tab. 1 tab. 2. Its ranking is greedy, and the authors say this may not be the best choice 2605.04666 p. 5.

SVM with default settings on image statistics

Card: Impact of canny edge detection preprocessing on performance of machine learning models for Parkinson's disease classification. Each spiral image is reduced to eleven statistics. Recursive feature elimination keeps, for each model, the features ranked 2 or lower 2609.07408 p. 9 p. 10. The SVM is one of nine classifiers run with library default settings 2609.07408 p. 11. The feature selection runs on the whole training set before cross-validation, so the reported CV scores may be optimistic 2609.07408 alg. 3 p. 22. Its AUC is 0.85 on the original training images and 0.83 after Canny preprocessing 2609.07408 fig. 12 fig. 13. On the augmented set DS2 it is 0.94 2609.07408 fig. 13. Its model size grows from 7 KB on DS0 to 113–304 KB on DS2 2609.07408 fig. 40, and its prediction time on DS2 is 9,000–30,000 ms, the highest of the nine models there 2609.07408 fig. 41. The paper does not define how size or time were measured 2609.07408 p. 19 p. 20.

Grid-searched SVM on grasp-sensor statistics

Card: Single-grasp Deformable Object Discrimination: the Effect of Gripper Morphology, Sensing Modalities, and Action Parameters. Each squeeze gives a time series from the gripper sensors. The paper computes per-channel statistics, among them skewness, maximum, kurtosis, median, standard deviation and sum, and standardises the resulting vector 2204.06343 §4.1. The SVM is tuned over kernel, C and gamma, separately for each gripper, action and object set 2204.06343 §4.2.2. The stated C range is 1e-5 to 1e-3 2204.06343 §4.2.2.

Author-reported accuracy depends on the gripper and the set. On the objects set with the qb SoftHand, the best SVM reaches about 86% at the slow speed 2204.06343 §5.1. On 20 polyurethane foams with the Barrett Hand it reaches 94–99% across actions, with the SVM and the LSTM 2204.06343 §5.2 fig. 5. On the same foams with the Robotiq 2F-85 it reaches 68–77% across speeds 2204.06343 §5.2. Transfer across finger configurations fails for the qb SoftHand and the Barrett Hand 2204.06343 §5.4 fig. 6. The paper calls the SVM much less demanding than the LSTM, but gives no timings or sizes 2204.06343 §7.

Terrain classifier on force-torque and IMU statistics

Card: Field Assessment of Force Torque Sensors for Planetary Rover Navigation. Each force-torque sensor (FTS) contributes 35 inputs, from six channels and a lever estimate, each with five statistics. Each IMU contributes 15 accelerations with the same statistics, so the full input has 225 values 2411.04700 §3. The SVM is a support vector classifier, tuned by grid search over C, gamma and the kernel 2411.04700 §3.1. The grid covers C in {0.1, 1, 10, 100}, gamma in {1, 0.1, 0.01, 0.001}, and the linear, RBF, polynomial and sigmoid kernels 2411.04700 §3.1. The data are split 75/25 into training and test sets 2411.04700 §3.1 tab. 2.

The best test accuracies are 85.84% with IMU only (polynomial kernel), 95.58% with FTS only (linear kernel) and 95.87% with both (linear kernel) 2411.04700 tab. 2 tab. 3 tab. 4. The FTS-only and combined SVMs reach 100% training accuracy, so the test figures are the ones to read 2411.04700 tab. 3 tab. 4. The paper does not state whether the SVM inputs were scaled, and the code scales only when a flag is passed 2411.04700 §3.1 svm.py. Timing on one laptop CPU gives 24.3 s for the full grid search, 0.13 s for one fit with the best parameters, and 7.6 ms to predict the 339-sample test set 2411.04700 tab. 8. The field test uses one rover, a half-scale model of the ExoMars rover, on four terrain classes 2411.04700 §1 §3.1.

Support vector regressor as a baseline for joint angles

Card: Online Adaptation for Myographic Control of Natural Dexterous Hand and Finger Movements. The paper regresses the wrist and finger angles from forearm sensor signals. Its baseline is a support vector regressor (SVR) with a Gaussian radial basis kernel, gamma = 0.01 and C = 1, on a single 200 ms feature window 2412.17991 §2.1.6 tab. 1. It is the only non-sequential baseline 2412.17991 §2.1.6. On the standard session with nine subjects it reaches 11.3° angular RMSE and r² 0.72. The temporal convolutional and LSTM models reach 8.4° and 8.0° with r² 0.85 2412.17991 §3.1. On the free-form offline session with eight subjects the SVR reaches 15.4°, against 10.4° and 9.9° for the two sequence models 2412.17991 §3.2. The card gives the SVR a response delay of 120 ms on average, and under 25 ms for the sequence models 2412.17991 §3.2 fig. 6.

RBF SVM as the query model in active learning

Card: A survey of active learning algorithms for supervised remote sensing image classification. The loop trains a classifier on a small labelled set of pixels, scores the unlabelled pool, and sends the q best pixels to a user for labelling. Each round then repeats 2104.07784 alg. 1. With an RBF-kernel SVM, the paper tunes the kernel and penalty by 5-fold cross-validation, but the grid and the chosen values are not stated 2104.07784 §6.2. Margin sampling takes, for a one-against-all SVM, the smallest absolute decision value over classes, which is the pixel closest to a boundary 2104.07784 §4.1. The paper reports margin sampling as generally the best of the SVM heuristics it tested, against random sampling, on three datasets 2104.07784 fig. 4 §7. Adding a diversity term to MCLU seems to help slightly, except on AVIRIS Indian Pines 2104.07784 fig. 6 §7. The results appear only as curves, so no numbers are given in the text 2104.07784 §7. The paper also says that KL-max needs one retrained model per candidate and is hard to use with SVMs 2104.07784 §5.1.

When to use which

Implementation sketch

Three pipelines are sketched. The first is the RBF pipeline after feature selection. The grid and scaling are open choices the paper leaves unstated. The second is the linear pipeline on sparse text. The third is one round of margin sampling with an RBF SVM. The arithmetic pipeline needs its scorers and beam search, and the theory card has no algorithm to implement.

# RBF pipeline. Per video: maximum signed value over frames, per selected feature  [§5.3]
x = max_over_frames(selected_feature_values)  # length 2 * M for M features
# Grid search over C and gamma; ranges not stated  [§5.3]
best = grid_search(C_range, gamma_range, cross_validate)
# Scaling before the kernel: not stated  [§5.3]
# Train an RBF-kernel SVM with the chosen settings  [§5.3]
model = train_svm(X_train, y_train, kernel="rbf", C=best.C, gamma=best.gamma)
# One label per video  [§5.3]
label = model.predict(x)

# Linear pipeline on TF-IDF text  [arxiv:2405.11619 p. 7-9]
# Fit the vocabulary and IDF on training text only; the paper does not say it did
vectoriser = fit_tfidf(train_texts)
X_train = vectoriser.transform(train_texts)
X_test = vectoriser.transform(test_texts)
# Linear kernel; C is not stated, so choose and report it  [arxiv:2405.11619 p. 9]
model = fit_linear_svc(X_train, y_train, C=C, random_state=42)
# Label 1 is spam, 0 is ham  [arxiv:2405.11619 p. 11]
label = model.predict(X_test)

# One round of margin sampling with an RBF SVM  [arxiv:2104.07784 §4.1] [alg. 1]
def margin_sampling_round(X_lab, y_lab, X_pool, q):
    # Retrain on the labelled pixels; C and gamma not stated  [§6.2]
    model = train_svm(X_lab, y_lab, kernel="rbf")
    # One-against-all decision values per pool pixel  [§4.1]
    scores = model.decision_function(X_pool)  # pool x classes
    # Smallest absolute value over classes: closest to a boundary  [§4.1]
    margin = abs(scores).min(axis=1)  # pool
    # Send the q smallest margins to the user for labelling  [alg. 1]
    return argsort(margin)[:q]

Contradictions

Kernel of the benchmark's SVM. The benchmark's text says the SVM uses an RBF kernel 2506.08274 §3.2. Its card records that the released classification code uses a linear kernel and that the RBF kernel is used only for regression (Impact of Feature Scaling, Paper vs code). Which kernel produced the reported SVM accuracy is therefore unresolved, and this page does not settle it.

SVM on augmented images. The image card's conclusion says the SVM is among the models that "excelled on augmented datasets" 2609.07408 p. 35. Its median DS2 accuracy for the SVM is 0.50 2609.07408 fig. 42, while the pooled accuracy from the DS2 confusion matrix is 0.86 2609.07408 fig. 16. The card computed the pooled value itself, and the paper does not explain the gap. The DS2 AUC of 0.94 2609.07408 fig. 13 is a different measure and does not resolve it.

Incompatible C ranges. The grasp card searches C from 1e-5 to 1e-3 2204.06343 §4.2.2. The terrain card selects C = 0.1 from {0.1, 1, 10, 100} 2411.04700 tab. 2. The two grids do not overlap, and the papers use different sensors, features and scaling. So the C values are not transferable between these tasks, and this page does not say which range is right for a given problem.

Apart from these, the cards use SVMs for different purposes, and their numbers are not comparable. The classification cards report accuracy, AUC or F1 on different tasks and datasets. The firewall card reports TPR at a fixed FPR. The joint-angle card reports angular RMSE and r² for a regressor. The theory card, the survey and the three reviews report no numbers for the SVM.

Kernel SVM and dimension. The mineral review says the SVM projects data into a space of smaller dimension 2103.07678 §5.2.2. The card for that paper notes that kernel SVMs usually map to a higher-dimensional space, and the paper gives no kernel setting. The page does not settle which description applies to the SVMs the reviewed studies used.

The survey's pitfalls section reads a comparative remark about SVM efficiency in its section 6 as a summary of cited work, not as a result of the survey 1904.08067 §6. This page does not use that remark.

Papers

• Geometric Feature-Based Facial Expression Recognition in Image Sequences Using Multi-Class AdaBoost and Support Vector Machines: RBF SVM on AdaBoost-selected landmark-motion features, 97.35% author-reported on CK+.
• Solving General Arithmetic Word Problems: binary and multiclass SVM scorers inside a constrained beam search over expression trees.
• Text Classification Algorithms: A Survey: survey that lists SVM among text classifiers with a string-subsequence kernel; no experiments, no SVM numbers.
• Implicit Bias of Gradient Descent on Linear Convolutional Networks: theory that gradient descent on fully connected linear networks converges in direction to the hard-margin SVM, under stated assumptions.
• Single-grasp Deformable Object Discrimination: the Effect of Gripper Morphology, Sensing Modalities, and Action Parameters: grid-searched SVM on grasp-sensor statistics; author-reported 94–99% on Barrett Hand foams, level with an LSTM.
• Novel Interpretable and Robust Web-based AI Platform for Phishing Email Detection: linear SVC on TF-IDF features for spam email, 0.991 accuracy author-reported on one split.
• ModSec-Learn: Boosting ModSecurity with Machine Learning: linear SVM and logistic regression over binary rule-firing features, learned weights in place of ModSecurity severity points.
• Field Assessment of Force Torque Sensors for Planetary Rover Navigation: SVM on FTS and IMU statistics for terrain classes, 95.87% author-reported test accuracy with both sensors.
• Online Adaptation for Myographic Control of Natural Dexterous Hand and Finger Movements: SVR with a Gaussian kernel as the only non-sequential baseline for joint-angle regression; 11.3° against 8.0–8.4° RMSE on the standard session.
• The Impact of Feature Scaling In Machine Learning: Effects on Regression and Classification Tasks: benchmark in which the SVM's accuracy is significantly sensitive to the scaler; the SVM kernel is disputed between text and code.
• Feature importance analysis for patient management decisions: linear SVM on the top 1, 3 or 30 temporal EHR features ranked by single-feature AUC; 69.6–89.8% author-reported AUC on 20 decisions.
• Semi-supervised learning with max-margin graph cuts: graph-inferred labels, then a kernel hinge-loss SVM on confident points; author-reported lower error than supervised SVM and manifold regularization on UCI.
• Impact of canny edge detection preprocessing on performance of machine learning models for Parkinson's disease classification: SVM with library defaults on eleven image statistics; AUC 0.94 on augmented images, with the accuracy figures in conflict.
• Class Imbalance Problem in Data Mining Review: review of class-imbalance remedies that names SVM-based methods; no experiments and no numbers.
• A survey of active learning algorithms for supervised remote sensing image classification: RBF SVM as the query model; margin sampling generally best in author-reported curves, no tabulated numbers.
• Machine learning based disease diagnosis: A comprehensive review: review of 102 studies in which SVM is among the most frequent algorithms; no experiments.
• A review of machine learning in processing remote sensing data for mineral exploration: review that describes SVM for lithology, alteration and mineral mapping; no experiments, no SVM numbers.