Review of Swarm Intelligence-based Feature Selection Methods

In plain words

Many datasets describe each example with far more measurements than are useful. Feature selection keeps only the useful ones. Trying every possible subset is impossible for all but small inputs, so people use search methods borrowed from nature. Swarm methods imitate groups of animals or insects that search together, such as birds, ants, bees, wolves and whales.

This paper is a survey. It explains the main ways to pick features, then lists 11 swarm-based methods and the feature selection work built on each one. It also runs a small comparison: one example method per swarm family, tried on six datasets with three standard classifiers. For wrapper methods the authors' own runs favour particle swarm optimisation in most cases, but the test is narrow, so the ranking should be read as a starting point rather than a verdict.

TL;DR

• A review of 11 swarm-intelligence feature-selection families, each with a section and a summary table §3 tab. 2.
• Experiments run one wrapper per family (11) and four filter methods, 10 runs each, on six UCI datasets with SVM, Naive Bayes and AdaBoost §4 §4.3.
• Author-reported: PSO has the highest average wrapper accuracy on SVM (89.35 %) and AdaBoost (88.92 %); ACO has the highest filter average on SVM (79.52 %) fig. 1 fig. 3 fig. 4.
• No code and no hyperparameters for the methods are given. Use it as a map of the literature, not as an implementation guide.

Problem

Feature selection keeps a subset of input features so that a learner works better and faster §1. The search space for n features has 2^n subsets, so exhaustive search is impractical even for medium datasets §1 §3. The paper also notes that the training set should contain at least 10 × n × C examples for n features and C classes, which is why reducing the feature count helps when data are scarce §2.

Filter methods score each feature with a statistic and are cheap, but they usually treat features as independent and tend to reach lower-quality subsets §2.1 §5. Wrapper methods train a classifier for each candidate subset, which is more accurate but expensive, especially on high-dimensional data §2.2 §5.

Method

The paper describes four stages of feature selection: generate a subset, evaluate it, check a stopping criterion, and validate the result on test data §1.

Families of methods §2:

1. Filter: univariate scores such as information gain, gain ratio, mutual information and Fisher score, or multivariate scores such as mRMR §2.1.
2. Wrapper: a search proposes subsets, and a classifier scores each one §2.2.
3. Hybrid: combines a cheap filter with a wrapper §2.3.
4. Embedded: selection happens inside a learning algorithm §2.4.
5. Graph-based: uses similarity structure among features to rank or group them §2.5.

Swarm-intelligence methods reviewed §3: each subsection summarises the feature selection papers built on one algorithm:

• PSO, Particle swarm optimisation §3.1
• ACO, Ant colony optimisation §3.2
• ABC, Artificial bee colony §3.3
• DE, Differential evolution §3.4
• GSA, Gravitational search algorithm §3.5
• FA, Firefly algorithm §3.6
• BA, Bat algorithm §3.7
• COA, Cuckoo optimisation algorithm §3.8
• GWO, Grey wolf optimiser §3.9
• WOA, Whale optimisation algorithm §3.10
• SSA, Salp swarm algorithm §3.11

Table 2 classifies the cited works by SI method, number of objectives (single or multi), type (filter, wrapper, hybrid) and application domain tab. 2. Table 1 compares feature selection methods on objectives, type, search strategy, application, weakness and strength §2.5 tab. 1.

Comparative protocol §4. The paper evaluates one wrapper per swarm family (11 in total) and four filter methods, not the algorithms as such §4.3. The loop below follows the order in the text.

# Protocol for one dataset and one method  [§4] [§4.1]
# Input:  features X (patterns x n), labels y, selection method, classifier
# Output: per run: accuracy and number of selected features

def evaluate(X, y, method, classifier):
    runs = []
    for run in range(10):    # ten independent runs  [§4]
        # Max-min scale each feature to [0, 1]; the text does this
        # before the split  [§4.1]
        X_scaled = max_min_scale(X)
        # Random split: 66 % train, 34 % test, redrawn each run  [§4]
        X_train, y_train, X_test, y_test = split(X_scaled, y, 0.66)
        # Selection uses the training part only  [§4]
        subset = method(X_train, y_train, classifier)
        model = classifier.fit(X_train[:, subset], y_train)
        # Accuracy on the test part  [eq. 1]
        acc = accuracy(model, X_test[:, subset], y_test)
        runs.append((acc, len(subset)))
    # Report mean and standard deviation over the runs  [§4]
    return summarise(runs)

Accuracy is the usual fraction of correct predictions, ACC = (TP + TN) / (TP + TN + FP + FN) eq. 1. The paper reports the mean and standard deviation of ten runs §4.

Hyperparameters

Dataset characteristics tab. 3:

Results

Author-reported. Setup: six UCI datasets §4.1; SVM, Naive Bayes and AdaBoost §4.2; 11 wrapper methods (one per family) and 4 filter methods §4.3; 10 runs each.

The text states that PSO performs better than the other wrapper methods in most cases §4.4, and that among the filter methods ACO shows the best balance of exploration and exploitation §4.5 item 4. Those statements rest on the averages above, without any significance test.

Paper vs code

No official code. The paper links no repository; its only URLs are for the UCI repository and Weka §4.1 §4.2. No repository search was run, because this is a survey and our rule for surveys is to write "No official code."

Pitfalls

• One method per family. Each swarm family is represented by one wrapper and one filter method §4.3. The averages say little about the family itself. The selected methods are named in §4.3.
• No method details for reproduction. Population sizes, iteration counts, stopping rules, fitness functions and the classifier used inside each wrapper are not stated for any method §4. Reproduce from the original papers cited in §3, not from this one.
• Scale before split. The text normalises each dataset and then splits it §4 §4.1. If min and max come from all rows, the test part leaks into scaling. Whether they come from the training part only is not stated.
• Imputation is vague. Missing values are filled with the average of the available data for that feature §4.1. The exact rule (global, per class, or per run) is not stated.
• Small test sets. Colon has 62 patterns tab. 3. A 34 % test split leaves about 21 test patterns per run (computed from §4 and tab. 3), which is consistent with the standard deviations of roughly 1 to 4 points in tab. 4.
• No significance testing. Ranking is by mean accuracy only. Gaps such as 0.16 points between PSO and COA on Naive Bayes §4.4 fig. 2 fall well within the per-dataset standard deviations of tab. 4.
• Dataset label. Table 3 is titled "medical datasets" tab. 3, while §4.1 introduces the six datasets from the UCI repository without calling them medical.
• Software inconsistency. §4.2 names Weka for the classifiers and §4.3 says all methods are implemented in C#. The paper does not say which code ran the classifiers.
• Time ranking mismatch. §4.4 says ACO is second for filter run time §4.4. Table 8's averages give WOA 9.20 and ACO 9.50, so WOA is second tab. 8.
• Filter labelled as wrapper. §4.4 calls the DE-based method a "multi objective wrapper" §4.4. §4.3 and tab. 2 list it under filter methods.
• Discussion claim without table support. §4.5 says size-aware and multi-objective methods perform better §4.5 item 3. No table isolates that effect.
• Figure values are not tabulated. The averages in fig. 1 to fig. 4 are shown only as bars. This card uses only the values the text states §4.4.

Lineage

• Builds on: nothing stated; the paper positions itself as the missing review of SI-based feature selection §1.
• Compared against: Basir and Ahmad (2014), which the paper says focuses only on traditional SI algorithms and their feature selection methods, and Brezočnik, Fister and Podgorelec (2018), which the paper says lacks experimental results §1.
• Superseded by: not known from the content in this wiki.