ModSec-Learn: Boosting ModSecurity with Machine Learning

In plain words

A web application firewall sits in front of a website and checks each incoming request for signs of an attack. The standard open-source one, ModSecurity, uses a large set of rules. Each rule matches one known attack pattern and carries a fixed severity score chosen by its authors. A request is blocked when the scores of the rules it matches add up past a limit. The scores are set by hand and are not tuned to the site being protected.

This paper keeps the rules but changes how they are weighted. It runs each training request through ModSecurity and records which rules fired. Then a standard machine-learning model learns how much each rule should count, using examples of harmless and malicious requests. The learned weights replace the fixed scores. The same training can also push some weights to exactly zero, which means that rule can be dropped.

The authors test this on SQL injection attacks. On their own test set, the learned models catch about 99% of the attacks at a 1% false alarm rate at the most thorough setting, while stock ModSecurity at that setting catches about 69%. These are the authors' numbers on their own data.

TL;DR

• ModSecurity blocks a request when the fixed severity points of the rules it matches add up past a threshold. The paper argues that these fixed points do not fit a given application §1 §2.
• ModSec-Learn turns each SQL-injection rule of the OWASP CRS into one binary feature and trains a linear SVM, a logistic regression or a random forest on those features §3.
• Learned weights replace the severity points. L1 regularization gives a sparse weight vector, so some rules can be dropped §4.4.
• Author-reported: at paranoia level 4, TPR at 1% FPR goes from 68.55% (vanilla ModSecurity) to 99.02-99.45% for the learned models tab. 1.
• Use it as a reference for learned rule weighting on a rule-based WAF. Check every number against the code notes below before relying on it.

Problem

ModSecurity with the OWASP Core Rule Set (CRS) decides by summing the severity points of matched rules and comparing the sum with a threshold §2. The authors list three shortcomings: the severities are heuristic and may not match the protected service; the rules only describe attacks, so legitimate traffic is not considered and false positives can be high; and rules can interfere or repeat each other §1.

The paranoia level (PL) enables nested groups of rules §2. The authors contrast learned rule selection with choosing a predefined PL §3 §4.4. Vanilla ModSecurity reaches 92.50% TPR at 1% FPR at PL1, 75.45% at PL2, and 68.55% at both PL3 and PL4 tab. 1 §4.2.

Method

ModSec-Learn has two parts: a feature extraction phase that encodes CRS rules as a vector, and a machine-learning model that learns how to combine those rules §3. The paper states the feature map as follows. Let D be the set of selected SQL-injection rules and d = |D|. A query z is mapped to x in {0,1}^d, where x_i = 1 when rule i fired on z and 0 otherwise §3. The paper has no numbered equations; the mapping is written inline in §3.

Models §3:

1. Linear SVM and logistic regression (LR), each with an L2 or an L1 penalty. Linear weights can be read as tuned severity scores, and with L1 the zero weights select a subset of rules instead of a predefined PL.
2. Random forest (RF), a non-linear model. The paper notes that a non-linear model is harder to put into ModSecurity itself.

Data §3: legitimate requests come from the open-appsec dataset (508,529 samples). Its SQLi part has 458 payloads, which the authors augmented from the HTTP Params dataset, a Kaggle SQLi dataset and SQLmap runs with tamper scripts, reaching 30,543 SQLi payloads. The evaluation uses 25,000 benign and 25,000 SQLi samples.

Split §4.1: the training set has 40,000 samples (20,000 per class). The test set has 10,000 samples (5,000 per class) with no overlap with the training set. Only the SQLi rules of the CRS are enabled. The CRS has 60 rules of this kind; 52 of them fire on the training samples, and the other 8 are discarded §4.1.

Setup §4.1: ModSecurity v3.0.10, CRS v4.0.0, pymodsecurity v0.1.0 (Python bindings), scikit-learn v1.4.0. The SVM and LR regularization strength C is set to 0.5 after testing five values. The paper states that other hyperparameters keep their defaults.

Reference pseudocode, written from the paper's text and checked against the code. It is not a copy of the code.

# ModSec-Learn: one learned weight per CRS SQLi rule  [§3]
# Input:  queries (strings), labels (1 = SQLi, 0 = benign), paranoia level pl
# Output: weights w (d,), bias b; detector score for a query

def featurise(query, pl):
    # Run the query through ModSecurity with the CRS SQLi rules at level pl;
    # the query is URL-encoded first  [§4.1] [code: src/models/modsec.py#L113]
    fired = modsecurity_rule_ids(quote_plus(query), pl)
    # Binary vector over a fixed, ordered list of d rule IDs; 1 = rule fired
    # [§3] [code: src/extractor.py#L89-L92]
    # [code: src/models/modsec.py#L257-L259]
    x = zeros(d)
    for rule_id in fired:
        x[position_of[rule_id]] = 1.0
    return x                                   # (d,), values in {0, 1}

# Build the training matrix from the labelled training queries  [§3] [§4.1]
X = stack([featurise(q, pl) for q in train_queries])   # (n, d)

# Linear SVM, one fit per penalty p in {l1, l2}; C = 0.5  [§4.1]
# [code: scripts/run_training.py#L60-L66]
svm = LinearSVC(C=0.5, penalty=p, class_weight="balanced",
                fit_intercept=False)
svm.fit(X, y)

# Logistic regression with the saga solver, same C and penalties  [§4.1]
# [code: scripts/run_training.py#L89-L96]
lr = LogisticRegression(C=0.5, penalty=p, class_weight="balanced",
                        solver="saga", max_iter=1000)
lr.fit(X, y)

# Detector: the learned weights replace the fixed severity points  [§3]
def detect(query, pl, w, b):
    x = featurise(query, pl)
    # A rule with weight 0 contributes nothing and can be dropped  [§4.4]
    return dot(w, x) + b

Stock ModSecurity baseline §2 §4.2:

# Baseline score: sum of severity points over the WAF's rule messages  [§2]
# [code: src/models/modsec.py#L191-L192] [code: src/models/modsec.py#L262]
def modsec_score(query, pl):
    total = 0.0
    for message in waf_rule_messages(query, pl):
        # CRITICAL=5, ERROR=4, WARNING=3, NOTICE=2  [§2]
        total += severity_points(message.severity)
    return total   # flag the request when total >= threshold  [§2]

Evaluation §4.2: for each PL, the authors sweep the decision threshold and plot ROC curves on the test set fig. 2. The headline metric is TPR at 1% FPR, the fraction of malicious requests detected when at most 1% of benign requests are flagged tab. 1. They chose 1% because it is a common value in the literature §4.2.

Interpretation §4.4: at PL4, the L1-regularized LR assigns weight 0 to 18 rules. The L2 model and ModSecurity use all rules §4.4. Some learned weights are negative, which the authors read as signs of legitimate behavior §4.4 fig. 3. fig. 1 shows the architecture: the ML model is trained on the 52 CRS rules as features, and sparse regularization can select rules in place of a PL fig. 1.

Hyperparameters

Results

Author-reported, on the authors' balanced test set of 10,000 samples §4.1. Setup: ModSecurity v3.0.10, CRS v4.0.0, SQLi rules only. The paper does not state the hardware. Baseline: vanilla ModSecurity at each PL.

TPR at 1% FPR, percent tab. 1:

Source: tab. 1. At PL1 all models match vanilla ModSecurity tab. 1.

Other results, also author-reported:

• At PL4 the authors report that the TPR at 1% FPR is 44.71% higher than vanilla ModSecurity for linear SVM, 44.93% higher for LR, and 45.07% higher for RF §4.3. See Pitfalls for how to read these figures.
• The authors say the learned models' detection rate is equal to or higher than vanilla ModSecurity's at every FPR, at every PL §4.3 fig. 2.
• With L1 regularization, LR reaches the same TPR as the L2 model with 18 rules at weight 0 at PL4 §4.4 tab. 1. The introduction also says 18 of the 60 CRS rules can be discarded §1.

Paper vs code

Code: https://github.com/pralab/modsec-learn, MIT license, at commit 10b8f53856fa896da2094ed3cf2ba2e703f37476. The repo is linked from the abstract and belongs to the authors' lab (PRALAB). Read scripts/run_training.py for training, src/models/modsec.py for the WAF wrapper and src/extractor.py for the features.

Differences:

• Feature count. The paper uses 52 features §4.1 fig. 1. The committed rule ID list data/crs_sqli_ids_4.0.0.json has 49 entries crs_sqli_ids_4.0.0.json. The committed weights file data/crs_sqli_weights_4.0.0.json has 52 keys crs_sqli_weights_4.0.0.json. Three of them (942361, 942522, 942560) are not in the ID list. The feature matrix is built from the ID list extractor.py, so the code's features have 49 columns. I did not load the committed joblib models, so I could not confirm which feature count they were trained on.
• Other hyperparameters. The paper says the other hyperparameters were left at their defaults §4.1. The code sets class_weight='balanced', fit_intercept=False and random_state=77 for the linear SVM run_training.py. It also sets max_iter=1000 for LR run_training.py. Those settings are not in the paper.
• Pre-processing. The code URL-encodes each payload with quote_plus before sending it to ModSecurity as a GET query modsec.py modsec.py. The paper does not mention this step.
• Rule files. The code loads REQUEST-901-INITIALIZATION.conf together with the SQLi file modsec.py. The paper says only SQLi rules are enabled §4.1 and does not mention the 901 file.
• Paranoia level. Each PL is set by a crs-setup-plN.conf file that sets tx.blocking_paranoia_level crs-setup-pl1.conf. This matches the paper's nested PL description §2.
• Severity mapping. The code maps ModSecurity's numeric severities 2 to 5 to scores 5 to 2 modsec.py. This matches the paper's list §2.
• Split. The build script samples 25,000 per class and makes an 80/20 split build_dataset.py. This gives 20,000 and 5,000 per class, as the paper states §4.1.
• ROC. The paper sweeps the threshold §4.2. The code uses the continuous ModSecurity score as the ROC input run_experiments.py modsec.py. The threshold of 5.0 affects only the binary status modsec.py.
• Table values. No script in the repo prints the TPR-at-1%-FPR table. src/utils/plotting.py computes the partial AUC at max_fpr=0.01 and discards the result plotting.py. The table cannot be reproduced from the repo as shipped.
• Sparsity. The "18 rules at weight 0" result §4.4 depends on the committed LR-L1 model at PL4. I did not load that model, so this claim is not checked.

Pitfalls

• Relative, not absolute, improvement. "More than 45%" in the introduction and conclusion §1 §6 is a relative gain. For RF at PL4, 99.45 against 68.55 is about 45% more, but the gap is about 31 percentage points.
• Text and table disagree for SVM. The text gives 44.71% higher for linear SVM at PL4 §4.3. The table values (99.02 against 68.55) give about 44.45% tab. 1. Use the table values, and treat the text figure as unreliable.
• "More than 30%" of rules. The abstract and conclusion §6 say more than 30% of CRS rules are discarded. The introduction says 18 of 60 §1, which is exactly 30%. Against the 52 active rules it is about 35%. The denominator is not stated.
• Balanced test set. The test set has 5,000 benign and 5,000 malicious samples §4.1. The 1% FPR is measured on that mix. The paper does not say how the class mix of production traffic compares, so the FPR here is not a direct estimate of false positives in production. The paper says the benign data is "based on real-world traffic" §3.
• Tuning data for C is not stated. The paper tested five values of C and found 0.5 best §4.1. It does not say whether the choice used the test set. If it did, the test numbers are optimistic.
• Unseen rule IDs crash extraction. The feature code calls list.index on the rule ID list extractor.py. A rule that fires on a new query but is missing from the list raises ValueError. It is not ignored. A new deployment needs the rule ID list rebuilt, or a guard in the code.
• Debug mode gives all-zero features. In RulesLogger.__call__, the elif branch that records rule IDs does not run when debug=True modsec.py. Features built in debug mode are all zero.
• Score accumulates per message. The baseline adds severity points for every rule message, but the feature records each rule ID once modsec.py. If one rule produces several messages, the baseline counts each of them, while the feature stays 1. I did not test this case.
• Column order matters. The weights map to the order of the ID list in the JSON file extractor.py. Keep that order when you port the model. The plot script sorts the IDs analyze_rules.py.
• Negative weights. Some learned weights are negative §4.4. Do not clip them to zero. Clipping changes the decision.
• Hidden cost. Each feature needs a full WAF run per query extractor.py. The README calls the ModSecurity build "tricky" README.md. The learned scoring is one dot product, but the feature extraction is the expensive part.
• Only SQLi tested. The paper says the methodology can be applied "as is" to other rule sets §6 and that the evaluation could be repeated on other WAFs §6, but it reports only SQLi results on ModSecurity.
• Dataset provenance. The data mixes open-appsec, HTTP Params, a Kaggle set and SQLmap output §3. The MIT license covers the code. I did not check the dataset's license, so do not redistribute the data from this card.

Lineage

• Builds on: ModSecurity v3.0.10 and the OWASP CRS v4.0.0 §4.1. The learned models use scikit-learn's linear SVM, LR (saga solver) and random forest §4.1. The paper cites the SVM, LR and RF methods and the saga solver §3 §4.1.
• Compared against: vanilla ModSecurity at PL1 to PL4 tab. 1.
• Related work named in §5 (no arXiv link checked): Tran et al. (ML with CRS rules), Nguyen et al. (hybrid ModSecurity and ML), Folini et al. (unsupervised anomaly detection with CRS), and Singh et al. and Sobola et al. (ModSecurity PL studies). The authors say they are the first to study regularization for rule selection §5.
• Superseded by: not stated.