Threat analysis of IoT networks Using Artificial Neural Network Intrusion Detection System

In plain words

Internet-of-things gadgets such as sensors and smart devices talk to a server over a network, and an attacker can flood that server with junk traffic until it stops responding. An intrusion detection system watches the traffic and raises an alarm when the pattern looks like such an attack.

This paper builds that watcher with a small neural network. The network is fed numbers that describe the traffic and learns, from labelled examples, which patterns are normal and which are floods. The authors built a small test network of sensors and a server, ran normal traffic and flood attacks through it, and trained the network on the recorded traffic.

The reported accuracy is high, but the card has to be read with care: the paper leaves out several details an implementer would need, such as which measurements form the input, and the training settings. Its test set is also small and comes from one simulated setup.

TL;DR

• A three-layer sigmoid MLP (6 inputs, 3 hidden units, 1 output) is trained by gradient descent on labelled IoT traffic to separate normal from DDoS/DoS traffic §III fig. 1.
• Training data: 2313 samples; validation and test: 496 each §V tab. 2.
• Author-reported accuracy is 99.4% over all 3305 samples §V fig. 3.
• Relevant if you want a simple, supervised IDS baseline. Not usable as-is: the input features, optimiser settings and decision threshold are not stated.

Problem

Sensor nodes in an IoT network depend on a server that analyses and answers their data. A DoS or DDoS flood can make that server unresponsive. The sensors then cannot adapt, which can lead to a fault in the monitored system §IV. The authors want an early warning, so that a response team can act before the network is disrupted §IV §V. The paper treats the detector as an offline analysis of captured traffic §I.

Method

Network and data

The testbed has five sensor nodes: four act as clients and one as a server relay node. A network tap captures the traffic without changing it §IV fig. 2. The attacker targets only the server node §IV:

• DoS: one host sends more than 10 million packets.
• DDoS: up to three hosts, each sending more than 10 million packets at wire speed.
• The attack packets are UDP packets crafted by a custom C script §IV.

The samples are split into 2313 training, 496 validation and 496 test samples §V. Table II gives the class counts: 2121 DDoS/DoS (64.18%) and 1184 normal (35.82%) tab. 2. How a sample is built from the traces (its features and time window) is not stated.

Network architecture

A feed-forward network with one input layer of six units, one hidden layer of three units and one output unit, each unit with a bias. Every hidden and output neuron uses a unipolar sigmoid §III fig. 1. Let x_i be input i, w^{(1)}_{ij} the weight from input j to hidden unit i, and b^{(1)}_i the hidden bias. The forward pass is eq. 1-4:

a^{(2)}_i = f\left(\sum_{j=1}^{6} w^{(1)}_{ij} x_j + b^{(1)}_i\right), \quad i = 1,2,3 eq. 1-3

h_{w,b}(x) = f\left(\sum_{i=1}^{3} w^{(2)}_{1i} a^{(2)}_i + b^{(2)}_1\right) eq. 4

Here f is the sigmoid, and h_{w,b}(x) is a single real number. The paper says the network has a "stochastic learning algorithm with a mean square error function" §III.

Backward pass

The paper describes four steps: run the forward pass, compute the output error, compute the hidden errors, then compute the partial derivatives §III-B.

The output error uses half the squared error for one sample, differentiated with respect to the output's pre-activation z eq. 5:

\delta^{(3)}_1 = -(y - a^{(3)}_1)\, f'(z^{(3)}_1) eq. 5

Hidden errors pass the next layer's errors back through the weights and multiply by f' eq. 6:

\delta^{(l)}_i = \Big(\sum_j w^{(l)}_{ji}\, \delta^{(l+1)}_j\Big)\, f'(z^{(l)}_i) eq. 6

The bias derivative equals the layer's error term eq. 8:

\frac{\partial}{\partial b^{(l)}_i} J(w,b;x,y) = a^{(l)}_j\,\delta^{(l+1)}_i = \delta^{(l+1)}_i eq. 8

Training repeats gradient-descent steps that reduce J(w,b) §III-B.

# Forward pass, one example  [§III-A] [eq. 1-4]
# Input:  x (6,); weights w1 (3, 6), w2 (1, 3); biases b1 (3,), b2 (1,)
# Output: h, a score in (0, 1) for one example

def forward(x, w1, b1, w2, b2):
    # Hidden layer: three sigmoid units  [eq. 1-3]
    z_hidden = w1 @ x + b1                     # (3,)
    hidden = sigmoid(z_hidden)                 # (3,)
    # Output unit: sigmoid of a weighted sum of the hidden units  [eq. 4]
    z_out = w2 @ hidden + b2                   # (1,)
    out = sigmoid(z_out)                       # (1,)
    return z_hidden, hidden, z_out, out

# Backward pass, one example  [§III-B] [eq. 5-6] [eq. 8]
# Input:  y, the target for the example (encoding not stated, see Pitfalls)
# Output: gradients for w1, b1, w2, b2

def backward(x, y, z_hidden, hidden, z_out, out, w2):
    # Output error: -(y - a) * f'(z)  [eq. 5]
    delta_out = -(y - out) * sigmoid_prime(z_out)   # (1,)
    # Hidden errors: send the output error back through w2, times f'  [eq. 6]
    delta_hidden = (w2.T @ delta_out) * sigmoid_prime(z_hidden)   # (3,)
    # Bias gradients equal the error terms  [eq. 8]
    grad_b2 = delta_out                        # (1,)
    grad_b1 = delta_hidden                     # (3,)
    # Weight gradients: error times the activation feeding that weight
    # The paper's weight-derivative equation is not legible in the PDF text
    # layer (see Pitfalls); this follows the chain rule from eq. 5-6
    grad_w2 = delta_out * hidden               # (1, 3)
    grad_w1 = outer(delta_hidden, x)           # (3, 6)
    return grad_w1, grad_b1, grad_w2, grad_b2

# Gradient-descent update  [§III-B]
# The learning rate is not stated in the paper.
# Input:  params and grads as lists of arrays of the same shapes
# Output: updated params

def update(params, grads, learning_rate):
    # Move each parameter a small step against its gradient  [§III-B]
    return [p - learning_rate * g for p, g in zip(params, grads)]

Hyperparameters

Results

Author-reported. Setup: a simulated (abstract) five-node IoT network, one classifier, no baseline method §IV §V. Accuracies are as printed in the confusion-matrix figure; counts are read from its cells fig. 3.

The paper reports no precision, F1, ROC curve, false-alarm rate, latency or per-attack-type breakdown. The abstract's claim to detect "various DDoS/DoS attacks" is supported only by the combined DoS/DDoS class §V fig. 3.

Paper vs code

No official code. Checked: the paper has no code or project link; the arXiv abstract page lists no code, and its comments give only the venue (IEEE ISNCC 2016). A GitHub API search for the title returned HTTP 403 in this session, and a GitHub web search for the exact title returned zero repositories. The search for the author names was blocked. The result is therefore not a complete search.

Pitfalls

• Input features are not named. The network has six inputs labelled x_1 \dots x_6 fig. 1, and the text describes captured packet traces §I §IV. It never says which six values are extracted or how a sample is built from the traces. Without this, the model cannot be reproduced.
• Weight-matrix shape is inconsistent. The text gives w^{(1)} \in \mathbb{R}^{6\times 3} and w^{(2)} \in \mathbb{R}^{3\times 1} §III. The index convention in eq. 1-4 (w^{(1)}_{ij} with i the hidden unit and j the input) gives a 3 by 6 matrix. Use the index convention, and check shapes in code.
• Some equation text does not extract cleanly. Eq. 6 superscripts and summation limits are hard to read in the PDF text layer. The weight-derivative equation that should follow "the desired partial derivatives are calculated as" is not legible in the text layer; only eq. 8 (the bias derivative) is printed. Check the PDF before implementing eq. 6 and the weight gradient.
• Error-term order in the text. Section III-B says errors are computed "for l=2, l=3". eq. 5 is the output error and eq. 6 the hidden error; the text does not give the order in which to apply them. Apply eq. 5 first.
• Loss scale and aggregation. The cost is named "mean square error" and eq. 5 differentiates a half-squared error for one example. Whether the loss is summed or averaged over a batch is not stated. This changes the effective learning rate, which is also not stated.
• Label encoding and threshold. §III defines targets y \in \{1..C\}, but the network has one sigmoid output. The paper does not say how the two classes are encoded in y, nor the threshold that separates them. Table II and the confusion-matrix totals imply that the 1184 normal samples are one class and the 2121 attack samples the other, but not which label is which (inferred from counts, not stated).
• Confusion-matrix labels conflict with the text. §V calls true positives attacks "rightly classified" and false positives normal events "classified rightly". Those definitions do not match the usual meaning. Reading the cells of fig. 3: 20 attack samples were labelled normal (2121 − 2101), and no normal sample was labelled attack. Use the counts, not the text's TP and FP labels.
• Split procedure and leakage. The paper does not say whether the split is random, stratified or by capture run. If traffic from the same flood appears in both training and test sets, the accuracy overstates generalisation to new attacks. The validation set's role in training is not described.
• Simulated or testbed. The abstract says the IDS was "validated against a simulated IoT network". §IV describes a five-node setup with a network tap and a custom attack script, and does not say whether the nodes are virtual or physical. One network, one attack script, and one class of attack are represented, so the generalisation claim is untested.
• Class balance. Attacks are 64.18% of samples tab. 2. A classifier that always predicts attack would score 64.2%. The paper does not report that baseline, and it does not report precision or false-alarm rate, which the imbalance makes more informative.
• Offline, not real-time. §I calls the ANN an "offline IDS". §IV and §V say the early warning helps the response team. No detection latency, throughput or per-packet cost is measured.
• Background sections are not method. Section II (the HIDS/NIDS comparison in tab. 1, taken from the cited work [13]) and the list of detection techniques describe other approaches. They do not describe this paper's method.
• Version. This card follows arXiv v1 (2017-04-07), the version recorded by wikillm fetch. The arXiv comments give the venue as IEEE ISNCC 2016. The published version was not checked and may differ.
• Software and compute are not stated. No software, hardware, runtime or energy cost is given.

Lineage

• Builds on: earlier MLP and neural-network intrusion work cited in §III and §II (KDD99-based neural IDS work, MLP intrusion detection with SOM comparison). None of those papers has a card here, so their methods are not described.
• Compared against: no comparison method is reported in this paper.
• Superseded by: not stated. No later work is covered in this wiki.