Byzantine-Robust Aggregation for Securing Decentralized Federated Learning

In plain words

Federated learning trains one model across many devices without pooling their raw data. The usual setup has a central server that collects and averages the device models. This paper removes the server, so each device exchanges models only with its neighbours and averages what it hears. The risk is that a device sending bad models, by fault or on purpose, can pull its neighbours off course. The standard protections were designed for a server that sees every device.

WFAgg is a filter-then-average rule for this setting. Each device checks the models it received with three different tests: is the model far from the typical one, does it point in an unusual direction, and does it jump abruptly compared with that neighbour's earlier models. A model has to pass enough of these checks to count. The device then blends its own model with the average of the accepted ones, so no single bad neighbour takes over.

The authors test it on handwritten-digit classification with 20 devices, under attacks that add noise, flip signs or flip labels. Their results come from one small benchmark, so treat them as a first look rather than proof.

TL;DR

WFAgg is a Byzantine-robust aggregation rule for decentralized federated learning, where nodes exchange models with neighbours and no server exists §1. Each node runs three filters (distance to the coordinate-wise median, cosine to the median after norm clipping, and a temporal change test) and weights each neighbour model by the filters that accept it alg. 1. On MNIST with 20 nodes on an 8-regular graph, the authors report 94.8–95.8% benign accuracy with two malicious neighbours under seven attacks, against 87.1% for Multi-Krum under IPM-0.5 tab. 1. All results are author-reported, use IID data and one dataset, and the released code differs from the paper in several details (see Pitfalls).

Problem

Decentralized federated learning removes the single point of failure of a server §1. But most Byzantine-robust aggregators were designed for a server that sees all clients, and the authors ask whether they work equally well when each node only has its neighbours' models §1 §3.2. In a decentralized node the number of received models can be small and the share of malicious neighbours can be high, which weakens rules that rely on counting or on a central statistic §4.2. The setting assumes a static topology, that every neighbour model arrives in each round, and IID data §4.1 §5.1.

Method

Setting §4.1

1. Clients form an undirected graph. Each client trains on its own data, then sends its model to all neighbours (gossip-style, not to one random node).
2. Each client receives the models of its neighbours and runs the aggregation rule on them.
3. Rounds repeat from local training. The paper assumes every neighbour model arrives §4.1.

WFAgg alg. 1 §4.2

1. Run three filters on the same received set: WFAgg-D (distance) alg. 2 §4.3, WFAgg-C (cosine) alg. 3 §4.4 and WFAgg-T (temporal) alg. 4 §4.5. Each returns the subset of neighbour models it accepts.
2. Each neighbour model collects a vote from every filter that accepts it. The votes are τ₁ = 0.4 for D, τ₂ = 0.4 for C and τ₃ = 0.2 for T in the experiments §5.1.
3. Any model whose total vote is below the smallest sum of two τ values is set to zero. With these τ values that threshold is 0.6 alg. 1, lines 219–221.
4. Aggregate with WFAgg-E: the local model is blended with the normalised, weighted mean of the neighbour models, with smoothing factor α = 0.8 eq. 3 §5.1.

WFAgg-D alg. 2 §4.3

1. Take the coordinate-wise median of the neighbour models as a reference §4.3.
2. Compute each neighbour's squared Euclidean distance to the reference.
3. Keep the K − f − 1 neighbours with the smallest distances, where f is the estimated number of malicious nodes alg. 2.

WFAgg-C alg. 3 §4.4

1. Compute the reference as in WFAgg-D, and the median of the neighbour model norms.
2. Clip each neighbour model so its norm does not exceed that median norm alg. 3.
3. Compute each clipped model's cosine distance to the reference eq. 2.
4. Keep the K − f − 1 neighbours with the smallest cosine distance.

WFAgg-T alg. 4 §4.5

1. During the first T_th rounds, accept nothing alg. 4.
2. For each neighbour, keep its last model, and its history of squared distance and cosine change between consecutive models.
3. Compute the current squared distance and cosine change against the neighbour's previous model alg. 4.
4. Compute an exponentially weighted mean and standard deviation of the history over a window of W rounds alg. 4 §4.5.
5. Accept the neighbour when both current values lie within one standard deviation of their means alg. 4.

# WFAgg for one benign node  [alg. 1] [§4.2]
def wfagg(own, neighbours, f, window, transient, alpha):
    # Run the three filters on the same received set  [alg. 1, lines 203-207]
    accepted_d = wfagg_d(neighbours, f)
    accepted_c = wfagg_c(neighbours, f)
    accepted_t = wfagg_t(neighbours, window, transient)

    # Sum the filter votes per neighbour  [alg. 1, lines 209-218]
    weights = votes(neighbours, accepted_d, accepted_c, accepted_t)

    # Zero weights below the smallest sum of two tau values  [alg. 1, lines 219-221]
    # With tau = (0.4, 0.4, 0.2) that threshold is 0.6
    weights = zero_below(weights, threshold=min_pair_sum(TAUS))

    # Blend the local model with the weighted neighbour mean  [eq. 3] [alg. 1, line 224]
    return wfagg_e(own, neighbours, weights, alpha)

# WFAgg-E: smoothed blend of own model and neighbour mean  [eq. 3] [alg. 1, line 224]
def wfagg_e(own, neighbours, weights, alpha):
    # All-zero weights leave the local model unchanged
    # [code: src/machine_learning/aggregation/byzantine_robust_algorithms.py#L439-L440]
    total = sum(weights)
    if total == 0:
        return own
    # Normalised weighted mean of the neighbour models  [eq. 3]
    mean = sum(w / total * m for w, m in zip(weights, neighbours))
    # Smoothing: (1 - alpha) own + alpha mean  [eq. 3]
    return (1 - alpha) * own + alpha * mean

# WFAgg-D: distance to the coordinate-wise median  [alg. 2] [§4.3]
def wfagg_d(neighbours, f):
    # Reference: coordinate-wise median of the neighbour models  [alg. 2, line 256]
    reference = median(neighbours)
    # Squared Euclidean distance to the reference  [alg. 2, line 258]
    distances = [squared_norm(m - reference) for m in neighbours]
    # Keep K - f - 1 closest neighbours  [alg. 2, line 261]
    return smallest(neighbours, distances, len(neighbours) - f - 1)

# WFAgg-C: cosine distance after norm clipping  [alg. 3] [§4.4]
def wfagg_c(neighbours, f):
    reference = median(neighbours)
    # Clip each model's norm to the median norm  [alg. 3, lines 286-288]
    magnitude = median([norm(m) for m in neighbours])
    clipped = [m * min(1, magnitude / norm(m)) for m in neighbours]
    # Cosine distance to the reference  [alg. 3, lines 289-291] [eq. 2]
    distances = [1 - cosine(c, reference) for c in clipped]
    # Keep K - f - 1 closest neighbours  [alg. 3, line 291]
    return smallest(neighbours, distances, len(neighbours) - f - 1)

# WFAgg-T: temporal test against each neighbour's history  [alg. 4] [§4.5]
def wfagg_t(histories, window, transient, round_index):
    # Accept nothing during the transient rounds  [alg. 4, line 310]
    if round_index <= transient:
        return []
    accepted = []
    for neighbour, history in histories.items():
        # Change from the neighbour's previous model  [alg. 4, lines 314-315]
        distance = squared_norm(history.model - history.previous)
        cosine = 1 - cosine_similarity(history.model, history.previous)
        # Mean and spread over the last `window` values  [alg. 4, lines 312-313]
        mean_d, std_d = ewm_stats(history.distances[-window - 1:-1])
        mean_c, std_c = ewm_stats(history.cosines[-window - 1:-1])
        # Accept when both lie within one std of their means  [alg. 4, line 316]
        if abs(distance - mean_d) <= std_d and abs(cosine - mean_c) <= std_c:
            accepted.append(neighbour)
    return accepted

Key equations (numbers counted in order of appearance in the source §4.1–§5.3):

• Global FL objective, a sum of client losses weighted by data share eq. 1.
• Cosine distance α(x, y) = 1 − cos(x, y), in [0, 2] eq. 2.
• WFAgg-E aggregation: θ̄ᵢ = (1 − α) θᵢ + α Σⱼ w′ᵢⱼ θⱼ, with normalised weights w′ eq. 3.
• IPM attack vectors and their mean eq. 4 eq. 5.
• Accuracy (micro-averaged) eq. 6 and the R²-style convergence metric for benign models eq. 7.

Hyperparameters

Results

Author-reported. Setup §5: MNIST, 20 nodes, decentralized runs on an 8-regular graph with at most two malicious neighbours per node, CNN as above, 10 rounds, against centralized and decentralized baselines (Mean, Trimmed-Mean, Median, Krum, Multi-Krum, Clustering). Attacks: Noise, sign-flip, label-flip, IPM (ε = 0.5 and 100) and ALIE §5.2. Table 1 gives accuracy (%) on the test set; the decentralized columns are the mean over benign nodes with 0, 1 or 2 malicious neighbours. The caption does not say which model is scored in the centralized column tab. 1. The R² convergence results appear only as bar charts fig. 4 fig. 6, so no values are tabulated.

Full accuracy grid for all 12 rules. Values the paper colours red (well below the others) are marked *. Columns: centralized, then decentralized with 0, 1 and 2 malicious neighbours tab. 1.

Qualitative claims from the text, as reported by the authors §6.1–§6.2: in the centralized case Mean fails under Noise and IPM-100 (11.45 and 9.80), while Trimmed-Mean does not tab. 1. Krum and Multi-Krum resist Noise and IPM-100 but not IPM-0.5 in decentralized runs §6.1. The authors report that WFAgg mitigates all attacks, including the worst decentralized case with two malicious neighbours §6.2.2 fig. 7.

Paper vs code

Code: the official repository linked in the paper's footnote, https://gitlab.com/compromise3/decentralizedfedsim, commit 938a49307def89371feda02ba08787ec444f08e2 (latest on the default branch at compile time), licence GNU GPL v3.0 or later. The first author appears as the committer of the recent commits. The repository was read through GitLab's web view and API, not cloned; line numbers come from that view and may be off by a line. Because the licence is copyleft, no code is quoted here; everything is paraphrased.

Matches (checked against code):

• Topology: 20 nodes on a ring where each node links to 4 neighbours per side, degree 8 regular-8_decentralized_topology.json §5.1.
• Model: 2 conv layers, a pooling layer after each, 3 FC layers, ReLU MNIST.py §5.1.
• SGD with learning rate 0.01 and momentum 0.9 MNIST.py §5.1.
• Clustering: cosine distance, average-linkage agglomerative clustering into two groups, larger group kept byzantine_robust_algorithms.py §3.2.
• WFAgg-D and WFAgg-C follow the paper's steps, including norm clipping to the median norm in WFAgg-C byzantine_robust_algorithms.py alg. 2 alg. 3.
• WFAgg-E is the smoothing formula of eq. 3 byzantine_robust_algorithms.py.
• Zeroing threshold of 0.6 for τ = (0.4, 0.4, 0.2) byzantine_robust_algorithms.py alg. 1.
• Label-flip mapping ℓ → 9 − ℓ utils.py §5.2.

Differences:

• Noise scale. Paper: mean and variance both 0.1, "i.e." σ = 0.1 §5.2. Code passes 0.1 as the variance argument, which is converted to a standard deviation of √0.1 ≈ 0.316 utils.py utils.py.
• Label-flip scope. Paper: the labels of each training sample of Byzantine clients are flipped §5.2. Code: a random fraction of samples is flipped, with a default of 30%, and the change is written into the shared dataset object utils.py.
• Krum distance. Paper: sum of Euclidean distances to the K − M − 2 nearest models §3.2. Code: sum of squared Euclidean distances to max(1, n − f − 2) nearest models, where n counts neighbour models only byzantine_robust_algorithms.py.
• Trimmed-Mean. Paper: removes βK values from each end with β = 0.1 §3.2 §5.1. Code: removes int(β × n) values, where n is the number of models passed. In decentralized nodes the own model is appended for statistic rules, so n = 9 and nothing is removed byzantine_robust_algorithms.py decentralized_client.py.
• WFAgg-T window and history. Paper: mean and spread over a window of W = 3 rounds, using t−1 to t−W−1 alg. 4 §4.5. Code: no window parameter; it uses the two entries before the latest, with exponential smoothing at α = 0.01 and a ±1σ band byzantine_robust_algorithms.py. Per-neighbour history lists grow every round decentralized_client.py.
• Transient rounds. Paper: no model is accepted by WFAgg-T before T_th alg. 4. Code: for Alt-WFAgg the flag that controls whether first-round models are accepted is set to False, which matches the paper; the flag used by the main WFAgg call was not confirmed byzantine_robust_algorithms.py.
• Multi-Krum and Alt-WFAgg. Paper: m = K/4 §5.1 and Alt-WFAgg built from Multi-Krum and Clustering §6.2.2. Code: Multi-Krum's m comes from a runtime argument not in the repo, and Alt-WFAgg uses Multi-Krum with to_keep = n − 3 byzantine_robust_algorithms.py.
• Missing messages. Paper: all neighbour models are assumed to arrive §4.1. Code: it polls twice with a 5-second wait, then aggregates whatever arrived, with no padding decentralized_client.py.
• Own model in the rule. Paper: aggregation includes the own model with weight 1 − α eq. 3. Code: includes it for Mean, Median and Trimmed-Mean by appending it to the list, and passes it separately to the other rules decentralized_client.py.
• IPM reference set. Paper: the attacker uses the mean of benign models §5.2 eq. 4. Code: the mean of whatever models it is given, with ε from a runtime argument. Which models the attacker receives is not visible in the files read utils.py.
• Data partition. Paper: IID split across nodes §5.1. Code: the partition routine (in the utils module, not in the files read) was not located, so IID is not confirmed MNIST.py.
• Not in code. Batch size and seeds are not in the paper; the code sets batch size 32 for training. Rounds, Multi-Krum m, Trimmed-Mean β, α and the τ values are supplied at runtime, so the values used for the paper's table are not recorded in the repo main_sim_decentralized.py main_sim_decentralized.py.
• ALIE. Paper: values drawn from the range μ ± z_max·σ §5.2. Code: uniform draw in that range, with z_max from a runtime argument utils.py. Consistent with the wording.

R² and accuracy are not checked in code: the metric module was not read.

Pitfalls

• Trimmed-Mean removes nothing in the decentralized setting. With the paper's β = 0.1 and the code's int(β × n) rule, n = 8 or 9 gives zero trimmed values. The paper's Mean and Trimmed-Mean rows for decentralized nodes are nearly identical in Table 1, which fits this reading tab. 1. If you reproduce it, state whether the cut is floored, rounded or ceiled and how many values are removed per side; the paper does not say.
• Noise scale is ambiguous. The paper says σ = 0.1 §5.2; the code adds noise with standard deviation √0.1 utils.py. The table does not say which was used.
• WFAgg-T is under-specified. The paper gives no window extraction rule beyond "W rounds" and writes t−1 to t−W−1, which is W + 1 values, while the text says history should be non-consecutive and the formula compares with t−1 alg. 4 §4.5. The code uses a fixed two-entry slice byzantine_robust_algorithms.py. Pick one and document it.
• The two-filter rule depends on τ. "Accepted by two or more filters" follows from the 0.6 threshold only with τ = (0.4, 0.4, 0.2) alg. 1 §4.2. With τ = (0.5, 0.3, 0.2), a model accepted only by the distance filter gets 0.5, which is not below the threshold of 0.5, so it passes. Implement the threshold rule from Algorithm 1, not the verbal rule.
• All-zero weights are not defined in the paper. WFAgg-E on its own divides by zero (NaN). The combined rules return the local model instead byzantine_robust_algorithms.py byzantine_robust_algorithms.py.
• Two numbers disagree. The text gives 46.78% for Alt-WFAgg under Noise with two malicious neighbours §6.2.2. Table 1 gives 56.78% tab. 1. Use the table, and check the source before citing either.
• Krum scoring. Use squared distances, as the code does, if you want to reproduce the table. The paper's text says plain Euclidean distances §3.2.
• Missing details the table depends on. Batch size, weight decay, loss, normalisation, seeds, number of runs per cell and hardware are not stated. The code fills some of them (batch size 32, cross-entropy, no weight decay), but the values used for Table 1 are not recorded. Differences of a few tenths of a point in Table 1 carry little evidence without repeated runs.
• Memory grows with rounds. The paper says each node stores only the distance history and the last model per neighbour §4.5. The code appends to growing lists for every neighbour every round decentralized_client.py. Plan for bounded buffers if you deploy.
• Scope is narrow. One dataset (MNIST), IID data, 20 nodes, 10 rounds, one CNN, and the authors chose not to test non-IID data §5.1. The paper's own discussion attributes the high accuracies to the simplicity of MNIST §6.1.
• Cost. WFAgg runs three filters and a median per round. The paper's own analysis puts the filter costs in the same order as Krum, dominated by sorting for the median §4.3 §4.4. Scale tests before using it at larger d or K.
• Version. arXiv v2 is the accepted IEEE Access manuscript, not the version of record (DOI 10.1109/ACCESS.2025.3629864) p. 1. Check the published version before citing numbers.
• Code licence. GPL-3.0-or-later: copying code is not allowed here, so the pseudocode is our own and cites the paper's algorithms.

Lineage

• Builds on: FedAvg for the weighted average and the federated setting §2 §3.1; the centralized rules Krum and Multi-Krum, Clustering, Median and Trimmed-Mean §3.2; and the defence families of SignGuard, FLTrust and Siren, which the paper discusses but does not evaluate §3.3. No arXiv ids are given in this card.
• Compared against: Mean, Trimmed-Mean, Median, Krum, Multi-Krum and Clustering in centralized and decentralized settings §6.1 tab. 1.
• Superseded by: not known from content already in this wiki.