Paper · 2023-02-27 · cs.LG
Towards Interpretable Federated Learning
Survey of interpretable federated learning. Proposes a taxonomy by training stage (client, sample and feature selection, model optimisation, contribution evaluation), stakeholder, threat model and privacy technique. Reviews methods, faithfulness and cost metrics, and six open directions. The paper reports no experiments.
Federated learningInterpretable machine learningClient selection in federated learningShapley valuefederated-learninginterpretabilitysurvey
AI-compiled by claude-haiku-5-5 on 2026-11-13, not yet reviewed by a human. Every claim cites its source; check it when a detail matters.
In plain words
Federated learning lets several parties train one model together without handing over their raw data. The resulting models are usually large neural networks, so it is hard to see why they make a given prediction, or which party's data shaped the result. This survey asks what "interpretable" should mean in that setting. It sorts existing work into a map: which stage of training is explained (choosing participants, choosing samples, choosing features, the model itself, or how much each party contributed), who needs the explanation, what an attacker might do, and which privacy tools are used. It also covers how such methods are tested and what they cost, and it ends with open problems.
TL;DR
This is a survey of interpretable federated learning (IFL): methods that let parties in a federated setup explain their models, data or contributions to each other. It proposes a taxonomy by training stage, stakeholder, threat model and privacy technique. The covered families are influence and importance scores for clients and samples, feature selection, interpretable models and robust aggregation, and Shapley-based contribution scores. The paper reports no experiments, so use it as a map of the field and a list of open problems.
Problem
FL models are often deep networks with millions of parameters, which makes them hard for stakeholders to inspect, and this may reduce trust and adoption §1. Local data is hidden by design, which makes explanations harder to produce than in centralised settings §1. The authors also cite the GDPR right to "meaningful explanations" of automated decisions as a reason for interpretability §1. Explanations must also respect privacy: the paper sets the privacy targets for each stakeholder §2.3.
Method
This is a survey. Its contributions are a taxonomy §2.4 fig. 1, a review of IFL approaches §3, a review of evaluation metrics §4 and a list of future directions §5. The paper does not propose or test an algorithm.
Definition §2.1. As the authors define it, a party is interpretable when it can make its model- and data-related conduct clear to the other parties in an FL protocol, using plain terms. They extend an earlier, narrower definition that covered only client selection in personalised horizontal FL and tree-based vertical FL §2.1.
Stakeholders §2.2. Direct stakeholders are the FL server and the FL clients. Indirect ones include researchers, developers, regulators, policymakers and civil society. Each needs different explanations: the server needs the reasons behind a client's data choices and predictions; clients need the basis of their selection or exclusion and of their reward; developers need failure conditions for debugging §2.2.
Privacy targets and threats §2.3. Explanations to the server must not expose or reveal a party's local data or its distribution. Explanations to clients must also protect labels under vertical FL. Threat models are semi-honest participants, who follow the protocol but try to infer others' private information, and malicious participants, who manipulate their local models §2.3. Techniques named are differential privacy, homomorphic encryption and secure multi-party computation §2.3.
Taxonomy §2.4 fig. 1. IFL approaches are grouped by the stage at which they operate: client selection, sample selection, feature selection, model optimisation and contribution evaluation. Each group is then labelled with stakeholders, threat models, privacy targets and techniques, and metrics §2.4. The taxonomy figure is described only in words here.
3.1 Interpretable client selection §3.1. The server picks clients with high-quality data for each round.
- Importance-based §3.1.1. Model-deviation methods treat large deviations of local updates from the global update as a sign of noisy samples. Krum picks the local model closest to the others, and Bulyan combines Krum with a trimmed mean for malicious participants §3.1.1. Loss-based methods sum a client's sample losses and select clients above a threshold: cheap, but not an accurate measure of importance §3.1.1. Gradient-norm methods sum per-sample gradient norms, which is expensive, so an upper bound on the norm is used as a trade-off §3.1.1.
- Influence-based §3.1.2. Retraining-based methods remove or perturb clients and measure the performance change. They are unstable when dataset sizes and distributions vary, and expensive §3.1.2. Influence-function methods avoid retraining; Fed-influence sums sample influence values per client but requires transmitting Hessians §3.1.2. Hessian-vector products reduce the costs to O(np) computation and O(pk) communication, with p the parameter count, n the total training samples and k the number of clients §3.1.2.
3.2 Interpretable sample selection §3.2. Clients' samples differ in usefulness, so sample-level selection can save computation and speed up convergence §3.2.
- Logic-based §3.2.1. Explanations connect activated concepts to show the reasoning. Two examples: a logical-reasoning aggregation weighted by data quality, and a logic-rule method that picks chain-of-thought prompts for multi-domain LLMs by solving a bilevel program with variational expectation maximisation §3.2.1.
- Importance-based §3.2.2. Loss-threshold selection prioritises samples with high loss, but cannot separate erroneous samples, which also have high loss §3.2.2. Gradient-norm-bound selection picks important clients under a budget, and each client picks its important samples. A threshold such as the median gradient norm filters outliers §3.2.2.
- Influence-based §3.2.3. Naive per-sample influence costs O(np^2 + p^3) computation and O(kp^2 + np) communication. Approximations use Hessian-vector products, costing O(np) and O(pk), or the sign of the influence value, which has large errors when noise is high §3.2.3. A joint client and sample method uses bilevel meta-learning to separate hard, useful samples from noisy ones, with a convergence analysis §3.2.3.
3.3 Interpretable feature selection §3.3. Noisy or redundant features harm the global model and increase transmission §3.3.
- Model-agnostic §3.3.1. Supervised scores use mutual information, Gini impurity or F-statistics. One method runs mutual-information selection on clients and aggregates scores on the server with Bayes' theorem §3.3.1. MPC protocols compute Gini and F-statistic scores privately, under a malicious model where fewer than half the participants may be corrupted §3.3.1. Unsupervised selection uses a one-class SVM for outlier features, then hierarchical clustering with a variable threshold for non-IID data §3.3.1.
- Model-specific §3.3.2. LASSO enforces an l1 constraint but is limited to linear models and shrinks weights. An l2 variant with an autoencoder needs post-training thresholds. FedSDG-FS uses a Gaussian stochastic dual gate with an l0 relaxation to select features while the model is trained §3.3.2.
3.4 Interpretable model optimisation §3.4.
- Inherently interpretable models §3.4.1. FedForest uses gradient-boosted decision trees under horizontal FL §3.4.1. A non-model-sharing framework needs a shared public anchor dataset §3.4.1. In vertical FL, anonymous features protect privacy but reduce interpretability; one method expresses a tree's prediction as the intersection of sub-model results, so feature meanings can be disclosed while decision paths stay hidden §3.4.1.
- Interpretable robust aggregation §3.4.2. Against semi-honest participants, incentive mechanisms weight high-quality updates. One combines reputation with contract theory but assumes the server knows data quality. Stackelberg incentive games work only for IID data §3.4.2. Against malicious participants, outlier clients receive lower aggregation weights, and the robustness of a Byzantine estimator is analysed with influence values §3.4.2.
3.5 Interpretable contribution evaluation §3.5. Client contributions can be measured for the global model, and feature contributions for individual predictions §3.5.
- Client contribution §3.5.1. Utility games include fair value, labour union and Shapley-value schemes. Gradient-based fair value depends on the order in which clients join, so Shapley-value approaches are preferred §3.5.1. Efficient Shapley estimation works in two ways: faster evaluation within a round (gradient-based and local-embedding-based Shapley), and fewer rounds through truncation and guided Monte Carlo sampling §3.5.1. With semi-honest participants, these methods assume the server holds the model and a public test set. With malicious participants, secure Shapley computation uses ciphertext multiplication or blockchain-based secure aggregation §3.5.1.
- Feature contribution §3.5.2. Shapley-based feature methods use gradient-based estimation, but assume the server knows clients' feature IDs and can turn features off, which violates client privacy in vertical FL §3.5.2. Attention-based methods visualise which inputs the global model attends to §3.5.2. Activation-based methods extract subsequences that highly activate neurons and build a temporal evolution graph for time-series classification §3.5.2.
4. Evaluation metrics §4.
- Effectiveness §4.1. Post-interpretation performance measures the accuracy change after adjusting the model with the explanation. The authors doubt that performance gains alone show good interpretability §4.1. Faithfulness is tested with leave-some-out retraining: remove the entities an explanation calls important, retrain, and expect a significant drop §4.1. The procedure is in pseudocode below.
- Efficiency §4.2. Computation cost is measured in elementary operations, time and memory. Communication cost is measured in bytes of transmitted parameters. The authors note that no existing work evaluates interpretation quality or privacy exposure against these costs §4.2.
# Leave-some-out faithfulness check [§4.1]
# Input: clients, importance per client, count k (not stated in paper)
# Output: change in performance after removal
def faithfulness_check(clients, importance, k):
# 1. Rank the clients by their importance values [§4.1]
ranked = sort_by_importance(clients, importance)
# 2. Take out the entities the explanation calls important [§4.1]
removed = ranked[:k]
# 3. Retrain the FL model without them [§4.1]
model_without = retrain_fl(remaining_clients(clients, removed))
# 4. A large drop in performance supports faithfulness [§4.1]
return performance(model_without) - performance(model_full)5. Future directions §5. The paper lists six open problems: interpretable model approximation by mimic learning, to keep accuracy while improving interpretability §5; hard-sample-aware, noise-robust IFL, since thresholds for noisy samples are rarely known §5; interpretability for federated LLMs, where the patterns of interest are combinatorial §5; interpretability under complex threat models beyond semi-honest participants §5; trade-offs between privacy and efficiency for resource-constrained devices §5; and interpretability evaluation metrics that also measure privacy exposure §5.
Hyperparameters
| Name | Value | Where | Notes |
|---|---|---|---|
| learning rate, rounds, number of clients, dataset | not stated | §1 | Survey; the paper runs no experiments |
| k (entities removed in faithfulness test) | not stated | §4.1 | Procedure described without a value |
Results
Author-reported: none. The paper proposes a taxonomy and reviews published work qualitatively §1 §2.4 §3. The LaTeX source contains no result tables or benchmark numbers. The complexity figures in §3.1.2 and §3.2.3 are analytic estimates given in the text, not measurements.
Paper vs code
No official code. The paper's text and bibliography contain no code or project links §1. A GitHub repository search for "interpretable federated learning" returned only unrelated third-party projects, none under the authors' names.
Pitfalls
- No search protocol. The paper does not state how the surveyed works were selected, which databases or search terms were used, or the time window §1. Treat the coverage as the authors' selection.
- Experiments claimed, none shown. The contribution list says the authors "empirically evaluate" IFL algorithms §1, but the source has no experiments or result tables. Treat comparisons as qualitative.
- Complexity labels. The communication term O(p^2 k) for Fed-influence is labelled "computing operations" §3.1.2. Check the units before relying on it. Section §3.2.3 uses n and p without restating them; their meaning follows §3.1.2 (p parameters, n total training samples).
- Scope of the definition. The interpretability definition is the authors' extension of an earlier narrower one, which covered only client selection §2.1. Read the taxonomy with that wider scope in mind.
- Assumptions the authors flag. Shapley methods for semi-honest settings assume the server has the model and a public test set §3.5.1. Feature Shapley methods assume knowledge of clients' feature IDs §3.5.2. Stackelberg incentives only work for IID data §3.4.2. The authors note most IFL works assume semi-honest participants §5.
- Priority claim. The claim that this is the first IFL survey is the authors' own, "to the best of our knowledge" §1.
- Figure. The taxonomy figure is referenced as fig. 1 but only described in words here; the source image is not reproduced.
Lineage
- Builds on: not applicable (survey). It reviews the methods of the works it cites §3. The paper names them by bibliography key, and the bibliography gives no arXiv ids here, so no card links are made.
- Compared against: the paper contrasts IFL with interpretable AI for centralised ML, which it calls more challenging because local data is hidden and resources are constrained §1.
- Superseded by: not known.
Source and license
Adapted from “Towards Interpretable Federated Learning” by Anran Li, Rui Liu, Ming Hu, Yuanyuan Chen, Shipeng Wang, Lizhen Cui, Han Yu (arXiv:2302.13473), used under CC BY 4.0. Changes: condensed and restructured for implementation, with tables, equations and algorithms taken from the paper and its code, each cited to its source. Not affiliated with or endorsed by the authors.