Vertical Federated Learning: Concepts, Advances and Challenges

In plain words

Imagine a bank and an online shop that serve many of the same customers. The bank knows each customer's money habits, the shop knows what they buy, and neither may hand its records to the other. Vertical federated learning is a way for such parties to train one shared model anyway, by exchanging only partial results about the same people rather than their raw data.

This paper is a long review of that idea. It explains how the parties line up their shared customers without revealing the rest, how they pass intermediate numbers back and forth during training, and what an outside or dishonest party could still learn from those numbers. It also surveys ways to make the process faster, more accurate, more private and fairer, and lists real projects that use it.

Its own contribution is a checklist-style framework, called VFLow, that asks designers to balance privacy, cost and fairness together rather than optimise accuracy alone.

TL;DR

• A survey of vertical federated learning (VFL), in which parties share the same users but hold different features §2.1.
• Covers the basic training protocol, communication and effectiveness methods, privacy attacks and defenses, fairness and data valuation, applications and open challenges §2–§10.
• Proposes VFLow, a meta-objective that minimises task loss under privacy, efficiency and bias constraints §7.
• The paper reports no new experiments and no official code, so use it as a map of the literature and a design checklist, not as a benchmark.

Problem

Several organisations hold different attributes about the same set of people or companies, and each wants a joint model without exposing raw data or model parameters §2.1. Compared with horizontal federated learning, where parties share features and differ in samples, VFL has parties that keep both data and model local and exchange intermediate results, so inference also needs collaboration tab. 1. The paper argues that the surveys available before it covered horizontal FL or only a narrow part of VFL §1.

Method

The paper is a survey. Its only original technical content is the VFLow framework in §7. The sections below describe the material that an implementer would need.

Setting and formulation (§2.1). The training data are a set of N aligned samples. Each party k holds the features of its own block of columns; the K-th (active) party also holds the labels. The other K−1 parties are passive §2.1. The joint model Θ is split into a local model G_k per party, with parameters θ_k, and a global module F_K with parameters ψ_K that only the active party can access. The per-sample loss is the task loss of F_K applied to the concatenated local outputs, and the full objective adds a regularizer with strength λ eq. 1 eq. 2.

Variants (§2.1, tab. 2, fig. 3, fig. 4). The global module can be trainable (splitVFL) or a fixed aggregation function such as a sigmoid or an optimal split search for trees (aggVFL). The active party can also have no features of its own, which gives splitVFL_c and aggVFL_c. Local models may be trees, linear or logistic regression, SVMs, neural networks, K-means or EM models; kernel methods are also mentioned §2.1.

Training protocol (§2.2, alg. 1). Training has two phases. First, parties align their sample IDs with private set intersection, so that only the common IDs are revealed to the protocol §2.2. Second, parties iterate over mini-batches. Each party sends its local output on the batch to the active party. The active party updates its global module, computes the gradient of the loss with respect to every party's output, and sends each gradient back. Each party then computes the gradient for its own parameters from those output gradients and updates its model alg. 1 eq. 3. Homomorphic encryption, secure multi-party computation or a trusted execution environment can wrap the transmitted outputs and gradients, sometimes with a third party that holds the keys §2.2.

# General VFL training procedure  [alg. 1]
# Input: learning rate eta_1 (active party), eta_2 (all parties)

def vfl_iteration(batch_ids, features, labels, local_models, top_model):
    # Each party k computes H_k = G_k(x_k; theta_k) on the batch  [alg. 1]
    intermediates = [
        local_models[k](features[k][batch_ids])   # (B, h_k), h_k not stated
        for k in range(num_parties)
    ]
    # Active party computes the loss and updates its global module  [alg. 1]
    loss = task_loss(top_model(intermediates), labels[batch_ids])
    top_model.update(eta_1 * grad(loss, top_model.params))
    # Active party returns dl/dH_k to each party  [alg. 1]
    output_grads = grad(loss, intermediates)
    # Each party k computes its parameter gradient from its output gradient
    # and updates theta_k  [eq. 3] [alg. 1]
    for k in range(num_parties):
        local_models[k].update(
            eta_2 * local_grad(output_grads[k])
        )
    return loss

Tree-based VFL (§2.2.1). Each party holds partial trees that together form one tree. The active party aggregates the split information it receives and chooses the best split. GBDT methods (SecureBoost and successors) use additive homomorphic encryption on gradients and histograms; some use differential privacy; random-forest methods use bagging with encryption or a third party §2.2.1.

Communication efficiency (§3, tab. 3). Five families are summarised: multiple local updates per communication round (FedBCD and others, with convergence stated in tab. 3 as O(1/√T) for several entries), asynchronous coordination, one-shot communication, compression of transmitted data, and selection of samples or features (§3.1–§3.5).

Effectiveness (§4). Four families for improving accuracy when labels are scarce or parties hold little data: self-supervised, semi-supervised, knowledge-distillation and transfer-learning approaches. Each is described in prose with its own training objective, which the paper writes as separate equations §4.1–§4.4.

Privacy (§5). Threat models are honest-but-curious and malicious §5. Entity alignment is covered in §5.1 and protocol-level protections in §5.2. Attacks are grouped into label inference, feature inference and backdoor attacks §5.3.1 §5.3.2 §5.4. Defenses are grouped into cryptographic (§5.3.3), non-cryptographic such as adding noise, gradient discretisation and sparsification (§5.3.4), and specialised defenses (§5.3.5). The paper notes that noise, gradient discretisation and sparsification trade utility for privacy §5.3.4.

Fairness and data valuation (§6). Data valuation and incentive design for parties, explainability, and bias in VFL are reviewed, along with datasets used in the literature §6.1–§6.4.

VFLow (§7). VFLow is a framework for setting up, designing and tuning a VFL algorithm. It takes into account privacy, efficiency and fairness, plus a separate risk-evaluation module that measures attacks and defenses §7. The meta-objective minimises the task loss ℓ over the model Θ, given a VFL setting S, an effectiveness strategy A, a privacy defense P, a set of attacks K, an efficiency strategy E, and a fairness strategy R. It is subject to three constraints: privacy leakage M_p ≤ ε_p, efficiency cost M_e ≤ ε_e and bias M_b ≤ ε_b §7. The paper describes the solution set as a set of trade-offs between objectives, and links the formulation to constrained multi-objective federated learning §7.

Applications and challenges (§8–§10). Open-source platforms, standards, and industry uses in advertising, finance and healthcare are surveyed §8. Open challenges include trustworthy VFL, interoperability between platforms, and the use of blockchain against a single central party §9. The concluding section follows §10.

Hyperparameters

The paper gives no training configuration. The table lists every tunable the framework names.

Results

The paper reports no experiments of its own. The only quantitative content is a summary of convergence rates reported for cited methods in tab. 3, which the paper compiles from the original works. Author-reported.

Paper vs code

No official code. The arXiv metadata has no code link, the paper body has no repository link, and the source tree has no code. A GitHub repository search for the title and for "VFLow" returned HTTP 403 and could not be checked.

Pitfalls

• No experiments. Nothing here shows that VFLow improves any quantity. Its constraints (§7) have no concrete measure, threshold or algorithm, so an implementer must define M_p, M_e, M_b and the ε values from scratch §7.
• Notation drift in the training protocol. The text indexes a party's output with both i and k (for example, "G_i(x_k, θ_k)" in the protocol paragraph §2.2), and eq. 3 sums over H_{i,k} while alg. 1 writes H_k. Read them as one output per party, per mini-batch sample.
• Gradients are exchanged in the clear in the algorithm. alg. 1 sends the output gradients ∂ℓ/∂H_k back to every party. The encrypted variant is only described in prose §2.2, and the algorithm's encryption lines are commented out in the source. Any secure implementation needs the encrypted path designed separately.
• Tables use the paper's own categories. The contrast in tab. 1 is the "conventional" case; the paper warns that special cases may deviate §1.
• Convergence entries are summaries. tab. 3 lists rates as given by the cited works under their own assumptions; the paper does not check them.
• Fixed-version source. The fetched LaTeX is arXiv v4 (updated 2023-09-28); the license recorded by fetch is the one used here.

Lineage

• Builds on: the HFL/VFL/FTL taxonomy of Yang et al. (2019), cited as the first categorisation of FL by data partition §1. No card exists for it, so it is named but not linked.
• Compared against: earlier FL surveys, which the paper says covered HFL or a narrow part of VFL §1.
• Superseded by: not known from content already in this wiki.