---
kind: concept
slug: personalized-federated-learning
title: Personalized federated learning
summary: >-
  Federated learning where each client gets its own model. Three designs covered: a server-side hypernetwork maps a learned client embedding to that client's weights (pFedHN); a cloud model is fine-tuned per user with a CORAL term (FedHealth); a personal model is pulled toward the global one (Ditto). Results are author-reported.
tags: [federated-learning, personalization, hypernetwork, non-iid, transfer-learning, fairness, robustness]
papers: ["2103.04628", "1907.09173", "2012.04221", "2103.00710"]
related: [federated-learning, hypernetwork, federated-aggregation, clustered-federated-learning, transfer-learning, domain-adaptation, meta-learning, algorithmic-fairness, adversarial-robustness]
status: draft
updated_at: 2026-10-24
---

## Overview

Personalized federated learning (PFL) gives each client its own model, while
the data stays on the clients [arxiv:2103.04628 §1]. The motivation is that
clients often hold different data distributions, so one global model fits each
of them poorly [arxiv:2103.04628 §1] [arxiv:1907.09173 §1]. The objective is to
minimize the average loss over the personal models [arxiv:2103.04628 eq. 2].
Earlier personalized methods either exchange large amounts of parameters or
share little across clients [arxiv:2103.04628 §1, §2]. The general problem is
covered in [federated-learning](/concepts/federated-learning).

A survey splits the field into two strategies: adapt one global model per
client, or learn a separate model per client [arxiv:2103.00710 §II] [fig. 1].
It assumes horizontal FL [arxiv:2103.00710 §I-A] and notes that most PFL work
tests only one kind of non-IID skew [arxiv:2103.00710 §V] [tab. III].

Four cards are in this wiki for this concept. Three describe methods with
different designs, and the fourth is the survey. The method cards use different
datasets and baselines, so their numbers are not compared with each other.
Other methods the papers compare against, such as Per-FedAvg, pFedMe, LG-FedAvg
and APFL, are named below but have no cards here, so their content is not used.

## Variants

### Server-side hypernetwork (pFedHN)

A hypernetwork on the server maps a trainable embedding for each client to that
client's target-network weights [arxiv:2103.04628 §3.2]. The hypernetwork
parameters are shared, and only the embeddings are per client, so the server
holds a family of personal models [arxiv:2103.04628 §3.2]. The client returns
the change in its weights after several local steps, and the server passes that
change back through the chain rule to update both the shared parameters and the
embedding [arxiv:2103.04628 §3.2] [arxiv:2103.04628 alg. 1]. The hypernetwork
is never transmitted [arxiv:2103.04628 §3.2]. Card:
[Personalized Federated Learning using Hypernetworks](/papers/2103.04628).

The method is described in more detail on the [hypernetwork](/concepts/hypernetwork)
page.

### Personal classifier head (pFedHN-PC)

The hypernetwork produces only the feature extractor. Each client keeps its own
output layer locally, and that layer is never sent to the server
[arxiv:2103.04628 §3.3] [arxiv:2103.04628 eq. 4]. The authors use this variant
when clients solve different tasks, for example with different numbers of
classes [arxiv:2103.04628 §3.3]. Card: the same paper,
[Personalized Federated Learning using Hypernetworks](/papers/2103.04628).

### Personal model pulled toward the global model (Ditto)

Each device trains a personal model that is pulled toward the global model by
an L2 term with weight λ [arxiv:2012.04221 §3.1]. The global model is trained by
an ordinary federated solver, FedAvg in most experiments, and the server
averages the global updates [arxiv:2012.04221 §3.2] [arxiv:2012.04221 alg. 1].
The pull is toward the global model at the start of the round. Each device
sends back only its global update, so personal models never leave the device
and the messages match those of the global solver [arxiv:2012.04221 §3.2].
With λ = 0 the personal model is purely local, and as λ grows the objective
approaches the global one [arxiv:2012.04221 §3.1]. The survey files a
λ-controlled mix of local and global models under model interpolation
[arxiv:2103.00710 §IV-B]. Each device picks λ from a small grid using its own
validation data [arxiv:2012.04221 app. D.2]. That rule does not cover devices
with exactly 4 or 5 validation points [arxiv:2012.04221 app. D.2].

Robust aggregators can replace the server's averaging step. In the paper's
FEMNIST table, Ditto with clipping is the best entry in five of six columns
[arxiv:2012.04221 tab. 3]. Card: [Ditto: Fair and Robust Federated Learning
Through Personalization](/papers/2012.04221).

### Shared cloud model with per-user transfer (FedHealth)

FedHealth first builds one cloud CNN from federated updates, then gives each
user its own copy by transfer learning [arxiv:1907.09173 §3.2] [alg. 1]. Here a
"user" is an organisation or a device holder [arxiv:1907.09173 §3.2]. The copy
freezes the convolution and max-pooling layers and trains only the fully
connected layers [arxiv:1907.09173 §3.4] [§4.2]. Its loss is the classification
loss plus η times a CORAL term, with η = 0.01 [arxiv:1907.09173 eq. 5]
[arxiv:1907.09173 eq. 6] [§4.2]. The cloud weights reach users under additively
homomorphic encryption, and the server later aligns its model with the uploaded
user models [arxiv:1907.09173 §3.3] [alg. 1]. The aggregation rule is not
given, so the server side cannot be rebuilt from the text [arxiv:1907.09173
§3.3]. The alignment term is borrowed from domain adaptation
[arxiv:1907.09173 §3.4], which is covered on [domain-adaptation](/concepts/domain-adaptation).
Card: [FedHealth: A Federated Transfer Learning Framework for Wearable Healthcare](/papers/1907.09173).

## When to use which

| Situation | Use | Why |
|---|---|---|
| Clients share one architecture and each needs its own model, and a server-side network is acceptable | pFedHN | Only target-sized weight changes are exchanged, and the hypernetwork size does not change the per-round message [arxiv:2103.04628 §3.2]. Reported 90.83% on CIFAR10 with 10 clients, against 89.11% for LG-FedAvg and 76.65% for Per-FedAvg [arxiv:2103.04628 tab. 1] |
| Clients have different numbers of classes or different output heads | pFedHN-PC | The output layer stays local [arxiv:2103.04628 §3.3]. Reported 92.47% on CIFAR10 with 10 clients, the highest value in that column [arxiv:2103.04628 tab. 1] |
| Clients need models of several sizes from one server | pFedHN | One hypernetwork produced three target sizes, and it outperformed per-size baselines on all three sizes in the reported test [arxiv:2103.04628 §5.2] [arxiv:2103.04628 tab. 2] |
| A new client joins after training | pFedHN | Freeze the hypernetwork and fit only a new embedding on the new client's data [arxiv:2103.04628 §5.3]. Steps and learning rate are not stated, and the result is shown only as a figure [arxiv:2103.04628 fig. 2] |
| Each device needs a personal model next to a single global model, and a federated solver such as FedAvg is already in use | Ditto | Personal models stay on the device, so the messages match the global solver's [arxiv:2012.04221 §3.2]. Reported FEMNIST clean accuracy .836 for Ditto, against .827 for Per-FedAvg (HF) and .804 for the global model [arxiv:2012.04221 tab. 2] |
| Some devices are corrupted, or accuracy must be even across devices | Ditto | Reported about 6 points higher mean benign accuracy than the strongest robust baseline, and about 10% lower spread than the fair baseline TERM [arxiv:2012.04221 §1] [§4.1] [§4.2]. With half of the devices corrupted by label attacks on Fashion MNIST, Ditto reported .937 against .855 for the global model [arxiv:2012.04221 tab. 1]. Strong attacks use a different λ grid [arxiv:2012.04221 app. D.2] |
| A shared model already exists and each client needs a personal update from its own labelled data | FedHealth-style transfer | Only the fully connected layers are trained, and the early layers stay fixed [arxiv:1907.09173 §3.4] [§4.2]. Reported 99.4% mean accuracy on five isolated subjects, against 94.1% for the shared model with no transfer [arxiv:1907.09173 tab. 2] |
| Devices cannot keep state between rounds | Check first | Ditto needs each device to keep its personal model across rounds [arxiv:2012.04221 §3.2]. A device that is selected rarely would then rely on an old personal model. This follows from the design and was not tested in the paper |
| The server must aggregate encrypted updates | Check first | The paper names additively homomorphic encryption but no scheme, key size or aggregation rule, so the federated steps cannot be rebuilt from the text [arxiv:1907.09173 §3.3] [§4.2] |
| Omniglot-style splits with one alphabet per client | Check first | pFedHN reported 72.03% on Omniglot with 50 clients, below Per-FedAvg at 76.46%, while pFedHN-PC reported 81.89% [arxiv:2103.04628 tab. 1] |
| Hard resource limits on per-round traffic | Check first | Each round still sends a full target-sized vector down and back, so traffic scales with the target network and the number of rounds [arxiv:2103.04628 §3.2] [arxiv:2103.04628 §5] |
| Evidence on many clients or on other tasks | Check first | The FedHealth result comes from one dataset, with five test subjects and five averaged runs and no spread reported [arxiv:1907.09173 §4.1] [§4.2] |
| You need a map of the methods or of the non-IID benchmarks, not a method to run | Survey first | The survey sorts the methods and lists benchmark suites and metrics, but it reports no experiments of its own [arxiv:2103.00710 §V] [tab. III] [tab. IV] |

Every number above is author-reported, on the paper's own splits and seeds. None
of them has been reproduced here.

## Implementation sketch

Language-agnostic pseudocode of the common core for pFedHN. Where the paper's
printed algorithm and its code disagree on the update sign, the code's sign is
used; see Contradictions.

```pseudocode
# One pFedHN communication round  [alg. 1] [§3.2]
# Input:  hypernet (shared params phi, per-client embeddings v),
#         client i, outer lr alpha, inner lr eta, local steps K

def pfedhn_round(hypernet, i, alpha, eta, K):
    # 1. Server builds the personal weights from the client embedding  [§3.2]
    theta = hypernet.generate(v[i])
    theta_local = copy_of(theta)
    # 2. Client runs K local mini-batch SGD steps  [alg. 1] [§5]
    for step in range(K):
        batch = client_i.sample_minibatch()
        theta_local = theta_local - eta * grad_loss(theta_local, batch)
    # 3. Weight change returned to the server  [§3.2]
    delta = theta - theta_local
    # 4. Chain rule through the hypernetwork, for phi and v[i]  [§3.2]
    grads = vector_jacobian(hypernet, v[i], delta)
    # 5. Step the shared parameters and the embedding  [alg. 1]
    hypernet.phi = hypernet.phi - alpha * grads.phi
    v[i] = v[i] - alpha * grads.v
```

For pFedHN-PC, the same loop runs on the feature extractor only, and the output
layer is updated on the client and never returned [§3.3].

Ditto's personal step is a separate loop, run on each selected device after
the global update in the round [alg. 1] [§3.2]:

```pseudocode
# Ditto personal step on one device  [§3.2]
# w: global model at the start of the round; v: personal model
# lam: pull toward w; eta: step size; s: local steps

def ditto_personal_step(v, w, grad_F, lam, eta, s):
    # Local loss gradient plus the L2 pull toward the global model  [§3.2]
    for _ in range(s):
        v = v - eta * (grad_F(v) + lam * (v - w))
    # v stays on the device; only the global update is sent  [§3.2]
    return v
```

FedHealth's per-user stage is a different loop. The user copy keeps the
convolution and pooling layers fixed, and it minimizes the classification loss
plus η times the CORAL loss on its own data [alg. 1] [eq. 6]. The server-side
alignment step is not specified in the paper, so it is left out here.

## Contradictions

The four cards report numbers on different datasets, splits and baselines, so
their results are not comparable. No two papers report conflicting numbers for
the same setting. Within each source, the following inconsistencies are noted:

- **pFedHN update sign.** The printed update in Algorithm 1 has the opposite
  sign to the code. The paper's surrogate-loss argument matches the code
  [arxiv:2103.04628 alg. 1] [arxiv:2103.04628 §3.2]. This is recorded from the
  card's comparison with the code, and the paper does not mention it.
- **pFedHN claimed margins.** The text claims gains of "2%–10%" over all
  competitors [arxiv:2103.04628 §5.1]. Computed from the table against the best
  baseline in each column, pFedHN gains 1.7 to 2.7 points on CIFAR10 and is 4.4
  points behind on Omniglot [arxiv:2103.04628 tab. 1].
- **Ditto table values.** Ditto's FEMNIST clean accuracy is .834 in Table 1 and
  .836 in Table 2, and its A1 50% value is .762 and .767 [arxiv:2012.04221 tab. 1]
  [arxiv:2012.04221 tab. 2]. The paper does not explain the difference.
- **Ditto λ rule against the released code.** The paper selects λ per device
  from the grids in [arxiv:2012.04221 app. D.2]. The released code instead tries
  {0.1, 1, 2} at every local step, and its run scripts fix λ = 1
  [code: flearn/trainers_MTL/ditto.py#L96-L113]
  [code: run_fashion_clean_ditto_lam1.sh#L25-L26]. Reproducing the dynamic-λ
  numbers needs a flag the scripts do not set by default.
- **FedHealth layer count.** Section 3.4 describes two fully connected layers
  with an alignment layer in place of fc2 [arxiv:1907.09173 §3.4]. Section 4.2
  describes three fully connected layers [arxiv:1907.09173 §4.2].
- **FedHealth headline gain.** The abstract's 5.3% is compared with "traditional
  methods". In the text it is the gap between FedHealth and the shared model
  with no transfer (99.4 − 94.1) [arxiv:1907.09173 §4.3]. That average is also
  the average of the best traditional method, SVM [arxiv:1907.09173 tab. 2].
  The gain is in percentage points, not a relative improvement.

## Papers

- [Personalized Federated Learning using Hypernetworks](/papers/2103.04628): a
  server-side hypernetwork produces each client's weights from a learned client
  embedding, with a personal-classifier variant.
- [Ditto: Fair and Robust Federated Learning Through Personalization](/papers/2012.04221):
  each device trains a personal model pulled toward the global model by an L2
  term, and the global model is trained by an ordinary federated solver.
- [FedHealth: A Federated Transfer Learning Framework for Wearable Healthcare](/papers/1907.09173):
  a shared cloud CNN built from encrypted federated updates, then per-user
  fine-tuning of the fully connected layers with a CORAL term.
- [Towards Personalized Federated Learning](/papers/2103.00710): a survey that
  sorts PFL methods into global-model personalization and learning personalized
  models, with benchmarks, metrics and open directions. It runs no experiments.