FedHealth: A Federated Transfer Learning Framework for Wearable Healthcare

In plain words

Wearables such as phones and wristbands record a lot about a person's movement and health, and each company stores that data on its own servers. Pooling it would train a better model, but the companies cannot share the raw data because of privacy rules. A single model trained on one company's data also fits other people poorly, because everyone moves differently.

FedHealth tackles both problems in two stages. First, every party trains a copy of the same model and only sends encrypted weight updates, so a shared model improves without anyone seeing the raw recordings. Second, each user takes that shared model and retrains its last layers on their own data, keeping the early layers fixed. A small penalty pushes the adapted model's statistics toward those of the shared model. The authors test this on activity recognition from smartphone sensors. Their reported gain is 5.3 points of average accuracy over the shared model alone.

TL;DR

FedHealth is a federated transfer learning pipeline: a cloud CNN is built from encrypted, federated user updates §3.3, then each user's model is personalised by fine-tuning its fully connected layers with a CORAL alignment term §3.4. On five isolated subjects of the UCI Smartphone dataset the authors report 99.4% mean accuracy against 94.1% for the cloud model without transfer tab. 2. The paper gives no encryption scheme, no server aggregation rule and no run-time numbers, so the federated part is hard to reproduce from the text. All results are author-reported on one dataset with five test subjects.

Problem

Wearable data sits in separate organisations and cannot be pooled because of privacy and regulation §1. A model trained on all data in one place is not an option under those constraints. A single server-trained model also fails on personalisation, because the distribution of one user's data differs from the cloud's §1 §3.4. The paper's objective is to make the accuracy of a federated model close to that of centralised training, within a small margin Δ, which is not given a value §3.1 eq. 1.

Method

The framework has four procedures §3.2 fig. 2: (1) train a cloud model on server-side data; (2) distribute it to users; (3) upload each user model, encrypted, for the server to update its model; (4) adapt each user model by transfer learning. Here "user" means an organisation or device holder.

Cloud model §3.3 eq. 2. A deep network is trained on the server data with a standard per-sample loss, for example cross-entropy for classification §3.3. The paper's experiments use a CNN for both cloud and user models §4.2.

Encrypted parameter sharing §3.3 §4.2. The server sends the cloud weights and biases to each user through homomorphic encryption. The paper describes only the additive property, ⟨a⟩ + ⟨b⟩ = ⟨a + b⟩, for real numbers §3.3. It says it follows Rivest et al. §4.2 and does not name a scheme, key size or encoding for reals.

User model §3.3 eq. 3. Each user trains its own network on its local data, starting from the cloud weights, with the same loss as the cloud model §3.3 eq. 3.

Server update §3.3 alg. 1. Each user model is uploaded in encrypted form. The server then aligns its previous model with the uploaded user models, possibly on a schedule (for example nightly) §3.3. The text does not give the aligning formula, so the aggregation rule is not stated.

Transfer learning for personalisation §3.4 fig. 3. Lower layers learn generic features and higher layers learn task-specific ones, so the paper freezes the convolution and max-pooling layers and fine-tunes the fully connected layers §3.4 §4.2. Section 3.4 describes the network as two convolution layers, two pooling layers, two fully connected layers (fc1, fc2) and a softmax, and replaces fc2 with a correlation-alignment layer placed before the softmax §3.4. Section 4.2 describes three fully connected layers §4.2. The card reports both and leaves the reconciliation to the implementer.

Alignment loss §3.4 eq. 5. The CORAL term compares second-order statistics: the squared Frobenius norm of the difference between source and target covariance matrices, scaled by 1/(4d²), where d is the embedding dimension eq. 5. Section 3.4 says the method has only the user data and the cloud model §3.4, which suggests source = cloud model and target = user data. The text never defines C_S and C_T explicitly. The user objective is the classification loss summed over the user's samples plus η times the CORAL loss eq. 6. The paper fixes η = 0.01 §4.2.

Softmax §3.4 eq. 4. The classifier output is a softmax over class scores eq. 4.

Procedure alg. 1. Build the cloud model eq. 2, send it encrypted, train user models eq. 3, upload and align the server model, send the aligned model and run transfer eq. 6 for each user, then repeat as new user data arrives alg. 1 §3.5. The paper also says the same approach can use gradient-boosted trees, which the authors describe as lightweight for wearable devices §3.5. No experiment reports this.

# FedHealth learning procedure  [alg. 1]
# Input:  public_data, user_datasets (one per user), eta
# Output: a personalised model for each user

def fedhealth(public_data, user_datasets, eta):
    # Build the cloud model on server-side data  [alg. 1] [eq. 2]
    server_model = train_cnn(public_data, cross_entropy)

    # Send the cloud model to each user, encrypted  [alg. 1] [§3.3]
    # The scheme is not stated beyond additive homomorphism
    user_models = []
    for data in user_datasets:
        user_models.append(encrypted_copy(server_model))

    # Train each user model on that user's own data  [alg. 1] [eq. 3]
    for index, data in enumerate(user_datasets):
        user_models[index] = train_cnn(user_models[index], data)

    # Upload the encrypted user models and align the server model to them
    # The aggregation rule is not stated  [alg. 1] [§3.3]
    server_model = align_to_users(server_model, user_models)

    # Fine-tune each user's model from the new cloud model  [alg. 1] [§3.4]
    # Convolution and pooling layers frozen; fully connected layers trained
    # Objective: classification loss + eta * CORAL loss  [eq. 6]
    personal_models = []
    for data in user_datasets:
        personal_models.append(adapt(server_model, data, eta))

    # Repeat the whole procedure as new user data arrives  [alg. 1]
    return personal_models

# CORAL alignment loss  [eq. 5]
# source_cov, target_cov: (d, d) covariance matrices
# d: dimension of the embedding features

def coral_loss(source_cov, target_cov, d):
    # Squared Frobenius norm of the difference, scaled by 1 / (4 d^2)  [eq. 5]
    difference = source_cov - target_cov
    return squared_frobenius_norm(difference) / (4 * d * d)

Key equations (paper numbering):

|\mathcal{A}_{FED} - \mathcal{A}_{ALL}| < \Delta eq. 1 (objective; Δ not given a value)

\arg\min_{\Theta} \sum_{i=1}^{n} \ell(y_i, f_S(\mathbf{x}_i)) eq. 2 (cloud model)

\arg\min_{\Theta^u} \sum_{i=1}^{n^u} \ell(y^u_i, f_u(\mathbf{x}^u_i)) eq. 3 (user model)

y_j = \frac{e^{z_c}}{\sum_{c=1}^{C} e^{z_c}} eq. 4 (softmax; the index on y is j but the sum is over c, see Pitfalls)

\ell_{CORAL} = \frac{1}{4d^2} \lVert C_S - C_T \rVert_F^2 eq. 5

\arg\min_{\Theta_u} \sum_{i=1}^{n_u} \ell(y^u_i, f_u(\mathbf{x}^u_i)) + \eta\, \ell_{CORAL} eq. 6

Hyperparameters

Values for the FedHealth model and its training. Items the paper does not state are listed as not stated.

Results

Author-reported. Setup: UCI Smartphone human activity recognition, 6 activities, 30 volunteers, 10,299 instances §4.1 tab. 1. The cloud model is trained on 25 users. Five isolated subjects (IDs 26–30) are the test users and their data is not shared §4.1. The baseline "NoFed" uses the server model directly, with no transfer §4.2. Hardware and run-time are not stated.

Table 2: per-subject accuracy (%) of the test subjects tab. 2

The table's AVG row matches the mean of the five rows. Subject labels P1–P5 are not explicitly mapped to IDs 26–30 in the text.

Table 3: classification report of NoFed and FedHealth for subject 2 tab. 3

The text says the matrices are for subject 2 §4.5; the caption does not name the subject. Per-class precision (P), recall (R) and F1 are read from the table. Confusion-matrix rows are in the table.

Off-diagonal confusion in C1: NoFed sends 7.4% to C2 and 22.2% to C3; FedHealth sends 3.7% to C2 and 7.4% to C3 tab. 3. The text says FedHealth "can reduce the misclassification rate, especially on class C1 (Walking)" §4.5.

Extensibility: transfer variants fig. 4 §4.4. The paper compares the alignment loss with plain fine-tuning and with MMD §4.4. The text states that transfer beats no transfer by 4% on average accuracy and that fine-tuning and MMD also give gains. The per-variant values are only in fig. 4 and are not restated here.

Headline comparison. The abstract reports a 5.3% accuracy improvement "compared to traditional methods". In the text this is the gap between FedHealth and NoFed (99.4 − 94.1) §4.3, given in percentage points. It equals the gap to SVM, the best traditional method on average (94.1) tab. 2.

Paper vs code

No official code. The paper gives no code link, and the arXiv abstract page has no code or project link either. Searches: the bibliography and LaTeX source for URLs (none); the arXiv abstract page; GitHub repository search for "fedhealth" (49 results, all third-party or student projects such as xiby/FedHealth and Encrypted-Lakshya/FedHealth, none owned by an author or linked from the paper); a search for "fedhealth" within the Jindong Wang account (none); and the Jindong Wang transfer-learning collection, which is a general resource and contains no FedHealth code.

Pitfalls

• Gain units. The 5.3% is percentage points of mean accuracy (94.1 → 99.4), not a relative improvement. The baseline is NoFed, which is the cloud model applied with no transfer §4.2.
• Federated steps not shown to run. The text describes encrypted upload and server alignment §3.3, and the experiments section says only that it follows Rivest et al. for homomorphic encryption §4.2. The experiments section does not describe running those steps. Whether the reported FedHealth numbers include them is not stated.
• Aggregation rule missing. The server "can align the old model with the model from each user" §3.3, and this is the only description. No formula, weighting or schedule details beyond "e.g. every night".
• Encryption underspecified. Only the additive property for real numbers is given §3.3. The paper names no scheme, key size, number encoding or precision, and gives no security argument for the claim that no information leaks.
• Layer structure conflicts. Section 3.4 describes two fully connected layers and an alignment layer that replaces fc2 §3.4. Section 4.2 says three fully connected layers §4.2. The position of the alignment term (before the softmax, or in place of fc2) is therefore ambiguous.
• Which tensor is aligned. Section 3.4 says the alignment is "between the inputs" §3.4, but after eq. 5 the covariance matrices are described as "of the source and target weights" eq. 5. The implementer must choose the tensor and the batch used to estimate each covariance. Neither is stated.
• Softmax notation. Eq. 4 indexes the output as y_j and sums over c, and calls z_c a "learned probability" eq. 4. z_c is the pre-softmax score. The index mismatch is in the source.
• Dimension d. Eq. 5 uses the embedding dimension d, but the layer it is taken from is not stated.
• Table 3 average. The NoFed precision average is printed as 0.96 tab. 3. The unweighted mean of the six class values is 0.95, and no weighting is stated. The other printed averages are within rounding of their unweighted means.
• Channel count. The text names 3-axis acceleration and 3-axis angular velocity §4.1, which is six axes, while Table 1 lists 9 channels tab. 1. The reconciliation is not stated.
• Split ambiguity. Section 4.1 describes a 70/30 split of volunteers. Section 4.2 describes a 70/30 split of training data into training and evaluation. Which data forms the test set for Table 2, and how the five isolated subjects are split, is not stated.
• Small test set. Five test subjects, five averaged runs, and no standard deviation. Per-subject gains over NoFed range from 3.9 to 7.4 points tab. 2, with no spread reported.
• Baselines. KNN, SVM and RF are "tuned using cross-validation" §4.2, but the grids, folds and chosen values are not stated.
• Eq. 1 is a goal. Δ has no value, and the paper gives no test for meeting it.
• Not evaluated. Compute cost, communication cost, the gradient-boosted-tree variant §3.5, incremental updates and blockchain storage §5 are discussed but not measured.
• Stated vs algorithm order. Section 3.5 says user and cloud models are updated "simultaneously", while the algorithm is written as sequential steps alg. 1.

Lineage

• Builds on: federated learning with a survey taxonomy §2 (Yang et al., 2019) and the federated transfer learning category §2; additively homomorphic encryption §3.3 (Rivest et al., 1978); freezing lower layers and fine-tuning higher ones, following Yosinski et al. §3.4; the CORAL second-order alignment loss, referred to as sun2016return eq. 5; and the weight-regularisation idea of Rozantsev et al. for domain adaptation without target data §3.4.
• Differs from: deep domain confusion §3.4 (Tzeng et al.), which needs both source and target data; SecureBoost §3.3, which the authors say they do not use for efficiency.
• Compared against: KNN, SVM, random forest, NoFed (cloud model only), fine-tuning and transfer with MMD §4.3 §4.4 (Wang et al., 2018).
• Superseded by: not stated in this wiki yet.