wikillm

Paper · 2026-11-06 · cs.LG

FedLoVA: Value-Only Aggregation for Federated LoRA Fine-Tuning of Large Language Models

FedLoVA keeps each client's query-side adapters private and sends only the value-side ones, which the server averages. Author-reported on RoBERTa-large with 3 clients, average GLUE accuracy is 88.13% versus 87.69% for FedLoRA, with half the parameters sent per round.

Low-rank adaptation (LoRA and variants)Federated learningFederated aggregationfederated-learningloraparameter-efficient-fine-tuning

arXivPDFCodeMarkdown15 min read

AI-compiled by claude-haiku-5-5 on 2026-11-21, not yet reviewed by a human. Every claim cites its source; check it when a detail matters.

In plain words

Fine-tuning a large language model usually means changing a small add-on, called a LoRA adapter, instead of all of the model. In federated learning, several devices each fine-tune their own copy on their own private text, and a central server combines what they send back. Sending the add-ons is cheap, but it still costs bandwidth every round.

The authors looked at the two parts of attention that LoRA usually modifies. One part, the value side, turned out to settle much faster during training than the query side. FedLoVA uses that difference: each device still trains both parts, but only the value add-on is uploaded and averaged. The query add-on stays on the device.

The result is half the upload per round, and the paper reports that accuracy holds up or improves on several language tasks. Those numbers come from the authors' own experiments, on two models and a fixed set of GLUE, sentiment and maths tasks.

TL;DR

  • FedLoVA trains LoRA adapters on the query and value projections on each client, but aggregates only the value adapters on the server §4.1 alg. 1.
  • The authors observe that value-side LoRA converges faster than query-side LoRA with RoBERTa-large, and also with BERT and Llama2 in the appendix, and argue the difference from curvature and data-dependence §3.2 §3.3 §7.3.
  • Author-reported: average GLUE accuracy 88.13% vs 87.69% for FedLoRA; 70.18% on multilingual sentiment vs 68.90% for FedLoRA tab. 1 tab. 2.
  • The uploaded parameter count per round is half that of FedLoRA tab. 4. The headline "up to 7.8×" total-communication figure is measured against FedLoQA, which is the authors' own variant, not against FedLoRA or LoRA-FAIR (see Pitfalls).
  • Convergence theory covers only the local objectives under smoothness and bounded-gradient assumptions, and the authors call it structural support, not a guarantee §4.2.

Problem

Full fine-tuning of an LLM in federated learning is too expensive for clients and too heavy to send each round. LoRA cuts the trainable and transmitted parameters, but in federated settings it is still applied to query and value projections on every client, and aggregating all of it costs the most bandwidth §1 §2.2.

Prior federated LoRA variants change which factor is shared. FFA-LoRA freezes A and shares B §2.2. FedSA-LoRA trains A and B locally and shares only A §2.2. The paper asks instead which attention projection, query or value, is worth sharing, and whether that choice changes convergence and accuracy under class-label and linguistic heterogeneity §1.

Method

Setting. Each client holds its own data. LoRA is applied to the query (W_q) and value (W_v) projections of every attention layer, with W_q = W_q,0 + B_q A_q and W_v = W_v,0 + B_v A_v eq. 5 eq. 6 §3.1. The frozen base weights are W_q,0 and W_v,0. B is zero-initialised and A is randomly initialised §3.1.

Motivation. Trained alone, LoRA on V reaches 95% of the final accuracy of LoRA on Q in about 3.3× fewer rounds on SST2 and 2.4× fewer on MNLI-mm, with RoBERTa-large and rank 8 §3.2 fig. 2. The authors explain this with the softmax: the output is linear in V but depends on Q through the softmax, which they argue gives worse-conditioned gradients §3.2. A separate argument uses Hessian blocks from Ormaniec et al. §3.3. The paper says those results concern the full projection matrices rather than the low-rank factors, and offer them as structural support rather than a proof §3.3.

FedLoVA procedure.

  1. The server holds global value adapters (A_v, B_v). Each client holds its own query adapters (A_q,k, B_q,k), which are never sent §4.1 alg. 1.
  2. In each round, the server sends the global value adapters to the selected clients §4.1 alg. 1.
  3. Each client runs local SGD on both its query and value adapters for E local steps on its own data §4.1 alg. 1.
  4. Each client uploads only its updated value adapters §4.1 alg. 1.
  5. The server replaces the global value adapters with the average of the uploads, weighted by each client's data size §4.1 alg. 1.
  6. After the last round, the output is the global value adapters plus each client's query adapters alg. 1.

Relative to FedLoRA, this halves the upload because query adapters are not sent. Relative to FedSA-LoRA, the paper says the number of exchanged parameters is the same and the local procedure is similar §4.1. Relative to FFA-LoRA, the query and value adapters are not frozen, which the authors argue gives more freedom in local optimisation §4.1.

Pseudocode
# FedLoVA: value-only aggregation  [alg. 1]
# Input:  clients k in K, rounds T, local steps E, learning rate eta
# Output: global value adapters; query adapters kept per client

def fedlova(clients, rounds, local_steps, eta):
    # Server keeps value-projection LoRA only  [alg. 1]
    global_value = init_lora_value()          # (A_v, B_v)
    # Each client keeps its own query-projection LoRA  [alg. 1]
    local_query = {k: init_lora_query() for k in clients}
    for t in range(rounds):
        # Pick the participating clients S_t  [alg. 1]
        selected = sample_clients(clients, t)
        uploads = []
        for k in selected:
            # Start from the current global value adapters  [alg. 1]
            value_k = copy(global_value)
            # Local SGD on query and value adapters  [alg. 1] [§5.1]
            for step in range(local_steps):
                batch = next_batch(data[k])
                loss = model_loss(batch, local_query[k], value_k)
                grad_q, grad_v = grads(loss)
                local_query[k] = local_query[k] - eta * grad_q
                value_k = value_k - eta * grad_v
            # Upload only the value adapters  [alg. 1]
            uploads.append((len(data[k]), value_k))
        # Server: average value adapters weighted by data size  [alg. 1]
        global_value = weighted_average(uploads)
    return global_value, local_query

Convergence analysis. Theorem 4.4 bounds the average squared gradient norm over T rounds for the local client objectives, under L-smoothness (Assumption 4.1), bounded stochastic gradient second moment G² (Assumption 4.2), and bounded LoRA factor norms with lower-bounded eigenvalues (Assumption 4.3) §4.2. The bound has an η-independent constant term, so the authors conclude convergence to a neighbourhood of a stationary point rather than to a stationary point §4.2. The paper states that query parameters are never shared, so no global objective is analysed §4.2.

Hyperparameters

Values for the NLU experiments unless stated. "Paper" is the cited paper text; "Code" is the official repository at code_commit.

NameValueWhereNotes
BackboneRoBERTa-large, 355M parameters, Hugging Face weights§5.1NLU experiments
LoRA targetsquery and value projections of attention§5.1 §3.1Following the LoRA paper
LoRA rank r8§5.1Ablation: 4 and 16 §7.10 tab. 14
LoRA scaling factor α16§5.1 fedlova_mnli_mm.yamlHow α scales the update is not written in the paper
LoRA dropoutnot statedfedlova_mnli_mm.yamlCode uses 0.05
Clients (NLU)3§5.1Code sets 50 for MNLI and SST2 configs (see Paper vs code)
Data splitnon-IID Dirichlet, β = 0.5§5.1Code: splitter: lda, alpha 0.5 fedlova_mnli_mm.yaml
Batch size128§5.1fedlova_mnli_mm.yaml
Sequence length128§5.1fedlova_mnli_mm.yaml
Communication rounds1000§5.1fedlova_mnli_mm.yaml
Local steps10§5.1Algorithm says E local epochs alg. 1; code uses batches fedlova_mnli_mm.yaml
OptimiserSGD; momentum and weight decay not stated§5.1
Learning rategrid {0.005, 0.01, 0.02, 0.05, 0.1}; best reported§5.1Selected rates in tab. 7 (see below)
Precisionhalf precision in code; not in the paper textfedlova_mnli_mm.yaml
Seeds5 for main results; 3 for ablations§5 §7.10
Hardware and softwareNVIDIA RTX 3090, PyTorch 2.1.0, CUDA 12.5§5
Convergence threshold95% of the final Q-projection accuracy§3.2 §7.3GSM8K uses a fixed 13% target §7.3
Multilingual sentiment3 clients, one language each (English, German, Chinese); Amazon reviews; same settings as NLU§5.2Code split domain fedlova_mls.yaml
GSM8K model and setupLlama3-8B; 3 clients; 400 rounds§5.3Code: 1000 rounds, batch 1, 30 local steps, lr 0.005 fedlova.yaml [#L30] [#L34] [#L37]

Learning rates selected per method for NLU tab. 7:

MethodMNLI-mMNLI-mmSST2QQPRTEMLS
FedLoRA0.010.010.020.0050.0050.01
FFA-LoRA0.050.050.050.020.020.05
FedSA-LoRA0.0050.0050.010.0050.0050.005
LoRA-FAIR0.0050.0050.020.0050.0050.01
FedLoQA0.0050.0050.0050.0050.0050.005
FedLoVA0.020.020.020.010.0050.01

Results

Author-reported. Setup: RoBERTa-large on GLUE (3 clients, non-IID β = 0.5, 1000 rounds, mean ± std over 5 seeds) §5.1 tab. 1. Multilingual sentiment and GSM8K are in tab. 2 and tab. 3.

GLUE accuracy (%) tab. 1

MethodMNLI-mMNLI-mmSST2QQPRTEAvg.
FedLoRA87.48 ± 1.1687.63 ± 0.4494.15 ± 0.7485.90 ± 0.5783.29 ± 0.2787.69
FFA-LoRA87.74 ± 0.5387.58 ± 0.1993.87 ± 0.7885.38 ± 1.0380.99 ± 0.8787.11
FedSA-LoRA87.09 ± 1.5786.95 ± 0.3294.18 ± 0.7182.82 ± 2.7783.07 ± 1.1486.82
LoRA-FAIR87.54 ± 0.3687.39 ± 0.1993.87 ± 0.3985.97 ± 0.6983.71 ± 2.3387.70
FedLoQA87.76 ± 1.4586.39 ± 0.9093.81 ± 1.5783.05 ± 1.6583.43 ± 1.3686.89
FedLoVA88.04 ± 1.4687.81 ± 0.4194.45 ± 0.4786.23 ± 0.6584.13 ± 1.3188.13

The authors say FedLoVA is comparable or better on every dataset. Within a fixed round count, FedLoRA and LoRA-FAIR are second best but use about twice the upload §5.1.

Communication-matched comparison tab. 8. For the baselines the authors read accuracy at round 500, where their cumulative upload equals FedLoVA's at round 1000. FedLoVA averages 85.14%, FedLoRA 83.74%, LoRA-FAIR 83.14% §7.6. The paper reports paired t-test p-values of 0.0150 (FedLoRA), 0.0037 (LoRA-FAIR) and 0.0287 (FedSA-LoRA) §7.6.

ApproachMNLI matchedMNLI mismatchedSST2QQPRTEMLSMean
FedLoRA (round 500)86.72 ± 0.8487.11 ± 0.3993.54 ± 0.8683.84 ± 0.4883.76 ± 0.5667.49 ± 0.2583.74
LoRA-FAIR (round 500)86.05 ± 0.3786.71 ± 0.2393.60 ± 0.8283.01 ± 0.3082.28 ± 3.5467.20 ± 0.7783.14
FedLoVA (round 1000)88.04 ± 1.4687.81 ± 0.4194.45 ± 0.4786.23 ± 0.6584.13 ± 1.3170.18 ± 0.2885.14

Multilingual sentiment (MLS), accuracy (%) tab. 2: FedLoRA 68.90 ± 0.24; FFA-LoRA 69.63 ± 0.18; FedSA-LoRA 68.61 ± 0.12; LoRA-FAIR 69.05 ± 0.36; FedLoQA 67.79 ± 0.19; FedLoVA 70.18 ± 0.28 tab. 2.

GSM8K accuracy (%), Llama3-8B, 400 rounds, 3 clients tab. 3: FedLoRA 43.59 ± 0.02; FFA-LoRA 44.95 ± 1.08; FedSA-LoRA 44.04 ± 2.61; FedLoQA 38.74 ± 2.83; FedLoVA 45.79 ± 1.46. LoRA-FAIR is not in this table §5.3.

Total communication cost tab. 4. Cost is parameters sent per round times rounds to reach a target accuracy. The target is the floor of the lowest mean accuracy among all methods for each dataset §5.4. Values are in millions of parameters, with the ratio to FedLoVA in brackets.

MethodMNLI-m (87%)MNLI-mm (86%)SST2 (93%)QQP (82%)RTE (80%)MLS (67%)
FedLoRA215.3 M (3.2×)132.6 M (3.1×)156.0 M (2.2×)237.1 M (3.3×)230.1 M (2.0×)287.0 M (3.1×)
FFA-LoRA184.9 M (2.7×)161.5 M (3.8×)352.2 M (5.0×)121.3 M (1.7×)186.0 M (1.6×)111.5 M (1.2×)
FedSA-LoRA129.9 M (1.9×)128.7 M (3.0×)102.6 M (1.5×)111.9 M (1.6×)128.3 M (1.1×)140.4 M (1.5×)
LoRA-FAIR411.8 M (6.0×)297.2 M (6.9×)237.9 M (3.4×)266.8 M (3.8×)280.0 M (2.5×)279.2 M (3.0×)
FedLoQA185.3 M (2.7×)333.1 M (7.8×)113.1 M (1.6×)101.4 M (1.4×)115.1 M (1.0×)151.7 M (1.6×)
FedLoVA68.3 M (1×)42.9 M (1×)69.8 M (1×)71.0 M (1×)113.9 M (1×)93.2 M (1×)

The appendix gives the underlying rounds to target and parameter counts tab. 11: FedLoVA needs 175, 110, 179, 182, 292 and 239 rounds, sending 0.39 M parameters per round. FedLoRA sends 0.78 M per round.

Convergence speed multipliers tab. 6 (ratio of Q to V rounds to reach γ of Q's final accuracy, RoBERTa-large): γ = 0.8: MNLI-mm 2.3, SST2 2.8; γ = 0.85: 2.3, 3.2; γ = 0.9: 2.3, 3.2; γ = 0.95: 2.4, 3.3.

Ablations (MNLI-m and SST2, 3 seeds, §7.10):

  • Heterogeneity tab. 12: MNLI-m IID FedLoVA 88.57 ± 0.10 (FedLoRA 88.15); β = 1.0 88.51 ± 0.04 (FedLoRA 87.46). SST2 IID 95.68 ± 0.23 (FedLoRA 95.37); β = 1.0 95.06 ± 0.09 (FedLoRA 94.57). IID and β = 1.0 runs are reported at 700 rounds.
  • Number of clients, severe non-IID tab. 13: MNLI-m 10 clients 86.90 ± 0.73 (FedLoRA 86.57); 50 clients 89.18 ± 0.10 (FedSA-LoRA 88.60). SST2 10 clients 96.53 ± 0.46 (FedLoRA 96.19); 50 clients 96.18 ± 0.21 (FedLoRA 96.17).
  • LoRA rank tab. 14: r = 4: MNLI-m 88.32 ± 0.77 (FedLoRA 87.80), SST2 94.19 ± 0.47 (FedLoRA 94.18). r = 16: MNLI-m 87.73 ± 0.79 (FedLoRA 87.06), SST2 94.45 ± 0.25 (FedLoRA 94.10). The ablation scaling factor is 2r §7.10.
  • Differential privacy tab. 15: Gaussian mechanism, clipping norm 0.005, (ε, δ) = (50, 1e-5) on transmitted local parameters. SST-2: FedLoVA 92.83 ± 2.15 (FedSA-LoRA 92.21 ± 3.17, FedLoRA 88.06 ± 5.84). MNLI-m: FedLoVA 85.73 ± 1.75 (FedSA-LoRA 85.73 ± 1.79, FedLoRA 80.41 ± 1.81). The paper says it makes no privacy-protection claim from this §5.5 §7.10.

Module choice tab. 5 (accuracy, %): SST2 K 91.13, Q 94.49, V 94.84, MLP 95.70, Q+V 95.49, Q+K+V 95.49. MNLI-mm K 41.81, Q 84.90, V 87.31, MLP 88.53, Q+V 87.80, Q+K+V 88.03. The paper picks Q+V because it is the strongest pair in accuracy, and Q+K+V adds only a small MNLI-mm gain at 1.5× the overhead of Q+V §7.2.

Learning-rate sensitivity tab. 9 (MNLI-mm and SST2, FedLoQA vs FedLoVA):

Learning rateMNLI-mm FedLoQAMNLI-mm FedLoVASST2 FedLoQASST2 FedLoVA
0.00586.39 ± 0.9087.51 ± 0.3393.81 ± 1.5793.57 ± 0.42
0.0137.58 ± 0.0087.52 ± 0.5093.60 ± 2.0194.06 ± 0.54
0.0233.09 ± 4.4987.81 ± 0.4156.04 ± 19.2594.45 ± 0.47
0.0535.70 ± 1.8835.70 ± 1.8857.02 ± 17.8575.21 ± 23.51

Paper vs code

Official repository: https://github.com/EnsiehKhazaei/FedLoVA, linked from the paper's abstract and author-owned. Cloned at code_commit. It is a fork of the FederatedScope framework, and the repository has no LICENSE file.

  • Transmitted set (agrees). The paper uploads only value adapters and keeps query adapters local §4.1. The code marks parameters whose names contain query (and classifier) as local fedlova_mnli_mm.yaml, and the LLM config marks q_proj as local fedlova.yaml. The keyword match is a substring test utils.py.
  • Classifier head (differs). The paper does not mention the classification head. The GLUE and MLS configs keep classifier local too fedlova_mnli_mm.yaml fedlova_mls.yaml.
  • Learning rates (differ). Paper tab. 7: FedLoVA MNLI-m 0.02, MNLI-mm 0.02, SST2 0.02, QQP 0.01, RTE 0.005, MLS 0.01. Code: MNLI 1e-2 fedlova_mnli.yaml; MNLI-mm 5e-3 fedlova_mnli_mm.yaml; SST2 1e-2 fedlova_sst2.yaml; QQP 5e-3 fedlova_qqp.yaml; MLS 1e-2 fedlova_mls.yaml.
  • Client count (differs). The paper uses 3 clients for the main NLU results §5.1. The MNLI and SST2 configs set 50 clients fedlova_mnli.yaml fedlova_sst2.yaml. The MNLI-mm, QQP and MLS configs use 3 fedlova_mnli_mm.yaml fedlova_qqp.yaml fedlova_mls.yaml. The configs therefore do not reproduce the main table as shipped.
  • RTE (missing). The paper reports RTE tab. 1, but there is no RTE config in the repository. The glue folder has configs for MNLI, MNLI-mm, SST2 and QQP only.
  • Seeds (differ). Each GLUE config carries a single seed (123, 456, 789 or 1) fedlova_mnli_mm.yaml. The five-seed loop described in the paper §5 is not in the repository.
  • Local work (differs). Algorithm 1 has E local epochs alg. 1. The experiments use 10 local steps §5.1. The code uses batch_or_epoch: batch fedlova_mnli_mm.yaml.
  • Aggregation weights (partly checked). The aggregator sums client sample sizes before averaging clients_avg_aggregator.py. I did not read the weighting formula itself.
  • Trainable filter (suspected bug, not run). _param_filter builds trainable_filter as lambda p: True if share_non_trainable_para else lambda p: p in trainable_names, which is a conditional with a lambda in the else branch trainer.py. With the default share_non_trainable_para = False cfg_fl_algo.py, the else-branch lambda is returned, which is truthy, so this check does not exclude frozen weights. Whether frozen base weights are actually transmitted was not tested.
  • DP path (absent). The paper's DP results use clipping norm 0.005 and (ε, δ) = (50, 1e-5) §7.10. The DP block in _para_weighted_avg is commented out clients_avg_aggregator.py, so the shipped aggregator adds no noise.
  • GSM8K settings (differ). Paper: 400 rounds §5.3. The LLM config: total_round_num: 1000 fedlova.yaml. Batch 1, 30 local steps and lr 0.005 are in the config fedlova.yaml [#L34] [#L37]. The paper gives none of these.
  • README paths (differ). The README runs federatedscope/glue/yamls/fedlova.yaml and federatedscope/mls/yamls/fedlova.yaml README.md [#L39]. Neither file exists; the files are suffixed by task.
  • Released run (not FedLoVA). The exp/ folder holds a log with method: FedAvg config.yaml and lr 0.02 [same config.yaml#L390]. It is not a FedLoVA result and I did not use it for any number here.
  • Initialisation and LoRA dropout. Initialisation (B zero, A random) follows the paper §3.1. I did not verify the code's initialisation. The code sets LoRA dropout 0.05 fedlova_mnli_mm.yaml; the paper does not state it.

Pitfalls

  • The "7.8×" headline is not against existing baselines. The 7.8× figure is FedLoQA on MNLI-mm (333.1 M vs 42.9 M) tab. 4. FedLoQA is the authors' own counterpart variant. Against the published baselines in the table, the largest ratio is LoRA-FAIR on MNLI-mm (6.9×); against FedLoRA it is 3.3× (QQP). The abstract says "existing federated LLM fine-tuning baselines".
  • "Up to 10×" is not in the table. The appendix says FedLoVA converges up to 10× faster §7.9. The largest round ratio in its own table is FedLoQA on MNLI-mm (854 vs 110 rounds, about 7.8×) tab. 11.
  • The cost metric depends on the chosen target. Target accuracies are the floor of the lowest mean accuracy among all methods §5.4. Rounds to target are not defined beyond that: it is not stated whether they come from mean curves or per seed.
  • Trainable count is unexplained. FedLoRA shows 1.83 M trainable parameters but 0.78 M communicated tab. 11. The paper does not say what the other ~1.05 M trainable parameters are. The code keeps the classifier local, which may account for part of the gap; I did not count the parameters.
  • Learning-rate sensitivity contradicts the claim at 0.05 on MNLI-mm. The text says FedLoVA stays strong at higher learning rates §5.1 §7.7. At lr 0.05 on MNLI-mm, FedLoVA drops to 35.70 ± 1.88 tab. 9. Both columns show exactly 35.70 ± 1.88 in that row, so it may be a copy error; check the source before relying on it.
  • Learning-rate selection uses the best grid point. The paper reports the best of the grid §5.1. The paper does not say whether the choice used a validation split or the reported test data. The released config's best_res_update_round_wise_key is val_accuracy fedlova_mnli_mm.yaml, which suggests validation, but this is not stated in the paper.
  • Paired t-test population is unclear. The text says "six NLU datasets" §7.6. Section 5.1 names four GLUE datasets, and Table 1 has five GLUE columns (MNLI-m and MNLI-mm separately) plus MLS. The number of pairs behind the p-values is not stated, and I did not recompute them.
  • Which query adapters are used at evaluation is not stated. Each client has a personal query adapter, and the output is the global value adapter plus those query adapters alg. 1. The paper does not say whether accuracy is measured per client with its own query adapter, or on a shared test set. The MNLI-mm config sets matched: False fedlova_mnli_mm.yaml; which split the paper reports is not stated.
  • Model naming is inconsistent. Section 5.3 uses Llama3-8B for GSM8K. Section 3.2 and Section 7.3 use Llama2 7B. Both cite the same Llama reference §3.2 §5.3 §7.3.
  • Ablation table caption is wrong. The LoRA-rank table caption says "r = 4 and r = 8", but its columns are r = 4 and r = 16 tab. 14.
  • The main Q vs V convergence figure is one configuration. It comes from RoBERTa-large with rank 8 §3.2 fig. 2. The seed count for that figure is not stated in the main text; the appendix BERT figure says three seeds §7.3 fig. 4.
  • Theory is about full matrices. The Hessian argument concerns full projection matrices, not the low-rank factors that are actually trained §3.3. The convergence theorem depends on Assumption 4.3, which restricts LoRA factor norms and eigenvalues; these are assumptions, not checked in the experiments §4.2.
  • Privacy claims are limited. The paper says sharing fewer parameters is only a side benefit and makes no formal privacy claim §4.1. The DP table measures utility under a DP mechanism, not leakage §5.5 §7.10.
  • Multilingual split. The paper says each of three clients gets one language (English, German, Chinese) §5.2. The MLS config splits by domain fedlova_mls.yaml. I did not check how the split is implemented, so whether clients are split by language is unverified.
  • Costs the abstract hides. Each client still trains both query and value adapters, so local compute matches FedLoRA; only the upload is halved. The paper reports no local-compute or wall-clock numbers beyond rounds to target.

Lineage

  • Builds on: LoRA §3.1; the paper follows the LoRA paper's choice of query and value projections §3.2. Federated LoRA variants FFA-LoRA §2.2, FedSA-LoRA §2.2 and LoRA-FAIR §2.2 are compared as baselines. No cards for these exist in this wiki.
  • Compared against: FedLoRA, FFA-LoRA, FedSA-LoRA, LoRA-FAIR and FedLoQA tab. 1.
  • Superseded by: not known.

Source and license

Adapted from “FedLoVA: Value-Only Aggregation for Federated LoRA Fine-Tuning of Large Language Models” by Ensieh Khazaei, Baturalp Buyukates, Dimitrios Hatzinakos (arXiv:2611.10248), used under CC BY 4.0. Changes: condensed and restructured for implementation, with tables, equations and algorithms taken from the paper and its code, each cited to its source. Not affiliated with or endorsed by the authors. Code citations refer to https://github.com/EnsiehKhazaei/FedLoVA at commit a00d32c321fa (none (no LICENSE file in repository)). Any quoted code is shown under that license.