wikillm

Paper · 2026-10-06 · cs.LG, cs.AI

MemFLoRA: Memory-Floor LoRA for CNN Adaptation at the Edge

A LoRA-style 1-D CNN adapter: frozen rank-r down-projection, trained up-projection scaled by the frozen source BatchNorm, eval-mode BatchNorm, so backward keeps only rank-r state. On three HAR datasets and two backbones, saved activations need 98.5-98.7% less memory and peak training footprint drops 94.9-97.3% versus full fine-tuning (author-reported); mean macro-F1 is 0.827 vs 0.798 for LoRA-C.

Low-rank adaptation (LoRA and variants)On-device trainingBatch normalizationDomain adaptationon-device-traininglow-rank-adaptationcnnhuman-activity-recognitionmemory-efficient-training

arXivPDFCodeMarkdown18 min read

AI-compiled by claude-haiku-5-5 on 2026-11-21, not yet reviewed by a human. Every claim cites its source; check it when a detail matters.

In plain words

A small network that reads body-sensor signals, for example to recognise what activity a person is doing, often meets data that differs from its training data: another person, or a sensor worn in another place. Adapting the model on the device would fix this. The obstacle is memory. In a convolutional network, the big cost during training is not the number of weights being trained but the intermediate values that must be kept until the backward pass. Standard low-rank adapters train few weights, yet they still keep full-width intermediate values, so they save little memory.

MemFLoRA is built around that memory cost. One small projection is frozen at random, a second projection that maps back to full width is trained, the backbone's normalisation is fixed to statistics measured on the new data, and the sign of each activation is stored as a single bit. What must be kept then scales with the small adapter rank instead of the layer width.

The authors report that this cuts stored activation memory by about 98.5% to 98.7% and peak training memory by about 95% to 97%, compared with full fine-tuning, while accuracy on three activity-recognition datasets matches or exceeds other low-rank adapters on average. These figures are the authors' own measurements.

TL;DR

  • MemFLoRA is a LoRA-style adapter for 1-D CNN backbones (T-ResNet and MobileNetV2) on sensor windows. It is designed so that the backward pass keeps only rank-r state §3.1.
  • Recipe: freeze the down-projection P (Gaussian, variance 1/r), train the zero-initialised up-projection U, multiply U's output per channel by the frozen source-BN gain, use eval-mode BN, and keep only ReLU sign bits §3.1-§3.4 eq. 8.
  • Headline (author-reported): at rank 2 and batch 64, saved-activation memory falls 98.5% (T-ResNet) and 98.7% (MobileNetV2) versus full fine-tuning, and peak training state falls 94.9% and 97.3% tab. 3. Mean macro-F1 over the 18 backbone-dataset-rank settings is 0.827 for MemFLoRA; the two other low-rank baselines reach 0.798 and 0.777 tab. 1 §4.2.
  • MemFLoRA-SG also trains P, using rematerialised inputs and one extra forward pass; it reaches mean macro-F1 0.839 at roughly 104-106% of full fine-tuning MACs tab. 5.
  • Several inputs an implementer needs are not in the paper (input normalisation, channel sets, labels, MobileNetV2 input layout and initialisation). The repository fixes them; see Pitfalls.

Problem

On-device adaptation is needed when a deployed model meets user-, sensor- or environment-specific shifts §1. Full fine-tuning updates all parameters and costs memory, compute and energy that embedded systems may not have §1. Parameter-efficient methods reduce gradient and optimiser memory but leave the saved-activation term essentially unchanged, so they do not reduce training memory in proportion to their parameter savings §3.1 eq. 1. For edge CNNs the saved activations dominate §1 §2.2.

A conventional low-rank convolutional adapter updates both of its factors. Its saved adapter input then stays full width, the same order as full fine-tuning §3.1. The paper's target setting is sensor-based human activity recognition where the target person, the body site or the sensor position differs from the source, with a budget of 50 adaptation steps §4.

Method

Memory floor criterion §3.1. Training memory is split into weights, gradients, optimiser state, saved tensors and temporary buffers eq. 1. For a conv layer with input width C and length T, the saved tensors have three parts: the ReLU state, the train-mode BatchNorm input, and the input of every convolution, including the frozen backbone §3.1. The paper defines the floor as persistent saved state of order B·r·T for trainable backward computation, independent of C. The ReLU gate is kept as a bit-packed mask §3.1 fig. 1.

Eval-mode BatchNorm §3.1 eq. 2. With frozen statistics, BN is a fixed per-channel affine map with gain a = γ / sqrt(σ² + ε) and shift b = β − a·μ. Its backward pass saves no activation §3.1.

Frozen down-projection and trainable up-projection §3.1 §3.2. The adapter for each convolution is Δ = U(P x), with P frozen and U trainable [eq. 3]. If P were trainable, its gradient would need the full-width input x. With P frozen, U needs only the rank-r bottleneck q = P x §3.1.

Fused forward §3.2 eq. 4 eq. 5. Each Conv-BN-activation site is one fused operator. The adapter output is added to the BN output before the activation eq. 4. Because BN in eval mode is affine, adding the adapter before BN gives the same result as adding a·Δ after BN eq. 5.

Scale matching §3.3 eq. 6 eq. 7 eq. 8. The bottleneck q is a random projection with uncontrolled per-channel magnitudes, so the raw adapter branch is not on the same scale as the normalised backbone term eq. 6. The paper compares two fixes: a trainable BatchNorm on the bottleneck eq. 7, and inheriting the frozen source gain a eq. 8. It adopts the second, because it adds no batch-statistic noise and no saved state. A raw-scale adapter fails to adapt at all in the ablation §3.3 tab. 4.

Convolutional geometry §3.4 eq. 9 eq. 10. P is a frozen 1×1 projection to rank r and U carries the k-tap spatial kernel eq. 10. The alternative places the frozen kernel in P and a 1×1 trainable U eq. 9. The paper adopts the trainable-kernel form. It costs a factor k in trainable parameters, r·C_out·k, but leaves the saved state at rank r §3.4.

Initialisation §3.4 eq. 11. P is Gaussian with variance 1/r, which makes each bottleneck channel have the same expected variance. U is zero, so adaptation starts exactly from the source model §3.4.

Projection-gradient recovery, MemFLoRA-SG §3.5 eq. 12. The gradient of P is the sum over batch and time of g·xᵀ, where g = ∂L/∂q eq. 12. SG keeps only the rank-r gradient g. It then replays the adapted network without autograd and contracts each rematerialised input x with g, layer by layer. The persistent state stays rank-bounded. The price is a second forward pass, plus one full-width temporary per layer that the peak-memory accounting includes §3.5.

Pseudocode
# MemFLoRA adapted Conv-BN-ReLU site  [§3.2] [eq. 8]
# Input: x (B, C_in, T); frozen W0; frozen P (r, C_in, 1); trainable U
# Output: y (B, C_out, T)

def adapted_site_forward(x):
    # Frozen convolution, then source BN with frozen statistics  [§3.1]
    base = bn_eval(conv(x, W0))             # (B, C_out, T)
    # Frozen 1x1 projection to rank r; the only input-dependent tensor
    # kept for U's gradient  [§3.1] [§3.4]
    q = conv1x1(x, P)                       # (B, r, T)
    # Trainable up-projection, zero at initialisation  [§3.4] [eq. 10]
    delta = conv_k(q, U)                    # (B, C_out, T)
    # Scale each output channel by the frozen source BN gain  [§3.3] [eq. 8]
    delta = a_source * delta
    # Add, then ReLU; only the sign bits are kept for the backward pass
    return relu(base + delta)               # [§3.1]
Pseudocode
# MemFLoRA-SG: exact gradient for the frozen projection P  [§3.5] [eq. 12]
# Input: x_batch; g = dL/dq captured at each site in the main backward
# Effect: P.grad gets the sum over batch and time of g times x transposed

def accumulate_projection_grad(model, x_batch):
    # Replay the adapted forward with autograd off, before the optimiser
    # step, so every site sees the same parameters  [§4] [§3.5]
    with no_grad():
        model(x_batch)
    # At each site, contract the rematerialised input with the captured g,
    # add the result to P.grad, then release the input  [§3.5]

Hyperparameters

Values are as stated in the paper unless marked "code" in the last column. Code citations are relative to the repository at code_commit.

NameValueWhereNotes
Adaptation optimiserAdam§4Code uses torch Adam with betas (0.9, 0.999), eps 1e-8 benchmark_training.py
Adaptation learning rate1e-3§4 01_main_accuracy_opportunity_t_resnet.shSame value is the default for Full FT benchmark_cli.py
Adaptation weight decay5e-4§4 benchmark_cli.pyCoupled L2 in torch Adam, not AdamW
Adaptation steps50§4One step = one Adam update on one mini-batch of 64 target windows benchmark_training.py
Batch size64§4 01_main_accuracy_opportunity_t_resnet.shUsed for pretraining, adaptation and AdaBN calibration
Seeds20§4 01_main_accuracy_opportunity_t_resnet.shMean and sample SD reported
Ranks r2, 4, 8§4 tab. 1
Projection P1×1, frozen, Gaussian, variance 1/r (std 1/√r)§3.4 eq. 11 bnpa_conv_bn_act_2d.pyBias off
Up-projection Uk×k, trainable, zero init, no bias§3.4 eq. 10 bnpa_conv_bn_act_2d.pyTakes the base conv's stride and padding (see Pitfalls)
Output scale1 (fixed)§3.2 bnpa_conv_bn_act_2d.pyThe α/r factor is commented out in the paper
Bottleneck BatchNormoff for MemFLoRA§3.3 methods.pyOn in the ablation and in other variants
Source-BN gain aγ / sqrt(σ² + ε), per channel, frozen§3.2 eq. 2 eq. 8
BN mode during adaptationeval, frozen statistics§3.1 benchmark_training.py
AdaBN calibration1 batch of target adaptation data§4 tab. 4 01_main_accuracy_opportunity_t_resnet.shMode ema_no_reset; the paper reports AdaBN as having a small effect §4.3
Pretraining epochs20§4Checkpoint kept at lowest validation loss (see Pitfalls) minimal_methods_benchmark.py
Pretraining optimiserAdam, lr 1e-3§4 ("all methods use Adam ... 0.001")Pretraining weight decay 5e-4 is not stated in the paper benchmark_cli.py
Window size / strideOpportunity 60 / 30; RealWorld 500 / 250; RealDisp 250 / 125tab. 2Opportunity code default is 90 / 45 opportunity.py
Sampling rate30 Hz (Opportunity), 50 Hz (RealWorld, RealDisp)tab. 2
Target split80% adaptation, 20% test§4Method differs by dataset (see Pitfalls)
Input channelsOpportunity 97; RealWorld imu9 default; RealDisp imu13 defaultnot stated in paperbenchmark_cli.py
Input normalisationnonenot stated in paperbenchmark.py
Opportunity labelscolumn ml_both_arms, majority rule per windownot stated in paper01_main_accuracy_opportunity_t_resnet.sh
MobileNetV2 initialisationImageNet weights, on by defaultnot stated in paperbenchmark_cli.py torchvision_mobilenet_v2.py
ActivationReLU (T-ResNet); ReLU6 also supported by the fused code§3.1 _common.py
SG projection optimiserAdam, lr 1e-2, weight decay 0, step every updatenot stated in papercommon.sh Class default is 1e-3 bnpa_sg.py
SG replaysame mini-batch, unchanged parameters, before the optimiser step§4

Results

Author-reported. Setup: two backbones (T-ResNet with 64 base feature maps, MobileNetV2), three HAR datasets with a shift each (Opportunity: subject; RealWorld: body location; RealDisp: sensor placement), ranks 2, 4 and 8, 50 adaptation steps, macro-F1 as mean ± sample standard deviation over 20 seeds §4 tab. 1. Dataset details as the paper gives them:

DatasetShiftSubjectsActivitiesHzWindow / stride
OpportunitySubject4173060 / 30
RealWorldBody location15850500 / 250
RealDispSensor placement173350250 / 125

tab. 2

Rank-independent baselines tab. 1. Zero shot, Full FT, Bias-tuning and BN-tuning do not depend on rank.

DatasetBackboneNo adaptationFull FTBias terms onlyBN terms only
OpportunityMobileNetV20.585 ± 0.0880.820 ± 0.0250.632 ± 0.0540.668 ± 0.043
OpportunityT-ResNet0.511 ± 0.0870.847 ± 0.0260.655 ± 0.0540.684 ± 0.052
RealWorldMobileNetV20.610 ± 0.0870.911 ± 0.0260.751 ± 0.0670.825 ± 0.049
RealWorldT-ResNet0.449 ± 0.1240.903 ± 0.0280.554 ± 0.1340.631 ± 0.125
RealDispMobileNetV20.791 ± 0.0000.894 ± 0.0130.835 ± 0.0030.865 ± 0.005
RealDispT-ResNet0.526 ± 0.0000.891 ± 0.0090.599 ± 0.0050.620 ± 0.005

tab. 1

Rank-dependent methods tab. 1.

DatasetBackboneRankLoRA-CLoRA-EdgeMemFLoRAMemFLoRA with SG
OpportunityMobileNetV220.698 ± 0.0370.730 ± 0.0320.747 ± 0.0250.758 ± 0.028
OpportunityMobileNetV240.728 ± 0.0320.760 ± 0.0270.764 ± 0.0280.774 ± 0.024
OpportunityMobileNetV280.756 ± 0.0280.789 ± 0.0250.771 ± 0.0280.785 ± 0.023
OpportunityT-ResNet20.746 ± 0.0440.710 ± 0.0520.768 ± 0.0460.773 ± 0.040
OpportunityT-ResNet40.773 ± 0.0390.746 ± 0.0460.793 ± 0.0340.797 ± 0.035
OpportunityT-ResNet80.801 ± 0.0300.774 ± 0.0380.812 ± 0.0330.815 ± 0.031
RealWorldMobileNetV220.802 ± 0.0440.817 ± 0.0460.874 ± 0.0380.882 ± 0.035
RealWorldMobileNetV240.846 ± 0.0370.866 ± 0.0350.886 ± 0.0350.892 ± 0.034
RealWorldMobileNetV280.880 ± 0.0320.889 ± 0.0340.896 ± 0.0340.901 ± 0.032
RealWorldT-ResNet20.689 ± 0.0960.632 ± 0.1080.826 ± 0.0650.847 ± 0.052
RealWorldT-ResNet40.779 ± 0.0670.717 ± 0.0850.859 ± 0.0470.871 ± 0.044
RealWorldT-ResNet80.847 ± 0.0430.799 ± 0.0560.875 ± 0.0450.883 ± 0.040
RealDispMobileNetV220.882 ± 0.0050.892 ± 0.0040.891 ± 0.0040.896 ± 0.004
RealDispMobileNetV240.890 ± 0.0050.898 ± 0.0040.888 ± 0.0100.897 ± 0.006
RealDispMobileNetV280.897 ± 0.0030.903 ± 0.0060.885 ± 0.0100.895 ± 0.007
RealDispT-ResNet20.738 ± 0.0090.631 ± 0.0050.742 ± 0.0170.768 ± 0.015
RealDispT-ResNet40.787 ± 0.0100.686 ± 0.0040.787 ± 0.0100.815 ± 0.009
RealDispT-ResNet80.827 ± 0.0100.742 ± 0.0060.828 ± 0.0110.847 ± 0.011

tab. 1 The paper's text gives the averages over these 18 settings as 0.827 (MemFLoRA), 0.798 (LoRA-C), 0.777 (LoRA-Edge) and 0.839 (MemFLoRA-SG) §4.2. Recomputing from the table gives the same four values. MemFLoRA is not uniformly ahead of LoRA-Edge: on RealDisp with MobileNetV2 it is lower at all three ranks (for example 0.885 against 0.903 at r = 8) tab. 1.

Memory at rank 2 tab. 3 fig. 2. Entries are peak / saved-activation memory in MB, with the optimiser-state column given separately.

BackboneMethodB = 1B = 8B = 32B = 64Opt. [MB]
T-ResNetFull FT9.02 / 0.9013.84 / 7.0935.08 / 28.3363.40 / 56.654.49
T-ResNetLoRA-C5.51 / 3.1011.53 / 9.1232.20 / 29.7959.76 / 57.350.10
T-ResNetLoRA-Edge2.93 / 0.607.05 / 4.7221.19 / 18.8640.04 / 37.710.02
T-ResNetMemFLoRA2.44 / 0.022.50 / 0.112.81 / 0.423.23 / 0.830.08
T-ResNetMemFLoRA-SG2.50 / 0.022.78 / 0.113.74 / 0.425.01 / 0.830.10
MobileNetV2Full FT43.20 / 16.12109.00 / 81.91346.93 / 319.85666.64 / 639.5617.97
MobileNetV2LoRA-C27.17 / 17.6190.00 / 80.44305.40 / 295.84592.61 / 583.050.30
MobileNetV2LoRA-Edge17.09 / 7.6669.77 / 60.34250.39 / 240.96491.22 / 481.790.16
MobileNetV2MemFLoRA9.63 / 0.2010.54 / 1.1113.66 / 4.2317.83 / 8.400.16
MobileNetV2MemFLoRA-SG9.91 / 0.2011.73 / 1.1117.97 / 4.2326.30 / 8.400.16

At B = 64 the reported reductions relative to Full FT are 98.5% and 98.7% in saved activations, and 94.9% and 97.3% in peak state, for T-ResNet and MobileNetV2 respectively. For SG they are 92.1% and 96.1% in peak state §4.1 (recomputed from this table: 3.23/63.40, 17.83/666.64 and 5.01/63.40, 26.30/666.64). The paper states that MemFLoRA at B = 64 is about the same peak as LoRA-Edge at B = 1, so the same memory budget supports a 64 times larger batch §4.1.

Design ablations, T-ResNet on Opportunity, 50 steps tab. 4. Entries are mean macro-F1 over target subjects. The paper says they come from an independent ablation sweep, and that differences from tab. 1 are within seed-level variation §4.3.

FactorVariantr = 2r = 4r = 8
InitNormal P0.7590.7770.802
InitOrthogonal P0.6730.7090.751
Geometry1×1 down, k×k up (adopted)0.7600.7770.802
Geometryk×k down, 1×1 up0.7310.7550.773
ScaleNone0.2000.1970.187
ScaleBottleneck BN (BN_r)0.7280.7720.805
ScaleSource gain a (adopted)0.7640.7920.808
AdaBNOff0.7530.7790.804
AdaBNOn0.7620.7830.801

tab. 4

Training cost at rank 2, Opportunity tab. 5 §4.3. Par. is the number of trainable parameters; MACs are AdaBN calibration plus 50 steps; steps to 85% counts the first checkpoint at which a run reaches 85 percent of the Full FT final score.

BackboneMethodPar. [10³]MACs [10⁶]Relative cost [%]Steps to 85%
T-ResNetFull FT560.9320.0——
T-ResNetLoRA-C12.8320.210022
T-ResNetLoRA-Edge2.3218.66824
T-ResNetMemFLoRA10.6222.57019
T-ResNetMemFLoRA-SG10.6331.710415
MobileNetV2Full FT2250403.0——
MobileNetV2LoRA-C37.1390.99732
MobileNetV2LoRA-Edge19.8283.27024
MobileNetV2MemFLoRA19.8286.07124
MobileNetV2MemFLoRA-SG19.8426.010623

tab. 5

Jetson Orin Nano (8 GB), rank 2, B = 64 tab. 6 tab. 7 fig. 4 §4.4. Table 6 gives peak training-state storage / saved-backward state in MB; the paper says the device profiler deduplicates storage and tracks lifetimes, so these figures differ slightly from tab. 3.

MethodT-ResNetMobileNetV2
Full FT54.52 / 47.77666.72 / 639.55
LoRA-C52.37 / 49.96592.61 / 583.04
LoRA-Edge35.12 / 32.79491.21 / 481.78
MemFLoRA3.22 / 0.8317.83 / 8.39

tab. 6

Peak requested CUDA memory including inputs, temporaries and workspaces, in MB tab. 7: Full FT 66.20 / 673.39; LoRA-C 64.08 / 601.45; LoRA-Edge 44.80 / 503.32; MemFLoRA 19.43 / 95.43 (T-ResNet / MobileNetV2). The paper attributes MemFLoRA's peak to transient buffers of the highest-resolution early layers in the forward pass, not to retained state fig. 4 §4.4.

Figures, in words. fig. 1 is the computation and saved-state diagram of one adapted site: the backbone branch and the adapter branch are added, and only the bottleneck and the ReLU mask are retained. fig. 2 is a stacked bar chart of peak training memory for MobileNetV2 at rank 2, batch 64, on Opportunity; the values in its description match tab. 3. fig. 3 shows adaptation trajectories at rank 2; the paper's text says the MemFLoRA curve closes in on Full FT as the steps increase, and SG is generally strongest among low-rank methods §4.2. Its description covers 50 to 250 steps, although the protocol in tab. 1 uses 50 steps. fig. 4 is the requested-memory timeline for one MobileNetV2 update.

Paper vs code

Code citations give file names only, in the repository at code_commit (MIT licence). Folders, relative to the repository root: src holds methods.py; src's adapters folder holds bnpa_conv_bn_act_2d.py, bnpa_sg.py, common.py and lora_edge.py; models holds adapter_injection.py and torchvision_mobilenet_v2.py; data holds benchmark.py, splits.py, realworld.py, realdisp.py and opportunity.py; experiments holds benchmark.py and minimal_methods_benchmark.py; scripts holds the 01_.sh and 02_*.sh runs and common.sh. The adapter, training loop, data split and the headline scripts were read directly. Paper numbers were not re-run.

  • MemFLoRA variant matches the paper. The headline method is bnpa_fa_postbn_bnr_off_scaled: 1×1 frozen P, k×k trainable U, bottleneck BN off, source-gain scaling after BN methods.py. The parameter setup matches the paper's description bnpa_conv_bn_act_2d.py.
  • Bottleneck BN flag is overridden. Script 01 passes --bnpa-bottleneck-bn on, but the method spec sets BN off, and the spec wins adapter_injection.py 01_main_accuracy_opportunity_t_resnet.sh. The flag therefore does not affect MemFLoRA.
  • SG runs use the pre-BN method. The SG script uses bnpa_q3_sg, which adds the adapter before BN methods.py bnpa_conv_bn_act_2d.py. In eval mode this is algebraically the same as adding a·Δ after BN eq. 5, so it matches the paper's form. The paper does not name the method used for MemFLoRA-SG.
  • Scale is fixed at 1. Forward uses scale 1.0; the α/r factor is absent bnpa_conv_bn_act_2d.py. This matches the paper, where the factor is commented out.
  • Initialisation. The paper writes N(0, 1/r). The code draws with standard deviation 1/√r, which is variance 1/r, so the two agree bnpa_conv_bn_act_2d.py. U is zero-initialised bnpa_conv_bn_act_2d.py.
  • Pretraining checkpoint is not the last epoch. The paper says the source model is pretrained for 20 epochs. The code keeps the epoch with the lowest validation loss on a 20% held-out split of source windows benchmark.py benchmark_training.py.
  • Sensitivity of SG optimiser. The paper gives no learning rate for P. The headline script sets Adam with lr 1e-2 and no weight decay for P, while U uses 1e-3 common.sh 01_main_accuracy_opportunity_t_resnet.sh.
  • Baseline implementations are the authors'. Script 01 runs lora_edge_optimized and lora_c 01_main_accuracy_opportunity_t_resnet.sh. The optimised LoRA-Edge variant saves the frozen tensor-train suffix projection rather than the input lora_edge.py. The paper's baseline numbers therefore come from these re-implementations, which the paper describes as activation-aware §4.
  • MobileNetV2 input layout is undocumented in the paper. A HAR window is treated as a one-channel image and repeated to three channels, and the network starts from ImageNet weights by default torchvision_mobilenet_v2.py torchvision_mobilenet_v2.py torchvision_mobilenet_v2.py benchmark_common.py.
  • Adapted sites on T-ResNet. Each block adapts conv8, conv5, conv3 and its shortcut convolution, giving 11 sites. Block-level BN layers before the convolutions are set to eval mode but not adapted adapter_injection.py adapter_injection.py. The paper says "all convolutional layers in T-ResNet" §4.
  • MobileNetV2 adapter placement. Pointwise-only placement matches the paper's statement that pointwise convolutions are adapted adapter_injection.py 02_main_accuracy_opportunity_mobilenet_v2.sh.
  • Data split differs by dataset. Opportunity uses a per-subject stratified random split benchmark.py splits.py. RealWorld and RealDisp use chronological target splits realworld.py realdisp.py. The paper states only the 80/20 ratio §4.

Pitfalls

  • Input normalisation is not stated and the code applies none. The data path records normalization: none benchmark.py. The paper says the bottleneck magnitudes are uncontrolled and must be scale-matched to the backbone term §3.3. My inference, not the paper's: normalising the sensor inputs would change those magnitudes. Keep the inputs raw to match the reported runs.
  • Channel sets are not stated. Opportunity uses 97 channels benchmark_cli.py. RealWorld defaults to the imu9 feature set and RealDisp to imu13 benchmark_cli.py benchmark_cli.py. The paper's Table 2 does not list these, so results are not reproducible from the paper alone.
  • Opportunity labels. The reported runs use the label column ml_both_arms with a majority rule for each window 01_main_accuracy_opportunity_t_resnet.sh. The paper gives neither.
  • Opportunity window defaults differ from the paper. Code defaults are 90 and 45 samples opportunity.py. The reported runs pass 60 and 30, which match the paper's Table 2. Pass them explicitly.
  • MobileNetV2 starts from ImageNet weights by default, and the HAR input is one channel repeated to three benchmark_cli.py torchvision_mobilenet_v2.py. Training from scratch or using another input layout gives different numbers, and the paper does not say which was used.
  • ReLU versus ReLU6. The paper speaks of ReLU throughout. The fused code also handles ReLU6, which clamps to [0, 6] _common.py _common.py. Check the activation of each site when porting.
  • Stride placement of the adapter. P is always stride 1 and U carries the base convolution's stride and padding bnpa_conv_bn_act_2d.py. The paper does not say where the stride goes. The adapted layers in the reported backbones have stride 1, so the choice does not affect these results. It matters for other backbones.
  • Adam weight decay is coupled L2. The paper says "weight decay 0.0005". The code uses torch's Adam, which adds the decay to the gradient. AdamW would behave differently benchmark_training.py.
  • Step counting. A step is one Adam update on one mini-batch of 64 windows. Accuracy is evaluated only after the 50th step in the headline script 01_main_accuracy_opportunity_t_resnet.sh benchmark_training.py.
  • AdaBN uses the adaptation data. The calibration batch comes from the shuffled target adaptation loader and is also used for adaptation, with mode ema_no_reset benchmark_adabn.py 01_main_accuracy_opportunity_t_resnet.sh. The paper says no test data is used for adaptation or calibration §4. Calibration with zero batches leaves source statistics in place benchmark_adabn.py.
  • SG costs. SG needs the gradient captured at each site during the main backward pass, and a replay forward under no_grad before the optimiser step. BN buffers are snapshotted and restored during the replay bnpa_sg.py bnpa_sg.py. In MACs, SG costs 104% (T-ResNet) and 106% (MobileNetV2) of full fine-tuning, against about 70% for MemFLoRA tab. 5.
  • Results are not uniformly better than LoRA-Edge. The abstract says "matching or exceeding" CNN PEFT baselines. Averages favour MemFLoRA, but per-setting results do not (see tab. 1 for RealDisp with MobileNetV2).
  • Memory accounting is method-specific. The paper's tab. 3 counts executed autograd save references and models some components. The Jetson measurements use a device profiler that deduplicates storage. They give 98.3% and 98.7% saved-state reductions and 94.1% and 97.3% peak reductions, against 98.5%, 98.7%, 94.9% and 97.3% in tab. 3 §4.4. Quote the source that matches your measurement method.
  • Equation, table and figure numbers were counted from the LaTeX source. The compiled PDF was not available. Check each number against the PDF before citing it outside this card.

Lineage

  • Builds on: LoRA [Hu2021LoRALA] and LoRA-FA, which keeps the down-projection fixed and updates only the up-projection [zhang2023lorafa] §2.1. The paper extends the asymmetric-trainability idea to CNN saved-state constraints §2.1.
  • Activation-memory context: TinyTL [NEURIPS2020_81f7acab], MCUNetV3 [Lin2022OnDeviceTU], POET [Patil2022POETTN], gradient checkpointing [Chen2016TrainingDN] (also used by MemFLoRA-SG) and activation compression methods §2.2.
  • Compared against: LoRA-C [Ding2024LoRACPF], LoRA-Edge [kwak2025loraedge] (the protocol is taken from LoRA-Edge §4), Full FT, Bias-tuning [ben-zaken-etal-2022-bitfit], BN-tuning and TinyTL §4.3.
  • Test-time normalisation: AdaBN [LI2018109] §2.1 §4.3.
  • Superseded by: not stated.

Source and license

Adapted from “MemFLoRA: Memory-Floor LoRA for CNN Adaptation at the Edge” by Mehmet Emre Akbulut, Johannes Geier, Ulf Schlichtmann (arXiv:2610.08669), used under CC BY 4.0. Changes: condensed and restructured for implementation, with tables, equations and algorithms taken from the paper and its code, each cited to its source. Not affiliated with or endorsed by the authors. Code citations refer to https://github.com/mehmetemreakbulut/MemFLoRA at commit 1efcf1ff0c55 (MIT). Any quoted code is shown under that license.