Multi-scale feature fusion

Overview

A convolutional backbone produces maps at several resolutions. Shallow maps keep fine spatial detail but carry weak semantics, so they are poor at classifying small objects. Deep maps are semantically strong but coarse. A multi-scale detector predicts from several of these maps. Without fusion, each map is used on its own, so a small object is judged from a map that has little context about it 1712.00960 §1.

The invariant most variants keep: the features that reach a prediction head combine several source layers, or the head reads a level chosen for the object sizes it must find. In detection, the heads are SSD-style in FSSD, YOLO-style in YOLO-Z and YOLO5Face, and a shared attention head in Dynamic Head. The variants differ in four choices. The first is which layers to use. The second is how to bring them to a common size and width. The third is how to merge them: by concatenation, by sum, by a product, or by a learned gate. The fourth is how to build the prediction levels from the merged features. Top-down pyramids, which merge layers one step at a time, are the main alternative the FSSD paper argues against 1712.00960 §1, §3.1. The decoder and skip-path variants below apply the same idea to segmentation, animation, change detection, infrared target detection, concealed-object detection and deblurring. Several papers fuse along other axes: AVOD fuses two sensor views for each anchor, D3Net fuses RGB and depth streams, the event-camera paper fuses warped frame features with events-driven flow, and TimeMixer fuses pooled copies of a time series. One DDOD variant changes only the loss across levels and leaves the features alone; it is listed under Other variants.

Variants

Single-step fusion followed by a pyramid (FSSD)

Three source layers of a VGG16 single-shot detector (conv4_3, fc7 and conv7_2) each pass through a 1x1 convolution to 256 channels. The two coarser maps are resized bilinearly to the finest grid (38x38 for a 300x300 input). The three maps are concatenated along the channel axis, then batch-normalised. The result is the single fused map. A stack of 3x3 convolution and ReLU blocks, each down-sampling or not, builds the pyramid, and the SSD-style heads read every level 1712.00960 §3.1, §4.2. The matching, loss and augmentation stay those of SSD 1712.00960 §3.2. Card: FSSD: Feature Fusion Single Shot Multibox Detector.

Reported results, author-reported, FSSD300 on VOC2007 test: 78.8 mAP with VOC07+12 training against 77.2 for SSD300, and 82.7 after COCO pre-training 1712.00960 tab. 3. Speed is 65.8 FPS against 85 FPS for SSD300 on one 1080 Ti 1712.00960 tab. 3.

Replacing the lowest-resolution map with a higher-resolution one (YOLO-Z)

YOLO-Z starts from the four YOLOv5 scales and changes their feature maps. Its variant XS_inc adds a higher-resolution map and keeps all original maps. Its variant XS_ex drops the lowest-resolution map and feeds the higher-resolution one to the neck and head in its place. Anchors in the head are adjusted to the new map resolutions. The paper does not give the resolutions of the maps 2112.11798 §3.3.3 fig. 5. Card: YOLO-Z: Improving small object detection in YOLOv5 for autonomous vehicles.

In the per-change table, XS_ex alone gives the largest small-object gain at each scale, from 3.84% to 6.96% relative to the YOLOv5 baseline of the same scale 2112.11798 supp. tab. 1 §4.3. At X, XS_inc is worse than the baseline on all objects [supp. tab. 1]. The full YOLO-Z models also change the backbone and neck. They gain 5.46% to 7.73% relative on small objects, at 0.9 to 3.7 ms more inference time on one GTX 1080 Ti with batch size 1 2112.11798 tab. 2. The combination is not always better than its best part: at S, the full model scores 0.925 on small objects against 0.9295 for S_XS_ex alone tab. 2 [supp. tab. 1].

The neck was also changed. The default PAN-style neck is simplified to an FPN, and a bi-FPN neck is tested as well 2112.11798 §3.3.2. Neither neck is better at every scale. On small objects, FPN is ahead only at S (0.894 against 0.889), and bi-FPN is ahead at M, L and X [supp. tab. 1]. The gaps are at most about 1.3 points, from single runs with no variance reported.

The study uses one dataset of traffic cones seen from a racing car, with about 4,000 images. The authors say the feature-map gains may depend on its high density of small objects, and they call for tests on other datasets 2112.11798 §3.2 §4.3 §4.7.

Adding a coarse output for large objects (YOLO5Face)

YOLO5Face adds an extra detection output at stride 64, called P6, for large faces 2105.12931 §3.6. It is coarser than the other outputs, so it adds a level at the top of the pyramid rather than a finer one. The P6 input shape must be a multiple of 64 on the shorter edge §3.1. Card: YOLO5Face: Why Reinventing a Face Detector.

In the ablation on YOLOv5s, adding P6 raises Easy from 94.31 to 95.29 and Medium from 92.52 to 93.61. Hard stays flat, at 83.15 against 83.13 2105.12931 tab. 2. The cost is 12.386M parameters and 6.28 GFLOPs, against 7.075M and 5.751 GFLOPs without P6 tab. 2. The paper does not isolate P6 on the larger models in the ablation.

Learned gates over levels in a unified head (Dynamic Head)

Dynamic Head stacks blocks that treat the pyramid as one tensor with three axes: levels, positions and channels. Each block applies three attentions in sequence. The scale-aware attention pools each level and turns the result into one gate per level, so the head can prefer a level for each input 2106.08322 §3.2 eq. 3. The spatial-aware attention samples a few deformable points per level and weights them §3.2 eq. 4. The task-aware attention switches each channel between two linear functions §3.2 eq. 5. The blocks replace the usual head of a detector such as ATSS §3.3. Card: Dynamic Head: Unifying Object Detection Heads with Attentions.

On ResNet-50 at the 1x schedule, evaluated on val2017, each attention alone gives 39.9 AP for the level gate, 41.4 for the spatial attention and 40.3 for the channel switch. The baseline is 39.0 and all three together reach 42.6 2106.08322 tab. 1. Six blocks give 42.6 AP and eight give 42.5 tab. 2. Six blocks add 21.50 GFLOPs to a 254.39 GFLOP baseline tab. 2. Gains on other detectors are 1.2 to 3.2 AP §4.2 tab. 3. The released code computes the level gate from a neighbourhood of levels and uses a different hard-sigmoid from eq. 3, so match the code if you need its numbers 2106.08322 §3.2.

Class-token exchange between the outer levels (HiFormer)

HiFormer encodes a 2D medical image with two branches. A three-level CNN feeds a Swin Transformer pyramid through 1×1 convolution skips 2207.08518 §3.1.1. The Double-Level Fusion (DLF) module then acts on the two outer levels of the Swin pyramid only: the largest-map branch and the smallest-map branch §3.1 §3.1.3. Each outer level's class token is taken as the average of its normalised tokens eq. 2. It is projected to the other level's width and attends to that level's patch tokens, with the projected class token as the only query, so the cost is linear in the token count §3.1.3 eq. 3. A decoder then reads the patch tokens of both outer levels, sums them after upsampling and predicts the mask §3.2. Card: HiFormer: Hierarchical Multi-scale Representations Using Transformers for Medical Image Segmentation.

Author-reported, Synapse, average over eight organs: 80.39 DSC and 14.70 HD for HiFormer-B, against 79.13 and 21.55 for Swin-Unet 2207.08518 tab. 2. Removing DLF gives 77.15 DSC and 16.88 HD, which the paper reads as a 3.24 DSC gain from DLF tab. 6 §5. Changing the DLF depth, width or head count moves DSC between 77.95 and 79.48 across the rows of the configuration ablation tab. 8. The released forward pass does not send the class-token output to the decoder, so this gain cannot be traced to the path the paper describes without checking the code. See Contradictions.

Two-view feature pyramid with mean fusion (AVOD)

AVOD fuses two views of the same scene at each scale, not two scales of one view. It builds one pyramid from a bird's-eye-view LiDAR map and one from the RGB image. Each decoder upsamples by transposed convolution and, at each of three scales, concatenates the encoder feature of the same size and fuses it with a 3x3 convolution. The output keeps the input resolution and has depth 32 1712.02294 §3.2. For each anchor, the proposal network crops both maps, reduces each with a 1x1 convolution, and averages the two crops §3.3. The second stage does the same at 7x7 §3.4. Card: Joint 3D Proposal Generation and Object Detection from View Aggregation.

On the moderate validation set, the pyramid extractor raises pedestrian AP from 39.5 to 58.8 and cyclist AP from 41.6 to 49.7. Car AP moves from 74.1 to 74.4 1712.02294 tab. 3. The paper also reports recall gains of 10.1% for pedestrians and 8.6% for cyclists when image features are fused into the proposal network, against a BEV-only proposal network §4. Each number comes from one run. The released code adds a random branch drop during training that the paper does not describe, and it divides the mean by the number of branches that survive 1712.02294 §3.3.

Decoder fusion of the three deepest levels (HarDNet-MSEG)

HarDNet-MSEG drops the highest-resolution encoder features. The authors argue that shallow features are expensive and that the deeper ones already carry spatial detail 2101.07172 §3.2. It keeps the stride-8, 16 and 32 levels, passes each through a receptive-field block §3.2.1 fig. 4, and fuses them by upsampling and element-wise multiplication §3.2.2 fig. 5. Card: HarDNet-MSEG: A Simple Encoder-Decoder Polyp Segmentation Neural Network that Achieves over 0.9 Mean Dice and 86 FPS. The released code also concatenates features and applies 3x3 convolutions between the fusion steps. The paper does not describe that 2101.07172 §3.2.2.

Author-reported, Kvasir-SEG: 0.904 mean Dice under setup A at 86.7 FPS, and 0.912 under setup B at 312x312 2101.07172 tab. 1 tab. 2. The card records no ablation of the fusion operator or of the dropped stride-4 level, so it cannot rank this decoder against other fusion designs.

Coarse-to-fine reverse guidance over three levels (SINet-V2)

SINet-V2 finds concealed objects in two phases. Three levels of a Res2Net-50 encoder, at strides 8, 16 and 32, each pass through a texture-enhanced module. A neighbor connection decoder then merges the three levels. Each step upsamples the coarser level and multiplies it with the finer one, giving a coarse map at stride 8 2102.10274 §4.2 eq. 1. Each later stage takes the coarser prediction through a sigmoid and reverses it, so the guidance marks the regions the coarser stage has not explained §4.3 eq. 2. Each stage has three group-reversal attention blocks, which mix the guidance into the features in groups of 32, 8 and 1 channels §4.3 §5.3. Each stage outputs a refined map plus the upsampled coarser map eq. 6. Card: Concealed Object Detection.

Author-reported, on the COD10K test set: S-measure 0.815 against 0.789 for PraNet, the next-best baseline on that measure 2102.10274 tab. 2. In the ablation on CAMO, adding the neighbor connection decoder raises E-measure from 0.869 to 0.882, and the asymmetric texture module raises it from 0.866 to 0.882 tab. 6. Reversing the guidance in the first block only, the default, scores 0.815 S-measure on COD10K. Reversing it in the first two blocks gives 0.807, and in all three gives 0.810 tab. 6. Each row is one run.

Skip-path fusion in encoder-decoder networks

The variants in this group merge features on the skip paths of an encoder-decoder, where each decoder scale reads the encoder features of the same scale. Four of them come from segmentation tasks, one from image animation and one from infrared target detection. Their numbers are not comparable with the FSSD mAP figures.

Residual units in a U-shaped decoder (Deep ResUnet). At each decoder level the features from below are upsampled, concatenated with the encoder features of the same level, and passed through one residual unit. Cropping is not needed because input and output have the same size 1711.10684 §2.1.3 §2.3. Card: Road Extraction by Deep Residual U-Net. On the Massachusetts roads test set, the paper reports a break-even point of 0.9187 against 0.9053 for U-Net 1711.10684 tab. II. It does not ablate the skip concatenation, so the number measures the whole network.

CNN skips into a transformer decoder (TransUNet). A ResNet-50 stem gives feature maps at 1/8, 1/4 and 1/2 of the input, and the decoder concatenates each one into a successive upsampling block 2102.04306 §3.2 §4.4. The ablation has no numbers for zero or one skip. The text says that more skips generally help and that three work best §4.4 fig. 2. On Synapse multi-organ CT the paper reports 77.48% average DSC against 75.57 for the best CNN row in its table 2102.04306 tab. 1. The card flags that the per-organ values of that CNN row do not average to the printed mean, so compare with care. Card: TransUNet: Transformers Make Strong Encoders for Medical Image Segmentation.

Dynamic conditional encoding and FF-Parser (MedSegDiff). In a diffusion segmenter, the image feature map at each scale is merged with the feature map of the current noisy mask. Both maps are layer-normalised and multiplied element-wise into an affinity map, which then scales the image features 2211.00611 §2.1 eq. 3. The paper applies this in the two middle encoder stages, at every sampling step §2.1. FF-Parser places a learned filter in the Fourier domain on each skip path 2211.00611 §2.2 eq. 4 eq. 5 eq. 6. Card: MedSegDiff: Medical Image Segmentation with Diffusion Probabilistic Model.

The ablation, Dice (%), compares no conditioning with conditioning alone, and then with FF-Parser added. Conditioning alone gives 84.6 → 86.7 on optic cup, 88.2 → 89.8 on brain tumour and 84.1 → 85.9 on thyroid nodule. FF-Parser on top gives 87.5, 90.5 and 86.6 2211.00611 tab. 2. The model size for the ablation is not stated tab. 2. Inference draws 25 masks of 100 steps each, fused with STAPLE §3.2. By arithmetic on those stated numbers, that is 2,500 denoiser calls per image; the paper reports no runtime.

Shallow features gated by deep features (SANet). The decoder reads three Res2Net levels. The features of a deeper level are upsampled, passed through ReLU, and multiplied element-wise into the shallower features. The paper calls this a gate that filters background noise, not self-attention, and says it weights each level by its contribution 2108.00882 §3.2 eq. 1 eq. 2. The text does not say how the channel counts are matched for the product §3.2. Card: Shallow Attention Network for Polyp Segmentation.

Author-reported, mean Dice on Kvasir: 0.904 against 0.898 for PraNet, and on ColonDB 0.753 against 0.712 2108.00882 tab. 1. In the ablation on ColonDB, adding the gate to the backbone raises mean Dice from 0.676 to 0.728, and on Kvasir from 0.853 to 0.882 tab. 2. The first row is not described in the text. The released code has no such gate and no switch for it, so these rows cannot be regenerated from the release model.py.

Multi-resolution occlusion masks (image animation). The dense motion network predicts an occlusion mask in each decoder layer, so the masks cover several scales 2203.14367 §3.2. The inpainting network warps each encoder level by the optical flow, multiplies it by the mask of that resolution, and joins it to the decoder through a skip connection §3.2. Card: Thin-Plate Spline Motion Model for Image Animation.

On TaiChiHD, the paper adds the masks after dropout 2203.14367 tab. 4. L1 goes from 0.048 to 0.046 and AED from 0.156 to 0.150, but AKD rises from 4.66 to 4.73. The paper reads this as the masks helping L1 and AED while raising AKD §4.3. Its two auxiliary losses then bring AKD down to 4.57 tab. 4. All numbers are author-reported.

Dense nested skips with attention and pyramid output fusion (DNA-Net). For single-frame infrared small targets, this network arranges five levels in a grid of nodes. Each node takes every earlier node of its own level, the node one level deeper (upsampled) and the node one level shallower (downsampled). It concatenates them, applies two convolutions, then channel and spatial attention 2106.00487 §3.2 eq. 2. For the output, the top-row node of each level is projected, upsampled to full size, concatenated and refined by one more block §3.3 eq. 7. Card: Dense Nested Attention Network for Infrared Small Target Detection.

On NUDT-SIRST, a synthetic dataset, the paper's Table V gives IoU, Pd and Fa of 85.01, 96.50 and 8.521 for a plain skip, against 87.09, 98.73 and 4.223 for the full model (Pd and Fa at the target level, IoU per pixel) [arxiv:2106.00487 tab. V]. Left-to-right stacking gives 85.89, 97.29 and 4.649 tab. V. With the attention block removed, the table compares merging by concatenation with merging by element-wise sum: 85.90 against 85.25 IoU, and 5.738 against 6.710 Fa 2106.00487 tab. VI. Keeping only the last output layer, with no pyramid output fusion, gives 86.26, 96.84 and 6.033 tab. VII. Each ablation is a single run.

One U-Net with several input and output scales (MIMO-UNet)

MIMO-UNet keeps the coarse-to-fine idea of stacked deblurring networks inside one U-Net 2108.05054 §1. Three changes separate it from a plain U-Net. The blurry image is also fed at half and quarter size. Each smaller input is turned into features by a shallow module, then fused into the encoder by a feature attention module. That module multiplies the two features element-wise, applies a 3×3 convolution and adds the result back to the encoder feature §3.1 fig. 4(b). Each decoder level outputs an estimate at its own scale, made from one convolution and the blurry input at that scale §3.2 eq. 5. Asymmetric feature fusion gives each decoder level the resampled encoder features of all three levels §3.3 eq. 6. The paper gives this structure only in a figure fig. 4(c). Card: Rethinking Coarse-to-Fine Approach in Single Image Deblurring.

On GoPro, the three ways of merging the half and quarter inputs score 31.66 dB with concatenation, 31.60 dB with element-wise sum and 31.73 dB with the feature attention module 2108.05054 tab. 3. In the component ablation, the multi-output decoder adds 0.17 dB and asymmetric fusion adds 0.17 dB, both over a baseline of 31.16 dB. All three together add 0.30 dB tab. 4 §4.3. The frequency loss adds a further 0.27 dB tab. 4. It is a training loss, not a fusion step, and is listed here because the authors ablate it with the others. The final model reaches 31.73 dB with 6.8M parameters, against 32.66 dB for MPRNet, which takes 0.162 s on the authors' machine against 0.008 s for MIMO-UNet tab. 1. The released code adds an upsampling layer and a concatenation before the merge convolution in each decoder block. Equation 5 of the paper does not show these steps MIMOUNet.py MIMOUNet.py.

Per-level side outputs with a learned fusion (DVA)

A VGG16 encoder with its last pooling layer removed gives three side outputs at conv3-3, conv4-3 and conv5-3. Each has its own deconvolution decoder that goes back up to the input size 1705.02544 §3.3.1 §3.3.2. Each side output has a cross-entropy loss, and so does the fused output §3.2 eq. 7 eq. 8. The three side maps are merged by a learned fusion layer §3.2. Card: Deep Visual Attention Prediction. The model predicts eye fixations from one RGB image and is trained on SALICON only §3.3.3.

On the TORONTO ablation, the whole model reaches s-AUC 0.76 and CC 0.72. Without deep supervision, the fused loss alone gives 0.71 and 0.68. Averaging the side outputs in place of the learned fusion gives 0.72 and 0.68. Bilinear upsampling in place of the learned deconvolutions gives 0.74 and 0.70 [arxiv:1705.02544 tab. 9]. The ablation uses one dataset.

Three streams with a test-time depth gate (D3Net)

For RGB-D salient object detection, D3Net runs three VGG-16 feature-pyramid streams: RGB, RGB-D with depth as a fourth channel, and depth alone 1907.06781 §4.1. Inside each stream, the levels merge from coarse to fine. The coarse map is upsampled, the finer map is reduced by a 1x1 convolution to 32 channels, and the two are concatenated before 3x3 convolutions §4.1 fig. 6. At test time the depth depurator unit takes the mean absolute difference between the RGB-D map and the depth map. It keeps the RGB-D map if the difference is at most t = 0.15, and otherwise uses the RGB map §4.2 eq. 2 eq. 3. Card: Rethinking RGB-D Salient Object Detection: Models, Data Sets, and Large-Scale Benchmarks.

Author-reported, NJU2K test: S-measure 0.900 against 0.879 for CPFP. On LFSD, CPFP is ahead on S-measure, with 0.828 against 0.825 1907.06781 tab. 4. In the ablation on SIP, the depth-gated model scores 0.860 and the RGB-D stream alone scores 0.862. The gated model is above the RGB stream alone at 0.831 and below the oracle upper bound at 0.872 tab. 5. Each stream is trained on its own, not jointly, in the released code train.py.

Splatted pyramids merged by a multi-level network (event decompression)

The model runs one U-Net per event window, and then decodes any number of times. It predicts per-pixel coefficients for K = 5 learned temporal basis functions, so each pixel's motion can be queried at any time 2312.00113 §3.1 eq. 2. A tri-plane synthesis gives a colour frame at a chosen time from the events and the first frame §3.2 eq. 3. RAFT estimates flow between the synthesized frame and the first frame §3.3 eq. 4. Feature pyramids and image pyramids of the first frame are softmax-splatted along both the motion flow and the latent flow. The splatted pyramids are concatenated with a pyramid of the synthesized frame, and a multi-level convolutional network merges them into the output §3.4 eq. 5. Card: Event-based Continuous Color Video Decompression from Single Frames.

Author-reported, BS-ERGB gap 1: 25.40 dB PSNR, against 23.41 dB for the best baseline in the table, RAFT (inpainted) 2312.00113 tab. 2. The ablation removes one branch at a time: synthesis-only gives 22.90 dB, motion-only gives 23.37 dB, and the full model gives 25.40 dB tab. 4. The card has no ablation that removes the pyramid splatting alone, so this table does not isolate the fusion step from the flow and synthesis branches.

FPN neck with PANet paths cut, three output scales (Edge YOLO)

Edge YOLO is a slimmed YOLOv4-style detector for vehicles. Its neck is a feature pyramid with top-down and bottom-up paths, using stride-2 sampling and lateral connections. Parts of YOLOv4's PANet are removed: the bottom-up path augmentation, adaptive feature pooling and the fully connected fusion. The neck shrinks from 61 layers to 17. Three heads read the 13×13, 26×26 and 52×52 maps 2205.14942 §III-B, p. 8–9 fig. 7 fig. 8. Card: Edge YOLO: Real-Time Intelligent Object Detection System Based on Edge-Cloud Cooperation in Autonomous Vehicles.

Author-reported, COCO2017 mAP@0.5: 0.473 for Edge YOLO against 0.657 for YOLOv4, with 25.27 MB against 245.78 MB and 26.6 FPS against 2.1 FPS on Xavier without TensorRT 2205.14942 tab. 5. The neck is changed together with the backbone and the training setup, and the card records no ablation of the neck alone, so the accuracy gap cannot be assigned to fusion. With TensorRT FP16, mAP@0.5 falls to 0.325 for Edge YOLO and 0.459 for YOLOv4 tab. 5.

Other variants

Top-down pyramids such as FPN merge layers in sequence. The FSSD paper compares against FPN in its tables, but those numbers are not recorded on this page, so it makes no ranking claim against FPN.

The YOLOv11 overview describes its detection neck as upsampling and concatenating features from several scales, with C3k2 blocks in place of C2f 2410.17725 §4.2. Card: YOLOv11: An Overview of the Key Architectural Enhancements. It reports no measurements of its own, so it does not show whether this fusion beats the earlier neck.

Per-level loss weights (DDOD). This variant does not fuse features. It weights the loss of each pyramid level by how many positive samples the level has received. The weight is 1.0 for the level with the most positives and 2.0 for the level with the fewest, with a linear ramp in between [arxiv:2107.02963 §4.3] eq. 3. Card: Disentangle Your Dense Object Detector. On ATSS, the linear ramp gives 39.8 mAP against 39.4 for constant weights, and a reversed ramp gives 37.6 2107.02963 tab. 2 §3.3. In the component ablation, the level weighting alone gives 40.0 against 39.4 2107.02963 tab. 4. The re-weighting schemes in its comparison table fall between 39.7 and 40.0 AP tab. 7. The card records no variance for these runs.

Edge features added to each object scale (EGNet)

A salient-edge branch builds edge features from the shallowest side output and the top-level location 1908.08297 §3.2.2 eq. 2. Each object-scale feature from the top-down path is then added to these edge features before a sub-prediction is made. The four sub-predictions are fused with weights §3.3 eq. 7 eq. 9. The paper gives no values or learning rule for those weights §3.3. Card: EGNet: Edge Guidance Network for Salient Object Detection.

The ablation trains on DUTS-TR and tests on SOD. The U-Net-style baseline gives a max F-measure of 0.851. Adding a top-down edge path gives 0.882, and the full model with one-to-one guidance gives 0.890 1908.08297 tab. 3. The variant named MRF_PROG in the ablation, which keeps the edge path but not the one-to-one guidance, also gives 0.882. These are salient-object numbers from one run each, not detection results. The released code merges the guided features at the feature level, not as the weighted sum of eq. 9 1908.08297 §3.3.

Per-scale differences with an MLP decoder (ChangeFormer)

For two dates of one scene, a shared transformer encoder gives four maps at 1/4 to 1/32 of the input 2201.01293 §2.1. Each scale has a learned difference module, which concatenates the two maps and applies a convolution, a ReLU and batch normalisation §2.1.3 eq. 5. The MLP decoder projects each difference to a common width and resizes it to 1/4 of the input. It then concatenates the four maps and fuses them with a linear layer §2.2.1 eq. 6 eq. 7 eq. 8. The paper has no ablation of this fusion against other choices, so the card gives no ranking. The released code instead adds the coarser fused map to each scale on the way down 2201.01293 §2.2. Card: A Transformer-Based Siamese Network for Change Detection.

Pooled scales with bottom-up and top-down mixing (TimeMixer)

This is a time-series forecaster, but it follows the same pattern. The past input is average-pooled into several shorter copies 2405.14616 §3.1. In each block, every scale is split into a seasonal and a trend part. Seasonal parts are mixed from fine to coarse and trend parts from coarse to fine §3.2 eq. 4 eq. 5. A linear predictor is then built for each scale and the predictions are summed §3.3 eq. 6. Card: TimeMixer: Decomposable Multiscale Mixing for Time Series Forecasting.

The ablations run on M4, PEMS04 and ETTm1 at prediction length 336 2405.14616 tab. 5. On M4 SMAPE, the official design scores 11.723. Removing the sum of per-scale predictors gives 12.503. Removing bottom-up seasonal mixing gives 13.051. Removing mixing across scales entirely gives 12.468. The dataset and metric differ from the detection results above, so these numbers do not transfer to vision.

When to use which

Memory use and training time are not reported on the FSSD card, so no claim is made about them.

Implementation sketch

The sketch follows the FSSD design. Layer choice, the target size, the width and the number of pyramid levels are configuration, not fixed by the method.

# Fuse several backbone maps into one map, then build the pyramid  [§3.1]
# Input:  source maps from the chosen layers
# Output: pyramid levels read by the same detection heads

def fuse_then_pyramid(sources, target_size, width, strides):
    # Reduce every source to a shared width with a 1x1 conv  [§4.2]
    reduced = [conv1x1(x, out_channels=width) for x in sources]
    # Resize every map to the finest grid, bilinear  [§3.1 "T", §4.2]
    sized = [resize(x, target_size) for x in reduced]
    # Concatenate along channels, then normalise  [§3.1 eq. 1] [§4.1.3]
    fused = batch_norm(concat(sized, axis=channels))
    # First pyramid level from the fused map  [§3.1 "φp"]
    levels = [conv3x3_relu(fused, stride=1)]
    # Each further level down-samples the previous one  [§4.2]
    for stride in strides:
        levels.append(conv3x3_relu(levels[-1], stride=stride))
    # Every level feeds the same SSD-style heads  [§3.1 eq. 3]
    return levels

Points where variants differ: the fusion operator (concatenation, element-wise sum, element-wise multiplication as in the HarDNet-MSEG decoder, a multiplicative gate on the skip paths, a ReLU-gated product of shallow and deep features as in SANet, a learned per-level weight as in Dynamic Head, class-token cross-attention between the outer levels as in HiFormer, or a reversed coarser prediction used as guidance in SINet-V2); whether normalisation is applied; the block that builds the levels; which levels are used at all; and the direction of mixing, which is bottom-up, top-down or both in TimeMixer. MIMO-UNet fuses at the input side as well, with a feature attention module on each smaller input. A YOLO-Z style change replaces the lowest level, and a P6 output adds one. AVOD adds a second view and averages the two crops. The FSSD ablations choose concatenation, batch normalisation and a simple block placed after the fused map 1712.00960 tab. 1, tab. 2. DVA adds a loss on each side output and fuses them with a learned layer. The event-decompression model warps pyramids with softmax splatting along flow before it merges them.

Contradictions

Between papers, no direct disagreement is recorded. The cards measure different tasks and datasets, so their numbers are not comparable. Within papers, both sides are given below; none is resolved here.

• FSSD gain over SSD300. The text reports 1.1 points on VOC2007 test 1712.00960 §4.2. The table gives 78.8 and 77.2, a gap of 1.6 1712.00960 tab. 3.
• FSSD SSD512 baseline. The text says the baseline improved "from 79.8% to 80.9%" 1712.00960 §4.2. The table lists SSD512 at 78.5 1712.00960 tab. 3.
• FSSD gain from batch normalisation. The text gives about 0.7 points 1712.00960 §4.1.3. The ablation rows give 78.6 and 78.1, a gap of 0.5 1712.00960 tab. 2.
• FSSD speed cost. The text says FSSD costs about 25% more time per image 1712.00960 §4.4. The table implies about 29%, from 85 and 65.8 FPS 1712.00960 tab. 3.
• FSSD speed hardware. The figure caption says speeds were measured on a Titan X. The FSSD speed in the table is given for a 1080 Ti 1712.00960 fig. 5 1712.00960 tab. 3. Compare speeds only within one hardware setup.
• FSSD initialisation. The method section names an SSD-trained model as the default starting point 1712.00960 §3.2. The experiments section starts VOC training from ImageNet VGG16 1712.00960 §4.2. The ablation gives 78.8 for VGG16 start and 78.6 for SSD start 1712.00960 tab. 2.
• FSSD learning schedule, paper vs released code. The paper gives a VOC learning rate of 0.001, reduced at 80k, 100k and 120k iterations 1712.00960 §4.2. The released FSSD script on the fssd branch uses 0.0005, reduced at 40k, 60k and 80k fssd_pascal.py. That code was read through a summary, so line numbers were not checked.
• MedSegDiff FF-Parser gain. The text says FF-Parser adds "near 1%" 2211.00611 §3.4. Table 2 gives 0.8, 0.7 and 0.7 points 2211.00611 tab. 2.
• MedSegDiff skip-path gating, paper vs code. The paper fuses with the eq. 3 affinity in two middle encoder stages 2211.00611 §2.1. The card infers that the paper's model is the UNetModel_v1preview class. Its forward pass never calls the affinity code, and it gates the skip features multiplicatively instead, at four skip levels unet.py unet.py. The ablation numbers are for the paper's description, and the code path that produced them is not stated.
• YOLO-Z abstract vs Table 2. The abstract claims up to 6.9% gain on small objects and "just a 3 ms" extra time. Table 2 gives 5.46% to 7.73% and +0.9 to +3.7 ms 2112.11798 tab. 2. The 6.9% figure matches single-change runs, not the YOLO-Z models 2112.11798 supp. tab. 1. The Conclusion says "close to 6%" §5.
• YOLO5Face headline numbers. For YOLOv5x6 the text gives validation values of 96.9, 96.0 and 91.6 (Easy, Medium, Hard). The validation row of the main table gives 96.67, 95.08 and 86.55 2105.12931 §4.5 tab. 3. The card advises using the table values.
• SANet shallow attention, paper vs code. The paper defines the ReLU gate on shallow features 2108.00882 eq. 1 eq. 2 and draws it in fig. 2. The released decoder has no such gate. It multiplies its 64-channel levels in a cascade and predicts with a 1×1 convolution model.py. The ablation numbers are for the paper's description, not for the code.
• HiFormer DLF output, paper vs code. The paper's decoder consumes the patch tokens of both outer levels 2207.08518 §3.2. Eq. 3 keeps those patch tokens unchanged, so the cross-level information lives only in the class tokens eq. 3. The released fusion returns each level's patch tokens unchanged and replaces only the class token utils.py, and the forward pass drops the first token of each level before the decoder HiFormer.py. By that reading, the cross-level output does not reach the decoder. This is a reading of the code; it was not run.
• HiFormer head count, paper vs code. The paper gives 6 heads on both outer levels for HiFormer-B 2207.08518 tab. 1 §5. The released config sets 6 and 12, so the 384-channel level gets 12 heads HiFormer_configs.py. Row A of the configuration ablation, with 12 heads on the small level, gives 79.00 DSC, against 80.39 for the default tab. 8.
• HiFormer repeated number, within the paper. The ResNet18 row of tab. 7 and the no-DLF row of tab. 6 both give 77.15 DSC and 16.88 HD 2207.08518 tab. 6 tab. 7. The paper does not explain the match.
• DDOD counts, paper vs code. The paper describes the level counts as a moving average up to the current iteration 2107.02963 §4.3. The released code keeps cumulative counts from the first iteration and never decays them, so late in training the weights hardly change 2107.02963 §4.3.
• Dynamic Head cost. The abstract claims no computational overhead. Table 2 shows +21.50 GFLOPs at six blocks, about 8.5% of the baseline, and the text calls this negligible 2106.08322 tab. 2 §4.2.
• Dynamic Head depth. The text says the head keeps benefiting up to eight blocks §4.2. Table 2 peaks at six blocks (42.6 AP), with eight at 42.5 2106.08322 tab. 2.
• DNA-Net Fa for ResNet-18 on NUDT. Table IV gives Fa 4.228 at tolerance under 4 pixels, and Table II gives 4.223 2106.00487 tab. IV tab. II. The Pd values in the two tables agree.
• DNA-Net NUAA Fa. Table VIII's real-only row gives Fa 4.223 for NUAA, while Table II gives 2.353 for NUAA and 4.223 for NUDT 2106.00487 tab. VIII tab. II.
• DVA fusion, paper vs code. The paper fuses the side maps with a 1x1 convolution in a weighted sum 1705.02544 §3.2 eq. 8. The released code concatenates the three pre-sigmoid logits and applies a 3x3 convolution with padding 1 attention_test.prototxt.
• DVA loss weights, paper vs code. The paper weights the side losses by 1/M 1705.02544 eq. 9. The training prototxt gives every loss weight 1 train_val.prototxt.
• D3Net joint training. The abstract says the parts are learned jointly. The released code trains each stream in a separate run, and no joint loss exists 1907.06781 abstract train.py.
• D3Net "exceeds all prior work". The abstract says D3Net exceeds all prior work on all five metrics. On LFSD, CPFP has the higher S-measure, max F, max E and the lower MAE 1907.06781 tab. 4.
• D3Net gate threshold, paper vs code. The paper's equation uses distance at most t 1907.06781 eq. 2. The code uses strict less-than eval.py.
• Event decompression, abstract vs tables. The abstract's 3.61 dB gain does not match the paper's tables. On BS-ERGB gap 3, table 2 gives 3.69 dB over DMVFN and 3.92 dB over RAFT (inpainted) 2312.00113 tab. 2. The release README reports 26.30 dB at gap 1 against 25.40 dB in the paper README.md.
• TransUNet CNN baseline row. The per-organ values of the R50-AttnUNet row average to about 72.10 by the card's check, against the printed 75.57. The aorta value is the likely outlier 2102.04306 tab. 1.
• TransUNet single skip scale, paper vs code. The text places the single skip at 1/4 resolution 2102.04306 §4.4. The code's first decoder block receives the 1/8 map 2102.04306 §4.4.
• TimeMixer ablation cases 176 to 178. The appendix says these cases reverse the directions. Table 5 shows 176 with top-down mixing on both parts and 177 with bottom-up on both, so only 178 swaps the two 2405.14616 app. F tab. 5.
• YOLOv11 C3k2 placement. Sections 4.1.1 and 4.2.1 say C3k2 replaces C2f in the backbone and in the neck. Section 4.3.1 adds C3k2 blocks in the head, and the text does not say whether these are the same block 2410.17725 §4.1.1 §4.2.1 §4.3.1.
• MIMO-UNet+ runtime. The text gives 0.014 s 2108.05054 §4.2. Table 1 gives 0.017 s 2108.05054 tab. 1.
• SINet-V2 guidance, paper vs code. The paper's equations use the first block's reversed map in every group-reversal block 2102.10274 §4.3 eq. 3 eq. 5. The released code keeps a running guidance map that each block updates in place, so the next block sees the updated map Network_Res2Net_GRA_NCD.py Network_Res2Net_GRA_NCD.py. The ablation numbers are for the paper's description, not for the code path.

Papers

• Deep Visual Attention Prediction: one VGG16 network with three per-level decoders, each with its own loss, and a learned fusion of the side maps, for eye-fixation prediction.
• FSSD: Feature Fusion Single Shot Multibox Detector: fuses three VGG16 layers in one step before building the SSD pyramid, and reports 78.8 mAP on VOC2007 test at 65.8 FPS.
• Road Extraction by Deep Residual U-Net: a residual U-Net with skip concatenation at each decoder level and no cropping, for road masks.
• Joint 3D Proposal Generation and Object Detection from View Aggregation: two-view pyramids with mean fusion of BEV and image crops in the proposal and second stages.
• Rethinking RGB-D Salient Object Detection: Models, Data Sets, and Large-Scale Benchmarks: three VGG-16 streams (RGB, RGB-D, depth) with top-down merging, and a test-time gate that drops the depth map when it disagrees with the depth-only stream.
• EGNet: Edge Guidance Network for Salient Object Detection: adds salient-edge features to each object-scale feature before prediction.
• HarDNet-MSEG: A Simple Encoder-Decoder Polyp Segmentation Neural Network that Achieves over 0.9 Mean Dice and 86 FPS: a segmentation decoder that discards the stride-4 features and fuses three deeper levels by upsampling and element-wise multiplication.
• TransUNet: Transformers Make Strong Encoders for Medical Image Segmentation: a transformer encoder with a CNN stem, and a decoder that receives three CNN skips.
• Concealed Object Detection: a three-level decoder that multiplies coarser and finer features, then refines a coarse map with reversed guidance, for camouflaged objects.
• YOLO5Face: Why Reinventing a Face Detector: adapts YOLOv5 to faces and adds a stride-64 output (P6) for large faces.
• Dense Nested Attention Network for Infrared Small Target Detection: dense nested skips, channel and spatial attention, and a pyramid output fusion for single-frame infrared targets.
• Dynamic Head: Unifying Object Detection Heads with Attentions: stacked blocks with level, spatial and channel attention over the feature pyramid.
• Disentangle Your Dense Object Detector: weights each pyramid level's loss by its positive count, alongside separate label assignment and a deformable first conv in each head branch.
• Rethinking Coarse-to-Fine Approach in Single Image Deblurring: a single U-Net with multi-scale inputs and outputs, and asymmetric fusion of encoder features, for single-image deblurring.
• YOLO-Z: Improving small object detection in YOLOv5 for autonomous vehicles: replaces the lowest-resolution map with a higher-resolution one, and tests FPN and bi-FPN necks, on a traffic-cone dataset.
• A Transformer-Based Siamese Network for Change Detection: per-scale difference modules feeding an MLP decoder for change masks.
• Shallow Attention Network for Polyp Segmentation: a Res2Net encoder-decoder whose shallow features are gated by ReLU-activated deep features, for polyp masks.
• HiFormer: Hierarchical Multi-scale Representations Using Transformers for Medical Image Segmentation: a CNN pyramid feeds a Swin pyramid, and the two outer levels exchange class tokens by cross-attention, for 2D medical segmentation.
• Thin-Plate Spline Motion Model for Image Animation: uses occlusion masks at several decoder scales to feed a skip-connected inpainting network for image animation.
• MedSegDiff: Medical Image Segmentation with Diffusion Probabilistic Model: gates image skip features with the noisy mask and adds a Fourier-domain filter on each skip path, for diffusion segmentation.
• Event-based Continuous Color Video Decompression from Single Frames: softmax-splats frame pyramids along motion and latent flow, then merges them with a synthesized-frame pyramid, to decode colour frames at any time.
• TimeMixer: Decomposable Multiscale Mixing for Time Series Forecasting: pools the input into scales, mixes seasonal parts bottom-up and trend parts top-down, and sums one predictor per scale.
• Edge YOLO: Real-Time Intelligent Object Detection System Based on Edge-Cloud Cooperation in Autonomous Vehicles: a slimmed YOLOv4 with an FPN-style neck that cuts parts of PANet, three output scales, and a Jetson-class target.
• YOLOv11: An Overview of the Key Architectural Enhancements: a review of the YOLOv11 neck and head with no experiments of its own.