Dynamic convolution

Overview

An ordinary convolution slides a window over the input and applies one learned filter set at every position. Dynamic convolution keeps the sliding window but generates the filters from the input at each position, so the filters adapt to local content 2602.02969 §II-B.1. The generated filters are produced by a small learned layer, so the block can be built from standard operations and placed where a convolution would go 2602.02969 §II-B.3.

Only one compiled card covers this concept so far, so this page describes one variant. Other dynamic-convolution work is not yet compiled here and is not covered by the evidence below.

Variants

DHiF: per-location high-frequency filter bank

The paper calls the block DHiF (dynamic high-frequency convolution). For each window it does the following 2602.02969 §II-B.1 2602.02969 §II-B.3:

1. normalises the local input (the paper names a normalisation layer but does not define it; see Notes for implementers);
2. projects the k² local values to k² filter vectors of length k² with a linear map and bounds every entry with tanh 2602.02969 eq. 5 2602.02969 eq. 6;
3. applies that k² × k² bank to the local patch of every channel 2602.02969 eq. 7;
4. adds the filtered response to the original patch and forms the output with per-filter weights 2602.02969 eq. 8.

The motivation is that a filter should respond to high-frequency detail rather than flat regions. The paper argues that kernels with positive and negative entries and a near-zero sum do this, and it enforces this by bounding each entry to (-1, 1) 2602.02969 §II-B.2. The bound is the only mechanism in the implementation; the near-zero sum is not enforced.

In the paper's residual block, DHiF replaces the first convolution, followed by batch norm and an activation, and the standard convolution comes after it 2602.02969 supp. §II-B 2602.02969 supp. eq. 1. The variant is evaluated on single-frame infrared small target detection with seven convolution-based networks: CNN and CNN-Transformer hybrids 2602.02969 §III-B. Card: Dynamic High-frequency Convolution for Infrared Small Target Detection.

Reported results (author-reported)

Each row of the main table is one network with its encoder convolutions replaced; each cell is a single reported value, with no variance given 2602.02969 tab. 1. For DNANet:

Source: 2602.02969 tab. 1. Units: IoU, nIoU and P_d in 10^-2; F_a in 10^-5 2602.02969 §III-A.

Across the 14 network-and-dataset rows, DHiF has higher IoU than the standard convolution in every row. The IoU gain ranges from +0.09 (MSHNet, NUAA-SIRST) to +40.24 (FC3Net, NUAA-SIRST), where the standard row is an outlier the paper does not explain 2602.02969 tab. 1.

Ablations, all on NUAA-SIRST with DNANet unless noted:

• Kernel size. DHiF has higher IoU than the standard convolution at k = 3 (77.68 vs 71.97), k = 5 (76.87 vs 72.36) and k = 7 (76.80 vs 74.17); the best DHiF IoU is at k = 3 2602.02969 supp. tab. II.
• Bound on the filters. With no non-linearity, GELU, LeakyReLU or sigmoid, IoU is 71.81, 71.12, 72.92 and 72.25. With tanh it is 77.68 2602.02969 supp. tab. III. GELU is below the no-activation baseline; LeakyReLU and sigmoid are above it; all three are below tanh.
• Position in the block. DHiF before the standard convolution gives 77.68 IoU; the reverse order gives 74.55 2602.02969 supp. tab. I.
• Noise. With salt-and-pepper noise on DNANet, IoU is 66.48 for the standard convolution and 72.08 for DHiF 2602.02969 supp. tab. V.
• Level. Replacing only encoder level 2 (two blocks) gives IoU 78.37; only level 5 gives 75.48 2602.02969 tab. 2. The main experiments use levels 2 and 3 2602.02969 §III-C.2; see Contradictions.

Cost on DNANet at 512 × 512: FPS 8.61 → 8.02 (about 6.9% fewer frames per second), parameters 17.92 M → 17.93 M, and runtime 0.1161 s → 0.1247 s 2602.02969 supp. tab. IV. The paper reports no peak-memory figure.

When to use which

The evidence is one paper on one task, with single-run numbers. Use the following as a guide, not a guarantee.

Implementation sketch

The sketch follows the paper's equations 2602.02969 eq. 5-2602.02969 eq. 8. Channel normalisation is left as a choice, as the paper leaves it open.

# DHiF forward pass for one feature map  [arxiv:2602.02969 §II-B.3]
# Input:  x (B, C, H, W); kernel size k
# Output: y (B, C_out, H, W)

def dhif(x, k, generator, out_weight):
    # Gather the k x k window at every location  [§II-B.1]
    patches = unfold(x, k)                    # (B, C, H, W, k*k)
    # Reduce the channels to one value per tap; the paper leaves this open
    local = normalise_over_channels(patches)  # (B, H, W, k*k)
    # Generate one k*k x k*k filter bank per location  [eq. 5]
    # Bound every entry to (-1, 1) with tanh  [eq. 6]
    bank = tanh(generator(local))             # (B, H, W, k*k, k*k)
    # Apply each location's bank to every channel's patch  [eq. 7]
    filtered = apply_bank(patches, bank)      # (B, C, H, W, k*k)
    # Add the input patch to the filtered response  [eq. 8]
    combined = patches + filtered             # (B, C, H, W, k*k)
    # Weighted sum over channels and taps, once per output filter  [eq. 8]
    return weighted_sum(combined, out_weight) # (B, C_out, H, W)

In the residual block, the DHiF output is followed by batch norm and an activation, then the standard convolution 2602.02969 supp. eq. 1. The shortcut path is not specified in the paper's equation.

Notes for implementers

• Normalisation is unspecified. The paper names a normalisation layer but does not define it 2602.02969 §II-B.3. The card for this paper records that the public code uses an L2 norm over channels; see its Paper vs code section. Results may depend on this choice.
• The sum over taps and channels. Equation 8 is written as a sum over k² only, but the weights have shape C × k² 2602.02969 eq. 8. Sum over channels and taps, or the output shape will be wrong.
• Cost scales with k⁴ per pixel. Each location builds a k² × k² bank, and the unfold step materialises a copy of the feature map 2602.02969 supp. tab. IV. Memory is not measured in the paper.
• Attention in the authors' block. The card records that the authors' DNANet block applies channel and spatial attention before the residual addition, which the paper's block equation does not show 2602.02969 supp. eq. 1. The paper's numbers may depend on this.

Contradictions

No disagreement between papers: one compiled card covers this concept. Within the paper and between the paper and its public code, these points differ:

• Levels replaced in DNANet. Table 2 lists levels 2 to 5, the text names levels 2 and 3, and the repository uses levels 2 and 3 2602.02969 tab. 2 2602.02969 §III-C.1 2602.02969 §III-C.2. The main-table IoU of 77.68 matches no single row of the level table.
• Learning-rate schedule. The paper says the rate halves after every 50 epochs; the code's first halving is at epoch 100 2602.02969 supp. §III-B.
• Initialisation. The paper cites Kaiming initialisation; the code uses the PyTorch default for convolutions 2602.02969 supp. §III-B.
• Input layer. The paper advises against DHiF at the input layer, while its public code uses it there for one network 2602.02969 §III-C.1.

Papers

• Dynamic High-frequency Convolution for Infrared Small Target Detection: introduces DHiF, a convolution with per-location tanh-bounded filter banks, and reports IoU gains in seven SIRST networks.