Fast Camouflaged Object Detection via Edge-based Reversible Re-calibration Network

In plain words

Camouflaged animals are hard to see because their colour and texture match the ground around them, and their outlines are faint. This paper's model, ERRNet, looks for them roughly the way the authors describe human attention: it first finds the edge of the object, then checks what the rest of the picture is, and uses the leftover area to sharpen its guess.

Concretely, an image network extracts features at several depths. A small edge module pulls contour details out of the shallowest features. A chain of refinement steps then starts at the deepest level with a rough guess of where the object is. Each step flips the guess, so that it shows what the guess has not yet explained, combines that with the edge details and the earlier guess, and produces a better mask for the next, shallower level. The last mask is the answer.

The method is tested on photos with hidden animals and on a few medical segmentation tasks. Its speed is reported at 79.3 frames per second. All numbers come from the authors.

TL;DR

• Camouflaged object detection: a per-pixel mask for objects that blend into the background §1.
• Encoder: ResNet-50 with five levels; an ASPP head on the top level gives a global prior §3 §3.1.
• Selective Edge Aggregation fuses the two shallowest levels into a 64-channel edge prior, trained with an edge loss §3.2.
• Reversible Re-calibration Units run top-down over three levels. Each reverses the current map and gates the encoder features with it §3.3 alg. 1.
• Author-reported: best scores on CHAM, CAMO and COD10K against 13 baselines, and about 6 points higher COD10K E-measure than SINet tab. 1 §4.5.
• The released test script does not output the paper's final map. See Paper vs code.

Problem

Camouflaged objects share texture, intensity and colour with their surroundings, and often have very ambiguous boundaries. The authors say that locating the object and its weak boundary is the key difficulty §1.

The paper argues that earlier methods focus on the target region and neglect the background, and that using the background as a contrast helps §3.3. It also argues that low-level edge information, if fused naively into high-level features, adds redundancy and inconsistency §3.2.

The experiments use benchmark datasets built for this task, plus polyp and lung-infection segmentation data in a later section §4.1 §4.6.

Method

Overview: an encoder produces five feature levels. An ASPP head on the deepest level gives a global prior, SEA gives an edge prior from the two shallowest levels, and three RRUs refine a coarse map from the deepest to the third level. The third-level map P_3 is the paper's final output §3 fig. 2 §4.2.

1. Input and encoder. Images are resized to 352 x 352 §4.3. A ResNet-50 without its fully connected layers gives E1 and E2 at H/4 (64 and 256 channels) and E3, E4, E5 at H/8, H/16 and H/32 §3. Channel counts for E3 to E5 are not stated in §3; the code uses 512, 1024 and 2048 ERRNet.py.

2. Global prior (ASPP). The head applies four atrous branches with dilation rates 1, 6, 12 and 18 to E5, concatenates them with E5, and reduces them with a convolution and a 1 x 1 filter §3.1 eq. 1. The result is one map, P_g. The code differs from this formula; see Paper vs code.

3. Selective Edge Aggregation (SEA). Each shallow input is reduced to 64 channels by a convolution block with 64 filters §3.2 eq. 2. The two reduced maps are multiplied element-wise, and a convolution gives a weight w_s eq. 3. Each reduced map is combined with w_s by a residual step, both are convolved, and the results are concatenated and convolved again to give the 64-channel edge prior F_e eq. 4. A convolution with one filter on the edge prior gives a coarse edge map P_e, supervised by ground-truth edges §3.2 ERRNet.py.

4. Reversible Re-calibration Unit (RRU). The paper runs one RRU at levels i = 5, 4, 3 in that order alg. 1. For each level:

   • the guidance is the neighbour prior P_{i+1} plus the global prior P_g, where the neighbour prior is zero at i = 5 alg. 1 lines 5-8;
   • the guidance is passed through 1 minus a sigmoid, and the one-channel result is repeated to match the channel count f_i of E_i §3.3 (i) alg. 1 lines 6-8;
   • the reversed map is multiplied with E_i, concatenated with the edge prior F_e, and passed through a convolution with one filter alg. 1 line 11 §3.3 (ii). The paper's result for level i is P_i. The "plus" in the guidance is written as a circled-plus operator that the text does not define.

5. Output. The network outputs the global map and one map per RRU level, each after a sigmoid §3 §3.3. The paper's segmentation result is P_3 §4.2.

6. Training objective. The loss is the sum of the edge loss and one mask loss for each of P_3, P_4, P_5 and P_g, with equal coefficients §3.4. The mask loss is a weighted binary cross-entropy plus a weighted IoU loss. The edge loss is a weighted binary cross-entropy only, because edge pixels are rare §3.4. The weight formulas are not restated in the paper; it refers to F3Net for them §3.4. They are therefore not stated in this card.

Pseudocode for the edge branch, following eq. 2 to eq. 4:

# Selective edge aggregation  [§3.2] [eq. 2-4]
# Input:  E1 (B, 64, H/4, W/4), E2 (B, 256, H/4, W/4)
# Output: edge prior F_e (B, 64, H/4, W/4), edge map P_e (B, 1, H/4, W/4)

def selective_edge_aggregation(E1, E2):
    # 1. Reduce both inputs to 64 channels  [eq. 2]
    E1_bar = conv_block_64(E1)
    E2_bar = conv_block_64(E2)
    # 2. Element-wise product gives the selective weight  [eq. 3]
    w_s = conv_block_64(E1_bar * E2_bar)
    # 3. Residual aggregation; the paper does not define the plus sign  [eq. 4]
    fused_1 = conv_block_64(E1_bar + w_s)
    fused_2 = conv_block_64(E2_bar + w_s)
    F_e = conv_block_64(concat(fused_1, fused_2))
    # 4. One-filter convolution gives the coarse edge map  [§3.2]
    P_e = conv_1filter(F_e)
    return F_e, P_e

Pseudocode for the reversible re-calibration cascade, following alg. 1:

# Reversible re-calibration cascade  [alg. 1] [§3.3]
# Input:  encoder maps E3, E4, E5; global prior P_g (B, 1, H/32, W/32);
#         edge prior F_e (B, 64, H/4, W/4)
# Output: coarse maps P_5, P_4, P_3; P_3 is the final mask  [§4.2]

def rru(E_i, guidance, F_e):
    # 1. Reverse the guidance: 1 minus its sigmoid  [alg. 1 lines 6-8]
    reversed_map = 1 - sigmoid(guidance)
    # 2. Repeat the one-channel map to E_i's channel count  [§3.3 (i)]
    F_R = repeat_channels(reversed_map, channels(E_i))
    # 3. Gate the encoder features  [alg. 1 line 11]
    gated = F_R * E_i
    # 4. Bring F_e to E_i's size; not stated in the paper  [code: ERRNet.py#L35]
    edge = resize(F_e, size_of(E_i))
    # 5. One-filter convolution over the concatenation  [alg. 1 line 11]
    return conv_1filter(concat(edge, gated))

def recalibrate(E3, E4, E5, P_g, F_e):
    # 6. Top level: guidance is the global prior alone  [alg. 1 line 6]
    P_5 = rru(E5, P_g, F_e)
    # 7. Next level: previous map plus global prior  [alg. 1 line 8]
    P_4 = rru(E4, resize(P_5, size_of(E4)) + P_g, F_e)
    # 8. Bottom level: P_3 is the paper's final mask  [§4.2]
    P_3 = rru(E3, resize(P_4, size_of(E3)) + P_g, F_e)
    return P_5, P_4, P_3

Hyperparameters

Results

Author-reported. Setup: three COD benchmarks (CHAM, CAMO, COD10K) and 13 baselines whose numbers are taken from the SINet report §4.5 tab. 1. Metrics: S-measure, mean E-measure, weighted F-measure and MAE, computed with the authors' toolbox §4.2.

Table 1, from tab. 1. Best in each column in bold in the paper; ERRNet row is the paper's model.

Speed, from §4.5: ERRNet 79.3 FPS with E-measure 0.867 on COD10K; SINet 5 FPS; CPD 62 FPS. The paper states that FPS for the three oldest baselines was not measured on the same hardware §4.5.

Table 2, ablation on CAMO and COD10K, from tab. 2. Components are listed as marked in the table: g-p global prior, n-p neighbour prior, r-o reverse operation, EAA edge addition, EAM edge multiplication, SEA selective edge aggregation, ASPP. The paper calls row #8 the full model.

Table 3, COD10K super-classes, from tab. 3. Each block is Sα, Eφ, M.

Table 4, polyp segmentation, from tab. 4. Kvasir and CVC-612 are seen datasets; ETIS and CVC-T are unseen. ‡ marks ERRNet retrained with the Res2Net backbone used by PraNet §4.6. The paper's header marks M with an upward arrow; see Pitfalls.

Table 5, COVID-19 lung infection segmentation, from tab. 5. Backbone column as printed.

Figures: fig. 1 plots speed against accuracy on COD; fig. 5 shows visual comparisons with SINet, EGNet, HTC, CPD and PFANet; fig. 6 shows failure cases with several camouflaged objects §4.5; fig. 7 compares four methods on two medical segmentation tasks §4.6. fig. 3 and fig. 4 visualise the priors and the effect of each prior §3.3.

Paper vs code

Official repository: GewelsJI/ERRNet, owned by the first author; the paper links it §4 footnote. Commit 22f78bb1abd16bcecfd3f74fd8aa0b707eb062d5. The repository contains only test.py, dataset.py and model/ (ERRNet, ASPP, ResNet). It has no training script, no loss, and no optimiser code. The README says the training script was taken from the later SINetV2 project [code: README.md#L11]. A retrained model "performs slightly differently" from the paper README.md.

1. Output map. The paper says P_3 is the final result §4.2. The released test script takes output[0], the global-prior map cam_out_g test.py ERRNet.py. The map at level 3, cam_out_2, is the fourth value returned ERRNet.py. Which map produced the paper's tables is not stated.

2. Post-processing. The paper says no post-processing is applied §4.3. The test script rescales each map to [0, 1] with its own minimum and maximum after the sigmoid test.py. This changes absolute values, so it affects MAE in particular.

3. Test sets. The paper evaluates COD10K, CAMO and CHAM §4.1. The test script evaluates CHAMELEON, CAMO, COD10K and NC4K test.py. The paper does not mention NC4K. CHAMELEON and CHAM are presumably the same dataset; the paper does not say so.

4. Edge branch (SEA). Eq. 2 to 4 describe one convolution per input, a product, one convolution, a residual sum and one more convolution §3.2. The code applies two 3 x 3 conv-ReLU blocks to each input before the product ERRNet.py ERRNet.py. After the product it applies a 3 x 3 conv and adds the first-stage output, not the paper's "E-bar plus w_s" ERRNet.py. The edge feature is a 3 x 3 conv over the concatenation ERRNet.py. The edge map is a 1 x 1 conv ERRNet.py ERRNet.py.

5. RRU body. The paper's refinement is one convolution with one filter alg. 1 line 11 §3.3 (ii). The code uses three 3 x 3 convolutions, with ReLU after the second and third, and a 3 x 3 output convolution to one channel ERRNet.py ERRNet.py. It also adds the guidance map to the output, x = x + crop_sal ERRNet.py, which Algorithm 1 does not contain.

6. Guidance sums. The paper writes the guidance as P_{i+1} plus P_g [alg. 1 line 8]. The code uses the same global map P_g at every level, adds the previous RRU output at each level, and doubles the global term's resolution for the third level ERRNet.py. The sum is taken on logits, and the sigmoid is applied inside the RRU ERRNet.py.

7. ASPP. The paper has four branches with dilation 1, 6, 12 and 18, a concatenation with E5, and a 1 x 1 final filter §3.1 eq. 1. The code has a 1 x 1 branch, a global average-pooling branch, three 3 x 3 atrous branches with dilation 6, 12 and 18, a 3 x 3 bottleneck to 256 channels, a position-attention module, a 3 x 3 conv to 64 channels, dropout 0.1 and a 1 x 1 conv to one channel aspp.py aspp.py. It does not concatenate E5 itself, and the position-attention module is not in the paper. The file's docstring says the module is borrowed from another repository aspp.py.

8. Edge supervision. The code computes the edge map at H/4 and upsamples it by four to the image size ERRNet.py. The edge loss is not in the repository.

9. Input normalisation. The code resizes to 352 x 352 and normalises with ImageNet mean and standard deviation dataset.py. The paper does not state normalisation.

10. Output resize. The test script upsamples the map to the ground-truth size with bilinear interpolation and align_corners=False test.py. The network's internal upsampling uses align_corners=True ERRNet.py.

11. Not in the code. The Adam settings, learning rate, batch size, epoch count, multi-scale augmentation and loss weights of §4.3 and §3.4 cannot be checked against code in this repository.

Pitfalls

• Which map to output. Follow the paper and use P_3 §4.2. The released evaluation uses the global map instead test.py. An implementation that copies the released script will not reproduce the paper's pipeline.
• Licence of the code. The LICENSE file is Apache-2.0 LICENSE. The README says the code is for research and education only, and that commercial use needs permission first README.md. Treat it as non-commercial: this card cites the code and quotes none of it.
• Abstract figure. The abstract's "about 6%" mean E-measure gain over SINet matches the COD10K gap alone (0.867 vs 0.806, tab. 1). Averaged over the three datasets, the gap is 4.8 points §4.5.
• Speed. 79.3 FPS is reported for 352 x 352 inference §4.3. Batch size, precision, whether pre- and post-processing are included, and the GPU used for that figure are not stated. The paper says the speeds were measured on the same RTX card §4.5, but training is said to use an RTX TITAN §4.3.
• Table 2 markings. The marked components in rows #1 and #2 do not match the text: row #1 is described as having no RRU but marks the neighbour prior, and row #2 is described as "global prior only" but marks the reverse operation tab. 2 §4.4. Check against the PDF before relying on those rows.
• Table 4 header. The header marks MAE with an upward arrow tab. 4, although lower MAE is better. The Kvasir S-measure is bold for PraNet (.915) against ERRNet (.911), while the text says ERRNet "outperforms all the competitors on four datasets consistently" §4.6.
• COVID gain. The text says ERRNet gives a 12% gain on Sen. over Inf-Net with Res2Net §4.6. The table gives .751 against .692 for Res2Net, a gap of 5.9 points, and .751 against .629 for Inf-Net with ResNet-50, a gap of 12.2 points tab. 5. The 12-point figure matches only the ResNet-50 comparison.
• Loss weights. The paper gives no weight formula and refers to F3Net §3.4. The weight map is therefore unknown here, and the edge loss weighting is not stated either.
• Training recipe. No LR decay, weight decay, warm-up or epoch schedule beyond "30 epochs" is given §4.3. The repository has no training code, and the README says the training script comes from SINetV2 README.md. Reproducing the reported numbers therefore means guessing that schedule.
• Source version. The fetched LaTeX is a journal-format file (elsarticle, review class) with \rev{} markup for revised text [tex: CamObjDet_PR.tex]. The arXiv metadata records v1 (2021-11-05). Later revisions may differ, so numbers here are from the fetched source. The README gives the journal reference Pattern Recognition 123 (2022), 108414 README.md.
• ASPP provenance. The ASPP file's docstring credits another repository aspp.py. Its licence is not checked here, so the code is not quoted.
• Medical results. Polyp and COVID results follow the protocols of the cited works, and some baselines are "N/A" or taken from those works §4.6. Compare only within a table.

Lineage

• Builds on: ResNet-50 for the encoder §3; ASPP for the global prior §3.1; the BCE and IoU loss formulas of F3Net, referenced without restatement §3.4.
• Compared against: SINet and 12 other COD baselines, with numbers taken from the SINet paper §4.5. Medical baselines follow PraNet and Inf-Net §4.6.
• Superseded by: not stated in this paper. The README points to a later project, SINetV2, which reuses this repository's training script README.md.