Paper · 2022-12-21 · cs.CV
DuAT: Dual-Aggregation Transformer Network for Medical Image Segmentation
A pyramid-Transformer (PVT) segmentation network for medical images. Its GLSA blocks split channels into a global-attention and a local depth-wise branch; its SBA module fuses shallow boundary and deep semantic features into a second side output. Author-reported mDice: Kvasir 0.924, ClinicDB 0.948, ISIC-2018 0.923, unseen ETIS 0.822.
Semantic segmentationHierarchical vision transformersLocal-global attentionmedical-image-segmentationpolyp-segmentationtransformerboundary-aware
arXivPDFCodeMarkdown17 min read
AI-compiled by claude-haiku-5-5 on 2026-11-13, not yet reviewed by a human. Every claim cites its source; check it when a detail matters.
In plain words
Doctors often need a computer to outline a polyp in a colonoscopy video frame, a skin lesion in a photo, or nuclei in a microscope image. Transformer networks are good at seeing the whole picture, but in doing so they tend to blur small objects and the thin edges of an object. Those small and edge details are exactly what matters in medical images.
DuAT tries to keep both kinds of information. One module looks at the whole image and also at small neighbourhoods, and the two views are combined. A second module takes the sharp edge information from the early layers and mixes it into the richer meaning from the later layers, so the outline of the object is kept. The network then produces its mask from these two streams.
The authors report that this beats several earlier medical segmentation networks on the datasets they tested, including unseen colonoscopy datasets. These are their own numbers, measured under their own setup.
TL;DR
- DuAT is a pyramid-Transformer encoder with two added modules: GLSA, which combines global and local spatial attention, and SBA, which aggregates boundary and semantic features §3.
- GLSA splits the channels of each deeper stage in half; one half goes through global attention, the other through local attention, and the outputs are concatenated §3.3 eq. 4 eq. 5.
- The model has two side outputs, one from the deep path and one from the SBA path, and both get a weighted IoU and weighted BCE loss §3.4 eq. 10 eq. 11.
- The authors report the best mean Dice of the compared methods on the seen polyp sets and ISIC-2018, and on the unseen ETIS set tab. 1 tab. 2.
- The official code is public, has no licence file, and differs from the paper in several module details; read "Paper vs code" before implementing.
Problem
Transformer encoders capture long-range context but tend to wash out local detail. Medical images contain small objects such as small polyps and ambiguous boundaries, and the authors say that loss of this detail gives over-smooth predictions for small objects and blurred edges §1. Earlier CNN and hybrid methods have the same limitation, because feeding local information straight into a Transformer lets the global context overwhelm it §1. The paper also notes that boundary cues from shallow layers are often fused with deep features in ways that add redundancy or inconsistency §3.2.
The paper assumes 2D images with binary masks and evaluates only 2D data. The authors state that extending to 3D is future work §5.
Method
DuAT has three parts: a pyramid Transformer encoder (PVT) §3.1, a GLSA module applied to stages 2 to 4 §3.3, and an SBA module that combines the first stage with the aggregated deep features §3.2 fig. 2. The paper says the encoder produces four feature maps. The first, F1, carries detailed boundary information, and F2 to F4 carry higher-level features §3.1.
- Encoder. A PVT produces four feature maps. F1 is the fine boundary map and F2 to F4 are the high-level maps §3.1.
- GLSA on F2 to F4. Each map's channels are split evenly into two groups. The first group goes to a global spatial attention (GSA) branch, the second to a local spatial attention (LSA) branch. The two outputs are concatenated and a 1×1 convolution gives 32 output channels §3.3 eq. 4 eq. 5 fig. 3.
- Semantic features. The GLSA outputs of the third and fourth stages are fused into a 32-channel map F^s at H/8. The paper says F^s fuses stages 3 and 4 but does not say how §3.2.
- Boundary features. F^b is the 32-channel map at H/4 taken from the first stage §3.2.
- SBA. Two re-calibration attention units (RAU) run in both orders, RAU(F^s, F^b) and RAU(F^b, F^s). Their outputs are concatenated and passed through a 3×3 convolution with batch norm and ReLU, giving Z with 32 channels at H/4 §3.2 eq. 1 eq. 2 eq. 3.
- Side outputs and loss. Two side outputs S1 and S2 are supervised with the same weighted loss, and the two losses are summed §3.4 eq. 10 eq. 11.
The paper's equations, reproduced from the source with their printed numbers:
T_1' = W_{\theta}(T_1), \quad T_2' = W_{\phi}(T_2) eq. 1
PAU(T_1,T_2)= T_1' \odot T_1 + T_2' \odot T_2 \odot (\ominus (T_1')) + T_1 eq. 2
Z = C_{3 \times 3}({\rm Concat} (PAU(F^s,F^b), PAU(F^b,F^s))) eq. 3
\mathcal{F}_{i}^1, \mathcal{F}_{i}^2 = {\rm Split}(\mathcal{F}_{i}) eq. 4
\mathcal{F}_{i}^{'} = C_{1\times1}({\rm Concat}(G_{sa}(\mathcal{F}_{i}^1), L_{sa}(\mathcal{F}_{i}^2))) eq. 5
Att_{G}(\mathcal{F}_{i}^1) = Softmax(Transpose(C_{1 \times 1} (\mathcal{F}_{i}^1))) eq. 6
G_{sa}(\mathcal{F}_{i}^1)= MLP(Att_{G}(\mathcal{F}_{i}^1)\otimes \mathcal{F}_{i}^1) + \mathcal{F}_{i}^1 eq. 7
Att_{L}(\mathcal{F}_{i}^2) = \sigma(C_{1 \times 1}(\mathcal{F}_c(\mathcal{F}_{i}^2))+\mathcal{F}_{i}^2) eq. 8
L_{sa} = Att_{L}(\mathcal{F}_{i}^2)\odot \mathcal{F}_{i}^2 + \mathcal{F}_{i}^2 eq. 9
\mathcal{L}(S,G) = \lambda_1 \mathcal{L}^{\omega}_{IoU}(S,G) + \lambda_2 \mathcal{L}^{\omega}_{BCE}(S,G) eq. 10
\mathcal{L}_{total} = \mathcal{L}(S_1,G) + \mathcal{L}(S_2,G) eq. 11
Symbols as defined in the paper: ⊙ and point-wise multiplication, ⊗ matrix multiplication, σ the sigmoid, F_c a cascade of 1×1 and 3×3 depth-wise convolutions with 32 channels §3.2 §3.3. The paper calls the unit RAU in the text and PAU in equation 2.
Pseudocode of the method as the paper describes it. Items marked "code only" come from the official implementation, because the paper does not state them.
# DuAT forward pass [fig. 2] [§3]
# Input: image (B, 3, 352, 352)
# Output: two maps at full resolution
def duat_forward(image):
# 1. Pyramid encoder, stages 1 to 4 [§3.1]
stage1, stage2, stage3, stage4 = encoder(image)
# 2. GLSA on stages 2 to 4, 32 channels each [§3.3] [eq. 4] [eq. 5]
glsa2 = glsa(stage2) # (B, 32, H/8, W/8)
glsa3 = glsa(stage3) # (B, 32, H/16, W/16)
glsa4 = glsa(stage4) # (B, 32, H/32, W/32)
# 3. Deep semantic map at H/8 [§3.2]; fusion method is code only
semantic = fuse_deep(glsa2, glsa3, glsa4) # (B, 32, H/8, W/8)
# 4. Boundary map at H/4, 32 channels [§3.2]; reduction is code only
boundary = reduce_to_32(stage1) # (B, 32, H/4, W/4)
# 5. Selective boundary aggregation [§3.2] [eq. 1] [eq. 2] [eq. 3]
z = sba(semantic, boundary) # (B, 32, H/4, W/4)
# 6. Side outputs: one map from the deep path, one from z [§3.4]
# Mapping from features to one channel is code only
side1 = head_deep(glsa2, glsa3, glsa4) # upsampled by 8 later
side2 = head_sba(z) # upsampled by 4 later
return side1, side2# GLSA block [§3.3] [eq. 4] [eq. 5]
# Input: features (B, C, H, W); output (B, 32, H, W)
def glsa(features):
# Split the channels into two equal groups [eq. 4]
first, second = split_channels_in_half(features)
# Global branch on the first group [eq. 6] [eq. 7]
global_out = gsa(first) # (B, 32, H, W)
# Local branch on the second group [eq. 8] [eq. 9]
local_out = lsa(second) # (B, 32, H, W)
# Concatenate, then a 1x1 convolution to 32 channels [eq. 5]
return conv1x1(concat(global_out, local_out))# Global spatial attention [§3.3] [eq. 6] [eq. 7]
# Input: x (B, 32, h, w)
def gsa(x):
# One weight per position, softmax over all h*w positions [eq. 6]
weights = softmax(flatten(conv1x1(x))) # (B, 1, h*w)
# Weighted sum of the features gives one context vector [eq. 6]
# The paper does not say how this vector is broadcast back to (h, w)
context = matmul(weights, flatten(x)) # (B, 1, 32)
# Two-layer MLP (expansion 2, ReLU, normalisation), residual [eq. 7]
return mlp(context) + x # (B, 32, h, w)# Local spatial attention [§3.3] [eq. 8] [eq. 9]
# Input: x (B, 32, h, w)
def lsa(x):
# Cascade of three 1x1 and three 3x3 depth-wise convolutions [eq. 8]
# The order of the layers is not stated in the paper
local = local_cascade(x) # (B, 32, h, w)
# Sigmoid gate from the local features plus the input [eq. 8]
gate = sigmoid(conv1x1(local) + x) # (B, 32, h, w)
# Gated input plus a residual connection [eq. 9]
return gate * x + x # (B, 32, h, w)# Re-calibration attention unit, called PAU in eq. 2 [§3.2] [eq. 1] [eq. 2]
# Inputs t1, t2: (B, 32, h, w), at the same resolution
def rau(t1, t2):
# Sigmoid of a 1x1 linear map of each input [eq. 1]
gate1 = sigmoid(conv1x1_theta(t1)) # (B, 32, h, w)
gate2 = sigmoid(conv1x1_phi(t2)) # (B, 32, h, w)
# t1 gated, t2 gated and reversed by gate1, plus t1 [eq. 2]
return gate1 * t1 + gate2 * t2 * (1 - gate1) + t1# Selective boundary aggregation [§3.2] [eq. 3]
# Inputs: deep (B, 32, H/8, W/8), boundary (B, 32, H/4, W/4)
def sba(deep, boundary):
# Bring the deep map to the boundary resolution; method is code only
deep = upsample_bilinear(deep, size=boundary)
# Two RAU fusions, in both orders [eq. 2] [eq. 3]
both = concat(rau(deep, boundary), rau(boundary, deep))
# 3x3 convolution, batch norm, ReLU [eq. 3]
return relu(batch_norm(conv3x3(both))) # (B, 32, H/4, W/4)# One training step [§4.2] [§3.4] [eq. 10] [eq. 11]
# Batch 16, AdamW, learning rate 1e-4, weight decay 1e-4 [§4.2]
def train_step(images, masks, model, optimizer):
# Two side outputs from the model [§3.4]
side1, side2 = model(images)
# Weighted IoU and weighted BCE for each side; lambda values not stated
# Each side is supervised separately and the two losses are summed [eq. 11]
loss = weighted_loss(side1, masks) + weighted_loss(side2, masks)
# Standard backward pass and parameter update
loss.backward()
optimizer.step()
return lossHyperparameters
| Name | Value | Where | Notes |
|---|---|---|---|
| Encoder | PVT, stages 1 to 4 | §3.1 §4.4 | Text says PVT; ablation says PVTv2. Code loads pvt_v2_b2 DuAT.py |
| Pretrained initialisation | not stated | §4.2 | Code loads ./pretrained_pth/pvt_v2_b2.pth DuAT.py. The weights are not in the repo |
| GLSA output channels | 32 | §3.3 | DuAT.py |
| SBA / RAU channels | 32 | §3.2 | Code uses 16 inside SBA DuAT.py |
| GLSA input split | "64 channels, split evenly" | §3.3 | Code uses the full stage width (128, 320, 512 for stages 2 to 4) DuAT.py |
| Loss | weighted IoU + weighted BCE, per side | §3.4 eq. 10 eq. 11 | λ1, λ2 not stated. Code uses weight 1 for both Train.py |
| Optimizer | AdamW | §4.2 | Train.py |
| Learning rate | 1e-4 | §4.2 | Train.py |
| Weight decay | 1e-4 | §4.2 | Train.py |
| Batch size | 16 | §4.2 | Train.py |
| Training input size (polyp) | 352×352 | §4.1 | dataloader.py |
| Training input size (ISIC-2018, 2018-DSB) | 512×384 and 256×256 | §4.1 | Code resizes all training images to 352×352; see "Paper vs code" |
| Epochs | not stated | §4.2 | Code default 100, but the loop runs 99 epochs Train.py Train.py |
| LR schedule | not stated | §4.2 | Code multiplies LR by 0.1 every 200 epochs utils.py Train.py |
| Multi-scale training | cited to PraNet and HarDNet, scales not stated | §4.2 | Code uses scales 0.75, 1.0, 1.25 Train.py |
| Gradient clipping | not stated | §4.2 | Code clamps each gradient element to ±0.5 Train.py utils.py |
| Augmentation | rotation and horizontal flip | §4.2 | Code: horizontal and vertical flip, each with p=0.2; rotation commented out dataloader.py |
| Colour augmentation | LAB colour transfer | §4.2 | Applied with probability 0.7 to a reference image dataloader.py |
| Hardware and framework | PyTorch, one NVIDIA RTX 3090 | §4.2 | |
| Input size at test | 352×352, then upsampled to ground-truth size | §4.2 | Test.py |
| Metrics | mean Dice, mean IoU, MAE | §4.2 | Authors used the MATLAB evaluation tool; see Pitfalls |
Dataset splits §4.1 tab. 3: Kvasir has 1000 images and ClinicDB 612, each split 80/10/10 into train, validation and test as stated. ColonDB (380), ETIS (196) and EndoScene (60) are test-only unseen sets. ISIC-2018 has 2596 images (2078/259/259) and 2018-DSB 670 (536/67/67).
Results
Author-reported. Setup: five polyp datasets, ISIC-2018 and 2018-DSB §4.1; PyTorch on one NVIDIA RTX 3090 §4.2; DuAT has 24.92 M parameters and 9.88 GFLOPs at 352×352 tab. 4. Baseline numbers are as printed in the paper; the paper does not say whether they were re-run.
Headline numbers from the authors:
- Seen sets: mean Dice 0.924 on Kvasir and 0.948 on ClinicDB §4.3 tab. 1.
- ISIC-2018: mean Dice 0.923 and mean IoU 0.867, which the text says is 1.0 and 1.5 points above the best baseline §4.3 tab. 1.
- Unseen ETIS: mean Dice 0.822, 3.5 points above Polyp-PVT §4.3 tab. 2.
Table 1, seen datasets tab. 1. Each column group is mean Dice (higher is better), mean IoU (higher) and MAE (lower):
| Method | Kvasir Dice | Kvasir IoU | Kvasir MAE | ClinicDB Dice | ClinicDB IoU | ClinicDB MAE | ISIC-2018 Dice | ISIC-2018 IoU | ISIC-2018 MAE | 2018-DSB Dice | 2018-DSB IoU | 2018-DSB MAE |
|---|---|---|---|---|---|---|---|---|---|---|---|---|
| U-Net | 0.818 | 0.746 | 0.055 | 0.823 | 0.755 | 0.019 | 0.855 | 0.785 | 0.045 | 0.908 | 0.831 | 0.040 |
| UNet++ | 0.821 | 0.743 | 0.048 | 0.794 | 0.729 | 0.022 | 0.809 | 0.729 | 0.041 | 0.911 | 0.837 | 0.039 |
| PraNet | 0.898 | 0.840 | 0.030 | 0.899 | 0.849 | 0.009 | 0.875 | 0.787 | 0.037 | 0.912 | 0.838 | 0.036 |
| CaraNet | 0.918 | 0.865 | 0.023 | 0.936 | 0.887 | 0.007 | 0.870 | 0.782 | 0.038 | 0.910 | 0.835 | 0.037 |
| TransUNet | 0.913 | 0.857 | 0.028 | 0.935 | 0.887 | 0.008 | 0.880 | 0.809 | 0.036 | 0.915 | 0.845 | 0.033 |
| TransFuse | 0.920 | 0.870 | 0.023 | 0.942 | 0.897 | 0.007 | 0.901 | 0.840 | 0.035 | 0.916 | 0.855 | 0.033 |
| UCTransNet | 0.918 | 0.860 | 0.023 | 0.933 | 0.860 | 0.008 | 0.905 | 0.83 | 0.035 | 0.911 | 0.835 | 0.035 |
| Polyp-PVT | 0.917 | 0.864 | 0.023 | 0.937 | 0.889 | 0.006 | 0.913 | 0.852 | 0.032 | 0.917 | 0.859 | 0.030 |
| DuAT (paper) | 0.924 | 0.876 | 0.023 | 0.948 | 0.906 | 0.006 | 0.923 | 0.867 | 0.029 | 0.926 | 0.870 | 0.027 |
Table 2, unseen datasets, for generalisation tab. 2:
| Method | ColonDB Dice | ColonDB IoU | ColonDB MAE | ETIS Dice | ETIS IoU | ETIS MAE | EndoScene Dice | EndoScene IoU | EndoScene MAE |
|---|---|---|---|---|---|---|---|---|---|
| U-Net | 0.512 | 0.444 | 0.061 | 0.398 | 0.335 | 0.036 | 0.710 | 0.627 | 0.022 |
| UNet++ | 0.483 | 0.410 | 0.064 | 0.401 | 0.344 | 0.035 | 0.707 | 0.624 | 0.018 |
| PraNet | 0.712 | 0.640 | 0.043 | 0.628 | 0.567 | 0.031 | 0.851 | 0.797 | 0.010 |
| CaraNet | 0.773 | 0.689 | 0.042 | 0.747 | 0.672 | 0.017 | 0.903 | 0.838 | 0.007 |
| TransUNet | 0.781 | 0.699 | 0.036 | 0.731 | 0.824 | 0.021 | 0.893 | 0.660 | 0.009 |
| TransFuse | 0.781 | 0.706 | 0.035 | 0.737 | 0.826 | 0.020 | 0.894 | 0.654 | 0.009 |
| SSformer | 0.772 | 0.697 | 0.036 | 0.767 | 0.698 | 0.016 | 0.887 | 0.821 | 0.007 |
| Polyp-PVT | 0.808 | 0.727 | 0.031 | 0.787 | 0.706 | 0.013 | 0.900 | 0.833 | 0.007 |
| DuAT (paper) | 0.819 | 0.737 | 0.026 | 0.822 | 0.746 | 0.013 | 0.901 | 0.840 | 0.005 |
Table 3, dataset statistics tab. 3:
| Dataset | Imaging | Images | Shape | Train | Valid | Test |
|---|---|---|---|---|---|---|
| Kvasir | Colon | 1000 | variable | 800 | 100 | 100 |
| ClinicDB | Colon | 612 | 384×288 | 488 | 62 | 62 |
| ColonDB | Colon | 380 | 574×500 | - | - | 380 |
| ETIS | Colon | 196 | 1225×966 | - | - | 196 |
| EndoScene | Colon | 60 | 574×500 | - | - | 60 |
| ISIC-2018 | Derm | 2596 | 512×384 | 2078 | 259 | 259 |
| 2018-DSB | Micro | 670 | 256×256 | 536 | 67 | 67 |
Table 4, parameters and FLOPs at 352×352 tab. 4:
| Model | Kind | Parameters, millions | GFLOPs |
|---|---|---|---|
| U-Net | CNN | 43.93 | 43.47 |
| UNet++ | CNN | 9.04 | 64.03 |
| PraNet | CNN | 30.50 | 13.01 |
| CaraNet | CNN | 44.59 | 21.65 |
| TransUNet | Transformer | 93.19 | 60.84 |
| SSformer | Transformer | 26.70 | 28.07 |
| TransFuse | Transformer | N/A | N/A |
| Polyp-PVT | Transformer | 25.08 | 10.00 |
| DuAT (paper) | Transformer | 24.92 | 9.88 |
Table 5, ablation on Kvasir (seen), ETIS (unseen) and ISIC-2018 (seen) tab. 5:
| Variant | Kvasir Dice | Kvasir IoU | Kvasir MAE | ETIS Dice | ETIS IoU | ETIS MAE | ISIC Dice | ISIC IoU | ISIC MAE |
|---|---|---|---|---|---|---|---|---|---|
| Baseline | 0.910 | 0.856 | 0.030 | 0.759 | 0.668 | 0.035 | 0.877 | 0.783 | 0.040 |
| + GSA | 0.912 | 0.860 | 0.029 | 0.772 | 0.675 | 0.030 | 0.887 | 0.803 | 0.038 |
| + LSA | 0.916 | 0.863 | 0.028 | 0.785 | 0.690 | 0.027 | 0.900 | 0.839 | 0.035 |
| + GSA + LSA (serial) | 0.914 | 0.863 | 0.028 | 0.786 | 0.695 | 0.025 | 0.909 | 0.845 | 0.034 |
| + LSA + GSA (serial) | 0.910 | 0.860 | 0.029 | 0.799 | 0.713 | 0.021 | 0.910 | 0.852 | 0.033 |
| + GLSA | 0.917 | 0.864 | 0.025 | 0.814 | 0.723 | 0.016 | 0.916 | 0.816 | 0.031 |
| w/o SBA | 0.917 | 0.864 | 0.025 | 0.814 | 0.723 | 0.016 | 0.916 | 0.816 | 0.031 |
| w/o GLSA (3×3 conv instead) | 0.915 | 0.863 | 0.026 | 0.790 | 0.696 | 0.023 | 0.901 | 0.800 | 0.033 |
| SBA + GLSA (full DuAT) | 0.924 | 0.876 | 0.023 | 0.822 | 0.746 | 0.013 | 0.923 | 0.867 | 0.029 |
Qualitative and size results, described in words: small-polyp size analysis, where DuAT's mean Dice is higher than Polyp-PVT and TransUNet on small polyps fig. 5 §4.3; boundary and small-object examples on ETIS, ISIC-2018 and 2018-DSB fig. 4; and ablation examples where SBA sharpens ambiguous boundaries and GLSA helps small objects fig. 6 §4.4.
Paper vs code
Code paths are relative to the repository at code_commit. Ownership: the paper does not link the repository and says the code will be provided after publication §4.2. The repository's owner display name matches the first author, and its README says it is the official implementation and lists the paper's authors [README §1, §4]. Its citation lists one author who is not in the arXiv author list, so the match to this paper is by name and README only. The repository has no LICENSE file, so no code is quoted here.
- Loss weights. The paper leaves λ1 and λ2 unstated eq. 10. The code uses weight 1 for the weighted-IoU and weighted-BCE terms, and adds a boundary weight map
1 + 5·|avgpool31(mask) − mask|that the paper does not describe Train.py. - Side outputs. The paper calls them S1 and S2 without saying which features produce them §3.4. The code's S1 comes from a 1×1 convolution over the three GLSA outputs, upsampled by 8 DuAT.py DuAT.py DuAT.py. S2 is the SBA output, upsampled by 4 DuAT.py DuAT.py.
- GLSA input width. The paper says the stage features have 64 channels before the split §3.3. The code applies GLSA to stages with 128, 320 and 512 channels, each split in half DuAT.py DuAT.py DuAT.py.
- GSA. The paper's equation 7 adds an MLP output to the input eq. 7. The code uses a context block whose output is
x + x·sigmoid(MLP(context)), with LayerNorm and a zero-initialised last layer DuAT.py DuAT.py DuAT.py DuAT.py. - LSA. The paper's equation 8 puts the input inside the sigmoid eq. 8. The code has no such term. It uses four 1×1 and three depth-wise 3×3 convolutions with batch norm, ReLU and SiLU, residual links inside the cascade, and a sigmoid gate on the result DuAT.py.
- SBA width and output. The paper says Z has 32 channels §3.2. The code reduces both inputs to 16 channels DuAT.py DuAT.py and ends in a 1-channel map DuAT.py DuAT.py.
- RAU and SBA gating. Equation 2 multiplies t2 by the reverse of gate1 eq. 2. The code's gating is different: it multiplies the upsampled, gated other input by
(1 − gate), and it adds two residual terms DuAT.py. - Input size. The paper resizes ISIC-2018 to 512×384 and 2018-DSB to 256×256 §4.1. The code resizes every training image to 352×352 dataloader.py and the default training size is 352 Train.py.
- Augmentation. The paper names rotation and horizontal flip §4.2. The code uses horizontal and vertical flips with p=0.2 each, rotation is commented out, and the
augmentationsargument is never used dataloader.py dataloader.py. The default--augmentationflag is False and has no effect Train.py. - Multi-scale training. The code trains each batch at three scales, 0.75, 1.0 and 1.25 Train.py. It takes one optimiser step per scale, so each batch costs three passes Train.py. The loss is logged at scale 1.0 only Train.py.
- Backbone name. The text says PVT §3.1; the ablation says PVTv2 §4.4. The code builds
pvt_v2_b2DuAT.py. - Metrics. The paper reports mean Dice, mean IoU and MAE §4.2. The repository's training script computes only a soft mean Dice on min-max normalised probabilities, with no threshold Train.py. Its test script saves maps and computes no metrics Test.py. The README says the paper used MATLAB evaluation [README §4.6].
- Learning-rate decay. The paper states no schedule §4.2. The code always multiplies the rate by 0.1 every 200 epochs, using constants, not the
--decay_rateand--decay_epochoptions Train.py Train.py.
Pitfalls
- The code does not run as shipped.
Train.pyandTest.pyimportlib.pvt, but the repository has onlylib/DuAT.pyandlib/pvtv2.pyTrain.py Test.py.Test.pyimportsscipy.misc, which recent SciPy versions no longer provide Test.py.Test.pyalso refers to a class namedPolypPVT, which is not defined in the repository. Uselib.DuATand fix the imports before training. The code also needsmmcv,timm,thopandalbumentationsDuAT.py DuAT.py dataloader.py utils.py. - Pretrained weights are not in the repository. The code loads
pvt_v2_b2.pthfrom./pretrained_pthand copies only keys that exist in the model, silently skipping the rest DuAT.py. Check that every backbone key was loaded. The README points to external downloads [README §4.3]. - Model selection uses a fixed test-like folder. After each epoch, the training script evaluates a folder called
testand saves the checkpoint with the best Dice as the final model Train.py. The paper does not say how checkpoints were selected. Keep a separate held-out set if you need unbiased numbers. - Dice is soft and unthresholded. The code's Dice is computed on min-max normalised probabilities without a threshold Train.py. This can differ from the MATLAB evaluation the paper used.
- Dataset folders are not included. The code expects folders named CVC-300, CVC-ClinicDB, Kvasir, CVC-ColonDB and ETIS-LaribPolypDB Train.py. The repository does not contain the data, so the paper's random 80/10/10 split §4.1 cannot be checked from the code. The paper also says it follows the PraNet and Polyp-PVT protocols §4.1, and those use fixed splits.
- Unused modules count as parameters.
ConvBranchcreates aChannelAttentionand aSpatialAttentionmodule thatforwardnever calls DuAT.py DuAT.py. They add parameters, so the parameter count in tab. 4 may not match an implementation without them. - Equation and table numbers come from source. No compiled PDF was available. Equation numbers were counted from the source, which gives 1 to 11. The loss equations carry the source labels
eq11andeq12, but by my count they print as 10 and 11, so the labels do not match the printed numbers. Check against the PDF. Tables are numbered 1 to 5 in source order, and figures 1 to 6. - Table anomalies. In Table 2, TransUNet's ETIS mean IoU (0.824) and TransFuse's (0.826) are higher than their mean Dice (0.731 and 0.737). Mean Dice is never below mean IoU for the same masks, so these are probably typos. Check them against the authors' data before using them.
- Text claims that the tables do not fully support. The text says DuAT beats all baselines on the unseen sets for all metrics §4.3. On ETIS, its MAE (0.013) ties Polyp-PVT (0.013) tab. 2. For 2018-DSB the text gives 1.0% for mean Dice and 0.03% for MAE over Polyp-PVT §4.3, while the table gives 0.9 points and 0.003 tab. 1.
- Ablation rows. The rows labelled "+GLSA" and "w/o SBA" are identical in every column tab. 5. They describe the same configuration. On ISIC-2018 the "+GLSA" mean IoU (0.816) is lower than the "+LSA" value (0.839) while its mean Dice is higher, so check that row.
- Naming. The text calls the unit RAU, but equation 2 writes PAU §3.2 eq. 2. Section 4.3 says "six" baselines and then lists eight entries, with TransFuse named twice §4.3.
- Cost. Each batch is processed three times at different scales, so training is about three forward and backward passes per batch Train.py. The authors report 24.92 M parameters and 9.88 GFLOPs at 352×352 tab. 4. These are author-reported and use the thop profiler DuAT.py.
- Unstated values to fill from the code or leave open. The paper gives no epoch count, learning-rate schedule, λ values, clipping rule or random seed. The code fills the first three (99 epochs, decay constants, element-wise clipping at ±0.5 utils.py). The seed is not set in the training script.
Lineage
- Builds on: the pyramid Transformer encoder (PVT / PVTv2) §3.1 §4.4; the reverse operation that the paper takes from PraNet §3.2; and the training protocols of PraNet and Polyp-PVT §4.1 §4.2. The paper cites these by name, and no card for them exists in this wiki.
- Compared against: U-Net, UNet++, PraNet, CaraNet, TransUNet, TransFuse, UCTransNet, SSformer and Polyp-PVT tab. 1 tab. 2 tab. 4.
- Superseded by: not known from the content in this wiki.
Source and license
Adapted from “DuAT: Dual-Aggregation Transformer Network for Medical Image Segmentation” by Feilong Tang, Qiming Huang, Jinfeng Wang, Xianxu Hou, Jionglong Su, Jingxin Liu (arXiv:2212.11677), used under CC BY 4.0. Changes: condensed and restructured for implementation, with tables, equations and algorithms taken from the paper and its code, each cited to its source. Not affiliated with or endorsed by the authors. Code citations refer to https://github.com/Barrett-python/DuAT at commit a075fe21fabb (none (no LICENSE file in the repository)). Any quoted code is shown under that license.