Hausdorff Distance Matching with Adaptive Query Denoising for Rotated Detection Transformer

In plain words

Object detectors built on transformers, such as DETR, find objects by guessing a fixed set of boxes and then deciding which guess belongs to which real object. This works well for upright boxes. Boxes that are tilted, such as aerial photos of ships or planes, are harder: the usual distance between two boxes gets confused near the angle wrap-around, and square-ish objects get several near-identical guesses with low confidence.

The paper changes two parts of this pipeline. First, when a guess is paired with a real object, it measures how far apart the two rotated boxes are by looking at their corners, using a Hausdorff distance. That gives a more faithful pairing than comparing box numbers directly. Second, the model trains on noisy copies of the true boxes to speed up learning. Early on these copies are useful, but later the model's own guesses are better than the noisy copies, so the paper keeps only those copies that the matching step still links to their original object.

The authors report that this lifts accuracy on several aerial and rotated-object benchmarks, mainly on AP50. These are the authors' own numbers.

TL;DR

• Extends DETR to rotated boxes by predicting a fifth value, the angle, as part of the ordinary box regression §3.2 app. A.
• Replaces the L1 term in the matching cost with a Hausdorff distance between box corners §3.4 eq. 3 tab. 7b.
• Adds adaptive query denoising: a noised query keeps its classification target only if Hungarian matching assigns it to its own ground truth §3.5 eq. 5 eq. 7.
• Author-reported AP50 on DOTA-v2.0, DOTA-v1.5, DOTA-v1.0 and DIOR-R with ResNet-50: 59.26, 71.96, 78.68 and 71.79 tab. 2 tab. 3 tab. 4 tab. 5.
• The code is CC BY-NC 4.0, so it is cited for comparison only. The paper's own text and a mismatch between one claim and its table are listed under Pitfalls.

Problem

Query-based detectors (DETR and its successors) match a fixed set of predicted boxes to ground truths. For rotated boxes the authors report two failure modes §3.3:

• The regression cost used for matching (L1 on the five box values) favours matches with the same angle even when another prediction has the higher overlap. This is tied to the angle boundary and to near-square shapes, and it produces duplicate low-confidence predictions fig. 2.
• Standard denoising training (DINO's contrastive denoising) keeps every noised query as a training target. The authors observe that many noised positives are poorly suited to accurate classification later in training, and they treat such targets as unhelpful §3.5 fig. 4.

The paper assumes the usual DETR setting: a fixed query budget, Hungarian matching and no NMS for the main predictions §3.1.

Method

The baseline is a rotated DETR built from Deformable DETR, DAB-DETR, DN-DETR and DINO, with a five-value box §3.2 app. A. The three contributions are described below.

Baseline. Each box is (x, y, w, h, θ), with the long-edge definition θ in [0, π). Every value is scaled to [0, 1] by dividing by the image width or height, and by π for the angle app. A. The paper replaces the GIoU term with a KLD-based IoU term (matching cost and loss) and sets its weight to 5 in all experiments except the patch-level ablation app. A. Deformable attention uses the rotated reference box, and the decoder updates the angle in logit space app. A.

The total training objective is a classification term plus a box term, and the box term is an L1 part plus an IoU part eq. 1 eq. 2:

\mathcal{L}_\mathrm{train} = \lambda_\mathrm{cls} \mathcal{L}_\mathrm{cls} + \mathcal{L}_\mathrm{box} eq. 1

\mathcal{L}_\mathrm{box} = \lambda_\mathrm{L1} \mathcal{L}_\mathrm{L1} + \lambda_\mathrm{iou} \mathcal{L}_\mathrm{iou} eq. 2

The paper does not give numeric values for these λ. The code uses the values listed under Hyperparameters.

Hausdorff matching cost. The cost used for the bipartite assignment replaces the L1 term with a symmetric Hausdorff distance between the corner sets of the predicted box and the ground-truth box §3.4 eq. 3. Corners are computed after scaling x, y, w and h by the image size and leaving the angle unscaled, so the angle enters only through the corner positions §3.4. The paper's default uses four corners per box; the appendix tests 8 and 32 points, with 32 giving the best AP75 and 4 the best AP50 app. C tab. 8b.

# Hausdorff matching cost for one image  [§3.4] [eq. 3]
# Input:  pred_boxes (N, 5) as (cx, cy, w, h, theta), gt_boxes (M, 5)
# Output: cost matrix (N, M)
def hausdorff_cost(pred_boxes, gt_boxes, img_w, img_h):
    # Scale x, y, w, h by the image size; keep theta in radians  [§3.4]
    scale = [img_w, img_h, img_w, img_h, 1.0]
    pred_corners = box_corners(pred_boxes / scale)  # (N, 4, 2)
    gt_corners = box_corners(gt_boxes / scale)      # (M, 4, 2)
    # For each corner of A, distance to the nearest corner of B; take max
    pred_to_gt = directed_hausdorff(pred_corners, gt_corners)  # (N, M)
    gt_to_pred = directed_hausdorff(gt_corners, pred_corners)  # (M, N)
    # Symmetric distance: the larger of the two directed distances  [eq. 3]
    return elementwise_max(pred_to_gt, transpose(gt_to_pred))

Adaptive query denoising (AQD). Denoising groups are built from each ground truth: positive and negative noised queries are generated from it by adding noise §3.5. In the paper's formulation, the positive queries and the predictions are matched together, so each ground truth either keeps its own positive query or is matched to a prediction instead eq. 6 eq. 7. A positive query keeps its classification target only if it matches its own ground truth. Otherwise it is trained as background eq. 5 alg. 2. Negative queries are always trained as background eq. 4.

The improved variant, labelled AQD* in the tables, also applies the box loss to filtered positives, so that they keep learning to localise eq. 8 alg. 2. The ablation shows that AQD* beats plain AQD tab. 7a. The headline RHINO model uses the Hausdorff cost together with AQD* tab. 7a app. B.

# Adaptive query denoising for one denoising group  [§3.5] [alg. 2]
# Input:  preds (N, 5), gts (M, 5), labels (M,),
#         pos_queries (M, 5), neg_queries (M, 5)
# Output: denoising loss for this group
def adaptive_dn_loss(preds, gts, labels, pos_queries, neg_queries):
    # Refine the noised queries through the decoder layers  [§3.5]
    pos_refined = refine(pos_queries)
    neg_refined = refine(neg_queries)
    # One matching over positive queries plus predictions  [eq. 6] [eq. 7]
    matches = hungarian(cost(concat(pos_refined, preds), gts))
    for each ground truth i:
        # Keep the label only if gt i matched its own positive query  [eq. 5]
        if matches[i] is positive query i:
            target[i] = labels[i]
        else:
            # Filtered positive: trained as background  [eq. 5]
            target[i] = background
        # Box loss on every positive, including filtered ones  [eq. 8]
        box_loss[i] = l_bbox(pos_refined[i], gts[i])
    # Negative queries are always background  [eq. 4]
    cls_loss = focal_loss(neg_refined, background) + focal_loss(target)
    return cls_loss + box_loss

The equations an implementer needs are the per-group denoising loss eq. 4, the positive-query loss with its match indicator eq. 5, the matching step eq. 7 and the improved positive-query loss eq. 8.

Hyperparameters

Values marked "paper" come from the text or appendix. Values marked "code" are in the official repository at the commit above and are not stated in the paper. For implementers who want the headline R-50 DOTA setting, use the 36-epoch configuration.

Results

Author-reported, single-scale training and testing on the test sets, with ResNet-50 unless stated. Each number is cited to its table. Values compare the paper's own rows; the authors' comparison set is the table itself.

Reported gains: +4.18 AP50 on DOTA-v2.0 and +4.59 AP50 on DOTA-v1.5 over the R-50 counterparts in the paper's tables §4.2. The text says +0.96 on DOTA-v1.0; see Pitfalls.

Paper vs code

The repository is SIAnalytics/RHINO (owner SIAnalytics, linked from neither the paper nor arXiv; found via GitHub search, described as the official implementation of the WACV 2025 paper). The authors' affiliation is SI Analytics Co. Ltd. [title block] at commit d9baf3bc1149ebef2ef03c89b26b93106b82075a. The repository's LICENSE file is CC BY-NC 4.0, so the code is non-commercial. No code text is quoted here; differences are cited by path and line only.

• IoU weight. The paper says the IoU weight of 5 applies to all experiments except patch validation app. A. The 12-epoch base config keeps 2.0 for both the matching cost and the loss rhino-4scale_r50_2xb2-12e_dota.py rhino-4scale_r50_2xb2-12e_dota.py. The 36-epoch configs used for the headline numbers set 5.0 rhino_phc_haus-4scale_r50_2xb2-36e_dota.py rhino_phc_haus-4scale_r50_2xb2-36e_dota.py.
• Hausdorff cost weight. The paper does not state it. The code sets 5.0 rhino_phc_haus-4scale_r50_2xb2-36e_dota.py.
• Denoising matching granularity. The paper writes one matching over all positive queries and predictions eq. 6 eq. 7. The code runs a separate matching for each denoising group. Each group's positive queries are matched against a copy of the full prediction set, and the groups are kept apart by a block-diagonal cost matrix with a large penalty off the blocks dn_group_hungarian_assigner.py dn_group_hungarian_assigner.py.
• Which AQD. The code gives every positive denoising query a box weight, including filtered ones rhino_phc_head.py. This is the AQD* behaviour eq. 8. The paper's headline RHINO numbers correspond to AQD* tab. 7a.
• Filtering rule. The code keeps a positive's classification target only when the denoising assignment returns that query's own ground truth rhino_phc_head.py rhino_phc_head.py. Filtered positives are moved to the negative set rhino_phc_head.py. This matches eq. 5.
• Angle in the Hausdorff cost. The code scales the angle by 1 for the Hausdorff cost, which matches the paper's factor match_cost.py §3.4. The angle scale for the regression head is π rotated_detr_head.py.
• Hausdorff points. The default is four corners match_cost.py match_cost.py, consistent with the paper. For more than four points the code places points along the edges and moves the cost computation to CPU at 700 or more corners match_cost.py.
• Denoising noise. The code adds noise to x, y, w and h only, consistent with the appendix. Negative queries get a larger offset in the range 1 to 2 rhino_layers.py. The paper does not describe the negative range.
• Group cap. The 36-epoch configs cap denoising groups at 30 rhino_phc_haus-4scale_r50_2xb2-36e_dota.py. Not in the paper.
• Batch size. The paper says total batch 8 on two GPUs (app. B). The 12-epoch base config is named 2xb2 but sets batch_size=4 per dataloader rhino-4scale_r50_2xb2-12e_dota.py. Use 8 total as the paper says; the file name does not match the number in the file.
• Denoising label noise and weight decay. Label noise 0.5, weight decay 1e-4 and gradient clipping at 0.1 appear only in the code rhino-4scale_r50_2xb2-12e_dota.py rhino-4scale_r50_2xb2-12e_dota.py rhino-4scale_r50_2xb2-12e_dota.py.
• KLD transform. The paper does not give the form of the KLD cost. The config uses log1p with τ = 1 and no square root. The transform maps d to 1 − 1 / (1 + log1p(d)) gaussian_dist_loss.py rhino_phc_haus-4scale_r50_2xb2-36e_dota.py.

No discrepancies found beyond these items on the parts checked.

Pitfalls

• Text versus table, DOTA-v1.0. The text claims +0.96 AP50 over the closest R-50 competitor §4.2. In Table 4 the highest R-50 row is PSD-SQ at 78.46, which gives a gap of 0.22. The +0.96 does not follow from the table. Use the table values.
• Matching formula. The paper writes the permutation over size M+N−1 while the sum runs over M+N indices eq. 6 eq. 7. The code uses one rectangular assignment per group, with M columns for ground truths and M+N rows (see Paper vs code). Implement the code's version, or check the appendix pseudocode alg. 2.
• Positive-query box loss. Filtered positives still receive a box loss, which is the AQD* choice eq. 8. Plain AQD eq. 5 drops that term. Paper and code agree on AQD*, but a reader following eq. 5 alone would implement the weaker version.
• Denoising matching cost. The assignment runs once per decoder layer and per image, and uses the CPU linear-sum solver rhino_ph_head.py dn_group_hungarian_assigner.py. The paper reports 25 h to 30 h of training on DOTA-v1.0 tab. 9. Budget for this.
• Hausdorff scale. Corners are measured in image-normalised units, so distances are small; the Hausdorff cost weight of 5 is not stated in the paper match_cost.py.
• Points versus AP75. More Hausdorff points raise AP75 but not AP50. The default of four points gives the best AP50 tab. 8b app. C.
• Limitation on COCO. The authors report that AQD on DINO trained on COCO lowers AP by 1.6 relative to the baseline; they attribute this to horizontal objects being less affected by noisy queries app. G.
• Table 7 mismatch. Table 7a reports 78.05 for DINO + Hausdorff + AQD, and Table 7c reports 78.06 for AQD at noise 1.0 tab. 7a tab. 7c. The paper does not say whether these are the same run. The difference is small.
• Unstated values. The λ weights, the number of training epochs behind the headline numbers and the DN label noise are not in the text. The Hyperparameters table gives the code values; the paper does not confirm them.

Lineage

• Builds on: DINO (contrastive denoising, mixed query selection, look forward twice) §3.2; DN-DETR (query denoising) §3.2; Deformable DETR (deformable attention, iterative refinement, two-stage) §3.2 app. A; DAB-DETR (queries as dynamic anchor boxes) §3.2 app. A; KLD loss for rotated boxes app. A. None of these has a wiki card yet.
• Compared against: ARS-DETR, AO2-DETR, D2Q-DETR, EMO2-DETR, DCFL, ReDet, Oriented R-CNN, RoI Transformer, PSD-SQ, PKINet, ViTAE-B with RVSA tab. 2 tab. 3 tab. 4 tab. 5.
• Superseded by: not known from content already in this wiki.