Inner-IoU: More Effective Intersection over Union Loss with Auxiliary Bounding Box

In plain words

Object detectors learn to place boxes by reducing a loss that measures how well a predicted box overlaps the true box. Inner-IoU keeps that idea but measures the overlap on smaller or larger versions of both boxes, each shrunk or stretched around its own centre. Boxes that already overlap well get a sharper signal from the smaller versions, which speeds up learning. Boxes that overlap poorly get more signal from the larger versions. The scale depends on the dataset and the detector, so there is no single fixed value. The method adds no new term to the loss, and the authors report small accuracy gains when they add it to existing box losses.

TL;DR

Inner-IoU replaces the IoU term of a box regression loss with the IoU of auxiliary boxes, which are the predicted and ground-truth boxes scaled by a ratio about their centres §3.2 eq. 15-21. Ratios below 1 are argued to speed up high-IoU regression and ratios above 1 low-IoU regression §3.1. The method is written for GIoU, DIoU, CIoU, EIoU and SIoU eq. 23-27, but the experiments use only CIoU and SIoU §4.2. Author-reported gains on VOC are 0.6 to 1.0 AP50 and 0.6 to 1.2 mAP50:95 points tab. 1. Each number is a single run, and the paper reports no variance.

Problem

The IoU loss gives an overlap-based signal for regression eq. 1-2, but the authors argue that it cannot adapt to different detectors and tasks and generalises poorly §1. Earlier work mostly keeps the IoU term and adds penalty terms: GIoU for non-overlapping boxes, DIoU and CIoU for centre distance and aspect ratio, EIoU for width and height, and SIoU for angle, distance and shape §1 §2.2. The paper instead changes the IoU term itself, with no added penalty §1. The experiments cover a general detection dataset (VOC) and a dataset of very small objects, with an average size of 12.8 pixels (AI-TOD) §4.2.

Method

Regression analysis §3.1. The paper compares IoU and its gradient for an actual box of size 10 with auxiliary boxes of sizes 8 and 12 §3.1 fig. 3. Three observations follow from that comparison:

• The IoU of auxiliary boxes changes during regression in the same way as the IoU of the actual boxes, so it still reflects regression quality §3.1 fig. 3.
• For high-IoU samples, the gradient of the smaller auxiliary box is larger in absolute value than the gradient of the actual box §3.1 fig. 3.
• For low-IoU samples, the gradient of the larger auxiliary box is larger in absolute value than the gradient of the actual box §3.1 fig. 3.

The authors conclude that a smaller auxiliary box helps high-IoU samples and a larger one helps low-IoU samples §3.1.

Inner-IoU §3.2. Given a ground-truth box and a predicted (anchor) box:

1. Scale each box's width and height by ratio about its own centre to get an auxiliary box. The centre is unchanged eq. 15-18.
2. Compute the overlap area of the two auxiliary boxes eq. 19.
3. Compute the union from the scaled areas, which is ratio squared times the sum of the two original areas, minus the overlap eq. 20.
4. Divide the overlap by the union to get the inner IoU eq. 21.
5. The base loss is one minus the inner IoU eq. 22.
6. For the penalised losses, the IoU term is replaced by the inner IoU, and the penalty terms are kept unchanged on the full boxes eq. 23-27.

IoU^{inner} = \frac{inter}{union},\quad union = (w^{gt} h^{gt} + w h)\,ratio^{2} - inter eq. 20-21

The paper says that ratio is typically between 0.5 and 1.5 §3.2. The penalty terms of the base losses are the standard ones: CIoU uses centre distance and aspect ratio eq. 5-7, and SIoU uses an angle term, a distance term and a shape term eq. 9-14.

# Inner IoU of two boxes  [eq. 15-21]
# pred, target: (x_center, y_center, width, height)
# ratio: scale of the auxiliary boxes, must be > 0  [§3.2]

def inner_iou(pred, target, ratio):
    # Auxiliary boxes: same centre, sides scaled by ratio  [eq. 15-18]
    aux_pred = scale_about_centre(pred, ratio)
    aux_target = scale_about_centre(target, ratio)

    # Overlap area of the auxiliary boxes, clamped at 0  [eq. 19]
    # [code: innerciou.py#L16-L17]
    inter = overlap_area(aux_pred, aux_target)

    # Union built from the scaled areas  [eq. 20]
    union = ratio**2 * (area(pred) + area(target)) - inter

    # Inner IoU  [eq. 21]
    return inter / union

# Penalised loss with the inner IoU  [eq. 23-27]
# base_loss: a penalised IoU loss on full boxes, e.g. CIoU or SIoU

def inner_penalised_loss(pred, target, ratio, base_loss):
    # Standard IoU of the full boxes  [eq. 1]
    iou = box_iou(pred, target)

    # Base loss, which already contains 1 - IoU  [eq. 2-5, 9]
    loss = base_loss(pred, target)

    # Swap the IoU term for the inner IoU; penalties stay on full boxes
    # [eq. 23-27]
    return loss + iou - inner_iou(pred, target, ratio)

# CIoU penalty on full boxes: centre term + alpha * v  [eq. 5-7]
# iou: standard IoU of the full boxes

def ciou_penalty(pred, target, iou):
    # Squared centre distance over squared enclosing diagonal  [eq. 5]
    centre_term = centre_dist_sq / enclosing_diag_sq

    # Aspect-ratio consistency term  [eq. 7]
    v = (4 / pi**2) * (atan(w_t / h_t) - atan(w / h))**2

    # Trade-off weight, treated as a constant in the code  [eq. 6]
    # [code: innerciou.py#L26-L27]
    alpha = v / ((1 - iou) + v)

    return centre_term + alpha * v

# SIoU penalty on full boxes: Delta + Omega  [eq. 9-14]
# dx, dy: centre offsets; dist: centre distance
# w_c, h_c: enclosing box sides; w_t, h_t: target sides

def siou_penalty(pred, target):
    # Angle term Lambda, using the axis with the smaller offset  [eq. 10]
    # [code: innersiou.py#L23-L30]
    angle = sin(2 * asin(min(abs(dx), abs(dy)) / (dist + eps)))

    # Gamma, which sets how strongly the distance terms act  [eq. 11]
    gamma = 2 - angle

    # Normalised centre offsets  [eq. 12]
    rho_x = (dx / w_c)**2
    rho_y = (dy / h_c)**2

    # Distance cost Delta  [eq. 11]
    delta = 0.5 * ((1 - exp(-gamma * rho_x)) + (1 - exp(-gamma * rho_y)))

    # Relative size differences  [eq. 14]
    omega_w = abs(w - w_t) / max(w, w_t)
    omega_h = abs(h - h_t) / max(h, h_t)

    # Shape cost Omega with theta = 4; exp(-omega) as in the code  [eq. 13]
    # [code: innersiou.py#L35-L37]
    omega = 0.5 * ((1 - exp(-omega_w))**4 + (1 - exp(-omega_h))**4)

    return delta + omega

Hyperparameters

Results

Author-reported. Setup: YOLOv7-tiny on VOC (train on VOC2007 and VOC2012 trainval, test on VOC2007 test), and YOLOv5s on AI-TOD, compared with SIoU §4.2. The gains in parentheses are the paper's own differences.

• Simulation: the paper reports that the inner variants converge faster than the baselines for ratio 0.8 on high-IoU samples and ratio 1.2 on low-IoU samples. This is shown as curves only, with no numbers §4.1 fig. 7.
• Training curves on VOC are described as better than the baselines from epoch 50 to 150. This is also shown only as curves §4.2 fig. 8.
• Example detections are described as more accurate, with fewer false and missed detections. This is shown only as images §4.2 fig. 2 fig. 6.

Paper vs code

The abstract links the repository, so it is used as the official code. It is small: two Python files with one function each, and a README that links only to external guides README.md. There is no training code, detector integration, configs, simulation code or weights. The ratios and results in the paper cannot be reproduced from the repository alone.

Checked against the paper:

• Inner IoU matches. The auxiliary boxes, overlap and union are built as in eq. 15-21 innerciou.py.
• The functions return the inner IoU minus the penalties, not a loss. Taking one minus that value gives the penalised losses in eq. 25 exactly. The caller is not in the repository innerciou.py.
• The SIoU angle term matches eq. 10. The code picks the smaller axis offset through a threshold of sqrt(2)/2. That gives the same value as the paper's minimum innersiou.py.
• The SIoU distance cost matches eq. 11. The code's expression is twice Delta, and the code multiplies it by 0.5 innersiou.py.
• The penalty terms use the full boxes in both files, as eq. 23-27 require innerciou.py innersiou.py.

Discrepancies and gaps:

• Shape cost sign. The paper typesets eq. 13 as (1 - e^(omega)) to the power theta. The code uses exp(-omega) innersiou.py. The paper's form appears to be missing a minus sign. This card uses the code's form.
• Alpha is treated as a constant. The code computes CIoU's alpha from the standard IoU, not the inner IoU, under torch.no_grad(), and adds 1e-7 in the denominator innerciou.py. The paper's eq. 6 says nothing about gradients or epsilon.
• Clamping. The code clamps the overlap width and height at 0 [code: innerciou.py#L16-L17] innersiou.py. The paper's eq. 19 has no clamp, though it is implied by the definition of overlap.
• Unused argument. Both functions take xywh=True, but never read it. Boxes are assumed to be (centre x, centre y, width, height). The paper does not state a box format innerciou.py innersiou.py.
• Default ratio. The code default is 1.0, which makes the inner loss equal to the base loss. The per-experiment ratios are not in the repository [code: innerciou.py#L1].
• Not in the repository: the YOLO training setup, the VOC and AI-TOD training settings, the epochs and the simulation code. Any check of tab. 1, tab. 2 or fig. 7 has to be done from the paper.

Pitfalls

• The penalty terms stay unscaled. Only the IoU term becomes the inner IoU. Scaling the distance, aspect, angle or shape terms by the ratio is not what the paper describes eq. 23-27 innerciou.py.
• Overlapping boxes can have disjoint auxiliary boxes. With a ratio below 1, two boxes that overlap can have disjoint auxiliary boxes. The clamp then gives the IoU term zero gradient innerciou.py. The paper does not discuss this case. For CIoU and SIoU the penalty terms still give gradients.
• No selection rule for the ratio. The paper shows several ratios on the test set and does not describe a validation split or a rule for choosing one §4.2. The reported gains may therefore be optimistic. A new dataset has no guidance beyond the typical range of 0.5 to 1.5 §3.2.
• Mixed AI-TOD result. The VOC text says the gains are above 0.5 points in AP50 and mAP50:95 §4.2. On AI-TOD, ratio 1.10 lowers mAP50:95 by 0.17 [tab. 2]. The AI-TOD gains are not uniform.
• No variance. Each row is one number. The paper gives no count of runs, seeds or standard deviations §4.2. The size of a 0.6-point gain cannot be judged from the paper alone.
• Only CIoU and SIoU were tested. GIoU, DIoU and EIoU are written into the method eq. 23-27, but no experiment uses them §4.2.
• Ratio must stay positive. The paper gives no bounds beyond the typical range, and the code has no check. A ratio of 0 or below makes the auxiliary boxes degenerate eq. 15-21.
• Extra cost is not measured. The paper reports no runtime or memory cost for the inner boxes §4.2.
• Figure and table numbers. Some figure labels in the LaTeX source do not match the numbers used in the text. This card follows the text's numbering for figures §4.1 §4.2.

Lineage

• Builds on: the IoU loss §1 [ref. 1 in the paper's bibliography], GIoU, DIoU, CIoU, EIoU and SIoU §1 §2.2. SIoU is arXiv:2205.12740 in the paper's bibliography. None of these has a card in this wiki, so none is linked. The size-versus-IoU analysis cites Xu et al. 2021 §3.1.
• Compared against: CIoU and SIoU §4.2 tab. 1 tab. 2.
• Superseded by: not stated.