--- kind: concept slug: oriented-object-detection title: Oriented object detection summary: >- Detect objects as rotated boxes (x, y, w, h, angle), mainly in aerial images where objects have arbitrary heading. Two variants: a rotation-equivariant backbone with rotated RoI pooling (ReDet), and an end-to-end transformer with rotation-aware attention (OrientedFormer). Choose by whether you can change the backbone, your compute budget and whether you want to avoid NMS. tags: [computer-vision, object-detection, remote-sensing] papers: ["2103.07733", "2409.19648"] related: [object-detection, object-queries, positional-encoding, bipartite-matching, equivariance-constraint, euclidean-equivariant-network] status: draft updated_at: 2026-10-19 --- ## Overview An oriented detector outputs, for each object, a class, a score and an oriented box (x, y, w, h, θ). The angle lets the box follow an object that is tilted in the image, which matters for aerial photos where objects have no preferred direction and are often packed closely [arxiv:2103.07733 §1]. The rotated box is the one thing every variant shares: its centre, size and angle are the state the detector predicts and the loss compares [arxiv:2409.19648 §3.2] [arxiv:2103.07733 §4]. The variants differ in where rotation is handled. One keeps the image backbone rotation-aware, so an object seen at a new angle yields a predictably rotated feature map [arxiv:2103.07733 §3] [arxiv:2103.07733 §4.1]. The other keeps a standard backbone and makes the transformer decoder aware of box angle, through the positional encoding, the attention bias and the sampling points [arxiv:2409.19648 §3.3] [arxiv:2409.19648 §3.5]. ## Variants ### Rotation-equivariant backbone with rotated RoI pooling (ReDet) The backbone is a ResNet-50 with FPN whose layers are equivariant to the discrete rotation group C8, so orientation becomes a channel axis and filters are shared across the eight directions [arxiv:2103.07733 §4.1]. The RPN and a RoI Transformer head produce rotated RoIs, and RiRoI Align pools each one in two steps: a spatial warp of the rotated box, then a cyclic shift of the orientation channels by the box angle with linear interpolation between the two nearest bins [arxiv:2103.07733 §4.2] [arxiv:2103.07733 eq. 8] [arxiv:2103.07733 eq. 9]. Card: [Rotation-equivariant detector for aerial objects](/papers/2103.07733). The reported backbone size is 12 Mb against 103 Mb for the R50-FPN baseline, at a DOTA-v1.5 score of 66.86 against 65.03 [arxiv:2103.07733 tab. 1]. The paper says the equivariant backbone has about one N-th of the baseline's parameters when channel counts match [arxiv:2103.07733 §4.1]. The released RiRoI kernel is a CUDA extension, so the method needs that build step. ### End-to-end transformer with rotation-aware decoder (OrientedFormer) A DETR-style decoder with two decoder layers and 300 queries, with no encoder [arxiv:2409.19648 §3.1] [arxiv:2409.19648 alg. 1] [arxiv:2409.19648 tab. 9]. Three changes adapt it to oriented boxes. The box is encoded as a 2-D Gaussian, so position, size and angle enter the positional encoding together [arxiv:2409.19648 eq. 2] [arxiv:2409.19648 eq. 6]. Self-attention gets a bias from the Wasserstein distance between box Gaussians [arxiv:2409.19648 eq. 8] [arxiv:2409.19648 eq. 9]. Cross-attention samples features at points that are rotated with the box and weighted across feature levels [arxiv:2409.19648 §3.5] [arxiv:2409.19648 eq. 14] [arxiv:2409.19648 eq. 16]. Matching is one-to-one with the Hungarian algorithm and the loss is a focal classification term, an L1 term and a rotated-IoU term [arxiv:2409.19648 §3.6] [arxiv:2409.19648 eq. 19]. Card: [OrientedFormer](/papers/2409.19648). Author-reported ResNet-50 results are 67.28 AP50 on DIOR-R against 66.12 for ARS-DETR, and 75.37 AP50 on DOTA-v1.0 single-scale against 74.16 for ARS-DETR [arxiv:2409.19648 tab. 3] [arxiv:2409.19648 tab. 4]. The authors also report convergence in 12 epochs against 36 for two end-to-end baselines [arxiv:2409.19648 §4.8]. The cost is higher: 529 GFLOPs against 186 for ARS-DETR and Deformable DETR-O, at 11.4 FPS on hardware the table does not name [arxiv:2409.19648 tab. 14]. ## When to use which | Situation | Use | Why | |---|---|---| | You can replace the backbone and want a smaller model file | Rotation-equivariant backbone | The equivariant ResNet-50 is 12 Mb against 103 Mb, with a higher DOTA-v1.5 score in the authors' runs [arxiv:2103.07733 tab. 1] | | You want to avoid rotation augmentation as the main source of rotation robustness | Rotation-equivariant backbone | The rotation-augmented baseline reaches 64.07 at 1x and 67.34 at 2x, against 66.66 for the equivariant model at 1x, which trains in 13 hours against 11 and 22 [arxiv:2103.07733 tab. 4] | | You want a DETR-style detector with no hand-written box suppression in the decoder | End-to-end transformer | The decoder is trained with one-to-one matching and no NMS [arxiv:2409.19648 §3.6]. The released code does use rotated NMS when choosing queries (see the [OrientedFormer card](/papers/2409.19648)) | | Compute or latency is tight | Rotation-equivariant backbone, if the backbone is the bottleneck | OrientedFormer costs 529 GFLOPs against 186 for its closest end-to-end baselines [arxiv:2409.19648 tab. 14]. Speed for the equivariant backbone is not stated, so no speed comparison between the two is possible from these cards | | Training budget is limited | End-to-end transformer | The authors report 12 epochs to reach their score, against 36 for two baselines [arxiv:2409.19648 §4.8] | | The top single-scale DOTA-v1.0 score matters more than the design | Check the table before choosing | On single-scale DOTA-v1.0 the cards show ReDet at 76.25 [arxiv:2103.07733 tab. 6] and OrientedFormer at 75.37, with Oriented R-CNN at 75.87 for a different method [arxiv:2409.19648 tab. 4]. The metric labels differ (see Contradictions), so these are not a like-for-like ranking | | Few labelled objects or unusual domains | Not stated | Neither card reports a low-data experiment | Both cards report results on DOTA and related sets. Neither reports results on a non-aerial oriented task, so transfer outside remote sensing is not supported by the cards. ## Implementation sketch The shared core is a rotated region read-out: sample features on a grid that lies in the box's own frame, then rotate that grid to the box angle. ReDet adds an orientation shift for its equivariant channels. OrientedFormer rotates the sampling offsets in its attention layer instead. ```pseudocode # Oriented box: centre, size, angle [arxiv:2103.07733 §4] [arxiv:2409.19648 §3.2] # Input: feature maps; candidate (x, y, w, h, theta); grid size S # Output: region feature of shape (S*S, C) def oriented_region_feature(features, box, S): # Regular grid inside the box, in the box's own frame local = grid_in_box(box.w, box.h, S) # (S*S, 2) # Rotate the grid by the box angle and move it to the box centre # [arxiv:2409.19648 eq. 14] (rotation about the centre, as in the code) c, s = cos(box.theta), sin(box.theta) px = box.x + c * local[:, 0] - s * local[:, 1] # (S*S,) py = box.y + s * local[:, 0] + c * local[:, 1] # (S*S,) # Bilinear sample at the rotated points [arxiv:2103.07733 §4.2 (a)] region = bilinear(features, px, py) # (S*S, C) # Equivariant backbones only: cyclic shift and interpolation of the # orientation channels by theta [arxiv:2103.07733 eq. 8] [eq. 9] if features.has_orientation_axis: region = orientation_align(region, box.theta) return region ``` The transformer variant trains with a rotated IoU term on matched boxes [arxiv:2409.19648 eq. 19], using one-to-one Hungarian matching. ## Contradictions - **ReDet's abstract and tables disagree.** The abstract gives gains of 1.2, 3.5 and 2.6 mAP over the previous best. The tables give 0.68, 1.90 and 2.62 against the strongest other row [arxiv:2103.07733 tab. 6] [arxiv:2103.07733 tab. 7] [arxiv:2103.07733 tab. 8]. Implement from the tables. - **ReDet labels one number two ways.** The single-scale DOTA-v1.0 score of 76.25 is labelled AP50 in one table and mAP in another [arxiv:2103.07733 tab. 5] [arxiv:2103.07733 tab. 6]. OrientedFormer reports AP50 throughout [arxiv:2409.19648 tab. 4]. Scores from the two cards are therefore not guaranteed to share a metric. This is not resolved here. - **The 80.10 ReDet result is not a pure equivariance result.** It uses rotation augmentation and test-time rotation [arxiv:2103.07733 §5.4]. Compare it with the single-scale rows only. - **ReDet's paper and code differ.** The paper's ablation over 1, 2 and 4 orientation channels [arxiv:2103.07733 tab. 3] cannot be reproduced from the released kernel, which always blends two channels. The paper also states 15 anchors per location [arxiv:2103.07733 §5.2]; the config uses 3. The code enlarges RoIs by 1.2 and 1.4 before RiRoI Align, which the paper does not describe. - **OrientedFormer's paper and code differ.** The code multiplies the self-attention bias by a learnable scale per head and divides the Wasserstein distance by an extra normalisation term. Neither is in the equations as printed [arxiv:2409.19648 eq. 9]. Implementing from the equations alone gives different attention behaviour. - **OrientedFormer's sampling-point text disagrees with its table.** The text says the count goes "from 4 to 32", while the table starts at 8 [arxiv:2409.19648 tab. 9]. The sampling exponent in the equation also does not match the paper's own definitions [arxiv:2409.19648 eq. 11] [arxiv:2409.19648 eq. 13]. Follow the definitions. ## Papers - [Rotation-equivariant detector for aerial objects](/papers/2103.07733): ReDet, a C8 rotation-equivariant ResNet-50 with FPN and RiRoI Align for rotated RoIs. - [OrientedFormer](/papers/2409.19648): end-to-end transformer detector for oriented boxes, with Gaussian positional encoding, Wasserstein attention bias and oriented cross-attention.