Salient object detection

Overview

Salient object detection (SOD) takes an image and outputs a map that gives every pixel a value for how likely it is to be part of the main foreground object. Models produce non-binary maps, while the ground truth (GT) is normally a binary mask 1708.00786 §1. This makes evaluation a design problem of its own: how should a soft map be scored against a hard mask?

The wiki has nine sources so far. One concerns the evaluation question, and the rest are models for image SOD or for neighbouring settings. The Structure-measure paper argues that the common scores (AP, AUC and the weighted F-measure Fbw) reduce a map to per-pixel TP/TN/FP/FN counts and so cannot tell apart errors that keep an object's shape from errors that break it 1708.00786 §2, §3. Its example: a square hole inside a digit and one cutting through the digit's outline give identical confusion counts 1708.00786 fig. 3. It proposes S-measure, which scores region-level and object-level structure instead 1708.00786 §4.

The EGNet paper is about a model. It trains one network to predict the object and its outline together, and feeds the outline features back into every scale of the object prediction 1908.08297 §3.2, §3.3. Its results are reported with max F-measure, MAE and S-measure.

The other sources extend the same per-pixel prediction to neighbouring settings. Two add a depth map to the colour image: D3Net 1907.06781 §4 and SwinNet, which also takes thermal images 2204.05585 §3.1. Human eye fixations 1705.02544 §1 and a neighbouring video frame, read alongside a static saliency map 1702.00871 §3.1, are two more. The remaining three apply it to camouflaged objects, which share texture and colour with their background, so that low-level edge and saliency cues mislead detectors 2102.10274 §1, §2 2105.09451 §1, §2.2 2103.04011 §1. SINet-V2 is a search-and-identification cascade for concealed objects. The Anabranch Network (ANet) multiplies a saliency-style map by an image-level camouflage probability, so the output is scaled by how likely camouflage is present at all 2105.09451 §4.2. COD-Rank adds a third head that ranks how hard each camouflaged object is to spot 2103.04011 §3.3.

The mask-based measures below share one contract: a non-binary prediction is compared with a binary GT of the same size, and the output is a scalar per image that is averaged over a dataset. Eye-fixation work is the exception. Its GT is a fixation map and its metrics (NSS, CC, AUC variants, EMD) are different 1705.02544 §4.2, so its numbers do not sit with the mask-based ones.

The S-measure paper ranks ten published models (ST, DRFI, DSR, DCL, rfcn, MC, MDF, DISC, DHS, ELD) with its measure 1708.00786 §5.5. Their methods are outside this wiki and have no cards.

Variants

Pixel-wise measures: AP, AUC, Fbw

These are the baselines the S-measure paper compares against 1708.00786 §5, tab. 1. Each treats pixels independently: the map is compared to the GT through confusion counts, either over thresholds (AP, AUC) or with per-pixel weighting (Fbw) 1708.00786 §2. The card does not give their formulas. The paper's criticism is that structure does not enter the score, and it shows Fbw ranking a blurry detection that only covers the top of a pyramid above one that keeps the whole shape 1708.00786 §3, fig. 3.

Structure-measure (S-measure)

S = α·S_o + (1 − α)·S_r with α = 0.5 1708.00786 eq. 10, §4.3.

• Region term S_r: split map and GT into four blocks at the centroid of the GT foreground, compute SSIM on each block without the usual stabilising constants, and sum with block weights 1708.00786 §4.1, eq. 3, eq. 4.
• Object term S_o: inside the GT foreground, reward a high and evenly spread predicted mean; on the GT background, do the same on the inverted map; weight the two by the foreground area fraction 1708.00786 §4.2, eq. 7, eq. 8, eq. 9.

The paper's experiments use λ = 0.5 and K = 4 blocks 1708.00786 §5. Runtime is 5.3 ms per image on one CPU thread in Matlab 1708.00786 §5. Card: Structure-measure: A New Way to Evaluate Foreground Maps.

Reported comparison across four datasets (lower is better on all three meta-measures; MM1 = 1 − Spearman ρ against a SalCut application ranking, MM2 = % of images where a generic map beats the state-of-the-art mean, MM3 = % of cases where a wrong GT raises the score) 1708.00786 §5.1, §5.2, §5.3, tab. 1:

1708.00786 tab. 1

In a user study with 45 viewers on 50 image pairs, people preferred the map that S-measure ranked first in 63.69% of trials against Fbw, 72.11% against AP and 73.56% against AUC 1708.00786 §5.6.

Edge-guided network: EGNet

EGNet runs two paths over one backbone 1908.08297 §3.2:

• Object path: a top-down pass over side features C3 to C6, where each level adds the upsampled level above it before a three-layer enhancement block 1908.08297 §3.2.1, eq. 3, tab. 1.
• Edge path: starts from the C2 side features, adds the top-level location, and is supervised by a cross-entropy edge loss 1908.08297 §3.2.2, eq. 2, eq. 4.

The edge features are added to each object-scale feature. Each of the four guided scales then predicts its own map, and the four maps are fused into the output 1908.08297 §3.3, eq. 7, eq. 9. Training sums the edge loss, deep supervision on every object side output and a loss on each guided map 1908.08297 eq. 6, eq. 10. At test time only the fused map is used 1908.08297 §4.1. The paper says its results come without any pre- or post-processing, unlike several baselines marked in its table 1908.08297 §4.4, tab. 2.

Author-reported max F-measure (β² = 0.3) 1908.08297 tab. 2:

The strongest baseline on DUTS-TE is PiCANet with ResNet, at 0.863 with post-processing 1908.08297 tab. 2. The ablation trains on DUTS-TR and tests on SOD and DUTS-TE. Adding the top-down edge path to the U-Net baseline lifts DUTS-TE max F from 0.855 to 0.879, and the full model reaches 0.893. Swapping in NLDF's edge penalty gives 0.866 [arxiv:1908.08297 §4.3, tab. 3]. The paper also reports the edge maps' own max F on DUTS, 0.539 against 0.429 for NLDF's edges 1908.08297 §4.3.3, tab. 4. EGNet also reports S-measure with γ = 0.5 1908.08297 §4.2, eq. 13; its ResNet DUTS-TE value is 0.875 1908.08297 tab. 2. Card: EGNet: Edge Guidance Network for Salient Object Detection.

RGB-D salient object detection: D3Net

D3Net adds a depth map to the colour image 1907.06781 §4. It has three VGG-16 feature streams: one on RGB, one on RGB-D stacked as four channels, and one on depth alone 1907.06781 §4.1. Each stream builds a feature pyramid and predicts its own map. At test time a depth depurator unit (DDU) compares the RGB-D map with the depth-only map by mean absolute difference. It keeps the RGB-D map when the difference is at most t = 0.15, and otherwise keeps the RGB map 1907.06781 §4.2, eq. 2, eq. 3, §4.3. The premise is that a poor depth map gives a cluttered scene, while a good one gives a clear object with closed boundaries 1907.06781 §4.2, fig. 7.

Each stream is trained alone with pixel-wise binary cross-entropy on the NJU2K and NLPR training pairs (1,485 and 700 pairs) [arxiv:1907.06781 §4.3, eq. 4]. The abstract says the parts are learned jointly, but the released code trains each stream in its own run (see the card's Paper vs code section) 1907.06781 abstract.

Author-reported S-measure on the test sets, D3Net against CPFP, the best published method in the paper's comparison 1907.06781 §5.5, tab. 4: NJU2K 0.900 against 0.879, NLPR 0.912 against 0.888, SIP 0.860 against 0.850. On LFSD, CPFP is ahead: 0.828 against 0.825. The abstract's claim of exceeding all prior work on all five metrics is therefore not borne out on every row 1907.06781 abstract, tab. 4. The ablation shows the gate is not always a gain: on SIP, D3Net scores 0.860, below the RGB-D stream alone at 0.862 1907.06781 tab. 5. Card: Rethinking RGB-D Salient Object Detection: Models, Data Sets, and Large-Scale Benchmarks.

Transformer RGB-D and RGB-T: SwinNet

SwinNet runs two Swin-B encoders, one on the colour image and one on the second modality 2204.05585 §3.1, §3.2. The paper describes the RGB-D case and says RGB-T uses the same design 2204.05585 §3.1. Swin's shifted windows let neighbouring windows exchange information, which the authors set against the limited long-range context of CNN backbones 2204.05585 §1.

At each of the four levels, a spatial attention map computed from the product of the two streams aligns them. Each stream is then re-weighted by channel attention that is computed on the aligned features and applied to the unaligned originals 2204.05585 eq. 1 to eq. 6. An edge feature built from the three shallowest levels of the depth stream guides the decoder 2204.05585 §3.4, eq. 7, eq. 8. Each level fuses the two streams by concatenating their sum and their element-wise product 2204.05585 eq. 9. Training sums a binary cross-entropy loss on the edge map and one on the saliency map. The edge targets are Canny edges of the ground truth 2204.05585 eq. 12, eq. 14, eq. 16, §3.6.1.

Training uses 384 × 384 inputs, Adam with learning rate 5 × 10⁻⁵ and batch size 3. The authors report about 26 hours on one RTX 2080 Ti 2204.05585 §4.2.

Author-reported S-measure is 0.941 on NLPR and 0.935 on NJU2K, against 0.931 and 0.922 for VST, the transformer baseline the paper singles out 2204.05585 §4.3, tab. 1. On RGB-T, S-measure is 0.912 on VT5000 against 0.883 for CGFNet 2204.05585 tab. 2. The RGB-T text says every metric is best, but CGFNet has the higher F-measure on VT1000 (0.906 against 0.896) 2204.05585 §4.3.2, tab. 2. In the paper's backbone ablation, Swin-B has the highest S-measure on all four RGB-D test sets against six other backbones 2204.05585 tab. 3.

Depth does not always help. On STERE, RGB alone scores S 0.923 against 0.919 for RGB plus depth, which the paper puts down to low-quality depth maps 2204.05585 §4.4.4, tab. 6. The model has 198.7 M parameters and 124.3 GFLOPs, and runs at about 10 FPS with I/O included, on hardware the paper does not state 2204.05585 §4.4.5, tab. 7. Card: SwinNet: Swin Transformer drives edge-aware RGB-D and RGB-T salient object detection.

Eye-fixation prediction: DVA

Deep Visual Attention Prediction (DVA) predicts where people look, not which object is salient. It is included here because its output is also a per-pixel map 1705.02544 §1. It uses a VGG-16 encoder with the final pooling layer removed, and three deconvolution decoders on conv3-3, conv4-3 and conv5-3. Each side output has its own cross-entropy loss, and a learned fusion layer merges the three maps 1705.02544 §3.1, §3.2, §3.3.2. It is trained on 10,000 SALICON images only 1705.02544 §3.3.3.

Scores use fixation metrics, not the mask measures above: NSS, CC, SIM, EMD and the AUC variants 1705.02544 §4.2. Author-reported: best on most metrics for MIT1003, TORONTO, PASCAL-S and DUT-OMRON, but not on MIT300, where DeeFix and SALICON score higher 1705.02544 §4.3, tab. 4, tab. 5. The DeeFix row on MIT1003 is fine-tuned on that dataset and is not comparable with the other rows 1705.02544 tab. 5. Runtime is about 0.1 s per image on one GPU 1705.02544 §3.3.3. Card: Deep Visual Attention Prediction.

Video saliency: dynamic FCN

Wang et al. extend the per-frame model to video with two networks 1702.00871 §3.1. The static network scores one frame. The dynamic network reads frame t, frame t+1 and the static map of frame t, and outputs the map for frame t. It computes no optical flow at test time, and the authors argue that flow, used by earlier video methods, dominates their running time 1702.00871 §1, §3.3, §5.4. The text does not say what happens to the last frame, which has no successor 1702.00871 §3.3.

The data is the hard part. Labelled video is small, so each annotated still image from MSRA10K and DUT-OMRON is turned into a two-frame training video, with ten simulated frames per image 1702.00871 §4, §5.1. The simulated frame is a warp of the image under a flow field built on superpixels 1702.00871 §4, eq. 5. Training then mixes these synthetic clips with about 3K real frame pairs from SegTrackV2 and the FBMS training set 1702.00871 §5.1.

Author-reported MAE, in percent (lower is better) 1702.00871 tab. 2: 7.65 on the FBMS test set and 6.36 on DAVIS for the full model. Training on image-derived video alone gives 9.27 and 7.53; video alone gives 24.5 and 23.9. Removing the static prior from the dynamic network gives 9.43 and 8.32 1702.00871 tab. 2. The paper plots its baselines and gives no numbers for them, so the card cannot back a numeric comparison 1702.00871 fig. 7. Because the metric is MAE in percent on video test sets, these values are not comparable with the image MAE values above or with S-measure. Speed is about 2 fps with all steps, on the authors' hardware 1702.00871 §5.4. Card: Video Salient Object Detection via Fully Convolutional Networks.

Concealed-object network: SINet-V2

SINet-V2 predicts a binary mask for concealed objects, a class-agnostic task 2102.10274 §2. It follows a search-then-identify pattern: a neighbor connection decoder finds a coarse region, and then three group-reversal attention (GRA) blocks per stage refine it 2102.10274 §4.1. The encoder is Res2Net-50, and only the top three feature levels are used 2102.10274 §4.2. Each stage reverses the coarser prediction, so the refinement targets the area the previous stage has not yet covered 2102.10274 §4.3, eq. 2.

Training uses Adam with learning rate 1e-4, batch size 36 and 100 epochs at 352 × 352 input, with the rate divided by 10 every 50 epochs. The authors report about four hours on one TITAN RTX 2102.10274 §4.4.2. Inference runs once with no post-processing, at about 45 fps on one GPU excluding I/O 2102.10274 §4.4.2.

Author-reported S-measure on three test sets 2102.10274 tab. 2: CHAMELEON 0.888, CAMO 0.820, COD10K 0.815. The next best S-measure on COD10K is PraNet's 0.789 2102.10274 tab. 2. Its dataset, COD10K (10,000 images), is described in 2102.10274 §3. Card: Concealed Object Detection.

Camouflage-gated segmentation: ANet

ANet pairs a saliency-style segmentation FCN with a classification stream 2105.09451 §4.2. The classifier has three fully connected layers, the first two with ReLU and 50% dropout, and a two-way softmax 2105.09451 §4.2, tab. 1. Its probability for "camouflaged" is a single scalar, and it multiplies every pixel of the segmentation map 2105.09451 §4.2. The authors use four FCNs: DHS, DSS, SRM and WSS 2105.09451 §5.3. Their names are only pointers here; their methods have no cards.

Training is two-step. The segmentation stream is first fine-tuned on CAMO, then frozen while the classifier is trained on CAMO-COCO, which adds non-camouflaged images 2105.09451 §5.1, §5.3. A joint variant that sums both losses is reported as mixed 2105.09451 §5.5.2, tab. 6.

Author-reported, with an adaptive threshold (mean plus standard deviation) 2105.09451 §5.2, tab. 3:

The paper's classifier accuracy on CAMO-COCO is 89.2–91.4% across the four ANet variants 2105.09451 tab. 4. Adding the classifier costs 3 to 10 ms per image on the paper's hardware 2105.09451 tab. 5. The authors expect ANet to be comparable with the FCN fine-tuned on CAMO, and say it can beat that FCN on CAMO-COCO 2105.09451 §5.4.2. Card: Anabranch Network for Camouflaged Object Segmentation.

Camouflage with difficulty ranks: COD-Rank

Lv et al. use one ResNet-50 network with three outputs: a fixation map for the discriminative region, a camouflaged-object mask, and a difficulty rank for each object 2103.04011 §3.2, §3.3. The fixation branch's output gives a reverse attention, 1 − F, which guides the mask branch [arxiv:2103.04011 §3.2]. The rank labels come from eye-tracker detection delays: six observers per image on the CAM-FR set of 2,000 training and 280 test images 2103.04011 §3.1. The paper gives the three rank levels but not the cut points between them 2103.04011 §3.1.

Author-reported, with the joint model trained on COD10K plus CAMO (4,040 images), COD10K S-measure is 0.793 against 0.776 for SINet on the same training set 2103.04011 tab. 4. On the smaller CAM-FR training set the joint model scores COD10K S-measure 0.760 against 0.733 for SINet 2103.04011 tab. 1. Its rank MAE is 0.049 jointly and 0.046 for a ranking-only model 2103.04011 tab. 5, so the joint model's ranking advantage is not shown. The released repository has no ranking branch (see the card's Paper vs code section). Card: Simultaneously Localize, Segment and Rank the Camouflaged Objects.

When to use which

Implementation sketch

Four cores. The first is the S-measure as its card documents it. Where the paper and the official code differ, the comment says which one is shown.

input: SM in [0,1] (float), GT binary, both H×W
m = mean(GT)
if m == 0: return 1 - mean(SM)          # code only, not in paper
if m == 1: return mean(SM)              # code only, not in paper

# object term
O_FG = 2·mean(SM[GT]) / (mean(SM[GT])² + 1 + 2λ·std(SM[GT]))
O_BG = same formula on (1 - SM)[not GT]
S_o  = m·O_FG + (1 - m)·O_BG

# region term
(X, Y) = rounded centroid of GT foreground
split SM and GT into 4 blocks at (X, Y)
for each block k: ssim_k = 4·x̄·ȳ·σ_xy / ((x̄² + ȳ²)(σ_x² + σ_y²))   # no C1, C2
w_k = foreground share of block k (paper) | pixel share of block k (code)
S_r = Σ w_k · ssim_k

S = 0.5·S_o + 0.5·S_r                    # code clamps negative S to 0

Implementation details that change the number: sample (N − 1) standard deviations and variances in the code, 1-based rounded centroid indexing, guarding empty blocks, and not substituting a library SSIM with constants and sliding windows 1708.00786 §4.1. The card's Pitfalls section lists each with code line references.

The second is the edge-guided core of EGNet. Shapes and the upsampling operator are simplified; the card has the full layer table.

input: image; side features C2..C6 from the backbone
# object path, top-down                     [arxiv:1908.08297 §3.2.1, eq. 3]
F6 = enhance(C6)
for i in 5..3: F_i = enhance(C_i + up(F_{i+1}, C_i))
# edge path: C2 plus top-level location     [arxiv:1908.08297 §3.2.2, eq. 2]
F_E = enhance(C2 + up(F6, C2))
edge_map = head(F_E)                        # trained with edge loss, eq. 4
# guide each object scale with edges        [arxiv:1908.08297 §3.3, eq. 7]
for i in 3..6: G_i = enhance'(up(F_i, F_E) + F_E); map_i = head'(G_i)
output = Σ β_i · map_i                      # β_i not stated, eq. 9

The third is D3Net's test-time depth gate. The paper's test is "at most t"; the released code uses a strict "below t", which the card records.

input: s_rgb, s_rgbd, s_depth in [0,1], H×W
# mean absolute disagreement between RGB-D and depth-only maps  [arxiv:1907.06781 §4.2, eq. 2]
d = mean(|s_rgbd - s_depth|)
# keep the RGB-D map only when the two agree within t = 0.15  [arxiv:1907.06781 §4.2, eq. 3, §4.3]
return s_rgbd if d <= t else s_rgb

The fourth is ANet's presence gate. Only the multiplication is shown; the fusion point and the classifier's input layer are not stated in the paper.

input: image; seg_map = FCN(image) in [0,1], H×W
# presence stream: three FC layers on FCN features  [arxiv:2105.09451 §4.2, tab. 1]
p_camo = softmax(fc3(drop(relu(fc2(drop(relu(fc1(feat))))))))[camo]
# fusion: scale every pixel by the image-level probability  [arxiv:2105.09451 §4.2]
out = p_camo · seg_map                      # which map and which index: not stated
# scoring: threshold mean + std, or fixed 0.5  [arxiv:2105.09451 §5.2]

The unstated values listed above are the ones an implementer has to choose and then report.

Contradictions

• λ inside the S-measure paper: the Fig. 2 caption gives λ = 0.25, the experiments use λ = 0.5 1708.00786 fig. 2, §5.
• Block weights: the text weights blocks by the GT foreground area they contain 1708.00786 §4.1; the card reports that the official code weights them by pixel area instead.
• Edge loss within the EGNet source: the paper's edge loss is plain cross-entropy over pixels 1908.08297 eq. 4. The EGNet card records that the released code weights edge and background pixels differently. The paper does not say whether its reported results used the balanced or the plain loss 1908.08297 §4.3.
• Size of the CAMO dataset: SINet-V2 says CAMO has 2,500 images 2102.10274 §2; ANet says CAMO has 1,250 images, split 1,000 for training and 250 for testing 2105.09451 §3.2, §5.1. The two papers use the same name, so numbers on "CAMO" cannot be compared until the sizes are resolved. Neither card resolves it.
• ANet-SRM on CAMO, two F values: SINet-V2 reports weighted F 0.484 for ANet-SRM on CAMO, with MAE 0.126 2102.10274 tab. 2. ANet reports F-beta 0.654 with the adaptive threshold, and MAE 0.126 [arxiv:2105.09451 tab. 3]. The MAE matches; the F values differ in metric and threshold, and the cards do not say which setting SINet-V2 used.
• The same baselines on COD10K, different numbers: SINet-V2 reports EGNet at S-measure 0.737 and the conference SINet at 0.771 on COD10K 2102.10274 tab. 2. COD-Rank reports EGNet at 0.751 and SINet at 0.776 on COD10K 2103.04011 tab. 4. Both papers retrained these baselines on their own training data (SINet-V2 on combined CAMO and COD10K training sets 2102.10274 §5.1; COD-Rank on 3,040 COD10K and 1,000 CAMO images 2103.04011 tab. 4). The gap is therefore a protocol difference, and neither side is wrong about its own table.
• Different metrics and test sets: SINet-V2 reports S-measure, E-measure, weighted F and MAE on CHAMELEON, CAMO and COD10K 2102.10274 §5.1; ANet reports F-beta, IoU and MAE on CAMO and CAMO-COCO [arxiv:2105.09451 §5.2]; COD-Rank reports S-measure, F-beta, E-measure and MAE on CAMO, CHAMELEON, COD10K and NC4K 2103.04011 §4.1; DVA uses fixation metrics on five eye-tracking sets 1705.02544 §4.2; the video model reports MAE in percent on DAVIS and FBMS 1702.00871 §5.2. Apart from the ANet-SRM MAE above and the shared COD10K baselines, no two papers share a metric and a test set, so the families cannot be ranked against each other.
• Video MAE, paper against released code: the paper reports 6.36% on DAVIS and 7.65% on FBMS 1702.00871 tab. 2. The card records that the authors' repository README reports 0.0587 on DAVIS and 0.0619 on FBMS for its Keras code, and calls these slightly better. The released code is a Keras re-implementation, while the paper says Caffe 1702.00871 §5.1. Neither card resolves which numbers match the paper's model.
• RGB-D numbers from D3Net and SwinNet: D3Net trains on 1,485 NJU2K and 700 NLPR pairs 1907.06781 §4.3. SwinNet's RGB-D training uses the same 2,185 pairs, and can add DUT for 2,985 2204.05585 §4.1.1. D3Net reports S-measure 0.900 on NJU2K and 0.912 on NLPR 1907.06781 tab. 4; SwinNet reports 0.935 and 0.941 2204.05585 tab. 1. The cards do not show whether the test splits and evaluation code match, so the gap is not resolved here.

Papers

• Structure-measure: A New Way to Evaluate Foreground Maps: structure-aware score for non-binary saliency maps; compares it with AP, AUC and Fbw on five datasets.
• EGNet: Edge Guidance Network for Salient Object Detection: object and edge paths over one backbone, with edge features guiding each object scale; author-reported max F-measure 0.893 on DUTS-TE with ResNet.
• Rethinking RGB-D Salient Object Detection: Models, Data Sets, and Large-Scale Benchmarks: three-stream RGB-D network with a test-time depth gate; adds the SIP phone-captured dataset and an 18-model benchmark.
• Deep Visual Attention Prediction: VGG-16 with three deconvolution decoders for eye-fixation maps; trained on SALICON, author-reported best on most metrics for four of five eye-tracking sets.
• Concealed Object Detection: SINet-V2, a search-then-identify network for concealed objects with reverse-guided refinement; author-reported S-measure 0.815 on COD10K.
• Anabranch Network for Camouflaged Object Segmentation: a camouflage classifier scales a saliency FCN's map; introduces the CAMO dataset; author-reported F-beta 0.826 on CAMO-COCO with ANet-SRM.
• Simultaneously Localize, Segment and Rank the Camouflaged Objects: one network for fixation, camouflage masks and difficulty rank; the ranking branch is not in the released code.
• Video Salient Object Detection via Fully Convolutional Networks: static and dynamic FCNs for video, with no optical flow; synthetic video from annotated images; author-reported MAE 7.65% on FBMS and 6.36% on DAVIS.
• SwinNet: Swin Transformer drives edge-aware RGB-D and RGB-T salient object detection: two Swin-B streams with per-level alignment and an edge-guided decoder; author-reported S-measure 0.941 on NLPR and 0.912 on VT5000.