--- kind: concept slug: guided-filter title: Guided filter summary: >- An edge-aware filter that uses a guide image to shape its output, and a differentiable form that can be trained inside a network to upsample a small output to full resolution. Covers the classical filter as post-processing and three learnable variants from the Deep Guided Filtering paper, the only card here. Numbers are author-reported, on MIT-Adobe FiveK and three computer-vision tasks. tags: [image-processing, joint-upsampling, guided-filter, dense-prediction] papers: ["1803.05619"] related: [feature-upsampling, guided-depth-upsampling, dilated-convolution, image-enhancement] status: draft updated_at: 2026-11-06 --- ## Overview The guided filter takes a guide image and an input image, and returns an output that follows the structure of the guide. For every local window it fits a linear model between the guide and the output [arxiv:1803.05619 §3.2] [eq. 1]. The filter comes from He et al. and is not on a card in this wiki; the paper uses it as its starting point [arxiv:1803.05619 §3.2]. The problem it addresses is cost on large inputs. A fully convolutional network (FCN) is expensive at full resolution in compute and memory. Shrinking the input, running the network and then upsampling loses edges, unless the upsampling is guided by the full-resolution image [arxiv:1803.05619 §1]. Earlier guided and bilateral upsamplers were hand-designed, could not be trained with the network, or were slow [arxiv:1803.05619 §2.1]. A plain guided filter could in principle be differentiated, but it was used only after training, so the network could not learn with it [arxiv:1803.05619 §3.3]. The paper's contribution is a guided-filter layer that is differentiable, and can therefore be trained end to end with a small FCN. The layer computes its coefficients on the low-resolution grid, upsamples them, and applies them to the full-resolution guide [arxiv:1803.05619 §3.2] [eq. 2]. The learnable convolutional versions also replace the fixed mean filter and linear model with trainable layers [arxiv:1803.05619 §3.5]. ## Variants All four variants come from one paper, and each is named in its paper's notation [arxiv:1803.05619 §4.2]. Card: [Fast End-to-End Trainable Guided Filter](/papers/1803.05619). ### Post-processing guided filter (DGF_s) The classical filter, with no learned parameters, applied after a separately trained FCN [arxiv:1803.05619 §4.2]. Training does not include the filter, so the network cannot adapt to it. ### Learnable layer with identity guidance (DGF_b) The differentiable layer is trained together with the FCN, using one loss [arxiv:1803.05619 §4.3]. The guidance is the identity, so the guide is the input image [arxiv:1803.05619 §3.4] [§4.2]. ### Convolutional layer (DGF_b^c) The mean filter is replaced by dilated 3x3 convolutions, with the dilation set to the radius r. The local linear model becomes a block of 1x1 convolutions that reads the local covariance and variance. The ε term is removed [arxiv:1803.05619 §3.5] [tab. 1]. Dilated convolutions are described on their own concept page. ### Convolutional layer with learned guidance (DGF^c) The convolutional layer above, plus a learned guidance map F(I). F(I) is a two-layer block: a 1x1 convolution, adaptive normalisation, a leaky ReLU and a second 1x1 convolution. Its first layer has 16 channels by default [arxiv:1803.05619 §3.4]. The paper's computer-vision table calls the full model DGF [arxiv:1803.05619 tab. 6]. ## When to use which The evidence here comes from one paper, with its own datasets and baselines. Each row below states what the paper's numbers support. | Situation | Use | Why | |---|---|---| | The FCN cannot be trained with the filter in the loop | DGF_s | It needs no joint training, because the filter has no learned parameters. On the paper's image tasks it has lower PSNR than DGF_b on all five [arxiv:1803.05619 tab. 2] | | Training from scratch, and the best PSNR among the four variants is the goal | DGF^c | Highest PSNR of the four variants on all five image tasks [arxiv:1803.05619 tab. 2] | | Training from scratch, without the learned guidance network F(I) | DGF_b^c or DGF_b | DGF_b^c is second to DGF^c on every image task in PSNR [arxiv:1803.05619 tab. 2]. The card does not give parameter counts or training times for F(I) | | Latency-critical, at high resolution | Any DGF variant | At 2048 x 2048 the paper reports 6, 6 and 21 ms for DGF_b, DGF_b^c and DGF^c, against 160 ms for CAN and 51 ms for DBL [arxiv:1803.05619 §5.4.1] | | Memory-critical, at high resolution | DGF_b or DGF_b^c | The paper reports memory staying under 1 GB up to 3072 x 3072 for the DGF_b variants [arxiv:1803.05619 §5.4.1]. The card records no memory figure for DGF^c | | Dense labelling or depth on your own data | Test before relying on it | The paper's computer-vision results come from its own splits, and the gains are small (see Contradictions) [arxiv:1803.05619 tab. 6] | | Need the best L0 smoothing or detail manipulation | Not this family, by these numbers | CAN is ahead of every DGF variant on both tasks in the paper's table [arxiv:1803.05619 tab. 2] | ## Implementation sketch The paper gives the layer as equations and a backward pass [arxiv:1803.05619 §3.3] [alg. 1]. The sketch below follows those equations in our own words. The repository has no LICENSE, so do not copy its code. ```pseudocode # Guided filtering layer, differentiable [§3.2] [§3.3] [eq. 1] [eq. 2] # Inputs: guide_lr, target_lr (B, C, h, w); guide_hr (B, C, H, W) # Output: output_hr (B, C, H, W) def guided_filter_layer(guide_lr, target_lr, guide_hr, radius, eps): # Window means from the mean filter of radius r [§3.3] mean_guide = box_mean(guide_lr, radius) mean_target = box_mean(target_lr, radius) # Local variance and covariance inside each window [§3.2] var_guide = box_mean(guide_lr * guide_lr, radius) - mean_guide ** 2 cov_gt = box_mean(guide_lr * target_lr, radius) - mean_guide * mean_target # Per-window linear coefficients; eps regularises the division [§3.2] coef_a = cov_gt / (var_guide + eps) coef_b = mean_target - coef_a * mean_guide # Bilinear upsampling of the coefficients is differentiable [§3.3] coef_a_hr = upsample_bilinear(coef_a, guide_hr.shape[2:]) coef_b_hr = upsample_bilinear(coef_b, guide_hr.shape[2:]) # Linear model applied to the full-resolution guide [eq. 2] return coef_a_hr * guide_hr + coef_b_hr ``` ```pseudocode # Deep guided filtering network, forward pass [§4] [fig. 3] # Input: image_hr (B, 3, H, W) in [0, 1] # Output: output_hr (B, 3, H, W) def dgf_forward(image_hr, low_size=64): # Shrink the input so its shorter side is low_size [§5.3] image_lr = resize_to_short_side(image_hr, low_size) # The FCN runs on the small image [§4.1] [tab. 1] output_lr = fcn(image_lr) # Guidance is the identity (DGF_b) or a learned F(I) (DGF^c) [§3.4] guide_lr = guidance(image_lr) guide_hr = guidance(image_hr) # The guided layer restores full resolution [§4] [eq. 2] return guided_filter_layer(guide_lr, output_lr, guide_hr) ``` Settings used in the paper's image-processing experiments: - Input I_l has a shorter side of 64 for every resolution [arxiv:1803.05619 §5.3]. - Training: 150 epochs at a shorter side of 512, then 30 more epochs at a random shorter side between 512 and 1672. Adam, learning rate 0.0001, batch size 1, L2 loss [arxiv:1803.05619 §5.3] [§4.3]. - Radius r = 1 and ε = 1e-8 by default [arxiv:1803.05619 §3.3]. In the computer-vision tasks the paper uses r = 4 for depth and segmentation, r = 8 for saliency, and ε = 1e-2 [arxiv:1803.05619 tab. 5]. - FCN: a context aggregation network with 24 channels and leaky ReLU slope 0.2 [arxiv:1803.05619 §5.2] [tab. 1]. The paper's text and its released code differ in several details that change results: layer count and dilations of the FCN, the first F(I) channel count, the ε default for the full model, output clamping, gradient clipping and the downsampling method. The paper card lists each difference with its code citation. Use the paper's values as the starting point, and check each difference against the card before relying on it. ## Contradictions - **Speed claim.** The paper's summary reports 10 to 100 times faster [arxiv:1803.05619 §5.4]. Its own timings at 2048 x 2048 give 51 ms for DBL and 160 ms for CAN, against 6 to 21 ms for the DGF variants [arxiv:1803.05619 §5.4.1]. Those figures are about 2.4 to 8.5 times for DBL and about 7.6 to 27 times for CAN. The 10 to 100 times claim is therefore not matched by these numbers against DBL, and only partly against CAN. - **Against CAN.** The paper states that DGF outperforms DBL on all five image tasks [arxiv:1803.05619 §5.4.2]. Its table shows CAN ahead on L0 smoothing and detail manipulation, and DGF^c ahead of CAN on style transfer, dehazing and retouching [arxiv:1803.05619 tab. 2]. - **Saliency.** On saliency, DenseCRF has a higher F-measure than DGF, 91.87% against 91.75%. DGF is ahead on segmentation, 73.58% against 72.69% [arxiv:1803.05619 tab. 6]. - **Inside the paper.** The text says both guidance layers of F(I) are 1x1 [arxiv:1803.05619 §3.4]. Table 1 gives the first layer a 3x3 kernel and dilation 1 [arxiv:1803.05619 tab. 1]. - **Upper bound.** The paper's upper-bound table replaces the FCN output with the low-resolution ground truth. Every DGF variant gains about 10 dB on retouching there, and the gap to the main table is the cost of the FCN output [arxiv:1803.05619 tab. 2] [tab. 3]. ## Papers - [Fast End-to-End Trainable Guided Filter](/papers/1803.05619): makes the guided filter a differentiable layer that can be trained end to end, with four variants (DGF) for image processing and dense prediction.