Paper · 2023-12-11 · cs.CV
Upscale-A-Video: Temporal-Consistent Diffusion Model for Real-World Video Super-Resolution
Upscale-A-Video (x4) adapts the pretrained SD x4 Upscaler to video. Temporal layers in the U-Net and a fine-tuned VAE decoder keep 8-frame windows consistent, and training-free RAFT-guided latent propagation at chosen denoising steps links windows. Author-reported: highest PSNR on four synthetic sets; best CLIP-IQA and DOVER on VideoLQ and AIGC30.
Image super-resolutionLatent diffusionOptical flowClassifier-free guidancevideo-super-resolutionlatent-diffusiontemporal-consistencyoptical-flowreal-world-restoration
arXivPDFCodeMarkdown16 min read
AI-compiled by claude-haiku-5-5 on 2026-11-13, not yet reviewed by a human. Every claim cites its source; check it when a detail matters.
In plain words
Upscaling a video with a diffusion model makes each frame sharp, but the result can shimmer from frame to frame. A diffusion model starts each step from random noise and works on a compressed version of the picture (a latent), and the decoder that turns that latent back into pixels can add flicker of its own. Both effects break the picture apart over time.
The authors start from an image upscaler and teach it to look at a few neighbouring frames at a time. They also retrain the decoder so that it flickers less. Frames that are far apart can still drift, so the model carries detail from earlier frames forward through the clip. It lines the pixels up using motion estimates, and it blends them in only where the motion check says they match. This step runs during some of the denoising steps (the steps that remove noise one at a time) and needs no training.
The user can describe the scene in text to guide the fine detail that gets added. They can also choose how much noise to add to the input. Lower noise keeps the result closer to the original, while higher noise lets the model invent more detail.
TL;DR
Upscale-A-Video is a text-guided latent diffusion model for 4x video upscaling, initialised from the pretrained SD x4 Upscaler §3.1. Temporal attention and 3D residual blocks in the U-Net keep 8-frame windows consistent §3.2. A VAE decoder fine-tuned with a spatial feature transform layer reduces flicker §3.2. A training-free, RAFT-guided latent propagation at user-chosen denoising steps links the windows §3.3. The author-reported results give the highest PSNR on all four synthetic benchmarks tab. 1. The paper describes bidirectional propagation, but the official inference code uses only the forward pass, and it uses different thresholds from the paper's defaults. Read Paper vs code before reproducing.
Problem
- Diffusion sampling is random, so frames sampled one after another can jump. In latent diffusion, the VAE decoder adds flicker to fine texture on top of that §1.
- Coherence is needed inside short segments and across the whole video, not only locally §3.
- Flow-guided long-range propagation helps video restoration with CNNs and transformers. Its memory cost limits diffusion models to short clips, so it does not carry over directly §3.3.
- Real-world inputs carry unknown mixed degradations: downsampling, noise, blur, flicker and compression §1. Training data uses the degradation pipeline of RealBasicVSR §4.1. The paper does not restate its parameters.
- The authors aim to carry what the image x4 upscaler learned over to video, which they say also makes training more efficient §3.1.
- Without the temporal and propagation modules, the ablation shows clear loss: warping error E*warp goes from 0.737 with both modules to 2.398 with neither tab. 2.
Method
fig. 1 gives the overview: the video is split into segments, a U-Net with temporal layers works inside each segment, a propagation module runs during chosen steps, and a fine-tuned VAE decoder finishes the frames.
Base model: latent diffusion from an image upscaler §3.1
- The base is the pretrained, text-guided SD x4 Upscaler. It is a latent diffusion model whose autoencoder downsamples by a factor of 4, and its denoiser is conditioned on the low-resolution image §3.1.
- The low-resolution image is also corrupted with noise of a chosen level tau, an augmentation that helps the model add fine detail §3.1.
- The network predicts the v-parameterisation, v = alpha * epsilon - sigma * x. Training minimises the squared error of Eq. 1, where c holds the text prompt and tau eq. 1 §3.1.
- To use video, 2D convolutions are inflated into 3D convolutions and the network starts from the image upscaler's weights §3.1.
Local consistency inside segments §3.2
- The U-Net gets temporal layers: temporal self-attention over the frames of a segment, plus 3D residual blocks built on 3D convolutions. Rotary position embedding (RoPE) gives the attention its time positions §3.2.
- Only the inserted temporal layers are trained, with the Eq. 1 loss. The pretrained spatial layers stay frozen §3.2. The segment length is eight frames §3.3.
- The VAE decoder gets temporal 3D residual blocks. Its spatial layers stay frozen, and only the new layers are trained §3.2.
- The fine-tuning loss is an L1 term plus an LPIPS perceptual term plus an adversarial term from a temporal PatchGAN discriminator. The loss weights are not stated §3.2.
- A spatial feature transform (SFT) layer takes the input video and changes the first decoder layer's features, which carries low-frequency colour information through to the output §3.2.
- The ablation rows (a) and (c) of tab. 2 compare the decoder with and without this fine-tuning. The qualitative temporal profile is in fig. 5.
Global consistency across segments: training-free recurrent latent propagation §3.3
- The optical flow between frames comes from RAFT §3.3. The paper says the flow resolution matches the latent resolution, so no resizing is needed §3.3.
- Each flow is checked by forward-backward consistency. Eq. 2 measures the mismatch at position p between one flow and the other flow warped along the first one. Positions whose mismatch is below a threshold delta form a mask M, which acts like an occlusion mask. The paper does not give delta eq. 2 §3.3.
- The predicted clean latent is updated through Eq. 3. Where the mask is 1, the warped latent from the previous frame is blended in with weight beta. Everywhere else the value stays put. Warping uses nearest-neighbour sampling. beta = 0.5 by default eq. 3 §3.3.
- The module runs in both directions, forward and then backward, and the backward pass uses the same process §3.3. fig. 2 illustrates the propagation.
- The module does not run at every step. The user picks T* steps. Early steps suit minor jitter, and late steps suit severe jitter such as in AI-generated video §3.3.
Conditions at inference §3.4
- Text prompts guide texture detail, for example animal fur or oil-painting strokes §3.4 fig. 12. Training used the prompts of WebVid10M, and null prompts for YouHQ, which has none §4.1.
- The noise level tau trades restoration against generation. Smaller values favour restoration and larger values give more detail §3.4 fig. 6.
- Classifier-free guidance is applied during inference and strengthens both the prompt and the noise-level effects §3.4.
Inference loop (own summary of §3.1 to §3.4)
# Local-global upscaling loop [§3.1] [§3.2] [§3.3] [§3.4]
# Input: low-res video (T, C, H, W); prompt; noise level tau
# Output: upscaled video (T, C, 4H, 4W)
def upscale_video(video, prompt, tau, propagation_steps):
# Add noise of level tau to the low-res condition [§3.1] [§3.4]
condition = add_noise(video, tau)
# Optical flow between neighbouring frames, both directions [§3.3]
flows_forward, flows_backward = raft(video)
# Start from Gaussian latents at the low-res spatial size [§3.1]
latents = random_normal(T, C_latent, H, W)
for step, t in enumerate(timesteps):
# The denoiser sees windows of 8 frames [§3.2] [§3.3]
v_pred = unet_on_windows(latents, condition, prompt, tau, t)
# Classifier-free guidance on the text branch [§3.4]
v_pred = guide(v_pred, guidance_scale)
# Predicted clean latent from the v-prediction [§3.1]
clean_latents = clean_estimate(latents, v_pred, t)
if step in propagation_steps:
# Blend warped earlier latents where flow checks pass [§3.3]
# Flow-check mask (Eq. 2), blend (Eq. 3) [eq. 2] [eq. 3]
clean_latents = propagate(
clean_latents, flows_forward, flows_backward)
# Move to the next, less noisy latent [§3.1]
latents = sampler_step(clean_latents, v_pred, t, latents)
# Decode with the fine-tuned VAE decoder [§3.2]
return vae_decode(latents, condition)Hyperparameters
Values not given in the paper are marked "not stated". Code defaults are listed separately and are not the paper's values.
| Name | Value | Where | Notes |
|---|---|---|---|
| Base model | SD x4 Upscaler, pretrained | §3.1 | Text-guided latent diffusion |
| Parameterisation | v-prediction | §3.1 eq. 1 | |
| Latent downsampling factor f | 4 | tab. 3 app. A.1 | |
| U-Net channels | 256, 512, 512, 1024 (multipliers 1, 2, 2, 4) | tab. 3 | Matches unet_video_config.json |
| U-Net depth | 2 | tab. 3 | Matches unet_video_config.json |
| U-Net attention heads | 8 | tab. 3 | Matches unet_video_config.json |
| Cross-attention dim / sequence length | 1024 / 77 | tab. 3 | Dim matches unet_video_config.json |
| Attention resolutions | 40, 20, 10 | tab. 3 | Not checked against code |
| VAE channels | 128 (multipliers 1, 2, 4) | tab. 3 | Matches vae_3d_config.json |
| Temporal positional encoding | RoPE | tab. 3 §3.2 | |
| 3D CNN kernel | 3, 1, 1 | tab. 3 | Not checked against code |
| Segment length | 8 frames | §3.3 §3.2 | Overlap and stride are not stated in the paper |
| Training patch | 8 x 320 x 320 x 3 (pixels); latent 8 x 80 x 80 x 4 | tab. 3 app. A.1 | Text in §4.1 says "cropped to 80x80"; see Pitfalls |
| Batch size | 384 | §4.1 | 32 x NVIDIA A100-80G |
| Optimiser, learning rate | Adam, 1e-4 | §4.1 | Schedule and warm-up not stated |
| U-Net iterations | 70K on WebVid10M and YouHQ, then 10K on YouHQ only | §4.1 app. C.1 | |
| U-Net data | WebVid10M subset (about 335K pairs) and YouHQ (about 37K clips, 1080x1920) | §4.1 app. B.1 | |
| VAE decoder fine-tuning data | 100K synthetic LQ-HQ video pairs | §4.1 | Optimiser, iterations, batch, loss weights: not stated |
| Degradation pipeline | As in RealBasicVSR | §4.1 | Parameters not restated |
| Propagation fusion weight beta | 0.5 | §3.3 | |
| Propagation warping | Nearest neighbour | §3.3 | |
| Consistency threshold delta | Not stated | §3.3 | Code values in Paper vs code |
| Propagation steps T* | User-chosen; middle steps are the default | §3.3 app. D.3 | Exact step indices for the defaults not stated |
| Noise level tau (main results) | Not stated | §3.4 | |
| Guidance scale | Not stated | §3.4 | |
| Sampling steps, sampler | Not stated | §3.1 | |
| Evaluation metrics | PSNR, SSIM, LPIPS, E*warp (x 10^-3); CLIP-IQA, MUSIQ, DOVER for real and AIGC data | §4.1 tab. 1 |
Results
Author-reported. Setup: four synthetic sets built with the training degradation (SPMCS, UDM10, REDS30 and YouHQ40, a 40-video split of YouHQ). Two sets without ground truth: VideoLQ (real videos) and AIGC30 (30 AI-generated videos) §4.1. Baselines: Real-ESRGAN, SD x4 Upscaler, ResShift, StableSR, RealVSR, DBVSR and RealBasicVSR §4.2. The paper's colour marking of the best and second-best values is not reproduced here.
Table 1: quantitative comparison tab. 1 (E*warp shown x 10^-3 as in the caption; "Ours" is Upscale-A-Video)
| Dataset | Metric | Real-ESRGAN | SD x4 Upscaler | ResShift | StableSR | RealVSR | DBVSR | RealBasicVSR | Ours |
|---|---|---|---|---|---|---|---|---|---|
| SPMCS | PSNR up | 22.89 | 23.19 | 23.27 | 22.71 | 23.88 | 24.28 | 24.51 | 25.32 |
| SPMCS | SSIM up | 0.669 | 0.631 | 0.667 | 0.657 | 0.681 | 0.726 | 0.717 | 0.741 |
| SPMCS | LPIPS down | 0.238 | 0.304 | 0.257 | 0.231 | 0.437 | 0.302 | 0.198 | 0.222 |
| SPMCS | E*warp down | 1.364 | 5.008 | 4.942 | 4.815 | 0.294 | 1.360 | 0.559 | 0.367 |
| UDM10 | PSNR up | 27.13 | 28.07 | 27.62 | 26.45 | 27.38 | 29.60 | 29.11 | 30.79 |
| UDM10 | SSIM up | 0.843 | 0.811 | 0.827 | 0.825 | 0.825 | 0.880 | 0.876 | 0.878 |
| UDM10 | LPIPS down | 0.190 | 0.186 | 0.222 | 0.181 | 0.278 | 0.155 | 0.172 | 0.133 |
| UDM10 | E*warp down | 1.462 | 1.710 | 2.196 | 2.797 | 0.531 | 1.943 | 0.602 | 0.446 |
| REDS30 | PSNR up | 22.40 | 22.98 | 23.00 | 23.72 | 23.05 | 24.37 | 23.91 | 24.41 |
| REDS30 | SSIM up | 0.591 | 0.572 | 0.580 | 0.635 | 0.603 | 0.633 | 0.636 | 0.631 |
| REDS30 | LPIPS down | 0.303 | 0.399 | 0.369 | 0.352 | 0.658 | 0.588 | 0.249 | 0.335 |
| REDS30 | E*warp down | 3.658 | 3.753 | 4.131 | 1.645 | 0.378 | 9.659 | 1.557 | 1.278 |
| YouHQ40 | PSNR up | 24.37 | 19.71 | 23.77 | 24.53 | 24.19 | 25.37 | 24.09 | 25.83 |
| YouHQ40 | SSIM up | 0.710 | 0.579 | 0.654 | 0.711 | 0.695 | 0.719 | 0.689 | 0.733 |
| YouHQ40 | LPIPS down | 0.272 | 0.442 | 0.376 | 0.271 | 0.484 | 0.430 | 0.306 | 0.268 |
| YouHQ40 | E*warp down | 1.856 | 3.399 | 4.426 | 1.529 | 0.485 | 1.149 | 1.052 | 0.737 |
| VideoLQ | CLIP-IQA up | 0.360 | 0.158 | 0.430 | 0.344 | 0.211 | 0.274 | 0.387 | 0.530 |
| VideoLQ | MUSIQ up | 49.48 | 26.21 | 40.95 | 44.23 | 24.52 | 29.15 | 55.33 | 57.99 |
| VideoLQ | DOVER up | 7.161 | 2.884 | 4.679 | 6.783 | 2.531 | 3.628 | 7.562 | 7.811 |
| AIGC30 | CLIP-IQA up | 0.430 | 0.329 | 0.569 | 0.467 | 0.276 | 0.290 | 0.565 | 0.674 |
| AIGC30 | MUSIQ up | 47.09 | 35.30 | 43.32 | 44.93 | 24.39 | 27.22 | 58.87 | 57.66 |
| AIGC30 | DOVER up | 9.710 | 5.646 | 7.042 | 9.668 | 3.285 | 3.523 | 10.68 | 11.67 |
What the authors claim from this table §4.2: the highest PSNR on all four synthetic sets; the lowest LPIPS on UDM10 and YouHQ40; the best E*warp on UDM10 and second-best on SPMCS, REDS30 and YouHQ40; and the highest CLIP-IQA and DOVER on VideoLQ and AIGC30. On MUSIQ, Upscale-A-Video is best on VideoLQ, while RealBasicVSR is best on AIGC30 tab. 1.
A user study with 20 participants compared Upscale-A-Video with StableSR, SD x4 Upscaler and RealBasicVSR on real and AI-generated videos. Each participant saw 10 random triplets and picked the better result. The paper reports a clear preference for Upscale-A-Video, with the counts in fig. 9 app. D.1.
Table 2: ablation on YouHQ40 tab. 2
| Exp. | Fine-tuned VAE decoder | Latent propagation | PSNR up | SSIM up | E*warp down |
|---|---|---|---|---|---|
| (a) | no | no | 23.82 | 0.6385 | 2.398 |
| (b) | no | yes | 25.47 | 0.7215 | 1.815 |
| (c) | yes | no | 25.75 | 0.7328 | 0.842 |
| (d) | yes | yes | 25.83 | 0.7326 | 0.737 |
Table 4: pretrained prior ablation, YouHQ40 tab. 4 app. D.2
| Metric | SD | SD (+ colour fix) | SD x4 Upscaler | SD x4 Upscaler (+ colour fix) |
|---|---|---|---|---|
| PSNR up | 19.03 | 23.81 | 25.83 | 26.07 |
| SSIM up | 0.590 | 0.632 | 0.733 | 0.737 |
| LPIPS down | 0.383 | 0.343 | 0.268 | 0.267 |
| E*warp down | 1.821 | 1.707 | 0.737 | 0.738 |
Table 5: propagation position, YouHQ40 tab. 5 app. D.3. The column headers name the step sets as the paper gives them. The paper does not say which decoder this table uses; see Pitfalls.
| Metric | w/o propagation | Early {4, 5, 6, 7} | Middle {14, 15, 16, 17} | Late {24, 25, 26, 27} |
|---|---|---|---|---|
| PSNR up | 23.82 | 24.18 | 24.53 | 24.10 |
| SSIM up | 0.639 | 0.646 | 0.671 | 0.670 |
| E*warp down | 2.398 | 1.931 | 0.638 | 0.618 |
The paper states that later propagation lowers the warping error but also lowers fidelity, so the middle position is the default app. D.3.
Paper vs code
The official repository is https://github.com/sczhou/Upscale-A-Video. The project page in the paper's author block links it (checked with WebFetch), and its owner is the first author. It is pinned at commit 10ca1d751cad25b8ac0b9b0cab415a82d9df1ac6 (dated 2024-09-27, after the arXiv v1 date). The repository contains inference code, the configs and the RAFT code. It contains no training code for the U-Net or the VAE decoder. The pretrained weights are not in the repository: the README links a Google Drive folder README.md and the clone's pretrained_models/ directory is empty.
- Bidirectional propagation. The paper says the module runs forward and backward, and that the two passes are combined §1 §3.3. The inference code creates the propagator with
learnable=Falseinference_upscale_a_video.py. In that mode the module returns only the forward result (outputs = outputs_f). The backward pass is computed and then discarded propagation_module.py propagation_module.py. - Consistency thresholds. The paper gives no value for delta §3.3. The module's defaults are alpha1 = 0.01 and alpha2 = 0.5 propagation_module.py. The pipeline calls it with alpha1 = 0.001 and alpha2 = 0.05 pipeline_upscale_a_video.py. The test is |sum of the two flows|^2 < alpha1 * (|f|^2 + |w|^2) + alpha2, not a single fixed threshold propagation_module.py.
- Order of flows in Eq. 2. Eq. 2 starts from the flow f(i-1 to i) and calls p "the position of the last frame latent", without saying which frame that is §3.3. The code's first term is the flow it also uses for warping, and it evaluates the mask on the grid of frame i propagation_module.py. This is a choice the code makes, not a contradiction; use the code's order if you reproduce it.
- Propagation steps. The paper's default is the middle of the schedule app. D.3. The CLI default is no propagation at all, since
-pis empty inference_upscale_a_video.py. The README sets steps by hand, for example-p 24,26,28with 30 steps README.md and-p 8,10,12with 30 steps README.md. Step indices are 0-based, because the loop enumerates the timesteps from 0 pipeline_upscale_a_video.py. - Text prompt. The paper uses a user-supplied prompt §3.4. By default the script runs LLaVA-13B to caption the first frame and prepends that caption to the prompt, and it appends the suffix "best quality, extremely detailed" inference_upscale_a_video.py inference_upscale_a_video.py CKPT_PTH.py. The flag
--no_llavaremoves the caption, but the suffix stays unless--a_promptis set to an empty string. The default negative prompt is "blur, worst quality" inference_upscale_a_video.py pipeline_upscale_a_video.py. - Noise level. The paper gives no value for the main results §3.4. The CLI default is 120, and its help text gives a range of 0 to 200 inference_upscale_a_video.py. The pipeline accepts up to 350 pipeline_upscale_a_video.py. The noise is added to the input and also passed as a class label, which matches the paper's conditioning pipeline_upscale_a_video.py.
- Other inference defaults not in the paper. Guidance scale 6 inference_upscale_a_video.py. 30 denoising steps inference_upscale_a_video.py. Seed 10 for the noise inference_upscale_a_video.py. DDIM-style scheduler with eta = 0 pipeline_upscale_a_video.py. The paper names neither the sampler nor the steps.
- Overlapping windows. The code uses windows of 8 frames with an overlap of 2, so the stride is 6 pipeline_upscale_a_video.py. The final window is shifted back so that it still has 8 frames pipeline_upscale_a_video.py. Where windows overlap, each new prediction is averaged in with a running 0.5/0.5 weight pipeline_upscale_a_video.py. The paper gives only the 8-frame length §3.3.
- Decoding. The VAE decodes the latents in chunks of 3 frames pipeline_upscale_a_video.py. The paper gives no chunk size. The temporal image-space propagation is commented out pipeline_upscale_a_video.py.
- Tiling. The script tiles automatically when the low-res area is 384 x 384 or more inference_upscale_a_video.py, with tiles of 256 and an overlap of 64 inference_upscale_a_video.py inference_upscale_a_video.py. The paper describes overlapping patches but gives no sizes app. C.2.
- Very large inputs. When both sides are at least 1280 px, the input is first downsampled by 4 inference_upscale_a_video.py. The output is then about the input size, not 4x larger. The paper does not mention this.
- Colour correction. The script's default is no colour correction inference_upscale_a_video.py. The paper's appendix shows that wavelet correction improves fidelity for the SD-prior variant app. C.3 tab. 4. The paper does not say whether the main results used it.
- Architecture hyperparameters. The U-Net channel multipliers, depth and head count match the paper's Table 3 unet_video_config.json. So do the VAE channels vae_3d_config.json. The temporal kernel and attention resolutions were not checked.
- Training. The repository has no training code for the U-Net or the VAE decoder. The only training files are LLaVA's. The training recipe in §4.1 and app. C.1 cannot be checked against code.
- Release status. The paper says the code and models "will be publicly released" app. A.1. The code is public, but the weights are on Google Drive and their licence was not checked. The YouHQ training set is linked in the README README.md.
- Licence. The code is under the S-Lab License 1.0, which permits non-commercial use only LICENSE. This card describes the code but quotes none of it.
Pitfalls
- Training crop size is ambiguous. The text says the training data is "cropped to 80 x 80 with a length of 8" §4.1. Table 3 gives an 8 x 320 x 320 pixel patch and an 8 x 80 x 80 latent tab. 3 app. A.1. Read the 80 x 80 as the latent size, which is 320 pixels at the 4x factor.
- The decoder recipe is incomplete. The optimiser, iteration count, batch size, loss weights and PatchGAN configuration for the VAE decoder fine-tuning are not stated §3.2 §4.1. The degradation parameters are not restated either.
- Uneven weights at the clip end. The running 0.5/0.5 average over windows gives unequal weights when a frame falls in three windows. For example, a 13-frame clip gives windows starting at frames 0, 5 and 5. Frames 5 to 7 then take 0.25 from the first window and 0.75 from the second pipeline_upscale_a_video.py. The paper does not describe this.
- Table 5 is not the main setting. The no-propagation row of Table 5 equals row (a) of Table 2 exactly, so Table 5 is most likely run without the fine-tuned VAE decoder tab. 2 tab. 5. The paper does not say so. Its middle-position PSNR of 24.53 is not comparable with the main result of 25.83 tab. 2.
- Prompt and caption defaults change the output. The default run adds a caption and a suffix that the paper does not describe inference_upscale_a_video.py. To match the paper's "no prompt" setting, pass
--no_llavaand an empty--a_prompt. The LLaVA model is a third-party dependency with its own terms. - Propagation thresholds and step set are the largest unknown. Delta is not given. The code's values are 10 times lower in alpha1 and 10 times lower in alpha2 than its module defaults, and they change the mask propagation_module.py pipeline_upscale_a_video.py. The step set for each result is not given either.
- Cost is not reported. The paper gives no runtime or memory figures [not stated]. The inference loop runs the U-Net once per window per step, and with guidance on the batch is doubled to carry the empty-prompt branch as well pipeline_upscale_a_video.py. Flow estimation and the optional LLaVA captioner add to that.
- Licence limits reuse. The code is non-commercial. The weights' licence, the WebVid10M and YouHQ data terms, and the LLaVA model's terms were not checked. Do not reuse any of them commercially without a separate check.
- Results are author-reported. The metrics are PSNR, SSIM, LPIPS and E*warp on synthetic data, and CLIP-IQA, MUSIQ and DOVER on real and AI-generated data. The non-reference metrics do not use ground truth. The user study numbers are only in the figure fig. 9.
Lineage
- Builds on: the pretrained SD x4 Upscaler and latent diffusion (LDM) §3.1; inflated 3D convolutions, as in other video diffusion models §3.1; RoPE §3.2; a spatial feature transform layer §3.2; a temporal PatchGAN discriminator §3.2; RAFT optical flow §3.3; the forward-backward consistency check §3.3; the RealBasicVSR degradation pipeline §4.1; and the decoder fine-tuning procedure from StableSR §4.1. None of these has a card linked here.
- Compared against: Real-ESRGAN, SD x4 Upscaler, ResShift, StableSR, RealVSR, DBVSR and RealBasicVSR §4.2 tab. 1.
- Superseded by: not known from wiki content.
Source and license
Adapted from “Upscale-A-Video: Temporal-Consistent Diffusion Model for Real-World Video Super-Resolution” by Shangchen Zhou, Peiqing Yang, Jianyi Wang, Yihang Luo, Chen Change Loy (arXiv:2312.06640), used under CC BY 4.0. Changes: condensed and restructured for implementation, with tables, equations and algorithms taken from the paper and its code, each cited to its source. Not affiliated with or endorsed by the authors. Code citations refer to https://github.com/sczhou/Upscale-A-Video at commit 10ca1d751cad (S-Lab License 1.0 (non-commercial)). Any quoted code is shown under that license.