🐼太真实 (2024-02-29 10:04):
#paper ProPainter: Improving Propagation and Transformer for Video Inpainting 本文介绍了一种新的视频修复技术——ProPainter,通过双域传播和掩码引导稀疏视频Transformer的设计,实现了高效而准确的视频修复。文章详细介绍了ProPainter的三个关键组成部分:循环流场完成、双域传播和掩码引导稀疏视频Transformer,并提供了相应的技术细节和实验结果。
ProPainter: Improving Propagation and Transformer for Video Inpainting
Shangchen Zhou, Chongyi Li, Kelvin C. K. Chan, Chen Change Loy
Abstract:
Flow-based propagation and spatiotemporal Transformer are two mainstream<br>mechanisms in video inpainting (VI). Despite the effectiveness of these<br>components, they still suffer from some limitations that affect their<br>performance. Previous propagation-based approaches are performed separately<br>either in the image or feature domain. Global image propagation isolated from<br>learning may cause spatial misalignment due to inaccurate optical flow.<br>Moreover, memory or computational constraints limit the temporal range of<br>feature propagation and video Transformer, preventing exploration of<br>correspondence information from distant frames. To address these issues, we<br>propose an improved framework, called ProPainter, which involves enhanced<br>ProPagation and an efficient Transformer. Specifically, we introduce<br>dual-domain propagation that combines the advantages of image and feature<br>warping, exploiting global correspondences reliably. We also propose a<br>mask-guided sparse video Transformer, which achieves high efficiency by<br>discarding unnecessary and redundant tokens. With these components, ProPainter<br>outperforms prior arts by a large margin of 1.46 dB in PSNR while maintaining<br>appealing efficiency.
回到顶部