🐼太真实 (2024-02-29 10:04):
#paper ProPainter: Improving Propagation and Transformer for Video Inpainting 本文介绍了一种新的视频修复技术——ProPainter,通过双域传播和掩码引导稀疏视频Transformer的设计,实现了高效而准确的视频修复。文章详细介绍了ProPainter的三个关键组成部分:循环流场完成、双域传播和掩码引导稀疏视频Transformer,并提供了相应的技术细节和实验结果。
ProPainter: Improving Propagation and Transformer for Video Inpainting
翻译
Abstract:
Flow-based propagation and spatiotemporal Transformer are two mainstreammechanisms in video inpainting (VI). Despite the effectiveness of thesecomponents, they still suffer from some limitations that affect theirperformance. Previous propagation-based approaches are performed separatelyeither in the image or feature domain. Global image propagation isolated fromlearning may cause spatial misalignment due to inaccurate optical flow.Moreover, memory or computational constraints limit the temporal range offeature propagation and video Transformer, preventing exploration ofcorrespondence information from distant frames. To address these issues, wepropose an improved framework, called ProPainter, which involves enhancedProPagation and an efficient Transformer. Specifically, we introducedual-domain propagation that combines the advantages of image and featurewarping, exploiting global correspondences reliably. We also propose amask-guided sparse video Transformer, which achieves high efficiency bydiscarding unnecessary and redundant tokens. With these components, ProPainteroutperforms prior arts by a large margin of 1.46 dB in PSNR while maintainingappealing efficiency.
翻译
回到顶部