负负 (2022-12-31 23:02):
#paper doi: 10.1109/ICCV.2019.00452. Dmytro Kotovenko et al., 2019, Content and Style Disentanglement for Artistic Style Transfer. 该项工作使用了一种生成对抗网络框架用来提取艺术油画作品中的内容(content)特征和风格(特征),并将这些特征应用在了艺术作品的风格迁移。除了生成对抗网络常用的损失函数之外(例如,MSE for Generator、 log(p)+log(1-q) for Discriminator),该团队在训练模型时考虑到了Triplet Loss —— 简单来说:如果存在梵高的两幅艺术作品A和B,以及莫奈的一幅作品C,那么在style encoder所编码的latent space下A应该离B更近,但离C更远,换句话说此时A样本作为一个“锚点”,编码器试图拉近B和A的距离而疏远C和A的距离;同理,Content编码器也通过这种Triplet loss的方式进行学习。虽然艺术风格迁移的问题已经提出了很长时间,但这篇文章的创新点在于,他提出的模型不仅生成了质量更高、更生动形象的作品,而且还在这一过程中学习到了不同艺术家的创作理念、创作风格,编码器学习到的“Style”这一抽象概念在latent space下是平滑的,能够较好地完成不同艺术家作品之间的风格迁移。
Content and Style Disentanglement for Artistic Style Transfer
翻译
Abstract:
Artists rarely paint in a single style throughout their career. More often they change styles or develop variations of it. In addition, artworks in different styles and even within one style depict real content differently: while Picasso's Blue Period displays a vase in a blueish tone but as a whole, his Cubist works deconstruct the object. To produce artistically convincing stylizations, style transfer models must be able to reflect these changes and variations. Recently many works have aimed to improve the style transfer task, but neglected to address the described observations. We present a novel approach which captures particularities of style and the variations within and separates style and content. This is achieved by introducing two novel losses: a fixpoint triplet style loss to learn subtle variations within one style or between different styles and a disentanglement loss to ensure that the stylization is not conditioned on the real input photo. In addition the paper proposes various evaluation methods to measure the importance of both losses on the validity, quality and variability of final stylizations. We provide qualitative results to demonstrate the performance of our approach.
翻译
回到顶部