muton
(2023-10-31 22:43):
#paper: https://doi.org/10.1073/pnas.2304085120 Modeling naturalistic face processing in humans with deep convolutional neural networks 大脑在加工信息的过程中,都是一个动态变化的过程,以往使用深度卷积神经网络可以模拟出大脑在记忆过程中的动态变化过程,但是对人脸材料而言,都是基于静态的材料进行解码,少有研究使用深度卷积神经网络的方法来解释大脑加工人脸的过程。由于人脸对于人类而言是具有特殊属性的一类材料,并且具有多维属性,如性别,表情,年龄等等,使用此方法解码是非常必要的。本文中作者使用700多个陌生面孔的自然刺激,每个视频长达4s,结合行为以及FMRI成像数据,对人脸加工过程进行了解码,结果发现,深度卷积神经网络模型在中间层/阶段可以很好的解码出分类情况,此阶段与行为结果也一致(行为结果更多体现出了分类信息),但是在全连接层可能更多体现了其他认知过程,如个性化信息等,神经信号的结果包含了更多动态和其他信息加工的信息。因此此模型可以很好的预测面部分类情况,但是对于随后的认知情况和动态变化情况并不能有一个很好的分类效果,因此,解码人脸动态加工过程的深度卷积神经网络模型仍有待进一步发展。
IF:9.400Q1
Proceedings of the National Academy of Sciences of the United States of America,
2023-10-24.
DOI: 10.1073/pnas.2304085120
PMID: 37847731
Modeling naturalistic face processing in humans with deep convolutional neural networks
翻译
Abstract:
Deep convolutional neural networks (DCNNs) trained for face identification can rival and even exceed human-level performance. The ways in which the internal face representations in DCNNs relate to human cognitive representations and brain activity are not well understood. Nearly all previous studies focused on static face image processing with rapid display times and ignored the processing of naturalistic, dynamic information. To address this gap, we developed the largest naturalistic dynamic face stimulus set in human neuroimaging research (700+ naturalistic video clips of unfamiliar faces). We used this naturalistic dataset to compare representational geometries estimated from DCNNs, behavioral responses, and brain responses. We found that DCNN representational geometries were consistent across architectures, cognitive representational geometries were consistent across raters in a behavioral arrangement task, and neural representational geometries in face areas were consistent across brains. Representational geometries in late, fully connected DCNN layers, which are optimized for individuation, were much more weakly correlated with cognitive and neural geometries than were geometries in late-intermediate layers. The late-intermediate face-DCNN layers successfully matched cognitive representational geometries, as measured with a behavioral arrangement task that primarily reflected categorical attributes, and correlated with neural representational geometries in known face-selective topographies. Our study suggests that current DCNNs successfully capture neural cognitive processes for categorical attributes of faces but less accurately capture individuation and dynamic features.
翻译
Keywords:
Related Links: