张德祥 (2023-03-20 10:45):
#paper doi: https://doi.org/10.1101/2022.05.17.492325 Inferring Neural Activity Before Plasticity: A Foundation for Learning Beyond Backpropagation 超越GPT需要从更底层的技术改进,BP是深度学习的核心,生物算法比BP更高效,生物算法是超越BP的一个途径,这篇论文给出了很好的解释及后续论文有一些实验及算法,效率已经可以匹配BP,仍然有更多的优点, 更多可以参考 https://mp.weixin.qq.com/s/lPzGvY6oOnwzVgxDr9ePpA
Inferring Neural Activity Before Plasticity: A Foundation for Learning Beyond Backpropagation
翻译
Abstract:
AbstractFor both humans and machines, the essence of learning is to pinpoint which components in its information processing pipeline are responsible for an error in its output — a challenge that is known ascredit assignment. How the brain solves credit assignment is a key question in neuroscience, and also of significant importance for artificial intelligence. It has long been assumed that credit assignment is best solved by backpropagation, which is also the foundation of modern machine learning. However, it has been questioned whether it is possible for the brain to implement backpropagation and learning in the brain may actually be more efficient and effective than backpropagation. Here, we set out a fundamentally different principle on credit assignment, calledprospective configuration. In prospective configuration, the network first infers the pattern of neural activity that should result from learning, and then the synaptic weights are modified to consolidate the change in neural activity. We demonstrate that this distinct mechanism, in contrast to backpropagation, (1) underlies learning in a well-established family of models of cortical circuits, (2) enables learning that is more efficient and effective in many contexts faced by biological organisms, and (3) reproduces surprising patterns of neural activity and behaviour observed in diverse human and animal learning experiments. Our findings establish a new foundation for learning beyond backpropagation, for both understanding biological learning and building artificial intelligence.
翻译
回到顶部