尹志 (2024-04-30 22:48):
#paper doi:https://doi.org/10.48550/arXiv.2211.07697,NeurIPS 2022 Workshop on Symmetry and Geometry in Neural Representations, 2022. Do Neural Networks Trained with Topological Features Learn Different Internal Representations? 作者主要讨论了使用拓扑特征训练神经网络和使用常规数据直接进行神经网络训练在表征上的区别。结论很有意思,比较容易猜到的是,两者确实有区别,特别是在作者选择的metrics下,这也说明了拓扑机器学习的价值。但作者发现在一些情况下,也存在可以利用简单的表征来替代拓扑特征训练的模型。当然,在具体的数据场景下怎么样提取出合适的拓扑特征显著区别于使用raw data可以提取的特征,这仍是一个开放的主题。
Do Neural Networks Trained with Topological Features Learn Different Internal Representations?
Sarah McGuire, Shane Jackson, Tegan Emerson, Henry Kvinge
Abstract:
There is a growing body of work that leverages features extracted via<br>topological data analysis to train machine learning models. While this field,<br>sometimes known as topological machine learning (TML), has seen some notable<br>successes, an understanding of how the process of learning from topological<br>features differs from the process of learning from raw data is still limited.<br>In this work, we begin to address one component of this larger issue by asking<br>whether a model trained with topological features learns internal<br>representations of data that are fundamentally different than those learned by<br>a model trained with the original raw data. To quantify ``different'', we<br>exploit two popular metrics that can be used to measure the similarity of the<br>hidden representations of data within neural networks, neural stitching and<br>centered kernel alignment. From these we draw a range of conclusions about how<br>training with topological features does and does not change the representations<br>that a model learns. Perhaps unsurprisingly, we find that structurally, the<br>hidden representations of models trained and evaluated on topological features<br>differ substantially compared to those trained and evaluated on the<br>corresponding raw data. On the other hand, our experiments show that in some<br>cases, these representations can be reconciled (at least to the degree required<br>to solve the corresponding task) using a simple affine transformation. We<br>conjecture that this means that neural networks trained on raw data may extract<br>some limited topological features in the process of making predictions.
回到顶部