There is a growing body of work that leverages features extracted via
topological data analysis to train machine learning models. While this field,
sometimes known as topological machine learning (TML), has seen some notable
successes, an understanding of how the process of learning from topological
features differs from the process of learning from raw data is still limited.
In this work, we begin to address one component of this larger issue by asking
whether a model trained with topological features learns internal
representations of data that are fundamentally different than those learned by
a model trained with the original raw data. To quantify ``different'', we
exploit two popular metrics that can be used to measure the similarity of the
hidden representations of data within neural networks, neural stitching and
centered kernel alignment. From these we draw a range of conclusions about how
training with topological features does and does not change the representations
that a model learns. Perhaps unsurprisingly, we find that structurally, the
hidden representations of models trained and evaluated on topological features
differ substantially compared to those trained and evaluated on the
corresponding raw data. On the other hand, our experiments show that in some
cases, these representations can be reconciled (at least to the degree required
to solve the corresponding task) using a simple affine transformation. We
conjecture that this means that neural networks trained on raw data may extract
some limited topological features in the process of making predictions.