Vincent (2023-08-31 23:50):
#paper https://doi.org/10.48550/arXiv.2306.03301. arxiv 2023, Estimating Conditional Mutual Information for Dynamic Feature Selection. 动态特征选择涉及到学习特征选择策略,以及使用任意特征对目标值进行预测。其中学习选择策略往往十分具有挑战性。这篇文章介绍了一种基于特征与预测目标的条件互信息(conditional mutual information)对特征进行优先级排序,该方法通过训练一个神经网络估算在给定特征集情况下,其他特征的预测能力(条件互信息),每一步选择最具信息的特征加入到已有特征集中。依次迭代下去直到满足停止条件(例如达到给定特征数量,不确定度,代价等)。此外,该框架同样能够利用先验信息。文章验证了该方法在表格与图像数据集测试中均有不错效果。
Estimating Conditional Mutual Information for Dynamic Feature Selection
翻译
Abstract:
Dynamic feature selection, where we sequentially query features to make accurate predictions with a minimal budget, is a promising paradigm to reduce feature acquisition costs and provide transparency into the prediction process. The problem is challenging, however, as it requires both making predictions with arbitrary feature sets and learning a policy to identify the most valuable selections. Here, we take an information-theoretic perspective and prioritize features based on their mutual information with the response variable. The main challenge is learning this selection policy, and we design a straightforward new modeling approach that estimates the mutual information in a discriminative rather than generative fashion. Building on our learning approach, we introduce several further improvements: allowing variable feature budgets across samples, enabling non-uniform costs between features, incorporating prior information, and exploring modern architectures to handle partial input information. We find that our method provides consistent gains over recent state-of-the-art methods across a variety of datasets.
翻译
回到顶部