This work considers the category distribution heterogeneity in federated<br>learning. This issue is due to biased labeling preferences at multiple clients<br>and is a typical setting of data heterogeneity. To alleviate this issue, most<br>previous works consider either regularizing local models or fine-tuning the<br>global model, while they ignore the adjustment of aggregation weights and<br>simply assign weights based on the dataset size. However, based on our<br>empirical observations and theoretical analysis, we find that the dataset size<br>is not optimal and the discrepancy between local and global category<br>distributions could be a beneficial and complementary indicator for determining<br>aggregation weights. We thus propose a novel aggregation method, Federated<br>Learning with Discrepancy-aware Collaboration (FedDisco), whose aggregation<br>weights not only involve both the dataset size and the discrepancy value, but<br>also contribute to a tighter theoretical upper bound of the optimization error.<br>FedDisco also promotes privacy-preservation, communication and computation<br>efficiency, as well as modularity. Extensive experiments show that our FedDisco<br>outperforms several state-of-the-art methods and can be easily incorporated<br>with many existing methods to further enhance the performance. Our code will be<br>available at https://github.com/MediaBrain-SJTU/FedDisco.