This work considers the category distribution heterogeneity in federatedlearning. This issue is due to biased labeling preferences at multiple clientsand is a typical setting of data heterogeneity. To alleviate this issue, mostprevious works consider either regularizing local models or fine-tuning theglobal model, while they ignore the adjustment of aggregation weights andsimply assign weights based on the dataset size. However, based on ourempirical observations and theoretical analysis, we find that the dataset sizeis not optimal and the discrepancy between local and global categorydistributions could be a beneficial and complementary indicator for determiningaggregation weights. We thus propose a novel aggregation method, FederatedLearning with Discrepancy-aware Collaboration (FedDisco), whose aggregationweights not only involve both the dataset size and the discrepancy value, butalso contribute to a tighter theoretical upper bound of the optimization error.FedDisco also promotes privacy-preservation, communication and computationefficiency, as well as modularity. Extensive experiments show that our FedDiscooutperforms several state-of-the-art methods and can be easily incorporatedwith many existing methods to further enhance the performance. Our code will beavailable at https://github.com/MediaBrain-SJTU/FedDisco.