Exploring connections of spectral learning priority and learning task

Spectral learning priority is a useful tool in analyzing a model’s focus during training, it describes how a model may understand a given image from the spectrum perspective. For example, to distinguish cats and tortoises, learning to recognize their shapes would be enough, such embedding will result in higher learning priority at low frequencies representing shapes; while learning to distinguish turtles and tortoises requires more effort to extract fine-grained features, leading to the corresponding learning priority shifting to higher frequencies.

This proposal aims at exploring how a learning task might affect the model’s learning priority. In particular, we are interested in finding the connections between the model’s learning priority and the task complexity in terms of number of classes, similarity of class statistics, etc. ImageNet-1K with 1.2M training and 50K validation images in 1K classes, and RadImgeNet with 1M training and 112K validation images in 165 classes will be the main datasets to be analyzed, but others are also possible.

You must have experience with deep learning frameworks (PyTorch or Tensorflow) and the HPC at ITU.

References

Lin, Z., Gao, Y., & Sang, J. (2022). Investigating and explaining the frequency bias in image classification. arXiv preprint arXiv:2205.03154.

Rahaman, N., Baratin, A., Arpit, D., Draxler, F., Lin, M., Hamprecht, F., … & Courville, A. (2019, May). On the spectral bias of neural networks. In International conference on machine learning (pp. 5301-5310).

Xu, Z. Q. J., Zhang, Y., Luo, T., Xiao, Y., & Ma, Z. (2019). Frequency principle: Fourier analysis sheds light on deep neural networks. arXiv preprint arXiv:1901.06523.

Hoffer, E., & Ailon, N. (2015). Deep metric learning using triplet network. In Similarity-Based Pattern Recognition: Third International Workshop, SIMBAD 2015, Copenhagen, Denmark, October 12-14, 2015. Proceedings 3 (pp. 84-92).

Deng, J., Dong, W., Socher, R., Li, L. J., Li, K., & Fei-Fei, L. (2009, June). Imagenet: A large-scale hierarchical image database. In 2009 IEEE conference on computer vision and pattern recognition (pp. 248-255).

Mei, X., Liu, Z., Robson, P. M., Marinelli, B., Huang, M., Doshi, A., … & Yang, Y. (2022). RadImageNet: an open radiologic deep learning research dataset for effective transfer learning. Radiology: Artificial Intelligence, 4(5).