Print Email Facebook Twitter What Can Style Transfer and Paintings Do For Model Robustness? Title What Can Style Transfer and Paintings Do For Model Robustness? Author Lin, Hubert (Cornell University) van Zuijlen, M.J.P. (TU Delft Human Information Communication Design) Pont, S.C. (TU Delft Human Information Communication Design) Wijntjes, M.W.A. (TU Delft Human Information Communication Design) Bala, Kavita (Cornell University) Date 2021 Abstract A common strategy for improving model robustness is through data augmentations. Data augmentations encourage models to learn desired invariances, such as invariance to horizontal flipping or small changes in color. Recent work has shown that arbitrary style transfer can be used as a form of data augmentation to encourage invariance to textures by creating painting-like images from photographs. However, a stylized photograph is not quite the same as an artist-created painting. Artists depict perceptually meaningful cues in paintings so that humans can recognize salient components in scenes, an emphasis which is not enforced in style transfer. Therefore, we study how style transfer and paintings differ in their impact on model robustness. First, we investigate the role of paintings as style images for stylization-based data augmentation. We find that style transfer functions well even without paintings as style images. Second, we show that learning from paintings as a form of perceptual data augmentation can improve model robustness. Finally, we investigate the invariances learned from stylization and from paintings, and show that models learn different invariances from these differing forms of data. Our results provide insights into how stylization improves model robustness, and provide evidence that artist-created paintings can be a valuable source of data for model robustness. Code and data are available at: https://github.com/hubertsgithub/style_painting_robustness To reference this document use: http://resolver.tudelft.nl/uuid:e97fd9a6-8a64-4f43-a111-ef3412d30680 DOI https://doi.org/10.1109/CVPR46437.2021.01088 Publisher IEEE, Piscataway ISBN 978-1-6654-4510-8 Source Proceedings - 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition, CVPR 2021: Proceedings Event 2021 IEEE/CVF Conference on Computer Visionand Pattern Recognition, 2021-06-20 → 2021-06-25, Virtual at Nashville, United States Series Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 1063-6919 Bibliographical note Accepted Author Manuscript Part of collection Institutional Repository Document type conference paper Rights © 2021 Hubert Lin, M.J.P. van Zuijlen, S.C. Pont, M.W.A. Wijntjes, Kavita Bala Files PDF Lin_What_Can_Style_Transf ... _paper.pdf 2.43 MB Close viewer /islandora/object/uuid:e97fd9a6-8a64-4f43-a111-ef3412d30680/datastream/OBJ/view