Rule of thirds-aware reinforcement learning for image aesthetic cropping

被引:0
|
作者
Li, Xuewei [1 ]
Zhang, Gang [2 ]
Wu, YuQuan [1 ]
Li, Xueming [3 ]
Zhang, YaQing [3 ]
机构
[1] Chinese Acad Sci, Inst Software, Sci & Technol Integrated Informat Syst Lab, 4 South Fourth St, Beijing 100190, Peoples R China
[2] Unit 95865 PLA, Beijing 102218, Peoples R China
[3] Beijing Univ Posts & Telecommun, Dept Digital Media & Art Design, 10 Xitucheng Rd, Beijing 100876, Peoples R China
来源
VISUAL COMPUTER | 2023年 / 39卷 / 11期
关键词
Image cropping; Reinforcement learning; Rule of thirds; Aesthetics image;
D O I
10.1007/s00371-022-02687-9
中图分类号
TP31 [计算机软件];
学科分类号
081202 ; 0835 ;
摘要
Image aesthetic cropping aims at improving the aesthetic quality by adjusting the composition of the image. Most cropping algorithms generate thousands of candidate windows, which is very time-consuming. Motivated by this challenge, we design a rule of thirds-aware reinforcement learning (RoTA-RL) model for image aesthetics cropping. The RoTA-RL model image cropping is a decision-making process of reinforcement learning. Through some simple actions, images with high aesthetic quality can be generated. Firstly, the deep global features and rule of thirds features are extracted. Secondly, the agent can predict the best cropping action by these features. Double deep Q-learning is used to update the parameters of the agent, and dueling deep Q-learning is used to devise the structure of the agent. Finally, a new reward function is proposed based on the aesthetic score and the rule of thirds score predicted by the improved aesthetic assessment network. The RoTA-RL model has been evaluated on public image cropping datasets, and the accuracy of intersection over union is improved by 5.78% on Flickr Cropping Dataset.
引用
收藏
页码:5651 / 5667
页数:17
相关论文
共 50 条
  • [1] Rule of thirds-aware reinforcement learning for image aesthetic cropping
    Xuewei Li
    Gang Zhang
    YuQuan Wu
    Xueming Li
    YaQing Zhang
    The Visual Computer, 2023, 39 : 5651 - 5667
  • [2] A2-RL: Aesthetics Aware Reinforcement Learning for Image Cropping
    Li, Debang
    Wu, Huikai
    Zhang, Junge
    Huang, Kaiqi
    2018 IEEE/CVF CONFERENCE ON COMPUTER VISION AND PATTERN RECOGNITION (CVPR), 2018, : 8193 - 8201
  • [3] Emotion Attention-Aware Collaborative Deep Reinforcement Learning for Image Cropping
    Zhang, Xiaoyan
    Li, Zhuopeng
    Jiang, Jianmin
    IEEE TRANSACTIONS ON MULTIMEDIA, 2021, 23 : 2545 - 2560
  • [4] Image Cropping with Composition and Saliency Aware Aesthetic Score Map
    Tu, Yi
    Niu, Li
    Zhao, Weijie
    Chen, Dawei
    Zhang, Liqing
    THIRTY-FOURTH AAAI CONFERENCE ON ARTIFICIAL INTELLIGENCE, THE THIRTY-SECOND INNOVATIVE APPLICATIONS OF ARTIFICIAL INTELLIGENCE CONFERENCE AND THE TENTH AAAI SYMPOSIUM ON EDUCATIONAL ADVANCES IN ARTIFICIAL INTELLIGENCE, 2020, 34 : 12104 - 12111
  • [5] COLLABORATIVE DEEP REINFORCEMENT LEARNING FOR IMAGE CROPPING
    Li, Zhuopeng
    Zhang, Xiaoyan
    2019 IEEE INTERNATIONAL CONFERENCE ON MULTIMEDIA AND EXPO (ICME), 2019, : 254 - 259
  • [6] Fast A3RL: Aesthetics-Aware Adversarial Reinforcement Learning for Image Cropping
    Li, Debang
    Wu, Huikai
    Zhang, Junge
    Huang, Kaiqi
    IEEE TRANSACTIONS ON IMAGE PROCESSING, 2019, 28 (10) : 5105 - 5120
  • [7] Rule-Aware Reinforcement Learning for Knowledge Graph Reasoning
    Hou, Zhongni
    Jin, Xiaolong
    Li, Zixuan
    Bai, Long
    FINDINGS OF THE ASSOCIATION FOR COMPUTATIONAL LINGUISTICS, ACL-IJCNLP 2021, 2021, : 4687 - 4692
  • [8] Learning the Relation Between Interested Objects and Aesthetic Region for Image Cropping
    Lu, Peng
    Zhang, Hao
    Peng, Xujun
    Jin, Xiaofu
    IEEE TRANSACTIONS ON MULTIMEDIA, 2021, 23 : 3618 - 3630
  • [9] Learning an Aesthetic Photo Cropping Cascade
    Wang, Peng
    Lin, Zhe
    Mech, Radomir
    2015 IEEE WINTER CONFERENCE ON APPLICATIONS OF COMPUTER VISION (WACV), 2015, : 448 - 455
  • [10] Aesthetic-guided Outward Image Cropping
    Zhong, Lei
    Li, Feng-Heng
    Huang, Hao-Zhi
    Zhang, Yong
    Lu, Shao-Ping
    Wang, Jue
    ACM TRANSACTIONS ON GRAPHICS, 2021, 40 (06):