Rule of thirds-aware reinforcement learning for image aesthetic cropping

Image aesthetic cropping aims at improving the aesthetic quality by adjusting the composition of the image. Most cropping algorithms generate thousands of candidate windows, which is very time-consuming. Motivated by this challenge, we design a rule of thirds-aware reinforcement learning (RoTA-RL) model for image aesthetics cropping. The RoTA-RL model image cropping is a decision-making process of reinforcement learning. Through some simple actions, images with high aesthetic quality can be generated. Firstly, the deep global features and rule of thirds features are extracted. Secondly, the agent can predict the best cropping action by these features. Double deep Q-learning is used to update the parameters of the agent, and dueling deep Q-learning is used to devise the structure of the agent. Finally, a new reward function is proposed based on the aesthetic score and the rule of thirds score predicted by the improved aesthetic assessment network. The RoTA-RL model has been evaluated on public image cropping datasets, and the accuracy of intersection over union is improved by 5.78% on Flickr Cropping Dataset.