Chaotic-based color image encryption using a hybrid method of reversible cellular automata and DNA sequences

In this manuscript, a new image cryptography algorithm based on new second-order recursive cellular automata (RCA), DNA sequences, and hyper-chaotic system is suggested. Different from the customary permutation-diffusion structure, this manuscript uses a newly three-layer structure, that is, diffusion-permutation-diffusion. Due to the characteristics of high correlation among image pixels that lead to statistical attacks, the aim of this initial diffusion that based on new recursive cellular automata is random redistribution of pixels through assigning a value new for each pixel. The next stage is to perform permutation based on Arnold cat's map, which aims to remove the high correlation among adjacent pixels in the image. The final step, which is based on DNA sequences and key stream, is to increase the complexity and security of the proposed algorithm. In addition, the initial values of the Chen hyper-chaotic system depend on the secret keys and pixels of the original image, which causes completely different chaotic sequences to be created when we encrypt different images. The simulation and empirical results show that the given algorithm has a proper key space and is resistant to various attacks that we test.