The denoising of hyperspectral image (HSI) plays a crucial role in the subsequent interpretation and application. The rise of artificial intelligence technology has brought new opportunities for hyperspectral image denoising, and its potential and advantages in this field are gradually changing the traditional denoising pattern. This paper proposes a jointly spatial and spectral difference constraints with low-rank tensor factorization. Firstly, the spatial and spectral difference is combined in the framework of low-rank tensor factorization, to fully mine global spatial-spectral information and improve the removal ability of complex distribution noise. Secondly, based on the premise of effectively preserving HSI intrinsic threedimensional structure, the spatial horizontal and vertical difference constraints are used to mine the local smoothness and similarity of spatial. Thirdly, the full-band spectral difference constraint could not only characterize the continuity and sparsity of the whole spectral domain, but also effectively characterize the noise distribution with linear structure. Finally, experiments on simulated and real HSIs show that the proposed method outperforms state-of-the-art methods in removing mixed noise performance.