A high-throughput unified transform architecture for Versatile Video Coding

Versatile Video Coding (VVC) offers a compression efficiency improvement of 50% and 75% compared to its predecessors, High Efficiency Video Coding (HEVC) and Advanced Video Coding (AVC), respectively. The VVC encoder software (VVENC), while highly efficient, remains exceedingly complex and operates at speeds that are not conducive to real-time encoding. Despite various speed-optimized versions being released since its standardization in 2020, the complexity remains substantial. This complexity primarily arises from the multiple transform selection (MTS) feature, which involves three transform types (DCT-II, DCT-VIII, and DST-VII) and various rectangular transform sizes ranging from 2×2\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$2\times 2$$\end{document} to 64×64\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$64\times 64$$\end{document}. In this research paper, we propose a unified transform architecture (UTA) that encompasses all transform types and sizes specified in VVENC. It supports both forward as well as inverse transform. The proposed architecture features a reusable one-dimensional transform system, consisting of two 32-point transform subsystems to perform the two-dimensional transform. This architecture can process up to 64 samples in parallel, achieving a high throughput. The architecture is implemented in VHDL and implemented on an Intel Arria 10 FPGA board, achieving a throughput of up to 332 fps at 3840×2160\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$3840\times 2160$$\end{document} resolution with 64×64\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$64\times 64$$\end{document} transform sizes. This makes the architecture a viable candidate for use as a co-processor with the VVENC software.