Performance Comparison of Video Quality Metrics

The development of digital video technology, due to its nature, introduced new approach to the objective video quality estimation. Basically there are two types of metrics for measuring the quality of digital video: purely mathematically defined video quality metrics (DELTA, MSAD, MSE, SNR and PSNR) where the error is mathematically calculated as a difference between the original and processed pixel, and video quality metrics that have similar characteristics as the Human Visual System (SSIM, NQI, VQM), where the perceptual quality is also considered in the overall quality estimation. The metrics from the first group are more technical ones and because the visual quality of perception is more complex than pixel error calculation, many examples show that their video quality estimation is deficiently accurate. The second group of metrics work in a different manner compared to previous, calculating the scene structure in the overall video quality estimation. This paper is concerned with experimental comparison of the performance of Structural Similarity (SSIM) and Video Quality Metric (VQM) metrics for objective video quality estimation. For the purpose of this experiment, more than 300 short video sequences were prepared. The measurements of these video sequences are used to draw the metrics dependence to common changes in processed video sequences. These changes include changes in: brightness, contrast, hue, saturation and noise. This paper pinpoints the key characteristics of each metric, gives the conclusion of the better performing one and gives directions for improvement of objective video quality estimation.