In this article, an optimal design of two-dimensional finite impulse response (2D FIR) filter with quadrantally even symmetric impulse response using fractional derivative constraints (FDCs) is presented. Firstly, design problem of 2D FIR filter is formulated as an optimization problem. Then, FDCs are imposed over the integral absolute error for designing of the quadrantally even symmetric impulse response filter. The optimized FDCs are applied over the prescribed frequency points. Next, the optimized filter impulse response coefficients are computed using a hybrid optimization technique, called hybrid particle swarm optimization and gravitational search algorithm (HPSO-GSA). Further, FDC values are also optimized such that flat passband and stopband frequency response is achieved and the absolute L1\documentclass[12pt]{minimal}
\usepackage{amsmath}
\usepackage{wasysym}
\usepackage{amsfonts}
\usepackage{amssymb}
\usepackage{amsbsy}
\usepackage{mathrsfs}
\usepackage{upgreek}
\setlength{\oddsidemargin}{-69pt}
\begin{document}$$L_1$$\end{document}-error is minimized. Finally, four design examples of 2D low-pass, high-pass, band-pass and band-stop filters are demonstrated to justify the design accuracy in terms of passband error, stopband error, maximum passband ripple, minimum stopband attenuation and execution time. Simulation results have been compared with the other optimization algorithms, such as real-coded genetic algorithm, particle swarm optimization and gravitational search algorithm. It is observed that HPSO-GSA gives improved results for 2D FIR-FDC filter design problem. In comparison with other existing techniques of 2D FIR filter design, the proposed method shows improved design accuracy and flexibility with varying values of FDCs.