The Brezis–Nirenberg Problem for the Fractional p-Laplacian in Unbounded Domains

In this paper we study the existence of nontrivial solutions to the well-known Brezis–Nirenberg problem involving the fractional p-Laplace operator in unbounded cylinder type domains. By means of the fractional Poincaré inequality in unbounded cylindrical domains, we first study the asymptotic property of the first eigenvalue λp,s(ωδ^)\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$${\lambda _{p,s}}(\widehat {{\omega _\delta }})$$\end{document} with respect to the domain (ωδ^)\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$(\widehat {{\omega _\delta }})$$\end{document}. Then, by applying the concentration-compactness principle for fractional Sobolev spaces in unbounded domains, we prove the existence results. The present work complements the results of Mosconi–Perera–Squassina–Yang [The Brezis–Nirenberg problem for the fractional p-Laplacian. Calc. Var. Partial Differential Equations, 55(4), 25 pp. 2016] to unbounded domains and extends the classical Brezis–Nirenberg type results of Ramos–Wang–Willem [Positive solutions for elliptic equations with critical growth in unbounded domains. In: Chapman Hall/CRC Press, Boca Raton, 2000, 192–199] to the fractional p-Laplacian setting.