In this paper, we study the following critical system with fractional Laplacian:
{(−Δ)su+λ1u=μ1|u|2∗−2u+αγ2∗|u|α−2u|v|βin Ω,(−Δ)sv+λ2v=μ2|v|2∗−2v+βγ2∗|u|α|v|β−2vin Ω,u=v=0in RN∖Ω,\documentclass[12pt]{minimal}
\usepackage{amsmath}
\usepackage{wasysym}
\usepackage{amsfonts}
\usepackage{amssymb}
\usepackage{amsbsy}
\usepackage{mathrsfs}
\usepackage{upgreek}
\setlength{\oddsidemargin}{-69pt}
\begin{document}$$\textstyle\begin{cases} (-\Delta)^{s}u+\lambda_{1}u=\mu_{1}|u|^{2^{\ast}-2}u+\frac{\alpha \gamma}{2^{\ast}}|u|^{\alpha-2}u|v|^{\beta} & \text{in } \Omega, \\ (-\Delta)^{s}v+\lambda_{2}v= \mu_{2}|v|^{2^{\ast}-2}v+\frac{\beta \gamma}{2^{\ast}}|u|^{\alpha}|v|^{\beta-2}v & \text{in } \Omega, \\ u=v=0 & \text{in } \mathbb{R}^{N}\setminus\Omega, \end{cases} $$\end{document} where (−Δ)s\documentclass[12pt]{minimal}
\usepackage{amsmath}
\usepackage{wasysym}
\usepackage{amsfonts}
\usepackage{amssymb}
\usepackage{amsbsy}
\usepackage{mathrsfs}
\usepackage{upgreek}
\setlength{\oddsidemargin}{-69pt}
\begin{document}$(-\Delta)^{s}$\end{document} is the fractional Laplacian, 0<s<1\documentclass[12pt]{minimal}
\usepackage{amsmath}
\usepackage{wasysym}
\usepackage{amsfonts}
\usepackage{amssymb}
\usepackage{amsbsy}
\usepackage{mathrsfs}
\usepackage{upgreek}
\setlength{\oddsidemargin}{-69pt}
\begin{document}$0< s<1$\end{document}, μ1,μ2>0\documentclass[12pt]{minimal}
\usepackage{amsmath}
\usepackage{wasysym}
\usepackage{amsfonts}
\usepackage{amssymb}
\usepackage{amsbsy}
\usepackage{mathrsfs}
\usepackage{upgreek}
\setlength{\oddsidemargin}{-69pt}
\begin{document}$\mu_{1},\mu_{2}>0$\end{document}, 2∗=2NN−2s\documentclass[12pt]{minimal}
\usepackage{amsmath}
\usepackage{wasysym}
\usepackage{amsfonts}
\usepackage{amssymb}
\usepackage{amsbsy}
\usepackage{mathrsfs}
\usepackage{upgreek}
\setlength{\oddsidemargin}{-69pt}
\begin{document}$2^{\ast}=\frac{2N}{N-2s}$\end{document} is a fractional critical Sobolev exponent, N>2s\documentclass[12pt]{minimal}
\usepackage{amsmath}
\usepackage{wasysym}
\usepackage{amsfonts}
\usepackage{amssymb}
\usepackage{amsbsy}
\usepackage{mathrsfs}
\usepackage{upgreek}
\setlength{\oddsidemargin}{-69pt}
\begin{document}$N>2s$\end{document}, 1<α\documentclass[12pt]{minimal}
\usepackage{amsmath}
\usepackage{wasysym}
\usepackage{amsfonts}
\usepackage{amssymb}
\usepackage{amsbsy}
\usepackage{mathrsfs}
\usepackage{upgreek}
\setlength{\oddsidemargin}{-69pt}
\begin{document}$1<\alpha$\end{document}, β<2\documentclass[12pt]{minimal}
\usepackage{amsmath}
\usepackage{wasysym}
\usepackage{amsfonts}
\usepackage{amssymb}
\usepackage{amsbsy}
\usepackage{mathrsfs}
\usepackage{upgreek}
\setlength{\oddsidemargin}{-69pt}
\begin{document}$\beta<2$\end{document}, α+β=2∗\documentclass[12pt]{minimal}
\usepackage{amsmath}
\usepackage{wasysym}
\usepackage{amsfonts}
\usepackage{amssymb}
\usepackage{amsbsy}
\usepackage{mathrsfs}
\usepackage{upgreek}
\setlength{\oddsidemargin}{-69pt}
\begin{document}$\alpha+\beta=2^{\ast}$\end{document}, Ω is an open bounded set of RN\documentclass[12pt]{minimal}
\usepackage{amsmath}
\usepackage{wasysym}
\usepackage{amsfonts}
\usepackage{amssymb}
\usepackage{amsbsy}
\usepackage{mathrsfs}
\usepackage{upgreek}
\setlength{\oddsidemargin}{-69pt}
\begin{document}$\mathbb{R}^{N}$\end{document} with Lipschitz boundary and λ1,λ2>−λ1,s(Ω)\documentclass[12pt]{minimal}
\usepackage{amsmath}
\usepackage{wasysym}
\usepackage{amsfonts}
\usepackage{amssymb}
\usepackage{amsbsy}
\usepackage{mathrsfs}
\usepackage{upgreek}
\setlength{\oddsidemargin}{-69pt}
\begin{document}$\lambda_{1},\lambda_{2}>-\lambda_{1,s}(\Omega)$\end{document}, λ1,s(Ω)\documentclass[12pt]{minimal}
\usepackage{amsmath}
\usepackage{wasysym}
\usepackage{amsfonts}
\usepackage{amssymb}
\usepackage{amsbsy}
\usepackage{mathrsfs}
\usepackage{upgreek}
\setlength{\oddsidemargin}{-69pt}
\begin{document}$\lambda_{1,s}(\Omega)$\end{document} is the first eigenvalue of the non-local operator (−Δ)s\documentclass[12pt]{minimal}
\usepackage{amsmath}
\usepackage{wasysym}
\usepackage{amsfonts}
\usepackage{amssymb}
\usepackage{amsbsy}
\usepackage{mathrsfs}
\usepackage{upgreek}
\setlength{\oddsidemargin}{-69pt}
\begin{document}$(-\Delta)^{s}$\end{document} with homogeneous Dirichlet boundary datum. By using the Nehari manifold, we prove the existence of a positive ground state solution of the system for all γ>0\documentclass[12pt]{minimal}
\usepackage{amsmath}
\usepackage{wasysym}
\usepackage{amsfonts}
\usepackage{amssymb}
\usepackage{amsbsy}
\usepackage{mathrsfs}
\usepackage{upgreek}
\setlength{\oddsidemargin}{-69pt}
\begin{document}$\gamma>0$\end{document}. Via a perturbation argument and using the topological degree and a pseudo-gradient vector field, we show that this system has a positive higher energy solution. Then the asymptotic behaviors of the positive ground state solutions are analyzed when γ→0\documentclass[12pt]{minimal}
\usepackage{amsmath}
\usepackage{wasysym}
\usepackage{amsfonts}
\usepackage{amssymb}
\usepackage{amsbsy}
\usepackage{mathrsfs}
\usepackage{upgreek}
\setlength{\oddsidemargin}{-69pt}
\begin{document}$\gamma\rightarrow0$\end{document}.