In this paper we study the following fractional Hamiltonian systems -tD∞α(-∞Dtαx(t))-L(t).x(t)+∇W(t,x(t))=0,x∈Hα(R,RN),\documentclass[12pt]{minimal}
\usepackage{amsmath}
\usepackage{wasysym}
\usepackage{amsfonts}
\usepackage{amssymb}
\usepackage{amsbsy}
\usepackage{mathrsfs}
\usepackage{upgreek}
\setlength{\oddsidemargin}{-69pt}
\begin{document}$$\begin{aligned} \left\{ \begin{array}{lllll} -_{t}D^{\alpha }_{\infty }(_{-\infty }D^{\alpha }_{t}x(t))- L(t).x(t)+\nabla W(t,x(t))=0, \\ x\in H^{\alpha }(\mathbb {R}, \mathbb {R}^{N}), \end{array} \right. \end{aligned}$$\end{document}where α∈12,1,t∈R,x∈RN,-∞Dtα\documentclass[12pt]{minimal}
\usepackage{amsmath}
\usepackage{wasysym}
\usepackage{amsfonts}
\usepackage{amssymb}
\usepackage{amsbsy}
\usepackage{mathrsfs}
\usepackage{upgreek}
\setlength{\oddsidemargin}{-69pt}
\begin{document}$$\alpha \in \left( {1\over {2}}, 1\right] ,\ t\in \mathbb {R}, x\in \mathbb {R}^N,\ _{-\infty }D^{\alpha }_{t}$$\end{document} and tD∞α\documentclass[12pt]{minimal}
\usepackage{amsmath}
\usepackage{wasysym}
\usepackage{amsfonts}
\usepackage{amssymb}
\usepackage{amsbsy}
\usepackage{mathrsfs}
\usepackage{upgreek}
\setlength{\oddsidemargin}{-69pt}
\begin{document}$$_{t}D^{\alpha }_{\infty }$$\end{document} are the left and right Liouville–Weyl fractional derivatives of order α\documentclass[12pt]{minimal}
\usepackage{amsmath}
\usepackage{wasysym}
\usepackage{amsfonts}
\usepackage{amssymb}
\usepackage{amsbsy}
\usepackage{mathrsfs}
\usepackage{upgreek}
\setlength{\oddsidemargin}{-69pt}
\begin{document}$$\alpha $$\end{document} on the whole axis R\documentclass[12pt]{minimal}
\usepackage{amsmath}
\usepackage{wasysym}
\usepackage{amsfonts}
\usepackage{amssymb}
\usepackage{amsbsy}
\usepackage{mathrsfs}
\usepackage{upgreek}
\setlength{\oddsidemargin}{-69pt}
\begin{document}$$\mathbb {R}$$\end{document} respectively, L:R⟶R2N\documentclass[12pt]{minimal}
\usepackage{amsmath}
\usepackage{wasysym}
\usepackage{amsfonts}
\usepackage{amssymb}
\usepackage{amsbsy}
\usepackage{mathrsfs}
\usepackage{upgreek}
\setlength{\oddsidemargin}{-69pt}
\begin{document}$$L:\mathbb {R}\longrightarrow \mathbb {R}^{2N}$$\end{document} and W:R×RN⟶R\documentclass[12pt]{minimal}
\usepackage{amsmath}
\usepackage{wasysym}
\usepackage{amsfonts}
\usepackage{amssymb}
\usepackage{amsbsy}
\usepackage{mathrsfs}
\usepackage{upgreek}
\setlength{\oddsidemargin}{-69pt}
\begin{document}$$W: \mathbb {R}\times \mathbb {R}^{N}\longrightarrow \mathbb {R}$$\end{document} are suitable functions. One ground state solution is obtained by applying the monotonicity trick of Jeanjean and the concentration-compactness principle in the case where the matrix L(t) is positive definite and W∈C1(R×RN,R)\documentclass[12pt]{minimal}
\usepackage{amsmath}
\usepackage{wasysym}
\usepackage{amsfonts}
\usepackage{amssymb}
\usepackage{amsbsy}
\usepackage{mathrsfs}
\usepackage{upgreek}
\setlength{\oddsidemargin}{-69pt}
\begin{document}$$W \in C^{1}(\mathbb {R}\times \mathbb {R}^{N},\mathbb {R})$$\end{document} is superquadratic but does not satisfy the usual Ambrosetti–Rabinowitz condition.