Magnetic structure of the antiferromagnetic Fulde-Ferrell-Larkin-Ovchinnikov state

The properties of incommensurate antiferromagnetic (AFM) order in the Fulde-Ferrell-Larkin-Ovchinnikov (FFLO) state are studied by solving the Bogoliubov-de Gennes (BdG) equations. The relationship between the electronic structure and the magnetic structure is clarified. We find that the magnetic structure in the AFM-FFLO state includes three cases. (I) In the strongly localized case, the AFM staggered moment is confined into the FFLO nodal planes where the superconducting order parameter vanishes. (II) In the weakly localized case, the AFM staggered moment appears in the whole spatial region, and its magnitude is enhanced around the FFLO nodal planes. (III) In the extended case, the AFM staggered moment is nearly homogeneous and slightly suppressed in the vicinity of the FFLO nodal planes. The structure of Bragg peaks in the momentum resolved structure factor is studied in each case. We discuss the possibility of an AFM-FFLO state in the heavy fermion superconductor CeCoIn(5) by comparing these results with the neutron scattering data of CeCoIn(5). Experimentally the magnetic structure and its dependence on the magnetic field orientation in the high-field superconducting phase of CeCoIn(5) are consistent with case (II).