Quantum anomalous Hall insulator stabilized by competing interactions

We study the quantum phases driven by interaction in a semimetal with a quadratic band touching at the Fermi level. By combining the density matrix renormalization group (DMRG), analytical power expanded Gibbs potential method, and the weak coupling renormalization group, we study a spinless fermion system on a checkerboard lattice at half-filling, which has a quadratic band touching in the absence of interaction. In the presence of strong nearest-neighbor (V-1) and next-nearest-neighbor (V-2) interactions, we identify a site nematic insulator phase, a stripe insulator phase, and a phase separation region, in agreement with the phase diagram obtained analytically in the strong coupling limit (i.e., in the absence of fermion hopping). In the intermediate interaction regime we establish a quantum anomalous Hall phase in the DMRG as evidenced by the spontaneous time-reversal symmetry breaking and the appearance of a quantized Chern number C = 1. For weak interaction we utilize the power expanded Gibbs potential method that treats V-1 and V-2 on equal footing, as well as the weak coupling renormalization group. Our analytical results reveal that not only the repulsive V-1 interaction, but also the V-2 interaction (both repulsive and attractive), can drive the quantum anomalous Hall phase. We also determine the phase boundary in the V-1-V-2 plane that separates the semimetal from the quantum anomalous Hall state. Finally, we show that the nematic semimetal, which was proposed for vertical bar V-2 vertical bar >> V-1 at weak coupling in a previous study, is absent, and the quantum anomalous Hall state is the only weak coupling instability of the spinless quadratic band touching semimetal.