Edge stability and edge quantum criticality in two-dimensional interacting topological insulators

The robustness of helical edge states in two-dimensional (2D) topological insulators (TIs) against strong interactions remains an intriguing issue. Here, by performing a sign-free quantum Monte Carlo (QMC) simulation of the Kane-Mele-Hubbard-Rashba model which describes an interacting 2D TI with two-particle backscattering on the edges, we verify that the gapless helical edge states are robust against a finite range of two-particle backscattering when the Coulomb repulsion is not strong. However, when the Coulomb repulsion is strong enough, the helical edge states can be gapped by infinitesimal two-particle backscattering, resulting in an edge magnetic order. We further reveal the universal properties of the magnetic edge quantum critical point (EQCP). At magnetic domain walls on edges, we find that a fractionalized charge of e/2 emerges. Implications of our results to recent transport experiments in an InAs/GaSb quantum well, which is a 2D TI with strong interactions, will also be discussed.