Topological edge states in correlated honeycomb materials with strong spin-orbit coupling

We study the topological nature of single layers of correlated honeycomb materials alpha-RuCl3 and A(2)IrO(3) (A = Li, Na) with strong spin-orbit coupling. An effective tight-binding model based on first principles band structure calculations including Hubbard and spin-orbit coupling is derived. Two pairs of propagating edge modes centered at the zone center and zone boundary are found when their one-dimensional boundary forms a zigzag shape, while the bulk has a gap with trivial time-reversal Z(2) invariants. The effects of strong electronic interactions and doping on the edge modes in these Mott insulators are discussed. We further suggest a heterostructure of alpha-RuCl3/IrCl3 to search for the proposed topological Mott phase.