SU(4) Heisenberg model on the honeycomb lattice with exchange-frustrated perturbations: Implications for twistronics and Mott insulators

The SU(4)-symmetric spin-orbital model on the honeycomb lattice was recently studied in connection to correlated insulators such as the e(g) Mott insulator Ba3CuSb2O9 and the insulating phase of magic-angle twisted bilayer graphene at quarter filling. Here we provide a unified discussion of these systems by investigating an extended model that includes the effects of Hund's coupling and anisotropic, orbital-dependent exchange interactions. Using a combination of mean-field theory, linear flavor-wave theory, and variational Monte Carlo, we show that this model harbors a quantum spin-orbital liquid over a wide parameter regime around the SU(4)-symmetric point. For large Hund's coupling, a ferromagnetic antiferro-orbital ordered state appears, while a valence-bond crystal combined with a vortex orbital state is stabilized by dominant orbital-dependent exchange interactions.