Quantum phase transition in the SU(4) spin-orbital model on the triangular lattice

Motivated by the absence of a cooperative Jahn-Teller effect in LiNiO2 and BaVS3, two layered oxides with triangular planes, we study the SU(4) symmetric spin-orbital model on the triangular lattice. Upon reducing the next-nearest-neighbor coupling, we show that the system undergoes a quantum phase transition to a liquid phase. A variational approach to this liquid phase shows that simple types of long-range correlations are suppressed, suggesting that it is stable against lattice distortions.