Hall conductance, topological quantum phase transition, and the Diophantine equation on the honeycomb lattice

We consider a tight-binding model with the nearest-neighbor hopping integrals on the honeycomb lattice in a magnetic field. Assuming one of the three hopping integrals, which we denote by t(a), can take a different value from the two others, we study quantum phase structures controlled by the anisotropy of the honeycomb lattice. For weak and strong t(a) regions, the Hall conductances are calculated algebraically by using the Diophantine equation. Except for a few specific gaps, we completely determine the Hall conductances in these two regions including those for subband gaps. In a weak magnetic field, it is found that the weak t(a) region shows the unconventional quantization of the Hall conductance, sigma(xy)=-(e(2)/h)(2n+1) (n=0,+/- 1,+/- 2,...), near the half filling, while the strong t(a) region shows only the conventional one, sigma(xy)=-(e(2)/h)n (n=0,+/- 1,+/- 2,...). From the topological nature of the Hall conductance, the existence of gap closing points and quantum phase transitions in the intermediate t(a) region is concluded. We also study numerically the quantum phase structure in detail and find that even when t(a)=1, namely, in graphene case, the system is in the weak t(a) phase except when the Fermi energy is located near the Van Hove singularity or the lower and upper edges of the spectrum.