The charge ordering process of a mixed valence LuFe2O4 system has been studied, focusing attention on the frustration effect associated with the charge configuration. It is revealed that the system undergoes successive phase transitions following the sequence of disorder --> 2D CDW --> 3D CDW. In the 2D-CDW state, the charge density wave defined by the wave vector (1/3, 1/3): is formed in the hexagonal layer. On the nodal lines of the CDW, the charge order is still undetermined due to the charge frustration. From a dielectric viewpoint, this phase is viewed as random stacking of 2D ferroelectric layers. In the SD-CDW state, antiphase stacking of the 2D ordered layer develops. At the same time, the in-plane charge order exhibits a doubly modulated long-period structure. From a dielectric viewpoint, this phase is characterized by an incommensurate antiferroelectric phase. Relevance of the charge ordering mechanism to the observed anomalously large dielectric constant in LuFe2O4 is discussed.