Three-dimensional finite element method is used to simulate the formation, self-assembly and shape transition of heteroepitaxial islands during Stranski-Krastonov growth. In the formulation, strain energy, surface energy, surface anisotropy and elastic anisotropy of a cubic lattice structure are taken into account. In the simulations, the SiGe/Si material system is used as a model system. An empirical surface energy as a function of surface orientation is proposed. The minimum energy surfaces are identified based on existing experimental observations. The simulation results show that the coupling of elastic energy relaxation, surface energy anisotropy and elastic anisotropy strongly influences the surface roughening morphology, self-assembly and shape transition of epitaxial islands, resulting in diverse evolution pathways. (c) 2006 Elsevier Ltd. All rights reserved.