In cellular networks, inter- cell interference is the main factor in the reduction of service quality for users, so inter-cell interference coordination (ICIC) has been widely studied to mitigate severe interference. However, in some previous work, cell- edge users are sacrificed to improve the performance of the overall system. Apart from this, most previous methods change the ICIC configuration frequently to achieve the optimal results, but in practice, the frequent ICIC reconfiguration results in large overhead for small cells. Thus, a centralized dynamic ICIC scheme is proposed in this work, including Q- learning assisted deep neural network based ICIC framework and Type- Balanced User Grouping algorithm. The simulation results show that the proposed ICIC scheme outperforms the benchmarks in both sparse and dense user distribution.