Semantic information in judgement documents has been an important source in Artificial Intelligence and Law. Sequential representation is the traditional structure for analyzing judgement documents and supporting the legal charge prediction task. The main problem is that it is not effective to represent the criminal semantic information. In this paper, to represent and verify the criminal semantic information such as multi-linked legal features, we propose a novel criminal semantic representation model, which constructs the Criminal Action Graph (CAG) by extracting criminal actions linked in two temporal relationships. Based on the CAG, a Graph Convolutional Network is also adopted as the predictor for legal charge prediction. We evaluate the validity of CAG on the confusing charges which composed of 32,000 judgement documents on five confusing charge sets. The CAG reaches about 88% accuracy averagely, more than 3% over the compared model. The experimental standard deviation also show the stability of our model, which is about 0.0032 on average, nearly 0. The results show the effectiveness of our model for representing and using the semantic information in judgement documents.
