Indexing source code and clone detection

Context: Searching source code is a common task in code recommendation systems as well as in many other areas. Clone detection is used in software maintenance and bug detection. Objective: The paper introduces an algorithm for building the index structure of abstract syntax trees. When the index structure is built, a pattern tree can be found in time linear in the length of the pattern. Furthermore, the paper describes DrDup2 and DrDupLex, two open-source tools that use the index structure to find Type-2 clones. Method: The index structure presented in this paper is based on the trie, which is a fundamental data structure in computer science. Evaluation of the presented clone detectors is done on BigCloneBench, which is a well-established benchmark for clone detection. Results: Comparison with three state-of-the-art clone detectors (NiCad, CloneWorks and SourcererCC) shows that DrDup2 and DrDupLex are able to beat them in precision, recall and running time. Conclusion: The presented index structure can be used for example to speed up searching for code fragments in code recommendation systems. It is also shown that it can be used to detect Type-2 clones with high precision and recall.