Mining Configuration Items From System Logs through Distant Supervision

IT operators face the big challenge of collecting and recording tens of thousands of configuration items (CIs) from large-scale IT infrastructure and managing their changes in the Configuration Management Database (CMDB). Existing automatic CI discovery tools such as IBM Tivoli rely on intrusive monitoring agents in each server, thus their setup and maintenance costs are significant. To solve these problems, we propose a non-intrusive CI mining approach through distant supervision, which can automatically discover CIs from system logs for managing configuration changes efficiently. It first labels CIs in logs through distant supervision of CMDB. Then, it discovers more CIs based on log clustering and alignment algorithms. The approach has been implemented with a distributed configuration management assistant tool named FineCI. Experiments on real-world system logs from a large bank show that our approach performs about 70% precision and 75% recall. Besides, through distributed implementation, the efficiency of our approach can be greatly improved.