DAC: Descendant-aware clustering algorithm for network-based topic emergence prediction

Topic emergence detection aids in pinpointing prominent topics within a given domain, providing practical insights into all interested parties on where to focus the limited resources. This paper employs the network-based topic evolution approach to overcome limitations in text-based topic evolution, providing prospective topic emergence prediction capabilities by representing emer-gent topics by their ancestors. A descendant-aware clustering algorithm is proposed to generate non-exhaustive and overlapping clusters, utilizing the pace of collaborations and structural sim-ilarities between topics with iterative edge removal and addition processes. Over 100 datasets specific to a research topic were extracted from the Microsoft Academic Graph dataset for the experiments, where the proposed algorithm consistently outperformed existing clustering algo-rithms in generating clusters with a higher likelihood of being ancestors to an emergent topic up to three years in the future. Regression-based cluster filtering using five structural cluster features and topic cluster qualities showed that the prediction performance can be enhanced by automat-ically classifying undesirable clusters from previously known data. The results showed that the proposed algorithm can enhance topic emergence predictions on a wide range of research do-mains regardless of their maturities, popularities, and magnitudes without having access to the data in the predicted year, paving a road to prospective predictions on emergent topics.