Clustering-based Transduction for Learning a Ranking Model with Limited Human Labels

Transductive learning is a semi-supervised learning paradigm that can leverage unlabeled data by creating pseudo labels for learning a ranking model, when there is only limited or no training examples available. However, the effectiveness of transductive learning in information retrieval (IR) can be hindered by the low quality pseudo labels. To this end, we propose to incorporate a two-step k-means clustering algorithm to select the high quality training queries for generating the pseudo labels. In particular, the first step selects the high-quality queries for which the relevant documents are highly coherent as indicated by the clustering results. The second step then selects the initial training examples for the transductive learning that iteratively aggregating the pseudo examples. Finally, the learning to rank (LTR) algorithms are applied to learn the ranking model using the pseudo training examples created by the transductive learning process. Our proposed approach is particularly suitable for applications where there is only little or no human labels available as it does not necessarily involve the use of relevance assessments information or human efforts. Experimental results on the standard TREC Tweets11 collection show that our proposed approach outperforms strong baselines, namely the conventional applications of learning to rank algorithms using human labels for the training and transductive learning using all the queries available.