The cross-lingual lexical substitution task

In this paper we provide an account of the cross-lingual lexical substitution task run as part of SemEval-2010. In this task both annotators (native Spanish speakers, proficient in English) and participating systems had to find Spanish translations for target words in the context of an English sentence. Because only translations of a single lexical unit were required, this task does not necessitate a full blown translation system. This we hope encouraged those working specifically on lexical semantics to participate without a requirement for them to use machine translation software, though they were free to use whatever resources they chose. In this paper we pay particular attention to the resources used by the various participating systems and present analyses to demonstrate the relative strengths of the systems as well as the requirements they have in terms of resources. In addition to the analyses of individual systems we also present the results of a combined system based on voting from the individual systems. We demonstrate that the system produces better results at finding the most frequent translation from the annotators compared to the highest ranked translation provided by individual systems. This supports our other analyses that the systems are heterogeneous, with different strengths and weaknesses.