Investigating Entity Linking in Early English Legal Documents

In this paper we investigate the accuracy and overall suitability of a variety of Entity Linking systems for the task of disambiguating entities in 17th century depositions obtained during the 1641 Irish Rebellion. The depositions are extremely difficult for modern NLP tools to work with due to inconsistent spelling, use of language and archaic references. In order to assess the severity of difficulty faced by Entity Linking systems when working with these documents we use the depositions to create an evaluation corpus. This corpus is used as an input to the General Entity Annotator Benchmarking Framework, a standard benchmarking platform for entity annotation systems. Based on this corpus and the results obtained from the General Entity Annotator Benchmarking Framework we observe that the accuracy of existing Entity Linking systems is limited when applied to content like these depositions. This is due to a number of issues ranging from problems with existing state-of-the-art systems to poor representation of historic entities in modern knowledge bases. We discuss some interesting questions raised by this evaluation and put forward a plan for future work in order to learn more.