Distributional Legacy: The Unreasonable Effectiveness of Harris's Distributional Program

This paper gives an overview of the influence that Zellig Harris's paper "Distributional structure" has had on the research area of distributional semantics, a subfield of natural language processing. We trace the development of the distributional paradigm through three generations of distributional semantics models, arriving at the large language models that currently are at the forefront of public awareness on AI, and that constitute the driving force in the current AI trend. We touch upon the discussion whether the hype around large language models is warranted or not, and we argue that much of the current (philosophical) discussion around the epistemology of distributional models can be resolved by recalling the main arguments in "Distributional structure".