A Version of Herbert A. Simon's Model with Slowly Fading Memory and Its Connections to Branching Processes

Construct recursively a long string of words w1,...wn, such that at each step k, wk+1 is a new word with a fixed probability p is an element of(0,1), and repeats some preceding word with complementary probability 1-p. More precisely, given a repetition occurs, wk+1 repeats the jth word with probability proportional to j alpha for j=1,...,k. We show that the proportion of distinct words occurring exactly l times converges as the length n of the string goes to infinity to some probability mass function in the variable l >= 1, whose tail decays as a power function when p<1/(1+alpha), and exponentially fast when p>1/(1+alpha).