Estimation of effective population size in continuously distributed populations: there goes the neighborhood

Use of genetic methods to estimate effective population size (Ne) is rapidly increasing, but all approaches make simplifying assumptions unlikely to be met in real populations. In particular, all assume a single, unstructured population, and none has been evaluated for use with continuously distributed species. We simulated continuous populations with local mating structure, as envisioned by Wright’s concept of neighborhood size (NS), and evaluated performance of a single-sample estimator based on linkage disequilibrium (LD), which provides an estimate of the effective number of parents that produced the sample (Nb). Results illustrate the interacting effects of two phenomena, drift and mixture, that contribute to LD. Samples from areas equal to or smaller than a breeding window produced estimates close to the NS. As the sampling window increased in size to encompass multiple genetic neighborhoods, mixture LD from a two-locus Wahlund effect overwhelmed the reduction in drift LD from incorporating offspring from more parents. As a consequence, [inline-graphic not available: see fulltext] never approached the global Ne, even when the geographic scale of sampling was large. Results indicate that caution is needed in applying standard methods for estimating effective size to continuously distributed populations.