Estimating the kappa coefficient and its variance under stratified random sampling

The kappa coefficient of agreement is frequently used to summarize the results of an accuracy assessment used to evaluate land-use or land-cover classifications obtained by remote sensing. The standard estimator of the kappa coefficient along with the standard error of this estimator require a sampling model that is approximated by simple random sampling. Formulas are presented for estimating the kappa coefficient and its variance for stratified random sampling. Empirical results demonstrate that these estimators have little bias, and confidence intervals perform well, often even at relatively small sample sizes.