Direction finding for multiuser correlated signals based on spatial signature estimation in cyclic cumulant domain

In smart antenna system (SAS), direction-of-arrival (DOA) estimation is one of the key issues in implementing SDMA. In this paper, we propose a multiuser spatial signature estimation algorithm, and apply it to DOA estimation for multiuser correlated sources in SAS. The algorithm constructs a new matrix called spatial signature matrix (SS Matrix) in cyclic cumulant domain, and estimates multiuser spatial signature through the decomposition of the SS Matrix. By exploiting the eigen-structure of the forward and backward smoothed covariance matrix of the estimated spatial signature, DOA of cochannel multiuser can be found, and the algorithm automatically groups them for each user. The validity of the algorithm is demonstrated by computer simulation.