Non-spectral problem for a class of planar self-affine measures

The self-affine measure mu M,D corresponding to an expanding matrix M is an element of M-n(R) and a finite subset D subset of R-n is supported on the attractor (or invariant set) of the iterated function system {phi(d)(x) = M-1 (x+d)}(d is an element of D). The spectral and non-spectral problems on mu M.D, including the spectrum-tiling problem implied in them, have received much attention i n recent years. One of the non-spectral problem on mu M,D is to estimate the number of orthogonal exponentials in L-2(mu M,D) and to find them. In the present paper we show that if a, b, c is an element of Z, vertical bar a vertical bar > 1, vertical bar c vertical bar > 1 and ac is an element of Z\(3Z), [GRAPHICS] then there exist at most 3 mutually orthogonal exponentials in L-2(mu M,D), and the number 3 is the best. This extends several known conclusions. The proof of such result depends on the characterization of the zero set of the Fourier transform mu M,D, and provides a way of dealing with the non-spectral problem. (C) 2008 Elsevier Inc. All rights reserved.