An overdetermined problem in Riesz-potential and fractional Laplacian

The main purpose of this paper is to address two open questions raised by Reichel (2009) in [2] on characterizations of balls in terms of the Riesz potential and fractional Laplacian. For a bounded C-1 domain Omega subset of R-N, we consider the Riesz-potential u(x) = integral(Omega) 1/vertical bar x - y vertical bar(N-alpha) dy for 2 <= alpha not equal N. We show that u = constant on partial derivative Omega if and only if Omega is a ball. In the case of alpha = N, the similar characterization is established for the logarithmic potential u(x) = integral(Omega) log 1/vertical bar x - y vertical bar dy. We also prove that such a characterization holds for the logarithmic Riesz potential u(x) = integral(Omega) vertical bar x - y vertical bar(alpha-N) log 1/vertical bar x - y vertical bar dy when the diameter of the domain Omega is less than e(1/N-alpha) in the case when alpha - N is a nonnegative even integer. This provides a characterization for the overdetermined problem of the fractional Laplacian. These results answer two open questions in Reichel (2009) [2] to some extent. Moreover, we also establish some nonexistence result of positive solutions to a class of integral equations in an exterior domain. (C) 2011 Elsevier Ltd. All rights reserved