A BAYESIAN-ANALYSIS FOR A CLASS OF PENALIZED LIKELIHOOD ESTIMATES

Given a vector y = (y(1),y(2),...,y(N))(T) of N observations sampled from a multivariate normal density with mean vector of the form X theta and covariance matrix upsilon I where theta = (theta(1),theta(2),...,theta(p))(T) is a vector of p less than or equal to N parameters and X is an NXp design matrix, we consider the estimator of theta defined as the value <(theta)over cap> which minimises Q = (y - theta)(T)(y - theta) + lambda theta(T) D0 where D is a pXp symmetric non-negative definite matrix and lambda greater than or equal to 0 is a smoothing parameter. For fixed values of upsilon and lambda, we define a prior probability distribution for theta for which the posterior mean of theta given the data y is <(theta)over cap>. We show that Jeffreys' prior for the parameters upsilon and lambda satisfies pi(upsilon, lambda) proportional to 1/upsilon lambda root R(lambda) where R(lambda) = (N+m-p)[Trace(H-2(lambda))+ m-p]-[Trace(H(lambda))+m-p](2) where H(lambda) = X(X(T)X+lambda D)(-1)X(T) and m is the rank of the matrix D. We report the results of a simulation study comparing the performance of the Bayes estimate using Jeffreys' prior with that of the estimate obtained using Generalized Cross Validation in the contexts of ridge regression and smoothing spline regression.