Let k be an algebraically closed field and let Hilb(d)(G)(P(k)(N)) be the open locus of the Hilbert scheme Hilb(d)(P(k)(N)) corresponding to Gorenstein subschemes. We prove that Hilb(d)(G)(P(k)(N)) is irreducible for d <= 9. Moreover we also give a complete picture of its singular locus in the same range d <= 9. Such a description of the singularities gives some evidence to a conjecture on the nature of the singular points in Hilb(d)(G)(P(k)(N)) that we state at the end of the paper. (C) 2009 Elsevier B.V. All rights reserved.