Personality disorders are characterized by interpersonal problems. These problems might arise not only in current, but also in past relationships. To test this hypothesis, recollections of abuse and neglect in childhood and representations of adult attachment to the romantic partner were related to dimensional ratings of the 10 DSM-5 personality disorders. Ultimately, 110 patients of a psychosomatic clinic took part in the study. All data were collected with self-report measures. Among the different types of child abuse and neglect, emotional abuse was shown to be most frequently correlated with personality disorders. Attachment dimensions avoidance and anxiety were both associated with personality disorders; anxiety, however, showed stronger correlations. The results reveal that personality disorders come along with enduring adversities in close relationships, mainly experiences of emotional abuse and attachment anxiety, starting in the family of origin and continuing in adult attachment bonds.