The principal result of this paper answers a long-standing question in the model theory of arithmetic [R. Kossak, J. Schmerl, The Structure of Models of Peano Arithmetic, Oxford University Press, 2006, Question 7] by showing that there exists an uncountable arithmetically closed family A of subsets of the set omega of natural numbers such that the expansion N(A) := (N, A)(A is an element of A) of the standard model N := (omega, +, X) of Peano arithmetic has no conservative elementary extension, i.e., for any elementary extension N*(A) = (omega*,...) of N(A), there is a subset of omega* that is parametrically definable in N*(A) but whose intersection with w is not a member of A. We also establish other results that highlight the role of countability in the model theory of arithmetic. Inspired by a recent question of Gitman and Hamkins, we furthermore show that the aforementioned family A can be arranged to further satisfy the curious property that forcing with the quotient Boolean algebra A/FIN (where FIN is the ideal of finite sets) collapses K I when viewed as a notion of forcing. (C) 2008 Elsevier B.V. All rights reserved.