In conformal field theory on a manifold with a boundary, there is a short-distance expansion expressing local bulk operators in terms of boundary operators at an adjacent boundary. We show how the coefficients of such an expansion are given solely by data appearing in the bulk theory on the sphere and torus. In particular, the coefficients of the identity operator, which fix the one-point functions, are determined by the elements of the matrix S which implements modular transformations on the torus. The other coefficients are related, in addition, to the elements of the matrices implementing duality transformations on the conformal blocks of the four-point functions on the sphere. Some examples are given.