A number of problems in theoretical physics share a common nucleus of a combinatoric nature. It is argued here that Hopf algebraic concepts and techiques can be particularly efficient in dealing with such problems. As a first example, a brief review is given of the recent work of Connes, Kreimer and collaborators on the algebraic structure of the process of renormalization in quantum field theory. Then the concept of k-primitive elements is introduced - these are particular linear combinations of products of Feynman diagrams - and it is shown, in the context of a toy-model, that they significantly reduce the computational cost of renormalization. As a second example, Sorkin's proposal for a family of generalizations of quantum mechanics, indexed by an integer k > 2, is reviewed (classical mechanics corresponds to k = 1, while quantum mechanics to k = 2). It is then shown that the quantum measures of order k proposed by Sorkin can also be described as k-primitive elements of the Hopf algebra of functions on an appropriate infinite dimensional abelian group.