Factorization-theoretic aspects of semigroups of matrices have received much attention over the past decade. Much of the focus has been on the multiplicative semigroups of nonzero divisors in rings of matrices; that is, factorization in rings of matrices. More recently, factorizations of upper triangular matrices over the nonnegative integers and over more general semirings have been considered. Here, we continue the study of the semigroup T-n(N-0)(center dot) of upper-triangular matrices over the nonnegative integers as well as the larger semigroup M-n(N-0)(center dot) of all square n x n matrices over the semiring of nonnegative integers. We extend the notion of divisor-closed semigroups to a noncommutative setting and show that each m <= n, T-m(N-0)(center dot) and M-m(N-0)(center dot) are almost divisor-closed in M-n(N-0)(center dot). After giving a characterization of irreducible elements in these matrix semigroups, we use the almost divisor-closed result along with precise computations, often in T-2(N-0)(center dot) and M-2(N-0)(center dot), to determine arithmetical invariants that measure the degree to which factorization in these semigroups is nonunique.