We investigate hierarchical properties and logspace reductions of languages recognized by logspace probabilistic Turing machines, Arthur-Merlin games, and games against nature with logspace probabilistic verifiers. Each logspace complexity class is decomposed into a hierarchy based on corresponding two-way multihead finite-state automata and we (eventually) prove the separation of the hierarchy levels (even for languages over a single-letter alphabet); furthermore, we show efficient reductions of each logspace complexity class to, or between, low levels of its corresponding hierarchy. We find probabilistic and probabilistic-plus-nondeterministic variants of Savitch's maze threading problem which are logspace complete for PL (the class of languages recognized by logspace probabilistic Turing machines) and, respectively, P (the class of languages recognized by polynomial-time deterministic Turing machines), and which can be recognized by one-way non-sensing two-head (or one-way one-head one-counter) finite-state automata with probabilistic and both probabilistic and nondeterministic states, respectively.