Consider weak solutions u of the 3D Navier-Stokes equations in the critical space u is an element of L-p(0, infinity; (B)over dot(q,infinity)(2/p+3/q-1)(R-3), 2 < p < infinity, 2 <= q < infinity and 1/p + 3/q >= 1. Firstly, we show that although the initial perturbations w(0) from u are large, every perturbed weak solution v satisfying the strong energy inequality converges asymptotically to u as t -> infinity. Secondly, by virtue of the characterization of w(0), we examine the optimal upper and lower bounds of the algebraic convergence rates for parallel to v(t) - u(t)parallel to(L2). It should be noted that the above results also hold if u is an element of C([0,infinity);(B)over dot(q,infinity)(3/q-1)(R-3)) with sufficiently small norm and 2 <= q <= 3. The proofs are mainly based on some new estimates for the trilinear form in Besov spaces, the generalized energy inequalities and developed Fourier splitting method.