The mechanism of force detection of Atomic-Force Microscopy (AFM) is theoretically investigated. First, a theoretical simulation of contact AFM images is performed, and a tip apex structure is studied. It is clarified how the AFM images and the force distributions change as the load varies. It is also revealed that the characteristics of the AFM images such as their detailed microscopic pattern, the symmetry, and the corrugation amplitude, depend strongly on the tip apex structure. Secondly, fundamental features of the atomic-scale friction in Frictional-Force Microscopy (FFM) are studied. Simulated FFM images are in good agreement with observed ones. Then we discuss the mechanism of the image pattern of FFM by an analytical method. It is revealed that the part of the boundary of the stable region of the cantilever basal position, appears as the boundary between the bright and the dark area of FFM images. Thus we clarify the physical meaning of the FFM image patterns. Lastly, we studied dynamics of the large amplitude cantilever oscillations in the noncontact AFM (nc-AFM). The oscillation of the cantilever is treated as a forced oscillation periodically interrupted by collisions with the surface. By solving this extremely nonlinear problem numerically, some remarkable features of the cantilever oscillation are revealed. We observed strange behaviors of the cantilever such as a bimodal state of dynamical touching and non-touching motion, as well as a fractional resonance features.