Because point clouds have reality measurements of physical objects, they are often used to reconstruct the as-built three-dimensional (3D) model of building construction sites through a modeling process called Scan-to-building information models (BIM). However, the reality measurements in point cloud data, such as actual color and deformed shapes of the original objects, could disappear during the solid modeling process. In addition, the conventional Scan-to-BIM pipeline still requires significant time and manual effort for object classification and shape representation. To address these problems, this study proposes a novel information modeling framework for point clouds, called point cloud information modeling (PCIM). PCIM can automatically recognize construction objects and their properties with deep learning approaches. Furthermore, it can store information in the original point cloud data with a hierarchical structure, rather than converting it to a solid or rigid model. To validate the overall PCIM concept, this research conducted a case study with an actual building construction project. The test results demonstrate that PCIM can be an effective tool for the as-is information modeling of structures and facilities during construction. (C) 2021 American Society of Civil Engineers.