In classic operation systems, processes are assigned different privileges according to the resources. The enforcement of privilege differentiation on diverse processes indicates that strict security management on the individual process, whose emphasis on the restriction on respective process, however, may also overlook the security risk among the processes. Specifically, one process can invoke another one and establish a session, during which the privileges of invoked process may be passed to the invoking process (e. g., by the inter-processes requests). Thus, it may result in the abuse of privilege and resource leakage. Moreover, the internetworking of the processes and their relations also complicate the tasks for the regulation on authorized privileges, and those can be obtained by inheritance. The management on the latter case (i. e., the inherited privileges) has not been well considered in the existing access control models, whose implementation also incur large overhead. In this paper, we propose a lightweight graph-based access control model to manage the privileges between the networked processes, which provides a general solution for the pervasive applicabilities such as process inter-invoking and network-based access control.