In physically (statistically) based theories for rubber-like materials, network models serve as a bridge that connects chain dynamics to continuum constitutive relations. However, there is no existing network model that accounts for the size-dependent mechanical properties of nano/microsize polymeric structures. The present work aims to fill this gap and derive a physically based strain gradient continuum. To establish a quantitative relation between the microscopic Helmholtz free energy due to polymer chain stretch and the macroscopic counterpart that depends on all details of the strain field, we connect strain and strain gradient measures to the positions of all chain ends. Taking the continuum displacement field to be interpolatory at the chain ends, a general framework is constructed, which is not restricted to any specific network structure. Applying the general framework to the commonly used 8-chain network model, we derive a first-order strain gradient elastic continuum, where size of the representative network turns out to be the characteristic length scale of strain gradient material. According to the scalar invariants of strain gradient tensor that remain at last, the assumption of parameter reduction in the simplified strain gradient elasticity theory is justified.