Accurate recognition of rock fractures is an important problem in rock engineering because fractures greatly influence the mechanical and hydraulic properties of rock structures. However, existing image segmentation methods for identifying rock fractures tend to be limited to handling only very simple fracture images, despite many real cases containing interfering objects or features such as dark surfaces, stripes (e.g., from foliation), infilling materials, scratches, shadows, and vegetation. Here, we propose a novel deep convolutional neural network to construct the first model that is applicable in the field. After selecting U-Net, a simple and powerful network for semantic segmentation, as a baseline network, we tested network architectures by applying atrous convolutions and extra skip connections to develop an optimal network specialized for rock fracture segmentation. The rate of erroneously detecting non-fracture objects or features was reduced by using the atrous convolution module, and more skip connections were appropriately added to increase the detection rate of the actual fractures. The model's performance gradually improved as these new techniques were added to the original model. Contrast-limited adaptive histogram equalization and a fully connected conditional random field were applied before and after the network, respectively, to enhance the model's performance. Evaluation of the proposed model using raw images of diverse site conditions shows that it can effectively distinguish rock fractures from various interfering objects and features. The source code and pre-trained model can be freely download from GitHub repository (https://github.com/Montherapy/Rock-fracture-segme ntation-with-Tensorflow).