Illegal construction not only seriously affects economic consumption and urban infrastructure (including monitoring equipment, 5G infrastructure, 6G devices, and other consumer electronics) but also hinders the development of smart cities. However, current illegal building detection methods have low detection accuracy and high calculation costs. Therefore, a method for detecting illegal building activities by identifying illegal building-related objects is proposed. A lightweight and highprecision detector and a dataset including 14,038 images of 29 categories of illegal building objects are proposed. The proposed detector is an enhanced version of YOLOv4. MobileNetV3 is utilized as the backbone to extract features, which greatly improves the detection accuracy. Further, depthwise separable convolution (DSC) is introduced to optimize the model structure and lower calculation costs and parameters. Furthermore, Mish was utilized to enhance the identification precision, generalization, and robustness of the detector. Experimental results revealed that the mean average precision (mAP) value of the detector is 88.79 with the dataset. Compared to other state-of-the-art (SOTA) detectors, this detector exhibited superior performance in identifying illegal construction objects, which can detect and prevent illegal building activities accurately and timely, enhancing the development of smart cities. Our project is released at https://github.com/QvQKing/YMMNet.git. IEEE