Ship detection in remote sensing imagery is crucial for various maritime applications such as surveillance and navigation. Convolutional neural networks (CNNs) and transformers have shown significant potential in object detection within the field of image processing. However, existing models applied directly to ship detection in synthetic aperture radar (SAR) imagery encounter challenges due to the varying sizes of ship targets. This often leads to issues such as low detection accuracy, missed detections, and false alarms. In this letter, we propose a new detection network, HMA-Net, to further address these issues. Initially, we introduce the Cwin module, which enhances interference resistance at a relatively low cost, enabling the model to more accurately capture target information. Subsequently, we design a multiscale ship feature extraction module, which uses a parallel multibranch structure to extract features of ships of various sizes and shapes. Finally, we introduce an adaptive fusion loss function that flexibly allocates loss calculation methods to detected targets, thereby enhancing the robustness of the model and achieving high-quality detection boxes. The proposed HMA-Net achieved improvements of 2.0% and 0.9% in mAP(.50:.95) over the baseline models on the SAR Ship Detection dataset and the High-Resolution SAR Images dataset, using only 3.52 M parameters.