MMAN: MULTI-TASK AND MULTI-SCALE ATTENTION NETWORK FOR CONCURRENTLY LOWER LIMBS SEGMENTATION AND LANDMARK DETECTION

Accurate bone segmentation and anatomical landmark detection are vital tasks for the clinical evaluation and treatment planning for patients with lower limbs X-ray films. To leverage the information between the two tasks and deal with the large-scale images, we propose an efficient end-to-end deep network, i.e., multi-task and multi-scale attention network (MMAN), to concurently segment lower limb bones and localize landmarks from large-scale X-ray films in one stage. The results demonstrate that our MMAN outperforms the other state-of-the-art methods for multi-task learning or single-task landmark detection using two separate stages. Our MMAN method has two main technical contributions. First, the local and global encoders are designed to capture multi-scale inputs and provide shared representations including local image details and global contexts, respectively. Second, a global-local attention module is designed to efficiently leverage the global context and learn task-specific information from shared representations under limited computational costs.