An implicit neural deformable ray model for limited and sparse view-based spatiotemporal reconstruction

Background: Continuous spatiotemporal volumetric reconstruction is highly valuable, especially in radiation therapy, where tracking and calculating actual exposure during a treatment session is critical. This allows for accurate analysis of treatment outcomes, including patient response and toxicity in relation to delivered doses. However, continuous 4D imaging during radiotherapy is often unavailable due to radiation exposure concerns and hardware limitations. Most setups are limited to acquiring intermittent portal projections or images between treatment beams. Purpose: This study addresses the challenge of spatiotemporal reconstruction from limited views by reconstructing patient-specific volume with as low as 20 input views and continuous-time dynamic volumes from only two orthogonal x-ray projections. Methods: We introduce a novel implicit neural deformable ray (INDeR) model that uses a ray bundle coordinate system, embedding sparse view measurements into an implicit neural field. This method estimates real-time motion via efficient low-dimensional modulation, allowing for the deformation of ray bundles based on just two orthogonal x-ray projections. Results: The INDeR model demonstrates robust performance in image reconstruction and motion tracking, offering detailed visualization of structures like tumors and bronchial passages. With just 20 projection views, INDeR achieves a peak signal-to-noise ratio (PSNR) of 30.13 dB, outperforming methods such as FDK, PWLS-TV, and NAF by 13.93, 4.07, and 3.16 dB, respectively. When applied in real-time, the model consistently delivers a PSNR higher than 27.41 dB using only two orthogonal projections. Conclusion: The proposed INDeR framework successfully reconstructs continuous spatiotemporal representations from sparse views, achieving highly accurate reconstruction with as few as 20 projections and effective tracking with two orthogonal views in real-time. This approach shows great potential for anatomical monitoring in radiation therapy.