The impact of domain randomization on cross-device monocular deep 6DoF detection

This work evaluates the use of synthetic data to train deep 6DoF pose estimation models that use a monocular RGB camera as input. We have compared different training strategies combining real and synthetic data (with domain randomization) to investigate how to better handle real-world challenges. We show that it is possible to obtain accurate models using less real data and suggest how to utilize this strategy. In this work, we have captured and made available two datasets: one real and one synthetic, totaling over 110,0 0 0 annotated frames. These datasets are organized according to the different cameras used and the challenges present in the sequences, all featuring textureless 3D printed objects. We also show that synthetic data can help models generalize, handling challenges such as fast motion, occlusion, illumination changes, color variation, scale changes, and unexpected geometry. Finally, we evaluated 70 different models to understand how a model trained for one camera sensor performs when used with a different sensor. To this end, we also suggest how to handle this challenge better by using synthetic simulations to supplement training.(c) 2022 Elsevier B.V. All rights reserved.