On the simulation of hand-held cameras in light field rendering

Camera motions are essential to numerous fields of arts and science, such as cinematography, video games, 3D simulations and many more. Some of these applications rely on physical cameras, while others use virtual cameras. Despite the efficiency of virtual cameras, their motion misses some of the subtle details of real camera movements. Since hand-held camera motion accounts for the noise factor produced by hand tremors and muscle fatigue, the resulting output is more realistic than what virtual cameras produce in general. Much research has been done to produce simulations for realistic hand-held cameras; however, implementing these techniques for light field rendering has not been investigated yet. Among the different solutions to produce realistic hand-held camera motions, databases could be set up by collecting data from real hand-held cameras, although this method requires extensive data to be recorded from different hand-held cameras in order to produce reliable results. In addition to generating databases for hand-held camera simulation, jitter could be used. Since hand tremors and muscle fatigue result in adding slight details to the camera motion paths, jitter models could be used to simulate deviations. A camera motion path is defined by a set of curve functions, which could be taken into account when adding noise models to produce the hand-held camera effect. In addition to the defined camera motion path, camera orientation, location, speed and acceleration can be also considered when adding jitter. For example, if the camera accelerates, jitter shall increase accordingly. Even though the work on hand-held camera simulation techniques is an ongoing research and valid solutions are already available, applying those techniques to light field visualization have not been performed yet. Contrary to rendering light fields from physical camera arrays, virtual scenes can be rendered for each ray of the display's light field. Rendering can be performed with both ray tracing and rasterization techniques. Both of these techniques involve camera (region of interest) placement and, therefore, they allow us to perform camera motion simulations. Rendering in such a manner also eliminates all sampling and conversion artifacts, thus making it more suitable for the evaluation of visual comfort. In this paper, we introduce the scientific considerations regarding our on-going long-term work on the simulation of hand-held camera motions for light field displays by means of jitter. The technical discussion covers every aspect of the procedure, from research goal and measurement utilities to visualization and quality assessment.