Interactive Visualization and Capture of Geo-Coded Multimedia Data on Mobile Devices

In digital community applications, geo-coded multimedia data including spatial videos, speech, and geo-narratives are collected and utilized by community users and researchers from multiple fields. It is often preferred that these data can be captured, visualized, and explored directly on mobile phones and tablets interactively. In this paper, we present a Geo-Video Mobile Application (GVM App) that collects geo-coded multimedia data for experts to process and analyze over an interactive visual exploration. This mobile App integrates user interactivity, AI-based semantic image segmentation, and audio transcription for effective data extraction and utilization. Then visualization functions are designed to quickly present geographical, semantic, and street view visual information for knowledge discovery. The users of this tool can include community workers, teachers, and tourists, and also span across multiple social disciplines in digital humanity studies.