SimMeme: Semantic-Based Meme Search

With the proliferation of social image-sharing applications, image search becomes an increasingly common activity. In this work we focus on a particular class of images that convey semantic meaning beyond the visual appearance, and whose search presents particular challenges. A prominent example are Memes, an emerging popular type of captioned pictures, which we will use in this demo to demonstrate our solution. Unlike in conventional image-search, visually similar Memes may reflect different concepts. Intent is sometimes captured by user annotations, but these too are often incomplete and ambiguous. Thus, a deeper analysis of the semantic relations among Memes is required for an accurate search. To address this problem, we present SimMeme, a semantic aware search engine for Memes. SimMeme uses a generic graph-based data model that aligns all the information available about the Memes with a semantic ontology. A novel similarity measure that interweaves common image, textual, structural and semantic similarities into one holistic measure is employed to effectively answer user queries. We will demonstrate the operation of SimMeme over a large repository of real-life annotated Memes which we have constructed by web crawling and crowd annotations, allowing users to appreciate the quality of the search results as well as the execution efficiency.