Semantic manipulation through the lens of Geometric Algebra

Semantic image manipulation is a complex problem defined as the ability to change high-level features while keeping the final result visually similar to the original. Recent deep generative solutions show that manipulating semantic features in latent space produces compelling results. Even if we consider those advances, the question remains: Can we algebraically operate high-level visual semantic features in images with meaningful operations? In this paper, we demonstrate the feasibility of interpreting and manipulating image pseudovectors ((n - 1) -dimensional subspaces) as the union of visual features (k-dimensional subspaces, for 0 < k < n) operated using Geometric Algebra (GA). Depending on how the latent space is organized, any GA operation would be applicable, enabling the solution to handle an open set of problems without retraining generative models for specific tasks. As a proof of concept, in this paper, we demonstrate how GA operations can be applied to manipulate subspaces in the latent space of faces to perform operations like putting on or taking off clothing accessories, transferring age characteristics, changing hairstyles, and performing semantic queries in sets of images.