Middleware Support for Edge Data Analytics over Heterogeneous Scenarios

Edge intelligence has been gaining traction and sophistication. The execution settings may range from a small cluster as an edge cloud to on-device inference on embedded devices, differing in hardware capability over orders of magnitude. The application scenarios for edge data analytics increasingly involve multimodal sensory input (e.g., combining audio, video, and text input) for richer contextual understanding. However, there is insufficient system support to handle the growing complexity and heterogeneity in edge analytics. Application development and deployment still require substantial domain knowledge of both multimodal inference and myriad execution environments. This paper presents Crystal, a framework to ease edge analytics development and deployment over diverse edge settings. Crystal presents developers with configuration interfaces to use builtin or custom libraries for common analytics modules and then compiles the application code. Internally, Crystal masks hardware heterogeneity with abstract resource types through containerization, while abstracting away application processing pipelines into generic flow graphs. On this basis, Crystal then supports a notion of degradable computing that adjusts the application flow to fit the available resource and streamlines the analytics processing by elimination modality redundancy. Crystal further interfaces with existing machine learning frameworks (e.g., TensorFlow) and containerization and orchestration tools (Docker and Kubernetes). Building atop Crystal reduces the application development effort by a factor of 10 in terms of lines of code; Meanwhile, Crystal automatically and gracefully adapts to settings from a Raspberry Pi to a small EC2 cluster, without manual effort or significant loss of analytics quality.