TinyWolf - Efficient on-device TinyML training for IoT using enhanced Grey Wolf Optimization

Training a deep learning model generally requires a huge amount of memory and processing power. Once trained, the learned model can make predictions very fast with very little resource consumption. The learned weights can be fitted into a microcontroller to build affordable embedded intelligence systems which is also known as TinyML. Although few attempts have been made, the limits of the state-of-the-art training of a deep learning model within a microcontroller can be pushed further. Generally deep learning models are trained with gradient optimizers which predict with high accuracy but require a very high amount of resources. On the other hand, nature-inspired meta-heuristic optimizers can be used to build a fast approximation of the model's optimal solution with low resources. After a rigorous test, we have found that Grey Wolf Optimizer can be modified for enhanced uses of main memory, paging and swap space among alpha,beta,delta and omega wolves. This modification saved up to 71% memory requirements compared to gradient optimizers. We have used this modification to train the TinyML model within a microcontroller of 256KB RAM. The performances of the proposed framework have been meticulously benchmarked on 13 open-sourced datasets.