Automatic Fish Classification System Using Deep Learning

Fish classification and detection in the wild remains an unsolved problem due to the variation of pose, uncontrolled environmental conditions and the limited amount of training data. We propose a deep neural fish classification system to automatically label fish using a camera without interference from human. In the system, classification is done in two independent branches. One branch, where the fish are detected, aligned, and fed to classifiers, aims to handle the variation of pose and scale of fish and extract discriminative features to distinguish fish. The other branch makes use of context information in the scenario to infer the type of fish, under the assumption that the types of captured fish are correlated with the environmental conditions around such as boat, fish gears, daylight and weather. The preliminary classification prediction made in the two branches is then reweighted and averaged as the final prediction. In the world-wide competition of "The Nature Conservancy Fisheries Monitoring" hosted by Kaggle, our model achieved top 0.7% rank on the final leaderboard among solutions submitted by 2,293 different competitive teams.