Vision capabilities have been significantly enhanced in recent years due to the availability of powerful computing hardware and sufficiently large and varied databases. However, the labelling of these image databases prior to training still involves considerable effort and is a roadblock for truly scalable learning. For instance, it has been shown that tag noise levels in Flickr images are as high as 80%. In an effort to exploit large images datasets therefore, extensive efforts have been invested to reduce the tag noise of the data by refining the image tags or by developing robust learning frameworks. In this work, we follow the latter approach, where we propose a multi-layer neural network-based noisy learning framework that incorporates noise probabilities of a training dataset. These are then utilized effectively to perform learning with sustained levels of accuracy, even in the presence of significant noise levels. We present results on several datasets of varying sizes and complexity and demonstrate that the proposed mechanism is able to outperform existing methods, despite often employing weaker constraints and assumptions.