Better Understanding of Convolutional Networks: The Lottery Ticket Hypothesis
A re-implementation of the iterative pruning algorithm proposed by J.Frankle and M.Carbin
The Lottery Ticket Hypothesis states that there exists, in a neural network, a smaller sub-network which, when trained in isolation, achieves the same or higher accuracy as the original network. As computational time is a limitation in training neural networks, being able to effectively find smaller sub-networks which perform well is desirable. In this study we re-implemented the algorithm used for finding such sub-networks in the original Lottery Ticket Hypothesis paper and attempted to replicate the experiments. We are able to successfully replicate the experiments using the proposed algorithm, as we find winning tickets that achieve the same accuracy as the original network, while they are able to learn faster. As in the original article, we also see the pattern that the initialization of the weights affects the outcome, as randomly re-initialized weights of the winning ticket result in worse performance and longer training time. Se the full report below. The source code can be found on Github here.