Our submission for the 15-418 Parallelism Competition is our evaluation of using Halide, a functional programming language designed for image processing and computational photography, to construct a neural network. Neural networks are an increasingly popular structure for machine learning, especially image recognition tasks. Because of Halide's design, parallelizing image processing tasks is incredibly simple, and we believe that it will translate well to evaluating neural networks.
Our final project paper can be found here.
For our project, we chose to construct the LeNet neural network, a popular network designed to classify images of digits in the MNIST dataset. This neural network consists of 6 layers. The first is the input layer The second and third are convolution / pooling layers. The fourth and fifth layers are fully connected layers. The final layer is a softmax layer, which calculates the class. An excellent example of this network, and the one we based our system off, can be found here. We were successful in our development of this system. In Halide, we were able to develop each layer and the connections between them, such that an image can be pipelined through the network and classified.
We have successfully ported our Halide neural network to the latedays cluster, and our remaining work will optimizing the code there. We will then compare it to other neural network implementations, such as Caffe and Theano.
Our initial results (using almost completely unscheduled, unoptimized Halide code) runs a single image through our neural network on a small laptop computer in ~0.8 seconds. Top-grade implementations such as Caffe claim to be able to classify images in ~1 millisecond on a high-end GPU such as an NVIDIA K40. While this is a significant difference, we believe the difference in computational resources will have a much larger impact. We hope to completely optimize and schedule our Halide code and perform direct time trials versus Caffe within the next two days.
Previous work on this project, including the checkpoint and proposal, can be found here.