The updated schedule can be found below at the bottom of this page.
Halfway through this project, I would say that we are well on our way to completion. Our understanding of neural networks has increased considerably, and we both understand how convolutional neural networks work and how the three main components (the convolution operators, max pool operators, and fully connected perceptrons) work to create the classifier. We have experience with Halide, and how it's functions, expressions, values, and range domains can be done to write these layers in few, but very meaningful lines of code. This will allow us to write very parallelizable code, which will help us be competitive.
At the parallelism competition, we will most likely be showing a graph of the speed of our Halide implementation versus some other implementations, as well as the results for our consideration of Halide as a tool for neural network training.
Right now, the majority of the work we have left to do is finish coding the nueral network evaluation and get the weights from Caffe's implementation/training of LeNet. Caffe has been proving problematic to get to compile on our local machines, but hopefully the Latedays implementation will work for us.
Halide Implementation of a Convolutional Neural Network - Philip Massey & Arjun Kar
Our project is to port the deep learning framework Caffe to the image processing language Halide. However, this is a daunting project. In the event that this proves too difficult / impractical with the current state of the framework or language, we may opt to instead do a simpler project. Based on which of the two resources are proving difficult, we will either port Caffe to a Xeon Phi, or develop a neural network in Halide.
This project is a topic in parallel machine learning.
Our particular project is looking into neural networks.
Neural networks are not simple algorithms; a nice introduction can be found here.
As a brief overview, neural networks are made up of many layers.
Each layer uses the weighted sum of the layers beneath it and computes a new value using the hyperbolic tangent function (as it is a universal basis function).
The first layer is simply an input feature vector, and the final layer is the output (either a value for regression or a percentage for classification).
What's amazing about this algorithm is that given enough hidden layers and training time, a neural network can represent ANY function.
Halide is a new programming language / C++ API that is designed for high-performance image processing.
It also has modes for both CPUs and GPUs and many languages.
Neural networks have many opportunities for parallelism.
One example is parallelizing the different nodes in each layer, and syncing between each layer.
This allows for parallelization on both forward and backward propagation.
However, other forms of parallelism can also prove interesting.
For example, parallelizing training examples and doing batch updates is a method we may explore.
Doing this would give an approximation to the correct solution of a neural network.
However, in many machine learning algorithms, this possible decrease in accuracy is well worth the decrease in training time.
The project is challenging because of both the subject matter and the project implementation. Deep learning through neural networks is a nontrivial application of machine learning. Simply developing the methods of forward and backward propagation to train the weights of hidden layers is a challenge. On top of this algorithm, this project requires a deep understanding of the Halide image processing language. Halide is a powerful API / language built on C++ used for image processing in an object-oriented method. Our primary goal, creating a neural network evaluator Halide, will test our understanding of everything above.
For this project, we will utilize the following resources:
By the end of this project, we hope to have a working implementation of a convolutional neural network written in Halide. We will then compare (and perhaps optimize) the performance of this algorithm with other implementations. In order to be consistant, we will use a well known neural network, LeNet. This also allows us to use another framework, such as Caffe, to traing the nueral network and discover the weights for us.
We will be working in unix environment, coding our project in C++ and Halide.
In order to complete this project in a timely manner, we will keep the following schedule:
This is the project page for a 15-418 class project at Carnegie Mellon University. It will (eventually) be updated to look pretty (probably).