[update] If you would like to skip the instructions and just install from an AMI, search the community for ami-b141a2f5 (Theano - CUDA 7) in the N. California region.

Theano is an amazing Python package for deep learning that can utilize NVIDIA's CUDA toolkit to run on the gpu. The gpu is orders of magnitude faster than the cpu for math operations (such as matrix multiplication), which is essential for many machine learning algorithms. While setting up AWS to run my thesis experiments, I realized many instructions were out of date. Hopefully these steps will help you get your deep learning models up and running on AWS. These instructions use Ubuntu 14.04 64-bit with Cuda 7.0 on a g2.2xlarge instance.

Creating an EC2 g2.2xlarge instance

Log into your AWS management console and click over to EC2. There are two options for creating your instance:

• On-Demand Instances: This option guarantees that your have your section of a machine for computing. Use this option if you cannot handle potential interrupts and are willing to pay more. As of this writing, a g2.2xlarge instance costs $0.65/hr. To create a dedicated instance, go to "Instances" and click "Launch Instance".
• Spot Instances: This option gives you left-over compute power at a much cheaper rate. Use this option if you can hande potential interrupts. Spot instances use a bidding system to determine who gets the left-over compute power; you can look at the history of rates under "Spot Requests". As of this writing, a g2.2xlarge spot instance costs $0.0642/hr (10% of the price!) and seems fairly stable over the last few months. I ended up using a few spot requests, bidding $0.07.

When choosing an AMI to use for the instance, pick the latest 64-bit Ubuntu (I used ami-df6a8b9b, which is Ubuntu 14.04). You should also set up the security group for your IP to have ssh access, and download a new private/public key file.

Installing Theano and its dependencies

Follow the instructions on Amazon to ssh into your instance. For Ubuntu on AWS, I found you have to use :

ssh -i [path/to/key.pem] ubuntu@[DNS]

Once you ssh into the instance, I recommend using screens (now updated to Tmux) to keep the session open so your program can run without the terminal window being open. Here are all the commands and the order I used to set up Theano:

1. update the default packages

sudo apt-get update

sudo apt-get -y dist-upgrade

2. create a new screen named theano (or look at using Tmux instead)

screen -S “theano”

3. install all the dependencies

sudo apt-get install -y gcc g++ gfortran build-essential git wget linux-image-generic libopenblas-dev python-dev python-pip python-nose python-numpy python-scipy

4. install the bleeding-edge version of Theano

sudo pip install --upgrade --no-deps git+git://github.com/Theano/Theano.git

5. grab the latest (7.0) cuda toolkit.

sudo wget http://developer.download.nvidia.com/compute/cuda/repos/ubuntu1404/x86_64/cuda-repo-ubuntu1404_7.0-28_amd64.deb

6. depackage Cuda

sudo dpkg -i cuda-repo-ubuntu1404_7.0-28_amd64.deb

7. add the package and install the cuda driver (this takes a while)

sudo apt-get update

sudo apt-get install -y cuda

8. update the path to include cuda nvcc and ld_library_path

echo -e "\nexport PATH=/usr/local/cuda/bin:$PATH\n\nexport LD_LIBRARY_PATH=/usr/local/cuda/lib64" >> .bashrc

9. reboot the system for cuda to load

sudo reboot

Wait a little bit for the reboot and then ssh back into the instance.

10. install included samples and test cuda

cuda-install-samples-7.0.sh ~/

cd NVIDIA\_CUDA-7.0\_Samples/1\_Utilities/deviceQuery
make
./deviceQuery

Make sure it shows that the GPU exists.

11. set up the theano config file to use gpu by default

echo -e "\n[global]\nfloatX=float32\ndevice=gpu\n[mode]=FAST_RUN\n\n[nvcc]\nfastmath=True\n\n[cuda]\nroot=/usr/local/cuda" >> ~/.theanorc

That's it! Now you can use git to pull whatever repo you need and test Theano. I recommend using the test script from Theano's site here.

These instructions were partially gathered from Erik Bernhardsson and Erik Hazzard, as well as the Ubuntu instructions from Theano.

Optimizations

Netflix has a great post showing that you can customize some functions in the Cuda kernel to speed up Theano by ~70%. I haven't done this, but it is definitely something to check out.