Lecture9b.slides.pdf - CS4487 Machine Learning Lecture 9b...

This preview shows 1 out of 5 pages.

CS4487 - Machine Learning Lecture 9b - Neural Networks, Deep Learning Dr. Antoni B. Chan Dept. of Computer Science, City University of Hong Kong Outline History Perceptron Multi-layer perceptron (MLP) Convolutional neural network (CNN) Autoencoder (AE) Example on MNIST Dataset Images are 28x28, digits 0-9 6,000 for training 10,000 for testing In [6]: n_train, nrow, ncol, trainimg = read_img( 'data/train-images.idx3-ubyte' ) _, trainY = read_label( 'data/train-labels.idx1-ubyte' ) n_test, _, _, testimg = read_img( 'data/t10k-images.idx3-ubyte' ) _, testY = read_label( 'data/t10k-labels.idx1-ubyte' ) # for demonstration we only use 10% of the training data sample_index = range ( 0 , trainimg . shape[ 0 ], 10 ) trainimg = trainimg[sample_index] trainY = trainY[sample_index] print trainimg . shape print trainY . shape print testimg . shape print testY . shape (6000, 28, 28) (6000,) (10000, 28, 28) (10000,)
Image of page 1

Subscribe to view the full document.

In [7]: # Example images plt . figure(figsize = ( 8 , 4 )) show_imgs(trainimg[ 0 : 50 ]) Pre-processing Reshape images into vectors map to [0,1], then subtract the mean In [8]: # Reshape the images to a vector # and map the data to [0,1] trainXraw = trainimg . reshape(( len (trainimg), -1 ), order = 'C' ) / 255.0 testXraw = testimg . reshape(( len (testimg), -1 ), order = 'C' ) / 255.0 # center the image data (but don't change variance) scaler = preprocessing . StandardScaler(with_std = False ) trainX = scaler . fit_transform(trainXraw) testX = scaler . transform(testXraw) Generate a fixed validation set use vtrainX for training and validX for validation In [9]: # generate a fixed validation set using 10% of the training set vtrainX, validX, vtrainY, validY = \ cross_validation . train_test_split(trainX, trainY, train_size =0.9 , test_size =0.1 , random_state =4487 ) # validation data validset = (validX, validY)
Image of page 2
MNIST - Logistic Regression (0-hidden layers) Training procedure We specify the validation set so that it will be fixed when we change the random_state to randomly initialize the weights. Train on the non-validation training data. Use a larger batch size to speed up the algorithm In [10]: nn = mlp . Classifier( layers = [ mlp . Layer( "Softmax" ) # one output layer ], valid_set = validset, # specify the validation set batch_size =50 , # use 50 images per update learning_rate =0.05 , n_iter =100 , callback = callbacks, random_state =4487 ) nn . fit(vtrainX, vtrainY) plot_nnstats(nnstats) predY = nn . predict(testX) acc = metrics . accuracy_score(testY, predY) print "test accuracy: " + str (acc) Examine the weights of the network params contains a list of weights/biases for each layer //anaconda/lib/python2.7/site-packages/theano/tensor/signal/downsample.py:6: UserWarning: downsample module has been moved to the theano.tensor.signal.po ol module. "downsample module has been moved to the theano.tensor.signal.pool module. ") [(10000, 10)] test accuracy: 0.8928
Image of page 3

Subscribe to view the full document.

In [11]: params = nn . get_parameters() print params Reshape the weights into an image input images that match the weights will have high response for that class. In [12]: W = params[ 0 ] . weights filter_list = [W[:,i] . reshape(( 28 , 28 )) for i in range (W . shape[ 1 ])] plt . figure(figsize = ( 8 , 4 )) show_imgs(filter_list, nc =5 , titles = " %d " ) MNIST - 1-hidden layer Add 1 hidden layer with 50 ReLu nodes each node is extracting a feature from the input image [Parameters(weights=array([[-0.0842584 , -0.05636672, 0.04469939, ..., -0.0 4113571, -0.02815367, -0.00659934], [-0.01111129, 0.04122496, 0.04119325, ..., 0.03417046, -0.08094409, 0.03806942], [-0.00935 , -0.02723137, 0.08016123, ..., 0.05052316, 0.07903875, -0.08166334], ..., [-0.02714266, 0.03911813, -0.02840337, ..., -0.06613068, -0.06183281, -0.08018292], [-0.08692493, -0.0517126 , -0.00311253, ..., 0.02136038, 0.02307253, 0.05877 ], [-0.08485767, 0.04953596, -0.02196686, ..., -0.04254714, 0.01364783, -0.04126686]], dtype=float32), biases=array([-0.9824140
Image of page 4
Image of page 5
You've reached the end of this preview.
  • Fall '16
  • Antoni B. CHAN
  • Artificial neural network, neural network

{[ snackBarMessage ]}

What students are saying

  • Left Quote Icon

    As a current student on this bumpy collegiate pathway, I stumbled upon Course Hero, where I can find study resources for nearly all my courses, get online help from tutors 24/7, and even share my old projects, papers, and lecture notes with other students.

    Student Picture

    Kiran Temple University Fox School of Business ‘17, Course Hero Intern

  • Left Quote Icon

    I cannot even describe how much Course Hero helped me this summer. It’s truly become something I can always rely on and help me. In the end, I was not only able to survive summer classes, but I was able to thrive thanks to Course Hero.

    Student Picture

    Dana University of Pennsylvania ‘17, Course Hero Intern

  • Left Quote Icon

    The ability to access any university’s resources through Course Hero proved invaluable in my case. I was behind on Tulane coursework and actually used UCLA’s materials to help me move forward and get everything together on time.

    Student Picture

    Jill Tulane University ‘16, Course Hero Intern