Monthly Archives: February 2014

Softmax Regression

WHAT IS SOFTMAX Softmax regression always as one module in a deep learning network, and most likely to be the last module, the output module. What is it? It is a generalized version of logistic regression. Just like logistic regression, it belongs to supervised learning, and the superiority is, the class label y can be more than […]

Posted in Algorithm, Machine Learning | Tagged , , , , | 1 Response

Seminar this week

I attended a seminar this Monday. The first part was by Prof. Michael Jordan from UC Berkeley, it was about statistics and big data, to be honest, I had no idea of what he was talking about, I barely understood those things. However, the second part was by Prof. Rob Fergus, about deep learning for computer vision, […]

Posted in Twaddle | Tagged , | Leave a comment

[UFLDL Exercise] Softmax Regression

I’m learning Prof. Andrew Ng’s Unsupervised Feature Learning and Deep Learning tutorial, This is the 4th exercise, which is using Softmax regression to build a classifier and classify MNIST handwritten digits. Just like my other UFLDL exercise posts, I’ll not go through the detail of the material. More details about this exercise can be found HERE. I’ll re-implement Softmax […]

Posted in Algorithm, Machine Learning | Tagged , , , , | 4 Responses

[UFLDL Exercise] PCA and Whitening

I’m learning Prof. Andrew Ng’s Unsupervised Feature Learning and Deep Learning tutorial, This is part 2 of the 3rd exercise, which is use PCA algorithm in a natural image dataset. Just like my other UFLDL exercise posts, I’ll not go through the detail of the material. More details about this exercise can be found HERE.

Posted in Algorithm, Machine Learning | Tagged , , , , , | 2 Responses

[UFLDL Exercise] PCA in 2D

I’m learning Prof. Andrew Ng’s Unsupervised Feature Learning and Deep Learning tutorial, This is part 1 of the 3rd exercise, which is use PCA algorithm in a simple 2D dataset. Just like my other UFLDL exercise posts, I’ll not go through the detail of the material. More details about this exercise can be found HERE.

Posted in Algorithm, Machine Learning, Uncategorized | Tagged , , , , | Leave a comment

[UFLDL Exercise] Sparse Autoencoder

I’m learning Prof. Andrew Ng’s Unsupervised Feature Learning and Deep Learning tutorial, I finished the first exercise, the tutorial is very professional and easy to learn. I don’t think I need to go through the detail of what Sparse Autoencoder is, I’ll put my code of the exercise here, if you have any question about it, feel […]

Posted in Algorithm, Machine Learning | Tagged , , , , , , | 6 Responses

Something About Convolution

I was in Communications Engineering major during my undergraduate years, I learnt Signals and Systems and Digital Signal Processing before, so I thought I was quite familiar with things like convolution, Fourier transform. However, recently I realized that what I knew is only sort of superficial things. I’m attending Prof. Eero Simoncelli‘s “Representation and Analysis of […]

Posted in Algorithm, Maths | Tagged , , , , , | Leave a comment

Configure X11 in Octave

When I plotting my first figure with octave yesterday, an error occurred and said: gnuplot> set terminal aqua enhanced title “*********” line 0: unknown or ambiguous terminal type; type just ‘set terminal’ for a list I searched on google and found this can be solved by make all gnuplot invocations to output to X11. This can […]

Posted in Something else | Tagged , | Leave a comment

K-Means Clustering

ABOUT UNSUPERVISED LEARNING In supervised learning problems, we deal with labeled data, means during the training process, we give our machine both Xs and Ys, by training, we ‘forge’ a system, which given new Xs, it is able to guess the output Ys. By using better algorithm and sophisticated methods, we can make the forged system […]

Posted in Algorithm, Machine Learning | Tagged , , , , , , , | Leave a comment

Back Propagation Neural Networks

I used to thought that BP Neural Networks algorithm belongs to supervised learning, however, after learned about Sparse Autoencoder algorithm, I realized it can also be used for unsupervised learning (use the unlabeled data itself as both input and output). BP neural networks is the base of a lot of other advanced neural networks algorithm, […]

Posted in Algorithm, Machine Learning | Tagged , , , , , , | 3 Responses