In my previous post, I introduced the basic ideas of Recurrent Neural Networks, as the 2nd post of RNNs, we’ll focus on long short-term memory method. LONG SHORT TERM MEMORY One of the very famous problems of RNNs is the vanishing gradient, the problem is that the influence of a given input on the hidden layer, … [Read more…]
RNNs Recurrent neural networks are very famous recently, they play as important roles as convolutional neural networks. RNNs can use their internal memory to process arbitrary sequences of inputs, so not only images, they work well on speech recognition and natural language processing tasks. There are several type of RNNs, as the beginning, we focus our attention on … [Read more…]
NAMED ENTITY RECOGNITION In Natural Language Processing, named-entity recognition is a task of information extraction that seeks to locate and classify elements in text into pre-defined categories. The following graph is stolen from Maluuba Website, it perfectly demonstrates what does NER do.
INTRODUCTION Sparse coding is one of the very famous unsupervised methods in this decade, it is a dictionary learning process, which target is to find a dictionary that we can use a linear combination of vectors in this dictionary to represent any training input vector. For better capture structures and patterns inherent in the input vectors, … [Read more…]
WHAT IS RBM Restricted Boltzmann Machine is one of the special cases of Boltzmann Machine, which restricted all visible-visible connections and hidden-hidden connections, which makes for each hidden unit, it connects to all visible units, and for each visible unit, it connects to all hidden units. Following is a figure which shows the model of RBM.
This post is about a new cluster algorithm published by Alex Rodriguez and Alessandro Laio in the latest Science magazine. The method is short and efficient, I implemented it using about only 100 lines of cpp code. BASIC METHOD There are two leading criteria in this method: Local Density and Minimum Distance with higher density. Rho above is the … [Read more…]
I chose “Dropped out auto-encoder” as my final project topic in the last semester deep learning course, it was simply dropping out units in regular sparse auto-encoder, and furthermore, in stacked sparse auto-encoder, both in visible layer and hidden layer. It does not work well on auto-encoders, except can be used in fine-tune process of stacked sparse … [Read more…]
The CIFAR-10 dataset can be found HERE. It is a very popular multi-channel image dataset for classifier training, as a simplify version of CIFAR-100, it is easier to use for newbies. Here’s a C++ version code for reading this dataset from .bin files into OpenCV matrices.
WHAT IS CNN A Convolutional Neural Network (CNN) is comprised of one or more convolutional layers, pooling layers and then followed by one or more fully connected layers as in a standard neural network. The architecture of a CNN is designed to take advantage of the 2D structure of an input image (or other 2D input … [Read more…]
During this spring break, I worked on building a simple deep network, which has two parts, sparse autoencoder and softmax regression. The method is exactly the same as the “Building Deep Networks for Classification” part in UFLDL tutorial. For better understanding it, I re-implemented it using C++ and OpenCV. GENERAL OUTLINE Read dataset (including training data … [Read more…]