Types of Deep Learning Architecture

Deep Learning

Machine learning technology powers many aspects of modern society from web searches to content filtering on social networks to recommendations on e-commerce websites. It is increasingly present in consumer products such as cameras and smart phones. Machine learning systems are used to identity objects in images, transcribe speech into text, match news items, posts or products with users, interests and select relevant results of search. These applications make use of a class of techniques called Deep Learning.

Deep learning also known as deep structured learning, hierarchical learning or deep machine learning. It is a branch of machine learning based on a set of algorithms that attempt to model high level abstractions in data. In a simple case, you could have two sets of neutrons:

i. Receives an input signal
ii. Send an output signal

Deep Learning Architecture

1. Generative Deep Architecture: These deep architectures which are intended to characterize the high order correlation properties of the observed or visible data for pattern analysis or synthesis purposes.

2. Discriminative Deep Architecture: These deep architectures which are intended to directly provide discriminative power for pattern classification. It is often by characterizing the posterior distributions of classes conditioned on the visible data. Hybrid deep architecture where the goal is discrimination but is assisted with the outcomes of generative architecture via better optimization and regularization. Discriminative criteria are used to learn the parameters in any of the deep generative models.

Types of Deep Learning Architecture

There are mainly three types of deep learning architecture:

1. Deep Feed-Forward Networks:

Deep feed forward networks also often called feed-forward neural networks or multilayer perceptions (MLPs). The goal of a feed-forward network is to approximate some function f”.

Example: For a classifier, y = f”(x) maps an input x to a category y. A feed-forward network defines a mapping y = f(x;e) and learn the value of the parameters e that result in the best function approximation.

2. Convolution Neural Networks:

In machine learning, a convectional neural network (CNN) is a type of feed-forward artificial neural network in which the connectivity pattern between its neurons. It is inspired by the organization of the animal visual cortex.

Individual cortical neurons respond to stimuli in a restricted region of space known as the receptive field. The receptive fields of different neurons partially overlap such that they tile the visual field. The response of an individual neuron to stimuli within its receptive field can be approximated mathematically by a convolution operation. Convolution networks were inspired by biological processes. The processes are variations of multilayer perceptrons designed to use minimal amounts of pre-processing. They have wide applications in image and video recognition, recommender systems and natural language processing.

3. Recurrent Neural Networks:

In a traditional neural network, we assume that all inputs and outputs are independent of each other. But for many tasks that’s a very bad idea. If you want to predict the next word in a sentence you better know which words came before it. RNNs are called recurrent because they perform the same task for every element of a sequence with the output being dependent on the previous computations.