Introduction to Deep Learning

Introduction 

Remember the time you came across a rare species of bird and your curious self took your phone out and clicked a picture of the bird in the google lens application. The google lens then extracted all the relevant data about the bird from the internet to let you know everything you wanted to know about the bird. Ever wondered what engineering goes behind such an intelligent application? 

The google lens identifying an object is a prime example of what Deep Learning is all about. Deep Learning is a subsidiary of Machine Learning and it’s completely based on Neural Networks. A neural network as the name suggests seems something that has to do with the human brain. It essentially tries to mimic how a human brain operates. And it’s complex engineering. And this comes at a high computational cost. Deep Learning algorithms are indeed computationally expensive. 

Source - link

Also See, Resnet 50 Architecture

The Idea 

A neuron plays an integral role in the functioning of the human brain. The idea behind deep learning is to build this network of neurons artificially. So how do we simulate the functioning network on a computer? We build an architecture wherein we have the input value, the output value, and some hidden layers in between. There are various architectures to simulate the neural network-

• Deep Neural Network: DNN is a multi-layer structure consisting of an input layer, the output layer, and the hidden layers in between the input and the output layer. The information moves from the input layer to the hidden layer calculating the outputs with initial weights and bias. The process is called forward propagation. The next phase is backpropagation. It is a technique of adjusting the weights and minimising the loss function. We have in-depth articles on both these techniques on our website. This architecture is well suited for non-linear relationships. Suppose we have to build a model that could identify the breed of a dog. So the input layer takes the input, the computation is done for the hidden layers, and then the output is generated. The individual elements that form a layer are called nodes. Each node performs a specific task. A DNN architecture could consist of hundreds or even thousands of hidden layers. Hence the name, Deep Neural Network. 

Source - link

More the layers, more complex functionalities it has to offer. To understand it better, let’s say you meet a person you have never met in your life. However, you can still identify him as a human based on certain characteristics specific to humans. That’s the basis of a DNN. Creative and Analytical elements are analyzed and grouped to ensure the object is identified correctly. 

• Deep Belief Network: DBNs are a type of DNN and are a multilayered architecture. DBNs are graphical representations that generate all possible outcomes for a given case. When trained without supervision, DBNs learn to reconstruct the given input probabilistically. This is called the learning step. The layers work as feature detectors, which on supervision can perform classification. DBNs can be said to be the composition of simple and unsupervised networks like restricted Boltzmann machines wherein each subnetwork’s hidden layer serves as the visible layer for the next.

• Recurrent Neural Network - RNNs are a type of Artificial Neural Network which uses sequential data. These deep learning algorithms are usually used for temporal and ordinal problems such as language translation. Their salient feature is that they can store data from prior inputs in the sequence which helps in generating better current outputs. In a regular feed-forward network, the hidden layers work independently of each other. That means the outputs from previous inputs aren’t memorised. This is because each independent layer is assigned different weights and biases. In an RNN, the layers are provided with the same set of weights and biases. This converts all the hidden layers into one single recurrent layer. Let us consider an example, suppose we input the word ‘network’ in a regular feed-forward network. It would have to process the word character by character since it doesn’t have any means to learn from its past inputs and outputs besides what it was initially trained on. An RNN copies the output to loop it into the network.

Source - link

There are various types of RNNs categorised based on their mapping of input to the output.

• One to One - When a single input is mapped to a single output.

Source - link

• One to many - When a single input is mapped to multiple outputs. 

Source - link

• Many to one - When multiple inputs are mapped to a single output. 

Source - link

• Many to many - When multiple inputs are mapped to multiple outputs. 

Source - link

ML vs DL - Understanding the difference

We know DL is a subset of Machine learning. But what more does it have to offer over machine learning and how does it differ exactly?

Understanding the working

Deep Learning models learn by observation and determining distinguishing features of certain objects which would then be used for classification. Suppose we train a model to determine shapes. How would we determine if a shape is a square? We will observe if a shape has features specific to a square. For example, if a shape has 4 sides, 4 right angles, diagonals bisecting each other at right angles, we can safely classify it as a square. Since deep learning models try to mimic the human brain and how it functions, this is essentially how they arrive at results. 

Advantages of deep learning

1. Highly reliable if implemented correctly.
2. Doesn’t require preprocessing of data. 

Disadvantages of deep learning

1. Requires a huge amount of data in the training phase. 
2. The computational cost incurred is high.

Applications

DL finds its uses across various sectors. 

1. Healthcare- Deep learning models can help detect various disorders like tumours. 
2. Auto-translation- Deep learning can detect a language and can automatically translate input text into some other language. Google translate works on principles of Deep Learning. 
3. Image Recognition- Deep learning models are vastly used for image recognition. Google lens is an excellent example of the purpose. 

Also read about, Artificial Intelligence in Education