Code360 powered by Coding Ninjas X Naukri.com. Code360 powered by Coding Ninjas X Naukri.com
Table of contents
1.
Introduction
2.
What is a CNN?
3.
What are GPUs?
4.
Training a CNN
5.
Why are GPUs preferred over CPUs in Deep Learning?
5.1.
Larger Memory Bandwidth
5.2.
Parallelization
5.3.
Fast Memory Access
5.4.
Explanation by Programming
5.4.1.
CPU
5.4.2.
GPU
6.
Frequently Asked Questions
6.1.
What is a GPU?
6.2.
What is an Artificial Neural Network?
6.3.
What is a CNN?
6.4.
Why are GPUs and CNN preferred over CPUs and CNN?
6.5.
Why are CPUs preferred over GPUs for scalar multiplication?
7.
Conclusion
Last Updated: Mar 27, 2024
Medium

GPUs and CNN

Master Python: Predicting weather forecasts
Speaker
Ashwin Goyal
Product Manager @

Introduction

Hello, ninjas! In this advancing world, you must have heard terms like artificial intelligenceneural networksdeep learning, matrices, etc. In this article, we are going to study GPUs and CNN. CNN stands for Convolutional Neural Network, and GPU stands for Graphics Processing Unit.

Introduction to GPUs and CNN

What is a CNN?

First of all, let us understand what a CNN is.

CNN stands for Convolution Neural Network. It is a version of the Artificial Neural Network(derived from the biological neural network). It is used to analyze and extract information, features, and patterns from grid-based matrix-like inputs. These inputs generally are images and videos.

It consists of multiple layers as follows:

  • Input layer
     
  • Convolution Layer
     
  • Pooling Layer
     
  • Fully Connected Layer
     
  • Output Layer
     

Below is the architecture of CNN.

convolution neural network architecture
  • The input layer takes the input.
     
  • The convolution layer filters the video or image to extract features.
     
  • The pooling layer reduces the size of the final feature map to make fast processing and prevent overfitting.
     
  • The input is fed into the Fully Connected layer, which applies mathematical functions. And finally, the output is generated.
Get the tech career you deserve, faster!
Connect with our expert counsellors to understand how to hack your way to success
User rating 4.7/5
1:1 doubt support
95% placement record
Akash Pal
Senior Software Engineer
326% Hike After Job Bootcamp
Himanshu Gusain
Programmer Analyst
32 LPA After Job Bootcamp
After Job
Bootcamp

What are GPUs?

GPU stands for Graphics Processing Unit. A GPU analyzes and processes graphical inputs like images, videos, games, etc. It reduces the processing time for graphical input. It can process many chunks of data simultaneously and is thus used in machine learning. It can be integrated with the system or have a separate hardware unit.

Training a CNN

Now, let us talk about the training of CNN.

Training a CNN is a heavy process because it requires significant hardware capacity to perform operations timely. Various operations take place in it, like the following.

  • Pre-processing the input data
     
  • Training the learning model
     
  • Storing the trained learning model
     
  • Final deployment of the model
     

Forward and backward passes are performed during the training of the data model.

In the forward pass, the input is processed, and the output is generated. In the backward pass, the generated output is analyzed, the error is calculated, and the neural network's input weights are adjusted accordingly.

Let us look at a diagram for this.

neural network

  • Here, we can see that we have many inputs (x1, x2, x3, ……., xn).
     
  • Also, the edges with which the input nodes are connected to the transfer function have associated weights(w1, w2, w3, …….., wn).
     
  • The transfer function processes the inputs and weights and provides the net input to the activation function, which finally, based on the threshold value, generates the output.
     
  • Afterward, if there is any error in the output, the weights are readjusted accordingly.

Why are GPUs preferred over CPUs in Deep Learning?

These operations in the neural network discussed above are matrix multiplications, and GPUs are faster in doing these than CPUs.

Three main features of GPUs give them an advantage over CPUs while being used for deep learning.

These are:-

1. Larger Memory Bandwidth

2. Parallelization

3. Fast Memory Access

Let us study these features one by one.

Larger Memory Bandwidth

Consider the CPU a speed boat with less loading capacity but a faster access rate. Now, consider GPU a cargo ship with a substantial loading capacity but a slower access rate.

When dealing with matrix multiplication, loads of data are required. Now, a CPU will fetch a small amount of data thousands of times, whereas a GPU will load much more in a single go. That is why GPUs and CNN are preferred more due to their larger memory bandwidth.

Parallelization

Now, although GPUs fetch loads of data in a single go, they have a slower access rate than CPUs, so while they are fetching, the processor would have already finished the computation on the previously fetched data and would be idle. So, to avoid this, GPU does parallelization.

This means that at the same time, more than one part fetches the data so the processor does not remain idle. In simple words, with context to the previous analogy, multiple cargo ships are loading the data.

Fast Memory Access

GPUs have hundreds of simple cores that are smaller and hence faster. CPUs have few complex cores. Although CPUs are faster for handling system operations.

Explanation by Programming

Let us see some code in Python justifying our discussion.

We can use the following commands to understand the difference in speed between a GPU and a CPU when dealing with scalar and matrix multiplications.

CPU

First, let us code for checking the speed of the CPU.

import torch

%%timeit
t=torch.randn(2000,2000)
res=torch.matmul(t,t)
del t,res

%%timeit
t=torch.randn(2,2)
res=torch.matmul(t,t)
del t,res


We imported the 'torch' library in Python because this is an ML library for creating deep neural networks.

Then, for CPU, we took two cases, one of scalar multiplication and the other of matrix multiplication.

We use the '%% timeit' line of code to get the time taken to compute the multiplications.

Also, we use the '.matmul' function, which returns the matrix product of two arrays that we have generated with random values using the '.randn' function.

Output

CPU time output scalar and matrix

The CPU takes around 4.31 microseconds to compute scalar multiplication and around 198 ms to compute matrix multiplication.

GPU

Now, we will perform the same using GPU with the following code.

import torch
device = torch.device('cuda' if torch.cuda.is_available() else 'cpu')
%%timeit
t=torch.randn(2,2).to(device)
res=torch.matmul(t,t)
del t,res


%%timeit
t=torch.randn(2000,2000).to(device)
res=torch.matmul(t,t)
del t,res


If you are using Google Colaboratory for the programs, select runtime options and then choose GPU for running the programs using GPU.

Here, we use 'cuda' because it provides GPU-based parallel processing for running the program.

Here also, we use the same '%%timeit', '.randn', and '.matmul' functions. We use the '.to(device)' function to move the model to the device that we have specified before.

Output

GPU time taken scalar and matrix

Here, we can see that the time GPU takes for scalar multiplication is around 59.8 microseconds, and the time taken for matrix multiplication is around 26.8 ms.

Hence, for matrix multiplication, GPUs are relatively much faster than CPUs, which is why we use GPUs and CNN.

Frequently Asked Questions

What is a GPU?

GPU stands for Graphics Processing Unit. It is an electronic circuit specially designed for handling graphics-related work like image processing, videos, games, designing, etc. These work faster when dealing with matrix-related operations.

What is an Artificial Neural Network?

An Artificial Neural Network or ANN is a network derived from the biological neural network. It is a collection of several units that take inputs, process them and give outputs like a biological network. It tries to replicate the human-brain mechanism.

What is a CNN?

CNN stands for Convolutional Neural Network. It is a version of ANN primarily used for extracting information from gird-based inputs like images or videos.

Why are GPUs and CNN preferred over CPUs and CNN?

It is because GPUs provide three main advantages over CPUs for deep learning. These are large memory bandwidth, parallelization, and fast memory access.

Why are CPUs preferred over GPUs for scalar multiplication?

It is because CPUs have a faster fetch rate and complex cores for handling functions like system operations.

Conclusion

In this article, we first studied what Convolutional Neural Networks and Graphics Processing Units are. Then, we moved to the integration of GPUs and CNN. Finally, we understood it with the help of a simple example of why we use GPUs and CNN.

We hope this article helped you understand why GPUs are used when dealing with CNNs. If you like to study these topics more, do read the following:-

To learn more about DSA, competitive coding, and many more knowledgeable topics, please look into the guided paths on Coding Ninjas Studio. Also, you can enroll in our courses and check out the mock test and problems available. Please check out our interview experiences and interview bundle for placement preparations.

Happy Coding!

Previous article
Convolution layer, Padding, Stride, and Pooling in CNN
Next article
Convolutional Neural Network in Keras
Live masterclass