Code360 powered by Coding Ninjas X Naukri.com. Code360 powered by Coding Ninjas X Naukri.com
Table of contents
1.
Introduction
1.1.
How vectorization works internally?
1.2.
Mathematical Intuition
1.2.1.
Non-vectorized
1.2.2.
Vectorized
2.
Vectorization in Machine Learning
2.1.
Importing Libraries
2.2.
Initializing both vectors with random values
2.3.
Non-Vectorized Implementation
2.4.
Vectorized Implementation
2.5.
Output
2.6.
Result
3.
Frequently Asked Questions
4.
Key Takeaways
Last Updated: Mar 27, 2024

Vectorization

Master Python: Predicting weather forecasts
Speaker
Ashwin Goyal
Product Manager @

Introduction

As the data increases or the complexity of a model increases, the machine learning algorithms tend to get slower, and generating predictions can take many hours or days, then we have to rely on computational resources (like GPUs or more CPUs) to reduce the waiting time.

Vectorization is the process of reducing the algorithm’s running time without using external computational resources.

The idea of Vectorization is to remove explicit ‘for loops’ from the code by replacing it with a vectorized implementation that yields the same output.

Also See, Resnet 50 Architecture

How vectorization works internally?

Every CPU has Single Instruction Multiple Dimension (SIMD) sets which means that we can apply the same instruction to multiple points simultaneously, thereby using the parallelism property of the CPU. So, if there are N elements in an array, we can simply execute the same instruction on multiple dimensions simultaneously instead of looping N times.

Also see, Artificial Intelligence in Education

Mathematical Intuition

Let us assume that our CPU can process four dimensions/points simultaneously. So, the time taken to process N dimensions would be N/4.

Non-vectorized

for i in range(0, N):
   prod[i] = array_one[i] * array_two[i]

 

Vectorized

for i in range(0, N, 4):
   prod[i] = array_one[i] * array_two[i]
   prod[i+1] = array_one[i+1] * array_two[i+1]
   prod[i+2] = array_one[i+2] * array_two[i+2]
   prod[i+3] = array_one[i+3] * array_two[i+3]

 

Vectorization in Machine Learning

In Machine Learning or Deep Learning, we often train our models on large datasets. So it becomes crucial that our code is time optimized, else the model will take several hours or days to yield results.

Let us take the example of the Logistic Regression ML algorithm,

We have the below equation,

 

 

The first step in calculating the value of Z is to multiply the two vectors W and X. We can multiply the two vectors in two ways vectorized and non-vectorized. Let’s see the python implementation of both.

Importing Libraries

import numpy as np    # Using numpy for doing operations on the arrays
import time    # Using time to determine the time taken by different operations

 

Initializing both vectors with random values

W = np.random.rand(5000000)    # one-dimensional array with five million randomly generated numbers

X = np.random.rand(5000000)    # one-dimensional array with five million randomly generated numbers

 

Non-Vectorized Implementation

# Non-vectorized demonstration

before_time = time.time()

Z = np.zeros((1,1))
for i in range(0, len(W)):
 Z += W[i]*X[i]

after_time = time.time()

print("Time taken by non-vectorized multiplication is: ", (after_time - before_time)*1000, " ms")

print(round(Z[0][0], 5))

 

Vectorized Implementation

# Vectorized demonstration

before_time = time.time()

Z = np.dot(W.transpose(), X)

after_time = time.time()

print("Time taken by the dot product is: ", (after_time - before_time)*1000, " ms")
print(round(Z, 5))

 

Output

Result

We can see from the above implementations that the non-vectorized implementation took around 14000 milliseconds (14 seconds) to give the output. In contrast, the vectorized implementation took only seven milliseconds which is 2000 times faster.

As the dimensions of the matrices increase, the time difference between the two approaches will also increase. Hence, it is good to always use vectorization wherever possible and eliminate for loops.

Read More - Time Complexity of Sorting Algorithms

Get the tech career you deserve, faster!
Connect with our expert counsellors to understand how to hack your way to success
User rating 4.7/5
1:1 doubt support
95% placement record
Akash Pal
Senior Software Engineer
326% Hike After Job Bootcamp
Himanshu Gusain
Programmer Analyst
32 LPA After Job Bootcamp
After Job
Bootcamp

Frequently Asked Questions

Q1. How can I make my code faster?

Ans. To decrease the time complexity of the code, we can eliminate ‘for loops’ and make use of vectorization using Numpy Functions such as:

  • np.exp(v): To get the exponential value of every element of vector v without looping through every element.
  • np.log(v): To get the logarithmic value of every element.
  • np.abs(v): To get the magnitude of every element of the vector v.

 

Q2. What are the advantages of vectorization?

Ans. The advantage of vectorization is that the code becomes faster and easier to debug.

 

Q3. What are the applications of vectorization?

Ans. We use vectorization to speed up many algebraic operations such as:

  • Dot Product
  • Outer Product
  • Matrix Multiplication
  • Element-wise Product
  • Add or Subtract by a Scalar

Key Takeaways

Congratulations on finishing the blog!! Below, I have some blogs suggestions for you. Go ahead take a look at these articles.

In today’s scenario, more & more industries are adapting to AutoML applications in their products; with this rise, it has become clear that AutoML can be the next boon in the technology. Check this article to learn more about AutoML applications.

Check out this link if you are a Machine Learning enthusiast or want to brush up your knowledge with ML blogs.

If you are preparing for the upcoming Campus Placements, don't worry. Coding Ninjas has your back. Visit this link for a carefully crafted and designed course on-campus placements and interview preparation.

Previous article
Binary Classification
Next article
Residual Analysis
Live masterclass