Code360 powered by Coding Ninjas X Naukri.com. Code360 powered by Coding Ninjas X Naukri.com
Last Updated: Mar 27, 2024
Difficulty: Easy

Introduction to TorchScript

Leveraging ChatGPT - GenAI as a Microsoft Data Expert
Speaker
Prerita Agarwal
Data Specialist @
23 Jul, 2024 @ 01:30 PM

Introduction

TorchScript isn't sound like something you need to become more familiar with. Or you are confused between PyTorch and TorchScript.

intro image

Well, you are in the right place; this article will introduce TorchScript and how it works. 

TorchScript

TorchScript is a subset of PythonTorch, used to create serialized and optimized models from the PyTorch library of Python. TorchScript can also be considered as an intermediate representation for a PyTorch model, which can be implemented or run in high-level performance environments like C++. in other words, it can also say that that the TorchScript is a high-performance static set of Python meant for the world of Machine Learning applications and algorithms.

TorchScript also provides tools to gather information related to the Model; the work is being done. It is very light in the case of memory dynamic and nature, which is very dynamic as it is similar to Pytorch. After loading the Model in TorchScript, the dependency of the Model on PyTorch starts decreasing. The method of applying TorchScript is either directly writing using @script annotations in the current Model, or else Python has the feature of automatically generating the code through tracing.

Get the tech career you deserve, faster!
Connect with our expert counsellors to understand how to hack your way to success
User rating 4.7/5
1:1 doubt support
95% placement record
Akash Pal
Senior Software Engineer
326% Hike After Job Bootcamp
Himanshu Gusain
Programmer Analyst
32 LPA After Job Bootcamp
After Job
Bootcamp

Parts of TorchScript

diagram

OPTIMIZATION

TorchScript is a subset of PythonTorch, used to create serialized and optimized models from the PyTorch library of Python. TorchScript can also be considered as an intermediate representation for a PyTorch model, which can be implemented or run in high-level performance environments like C++. in other words, it can also say that that the TorchScript is a high-performance static set of Python meant for the world of Machine Learning applications and algorithms.TorchScript also provides tools to gather information related to the Model; the work is being done. It is very light in the case of memory occupied and nature, which is very dynamic as it is similar to Pytorch. After loading the Model in TorchScript, the dependency of the Model on PyTorch starts decreasing. The method of applying TorchScript is either directly writing using @script annotations in the current Model, or else Python has the feature of automatically generating the code through tracing.

Types of JIT (.jit) 

  

 

JIT TRACE

JIT SCRIPT 

Definition

The work of the tracer is to run the supplied module and record the tensor operations performed, then turn this recording into a TorchScript module.

 

It allows you to write/ allow code directly into the torch script.

Torch extension

torch.jit.trace

torch. Jit. Script

Tensor

may/may not work on tensor 

It does not require tensor 

Drawback 

Omit all the control flows and data structures.

Does not omit control points and Data structures.

Serializable

The transformation of code from ClassType and Function or Functions type into Python source code. The module can be serialized as a TorchScript program, and the loading is done through the torch.jit.load().

torchscript workflow

Advantages of TorchScript

  • TorchScript code has its interpreter in which the code can be invoked and is a restricted python interpreter. The interpreter does not acquire the Interpreterlock, and thus many requests can be processed on the same instance simultaneously.
     
  • The format of TorchScript allows you to save the Model on the disk and transfer or load it into any other language, for example, R and other languages written in R.
     
  • TorchScript gives the representation in which their options for compiler's optimization on code make the code or the running code more effective.
     
  • TorchScript allows an interface between many backend/device runtimes, requiring a broader view of the program than individual operators. 
     
  • The TorchScript programs are created in libraries different than the torch. The R language objects can be directly converted to the touch script counterpart.
  • Serialization is applied to the Model on which torchscript is used in any way to save the Model and then use it later; this is also an advantage of torchscript over other similar components of Python.
     
  • TorchScript reduces the overhead and improves the Model's performance; thus, TorchScript can also be used for optimization.
     
  • TorchScript is used in complex functions and models, generally in neural networks and deep learning models. The GPU run time and the CPU run time of the Model are outperformed by TorchScript.

Comparing PyTorch and TorchScipt with Implementation

Example: BERT(BidirectionalEncoder Representation from Transformers)developed by the researchers and employees of Google AI. Here the BERT is utilized from the transformer's library. 

Step 1: Initialization of the BERT model/tokenizers and sample data for inference creation.

!pip install transformers
Code
from transformers import BertTokenizer, BertModel
import numpy as np
import torch 
from time import perf_counter 

def timer(f,*args): start = perf_counter() f(*args) 
return (1000 * (perf_counter() - start)) 
scriptor = BertTokenizer.from_pretrained('bert-base-uncase,' torchscript=True) 
script_model = BertModel.from_pretrained("bert-base-uncased", torchscript=True)

# Tokenizing input text 
= "[CLS] Who was Jim Henson? [SEP] Jim Henson was a puppeteer [SEP]." 
token_text = script ok. tokenize(text)

# Masking one of the input tokens masked_index = 8 
token_text[masked_index] = '[MASK]' 
indexed_tokens = scriptok.convert_tokens_to_ids(token_text)
 segments_ids = [6, 0, 4, 0, 3, 1, 2, 1, 1, 1, 1, 1, 1] 
# Creating a dummy input 
tokens_tensor = torch.tensor([indexed_tokens]) segments_tensors = torch.tensor([segments_ids])

Output:

output

Step 2: Creating the PyTorch models for inference on CPU/GPU

environment setup 

!python3 "/content/drive/My Drive/app/mnist_cnn.py."
import TensorFlow as tf
 tf. Test.gpu_device_name()


Output

output

Code

parser = argparse.ArgumentParser(description='Process some integers.') 
n_model = BertModel.from_pretrained("Bert-base-uncased") 
np. mean([timer(n_model,tokens_tensor,segments_tensors) 
for _ in range(1000)]) native_gpu = n_model.Cuda() 
tokens_tensor_gpu = tokens_tensor.Cuda() 
segments_tensors_gpu = segments_tensors.Cuda()
 Np. Mean ([timer(native_ GPU,tokens_tensor_gpu,segments_tensors_gpu) 
for _ in range(1000)])


Step 3: The Torch Script model for interference on CPU/GPU

pip install torchvision
 import torch import torchvision
traced_model = torch.jit.trace(script_model, [tokens_tensor, segments_tensors]) 
np. mean([timer(traced_model,tokens_tensor,segments_tensors) for _ in range(1000)])
 traced_model_gpu = torch.jit.trace(script_model.cuda(), [tokens_tensor.cuda(), segments_tensors.cuda()])
 np. mean([timer(traced_model_gpu,tokens_tensor.Cuda(),segments_tensors.Cuda( )) for _ in range(1000)])


Final Output

  Latency on GPU(MS) Latency on CPU 
PyTorch 17.89 82.11
TorchScript 14.67 85.33

Frequently Asked Questions 

Why is TorchScript used over PyTorch?

TorchScript is portable and more flexible than PyTorch, and It is also lightweight compared to PyTorch. 

Mention the Key Features of TorchScript.

The main reason behind using TorchScript is that it is optimized using .jit and has serializability properties.

Who or which organization owns TorchScript?

Google organization owns TorcchScript along with PyTorch and TensorFlow.

Is TorchScript efficient for visualization?

No, TensorFlow is preferred over TorchScript for visualization.

Are both PyTorch and TorchScript the same?

TorchScript is the extension of PyTorch, and it is much fast and more lightweight than PyTorch uses .jit to compile and run a model.

Conclusion 

This article contained a brief introduction, working, and implementation of TorchScript and Also covered both comparisons between PyTorch and TorchScript using jit methods.

Refer to our guided paths on Coding Ninjas Studio to learn more about DSA, Competitive Programming, JavaScript, System Design, etc. Enroll in our courses and refer to the mock test and problems available; take a look at the interview experiences and interview bundle for placement preparations.

Do upvote our blog to help other ninjas grow.

Merry Learning!

Topics covered
1.
Introduction
2.
TorchScript
3.
Parts of TorchScript
3.1.
OPTIMIZATION
3.1.1.
Types of JIT (.jit) 
3.2.
Serializable
4.
Advantages of TorchScript
5.
Comparing PyTorch and TorchScipt with Implementation
6.
Frequently Asked Questions 
6.1.
Why is TorchScript used over PyTorch?
6.2.
Mention the Key Features of TorchScript.
6.3.
Who or which organization owns TorchScript?
6.4.
Is TorchScript efficient for visualization?
6.5.
Are both PyTorch and TorchScript the same?
7.
Conclusion