Introduction to TorchScript

Parts of TorchScript

OPTIMIZATION

TorchScript is a subset of PythonTorch, used to create serialized and optimized models from the PyTorch library of Python. TorchScript can also be considered as an intermediate representation for a PyTorch model, which can be implemented or run in high-level performance environments like C++. in other words, it can also say that that the TorchScript is a high-performance static set of Python meant for the world of Machine Learning applications and algorithms.TorchScript also provides tools to gather information related to the Model; the work is being done. It is very light in the case of memory occupied and nature, which is very dynamic as it is similar to Pytorch. After loading the Model in TorchScript, the dependency of the Model on PyTorch starts decreasing. The method of applying TorchScript is either directly writing using @script annotations in the current Model, or else Python has the feature of automatically generating the code through tracing.

Types of JIT (.jit) 

Serializable

The transformation of code from ClassType and Function or Functions type into Python source code. The module can be serialized as a TorchScript program, and the loading is done through the torch.jit.load().

Advantages of TorchScript

• TorchScript code has its interpreter in which the code can be invoked and is a restricted python interpreter. The interpreter does not acquire the Interpreterlock, and thus many requests can be processed on the same instance simultaneously.
   
• The format of TorchScript allows you to save the Model on the disk and transfer or load it into any other language, for example, R and other languages written in R.
   
• TorchScript gives the representation in which their options for compiler's optimization on code make the code or the running code more effective.
   
• TorchScript allows an interface between many backend/device runtimes, requiring a broader view of the program than individual operators. 
   
• The TorchScript programs are created in libraries different than the torch. The R language objects can be directly converted to the touch script counterpart.
• Serialization is applied to the Model on which torchscript is used in any way to save the Model and then use it later; this is also an advantage of torchscript over other similar components of Python.
   
• TorchScript reduces the overhead and improves the Model's performance; thus, TorchScript can also be used for optimization.
   
• TorchScript is used in complex functions and models, generally in neural networks and deep learning models. The GPU run time and the CPU run time of the Model are outperformed by TorchScript.

Comparing PyTorch and TorchScipt with Implementation

Example: BERT(BidirectionalEncoder Representation from Transformers)developed by the researchers and employees of Google AI. Here the BERT is utilized from the transformer's library. 

Step 1: Initialization of the BERT model/tokenizers and sample data for inference creation.

!pip install transformers

Code
from transformers import BertTokenizer, BertModel
import numpy as np
import torch 
from time import perf_counter 

def timer(f,*args): start = perf_counter() f(*args) 
return (1000 * (perf_counter() - start)) 
scriptor = BertTokenizer.from_pretrained('bert-base-uncase,' torchscript=True) 
script_model = BertModel.from_pretrained("bert-base-uncased", torchscript=True)

# Tokenizing input text 
= "[CLS] Who was Jim Henson? [SEP] Jim Henson was a puppeteer [SEP]." 
token_text = script ok. tokenize(text)

# Masking one of the input tokens masked_index = 8 
token_text[masked_index] = '[MASK]' 
indexed_tokens = scriptok.convert_tokens_to_ids(token_text)
 segments_ids = [6, 0, 4, 0, 3, 1, 2, 1, 1, 1, 1, 1, 1] 
# Creating a dummy input 
tokens_tensor = torch.tensor([indexed_tokens]) segments_tensors = torch.tensor([segments_ids])

You can also try this code with Online Python Compiler

Run Code

Output:

Step 2: Creating the PyTorch models for inference on CPU/GPU

environment setup 

!python3 "/content/drive/My Drive/app/mnist_cnn.py."
import TensorFlow as tf
 tf. Test.gpu_device_name()

You can also try this code with Online Python Compiler

Run Code

Output

Code

parser = argparse.ArgumentParser(description='Process some integers.') 
n_model = BertModel.from_pretrained("Bert-base-uncased") 
np. mean([timer(n_model,tokens_tensor,segments_tensors) 
for _ in range(1000)]) native_gpu = n_model.Cuda() 
tokens_tensor_gpu = tokens_tensor.Cuda() 
segments_tensors_gpu = segments_tensors.Cuda()
 Np. Mean ([timer(native_ GPU,tokens_tensor_gpu,segments_tensors_gpu) 
for _ in range(1000)])

You can also try this code with Online Python Compiler

Run Code

Step 3: The Torch Script model for interference on CPU/GPU

pip install torchvision
 import torch import torchvision
traced_model = torch.jit.trace(script_model, [tokens_tensor, segments_tensors]) 
np. mean([timer(traced_model,tokens_tensor,segments_tensors) for _ in range(1000)])
 traced_model_gpu = torch.jit.trace(script_model.cuda(), [tokens_tensor.cuda(), segments_tensors.cuda()])
 np. mean([timer(traced_model_gpu,tokens_tensor.Cuda(),segments_tensors.Cuda( )) for _ in range(1000)])

You can also try this code with Online Python Compiler

Run Code

Final Output

Frequently Asked Questions 

Why is TorchScript used over PyTorch?

TorchScript is portable and more flexible than PyTorch, and It is also lightweight compared to PyTorch. 

Mention the Key Features of TorchScript.

The main reason behind using TorchScript is that it is optimized using .jit and has serializability properties.

Who or which organization owns TorchScript?

Google organization owns TorcchScript along with PyTorch and TensorFlow.

Is TorchScript efficient for visualization?

No, TensorFlow is preferred over TorchScript for visualization.

Are both PyTorch and TorchScript the same?

TorchScript is the extension of PyTorch, and it is much fast and more lightweight than PyTorch uses .jit to compile and run a model.

Conclusion 

This article contained a brief introduction, working, and implementation of TorchScript and Also covered both comparisons between PyTorch and TorchScript using jit methods.

Refer to our guided paths on Coding Ninjas Studio to learn more about DSA, Competitive Programming, JavaScript, System Design, etc. Enroll in our courses and refer to the mock test and problems available; take a look at the interview experiences and interview bundle for placement preparations.

Do upvote our blog to help other ninjas grow.

Merry Learning!