Table of contents
1.
Introduction
2.
What is Transfer Learning?
2.1.
Traditional vs Transfer Learning
2.2.
Key Steps to Transfer learning 
3.
Different types of transfer learning
3.1.
Transfer Learning on the basis of Algorithms involved 
3.2.
Transfer learning on the basis of knowledge being imparted
4.
Applications of Transfer learning 
5.
Frequently Asked Questions
6.
Key takeaways
Last Updated: Mar 27, 2024

Transfer Learning-Introduction

Author aniket verma
0 upvote
Career growth poll
Do you think IIT Guwahati certified course can help you in your career?

Introduction

Imagine if we didn’t transfer the learnings from one generation to another then the world couldn’t have made significant progress. Thus it’s a very common practice which has always brought improvement. Similarly, Deep Learning also works on the same principle and it tries to mimic the human brain. It uses the concept of Transfer Learning due to which we get such overwhelming results generally. Let’s explore more about Transfer Learning.

Source: Link

What is Transfer Learning?

Transfer learning is an important part of Deep Learning. It refers to re-usage of the pre-computed knowledge to improve the predictions. It aims to transfer knowledge to closely linked problems. Let’s consider the image classification example. What we do is simply feed the image dataset to a CNN which uses several convolution layers to process the images and extract features and then classify the images. 

In forming each convolution layer we transfer the previous layer features and process them to form the new layer and finally it’s fed to a Neural Network. Then the internal computation of Neural Networks also use transfer learning where weights are transferred from one layer to another to compute and update the weights.

Traditional vs Transfer Learning

The traditional learning mechanisms in machine learning included isolated learning and training where prior knowledge was not taken into account and is not retained.  

Source : link

In case of Transfer Learning, the prior computed knowledge like features, weights, etc, can be transferred from one layer to another which works well for even smaller amounts of datasets.

Key Steps to Transfer learning 

There are three important steps to perform transfer learning and they are as follows:

What to Transfer

This is the very first step and most important question to be answered while performing transfer learning. In this step all the important information and knowledge that can be transferred has to be identified by looking at the common information required by source and target.

When to Transfer

It’s important to look at the effect of the knowledge being transferred. It should have a positive impact on the results. Thus it’s important to recognize at what point it’s necessary to transfer knowledge.

How to Transfer

The third and the last step in the transfer learning process is to figure out how the transfer of knowledge will occur. This can be figured out by usage of existing algorithms or modifying them for our purpose.

Different types of transfer learning

There are various types of transfer learning which can be classified into two categories:

  1. Transfer learning on the basis of Algorithms involved
  2. Transfer learning on the basis of knowledge being imparted

 

Transfer Learning on the basis of Algorithms involved 

  1. Inductive Transfer learning 
    In this type of transfer learning, the domains of both source and target are same yet perform different tasks and utilise the source bias to improve the results of the target task. Some of the examples of the tasks are Classification, regression, etc.
     
  2. Unsupervised Transfer learning
    This type of transfer learning is similar to inductive transfer learning but difference is that the target domain deals with unsupervised tasks. The source and target have the same domains but their tasks are different from each other. Some of the examples of the tasks are Classification, dimensionality reduction, etc.
     
  3. Transductive Transfer learning 
    In this type of transfer learning, the domains of both source and target have some similarities and source domains have a lot of labelled data but the target tasks have unlabeled data. The tasks of both source and target are the same. Some of the examples of the tasks are Classification, regression, etc.

Transfer learning on the basis of knowledge being imparted

  1. Instance Transfer
    It occurs in inductive transfer learning that is used in algorithms like AdaBoost. Mostly the data instances in the source domain can’t be used directly. There are a very few instances when source domain data is used to improve the target tasks. 
     
  2. Feature Transfer
    This category deals with minimising the errors and achieving faster convergence. Hence feature transfer plays a very important role for which suitable supervised, or unsupervised methods are used.
     
  3. Parameter Transfer
    This category works on the assumption of models having the same parameters or having prior distribution of hyperparameters. While transferring of parameters takes place it’s important to introduce additional weightage to loss functions to improve the performance. 
     
  4. Relational Knowledge transfer  
    Mostly all algorithms are formulated on the basic assumption that the parameters are independent and are identically distributed. But it need not be the case every time. Therefore this category of transfer learning is utilised by relational knowledge transfer techniques. 

Applications of Transfer learning 

  • Image classification
    Image classification is a very common application of transfer learning which is generally used to make classifications in computer vision. 
     
  • Simulation learning 
    Simulation learning is another common usage of transfer learning in the gaming industry where the model trained for simulation games can be transferred to another simulation game of the similar type.
     
  • Healthcare
    Transfer learning is used a lot in Healthcare. Now the models trained for the MRI scans can also be used to analyse the CT scans. 

Frequently Asked Questions

  1. What are the various domains in which Transfer Learning is used?
    Transfer Learning is used in a large number of domains including NLP, Image Captioning, Dealing with Audio Data, Automobile Industry, etc. 
     
  2. What are the challenges faced while using Transfer learning?
    The common challenges faced while using transfer learning are negative transfer and unquantized bounds of transfer.

Key takeaways

This article gave a brief introduction to Transfer Learning. We saw what Transfer Learning is and how it works. We explored how it’s different from traditional learning and key steps to perform transfer learning. We then looked at different types of Transfer learning and its applications. To dive deeper into machine learning, check out our industry-level courses on coding ninjas.

Live masterclass