Code360 powered by Coding Ninjas X Naukri.com. Code360 powered by Coding Ninjas X Naukri.com
Table of contents
1.
Introduction
2.
What is PyTorch?
3.
Deep Autoencoder
4.
Implement Deep Autoencoder in PyTorch
4.1.
Step 1: Setting Up the Environment
4.2.
Step 2: Dataset Selection and Preprocessing
4.3.
Step 3: Building the Deep Autoencoder
4.4.
Step 4: Loss Function and Optimization
4.5.
Step 5: Training the Model
4.6.
Step 6: Evaluating Model Performance
4.7.
Output
5.
Frequently Asked Questions
5.1.
What is the Boltzmann Machine?
5.2.
What is the Sigmoid Function?
5.3.
What is Swish?
6.
Conclusion
Last Updated: Mar 27, 2024
Medium

Implement Deep Autoencoder in PyTorch

Leveraging ChatGPT - GenAI as a Microsoft Data Expert
Speaker
Prerita Agarwal
Data Specialist @
23 Jul, 2024 @ 01:30 PM

Introduction

An image is a picture that is formed by actual existence. The image is the main means for people to obtain information from the outside world—the key work of image recognition. 

This process involves delving into the representation of input images, allowing the machine to enhance its comprehension independently. The objective is to enable the machine to recognize and classify these images autonomously. The procedure encompasses external input samples subjected to various devices and filters, isolating pivotal features within the input images. 

Implement Deep Autoencoder in PyTorch

This comprehensive approach ultimately facilitates the accomplishment of image recognition and classification tasks. Image recognition involves a wide range of research content, such as license plate recognition, handwritten digit recognition, face recognition, and recognition and classification of parts in machining and accurate weather forecasting based on meteorological satellite photos.

In this article, we learn about implementing Deep Autoencoder in PyTorch.

What is PyTorch?

What is PyTorch?

It is a schematic diagram of Dropout. When training the Dropout method, the output value is calculated by the activation function of the hidden layer node. It is randomly cleared during the forward transmission. Pytorch is the input connected to the hidden layer neurons. 

The weights are randomly removed with a certain probability. It is a schematic diagram of a Pytorch. In other words, the fully connected six layers become sparsely connected. Using the Pytorch method, its connections are randomly selected during the training phase.

It should be noted that this does not mean that the weight matrix. It is set to a fixed sparse matrix. During the training process because, during each training. The weights are randomly cleared and not settled each time.

Get the tech career you deserve, faster!
Connect with our expert counsellors to understand how to hack your way to success
User rating 4.7/5
1:1 doubt support
95% placement record
Akash Pal
Senior Software Engineer
326% Hike After Job Bootcamp
Himanshu Gusain
Programmer Analyst
32 LPA After Job Bootcamp
After Job
Bootcamp

Deep Autoencoder

Autoencoders usually use a stacked structure in practical applications. Applying stacked autoencoders eliminates the huge workload of manually extracting data features. It improves the efficiency of feature extraction. It also exhibits powerful learning capabilities. 

Our model employs a 2-layer AE. The idea of Pytorch is introduced in the first layer of AE. Pytorch is a new regularization method proposed by Wan et al

After introducing the Pytorch idea, the input and hidden layers are no longer connected. But they are randomly connected so that the dynamic sparseness of the network can be achieved.

Implement Deep Autoencoder in PyTorch

Step 1: Setting Up the Environment

Install the required packages using the following command. Make sure you have Python and PyTorch installed on your system.

pip install torch numpy matplotlib

Step 2: Dataset Selection and Preprocessing

We'll make use of the torch-vision library to load the dataset.

import torch
from torchvision import datasets
from torchvision import transforms
import matplotlib.pyplot as plt
import torch.nn as nn
import torchvision
import torchvision.transforms as transforms
import matplotlib.pyplot as plt
tensor_transform = transforms.ToTensor()
 
dataset = datasets.MNIST(root = "./data", train = True,
   download = True, transform = tensor_transform)
 
loader = torch.utils.data.DataLoader(dataset = dataset, batch_size = 64,
   shuffle = True)
Step 3: Building the Deep Autoencoder

Step 3: Building the Deep Autoencoder

We are using PyTorch's nn. Module class, we'll build an encoder and a decoder. Our deep autoencoder's encoder and decoder parts will each have several fully connected layers. The architecture can be defined as follows:

class Autoenc(torch.nn.Module):
   def __init__(self):
      super().__init__()

      self.encoder = torch.nn.Sequential(
         torch.nn.Linear(28 * 28, 128),
         torch.nn.ReLU(),
         torch.nn.Linear(128, 64),
         torch.nn.ReLU(),
         torch.nn.Linear(64, 36),
         torch.nn.ReLU(),
         torch.nn.Linear(36, 18),
         torch.nn.ReLU(),
         torch.nn.Linear(18, 9)
      )

      self.decoder = torch.nn.Sequential(
         torch.nn.Linear(9, 18),
         torch.nn.ReLU(),
         torch.nn.Linear(18, 36),
         torch.nn.ReLU(),
         torch.nn.Linear(36, 64),
         torch.nn.ReLU(),
         torch.nn.Linear(64, 128),
         torch.nn.ReLU(),
         torch.nn.Linear(128, 28 * 28),
         torch.nn.Sigmoid()
      )
   def forward(self, x):
      encoded = self.encoder(x)
      decoded = self.decoder(encoded)
      return decoded

Step 4: Loss Function and Optimization

We use the Mean Squared Error loss function to determine how much the input and reconstructed data differ. We'll train our model using the Adam optimizer:

model = Autoenc()
loss_function = torch.nn.MSELoss()
optimizer = torch.optim.Adam(model.parameters(), lr = 1e-1, weight_decay = 1e8)

Step 5: Training the Model

Assessing how well our deep autoencoder has learned to reconstruct the input data after training is critical. To evaluate the accuracy of the reconstruction, we can compare the original and reconstructed images:

# Hyperparameters
epochs = 10
batch_size = 128
learning_rate = 0.001


# Lists to store losses and outputs
losses = []
outputs = []

# Training loop
for epoch in range(epochs):
    for (image, _) in loader:
        image = image.view(-1, 28 * 28)

        reconstructed = model(image)
        
        loss = loss_function(reconstructed, image)
        
        optimizer.zero_grad()
        loss.backward()
        optimizer.step()

        losses.append(loss.item())  # Append loss value to the list

    outputs.append((epoch, image, reconstructed))

Step 6: Evaluating Model Performance

Beyond image reconstruction, autoencoders can be used for a variety of purposes.

# Plotting the loss
plt.style.use('fivethirtyeight')
plt.xlabel('Iterations')
plt.ylabel('Loss')
plt.plot(losses[-100:])
plt.show()

# ...

# Displaying original and reconstructed images
with torch.no_grad():
    for epoch, image, reconstructed in outputs:
        plt.figure(figsize=(9, 2))
        for i in range(5):
            plt.subplot(2, 5, i + 1)
            plt.imshow(image[i].view(28, 28).detach().numpy(), cmap='gray')
            plt.subplot(2, 5, i + 6)
            plt.imshow(reconstructed[i].view(28, 28).detach().numpy(), cmap='gray')
        plt.show()
        break 

Output

Output
Output

Frequently Asked Questions

What is the Boltzmann Machine?

A network of symmetrically connected, neuronlike units that make stochastic decisions. Boltzmann machines have a learning algorithm that allows them to discover interesting features in datasets composed of binary vectors.

What is the Sigmoid Function?

A sigmoid function also called a logistic function, is an “S”-shaped continuous function with domain over all R. However, the range is only over (0, 1).

What is Swish?

Swish is unbounded above and bounded below. Unlike ReLU, Swish is smooth and nonmonotonic. The non-monotonicity property of Swish distinguishes itself from the most common activation functions.

Conclusion

In this article, we learn about implementing Deep Autoencoder in PyTorch. We also learn about Deep Autoencoder and also PyTorch. We concluded the article by discussing the definition and implementation.

To better understand the topic, refer to 

For more information, refer to our Guided Path on CodeStudio to upskill yourself in PythonData Structures and AlgorithmsCompetitive ProgrammingSystem Design, and many more! 

Head over to our practice platform, CodeStudio, to practice top problems, attempt mock tests, read interview experiences and interview bundles, follow guided paths for placement preparations, and much more!
Happy Learning!

Live masterclass