Table of contents
1.
Introduction
2.
RBM Probabilistic Model
3.
Training of RBM
4.
Steps Involved In Training
5.
FAQs
6.
Key Takeaways
Last Updated: Mar 27, 2024

Restricted Boltzmann Machine

Author Rajkeshav
0 upvote
Career growth poll
Do you think IIT Guwahati certified course can help you in your career?

Introduction

The Restricted Boltzmann machine is an undirected graphical model playing a significant role in deep learning frameworks. It was introduced as a harmonium initially, and it gained massive popularity in recent years in the context of the Netflix price where Restricted Boltzmann machines achieved state-of-the-art performance in collaborative filtering and have beaten most of the competition. Many hidden layers are learned efficiently by composing a restricted Boltzmann machine using the feature activation as the training data for the next. These are the neural networks that belong to the so-called energy-based model. It is an algorithm used for dimensionality reduction, classification, regression, collaborative filtering, feature learning, and topic modelling. 

source

Restricted Boltzmann machines are shallow and have two-layer neural nets that constitute the building blocks of deep learning networks. The first layer of the RBM is the Visible or Input layer, and the second is the Hidden layer. Each circle here represents a neuron-like unit called Node, and nodes are where calculations occur. Nodes connect across layers, but no two nodes of the same layer are linked, which means no Intra-communication, which is the restriction here. Each Node is a point of computation that processes Input and makes a stochastic decision about whether to transmit the Input or not. Each visible Node takes a low-level feature from an item in the data set to be learned. For example, from a data set of grayscale images, each visible Node would receive a one-pixel value for each pixel in one picture.

RBM Probabilistic Model

source

RBM is probabilistic as opposed to assigning discrete values to model assigns probabilities. At each point in time, RBM is in a specific state. The state refers to the importance of the neuron in the visible and hidden layers V and H. The following joint distribution gives the probability of observing a sure V and H state.  

source

Z is called the partition function, which is the summation of all possible pairs of visible and hidden vectors. So, this is the point where a restricted Boltzmann machine needs physics. The joint distribution is called the Boltzmann distribution.The image data set has a unique probability distribution for its pixel values depending on the kind of images in the set. Pixel values are distributed differently depending on which the data set includes MNIST handwritten numerals.

Training of RBM

source

The training of RBM differs from the training of regular neural networks via stochastic gradient descent. The two main training steps include Gibbs sampling and Contrastive divergence

We use the following equations to predict H's hidden values in Gibbs sampling given an input vector V. 

source

Knowing the hidden values, we will use the following equation to predict new input values v.  

source

This process repeats several times, and after each iteration, we obtain another input that recreates from the original input values. We update the weight Matrix during the contrastive divergence step. Both vectors are used to calculate the activation probabilities for the hidden deals. The difference between the cross products of those probabilities with input vectors results in the updated matrix, which is represented using the following equation. 

source

 Using the updated matrix, we can calculate the new weight with Gradient Accent, given by the following equation.

Source

Steps Involved In Training

Following are the steps involved in training to prediction in RBM.

  • Training the network on the data.
  • Taking the training data of a specific user during the interface time.
  • Using data to obtain the activation of hidden neurons.
  • Using the hidden neuron values to get activations of input neurons.
  • The new values of input neurons so the rating of the user would give.

FAQs

1. What is the Restricted Boltzmann machine in deep learning?

The Restricted Boltzmann machine is an undirected graphical model that plays a significant role in deep learning frameworks. Boltzmann machines achieved state-of-the-art performance in collaborative filtering and have beaten most of the competition.

2. What are the training steps involved in a Restricted Boltzmann machine?

The two main training steps include Gibbs sampling and contrastive divergence. 

3. How Restricted Boltzmann machines differ from Autoencoders?

 The Restricted Boltzmann machine uses a stochastic approach instead of deterministic; it uses stochastic units with a particular distribution. RBMs have two biases that distinguish them from other autoencoders.

4. How many layers are there in a Restricted Boltzmann machine?

 RBM has two layers; the Visible or Input layer and the Hidden layer.

5. Who discovered the Restricted Boltzmann machine?

 Geoffrey Hinton

Key Takeaways

In this blog, We discussed the architecture, the probabilistic model, and the training of the Restricted Boltzmann machine. We also looked at the steps involved in the activity for RBM.

You can also consider our Online Coding Courses such as the Machine Learning Course to give your career an edge over others.

 

Live masterclass