Introduction
The Restricted Boltzmann machine is an undirected graphical model playing a significant role in deep learning frameworks. It was introduced as a harmonium initially, and it gained massive popularity in recent years in the context of the Netflix price where Restricted Boltzmann machines achieved state-of-the-art performance in collaborative filtering and have beaten most of the competition. Many hidden layers are learned efficiently by composing a restricted Boltzmann machine using the feature activation as the training data for the next. These are the neural networks that belong to the so-called energy-based model. It is an algorithm used for dimensionality reduction, classification, regression, collaborative filtering, feature learning, and topic modelling.
Restricted Boltzmann machines are shallow and have two-layer neural nets that constitute the building blocks of deep learning networks. The first layer of the RBM is the Visible or Input layer, and the second is the Hidden layer. Each circle here represents a neuron-like unit called Node, and nodes are where calculations occur. Nodes connect across layers, but no two nodes of the same layer are linked, which means no Intra-communication, which is the restriction here. Each Node is a point of computation that processes Input and makes a stochastic decision about whether to transmit the Input or not. Each visible Node takes a low-level feature from an item in the data set to be learned. For example, from a data set of grayscale images, each visible Node would receive a one-pixel value for each pixel in one picture.
RBM Probabilistic Model
RBM is probabilistic as opposed to assigning discrete values to model assigns probabilities. At each point in time, RBM is in a specific state. The state refers to the importance of the neuron in the visible and hidden layers V and H. The following joint distribution gives the probability of observing a sure V and H state.
Z is called the partition function, which is the summation of all possible pairs of visible and hidden vectors. So, this is the point where a restricted Boltzmann machine needs physics. The joint distribution is called the Boltzmann distribution.The image data set has a unique probability distribution for its pixel values depending on the kind of images in the set. Pixel values are distributed differently depending on which the data set includes MNIST handwritten numerals.