Deep learning is a sub-category of machine learning which mirrors the functioning of our brains. Deep learning algorithms are similar to how the nervous system is structured, where each neuron is connected with the other and passes information. The Boltzmann Machine is a powerful generative model that uses a deep architecture, and in this article, we will learn all the types of Boltzmann Machine.
The Boltzmann Machine works on symmetric networks and decides whether the network will be on or off position by the random probability distribution. It belongs to the family of unsupervised learning; therefore, the model is solely responsible for any decision-making when some input features are provided.
To have a deeper understanding of Boltzmann Machine visit, this article.
Types of Boltzmann machine
There are three types of Boltzmann machine
Restricted Boltzmann machine
Deep Boltzmann machine
Deep Belief machine
Restricted Boltzmann Machine
The Restricted Boltzmann Machine is one of the types of Boltzmann machine. It is a bi-layered artificial network. They can learn a probability distribution over a set of inputs. As the name suggests, the Restricted Boltzmann machine restricts a few connections. We know that the Boltzmann machine has two layers, the hidden layer and the visible layer, which are connected with each other.
But the Restricted Boltzmann machine network is bipartite, which means that every node in the hidden layer is connected to the visible layer. Still, no two nodes of the visible layer are connected. This restriction provides a more efficient model.
As no two neurons of the same layer are connected, therefore each neuron has some random behaviour when activated. In RBMs, there are two additional layers of bias; hidden bias and visible bias. The hidden bias is responsible for producing the activation on the forward pass. The visible bias helps the machine rebuild the input during a backward pass or the reconstruction phase.
The image shows the initial step in training a Restricted Boltzmann Machine with numerous inputs. The product of inputs and weights are added to the bias. The final output is then passed through a sigmoid activation function, and the outcome determines if the hidden state gets activated or not. The sigmoid activation function here is:
Later comes the reconstruction phase. It is pretty similar to the initial step but only varies the direction.
It is one of the types of Boltzmann machines. Deep Boltzmann Machine is a generative model with an undirected connection between the layers. It contains a single visible layer and multiple layers of the hidden layer. Unlike the Restricted Boltzmann Machine, the layers are not interconnected; instead, they are intraconnected. The neuron in the deep Boltzmann machine is independent of each other, although they are dependent on the neighboring layer. The machine is trained layer-by-layer to make the model efficient.
The Deep Boltzmann Machine works by greedy layer-by-layer training. It depends on learning stacks of restricted Boltzmann machines. In layer-wise training, we increase the input twice for lower-level and top-level Restricted Boltzmann Machine. The lower level Restricted Boltzmann Machine inputs are doubled to compensate for the deficiency of top-down information into the first hidden layer. Likewise, we double the hidden units for the top-level Restricted Boltzmann Machine to pay for the lack of bottom-up input.
The Deep Belief Network is one of the types of Boltzmann machines. It is a generative model which uses multiple stacks of the deep architecture of the Restricted Boltzmann Machine. Each restricted Boltzmann machine performs a non-linear transformation on the input neurons and produces the outputs that serve as the input for the consecutive model. Deep Belief Networks can act as supervised or unsupervised as they have a generative model. Due to it, there is a lot of flexibility in Deep Belief Networks, and it is easier to expand.
Deep Belief Networks are pre-trained using a greedy algorithm like all other networks. The algorithm applies a layer-by-layer approach to learn the top-down weights, and these weights determine the dependencies between each layer. In this network, numerous steps of Gibbs sampling run on the top two hidden layers. This step essentially creates a sample from the Restricted Boltzmann Machine defined by the top two hidden layers. Later, a single pass of the previous sampling is used to find a piece from the visible layer throughout the rest of the model.
What are the uses of the Restricted Boltzmann Machine?
Restricted Boltzmann Machines are used for dimensionality reduction, classification, collaborative filtering, feature learning, topic modeling, and even many-body quantum mechanics.
What are the uses of the Deep Boltzmann machine?
The deep Boltzmann machine is used for efficient Deep Learning Approaches in Health Informatics.
What are the uses of the Deep Belief Network?
The Deep Belief Network is used to recognize, cluster, and generate images, video sequences, and motion-capture data.
Conclusion
Boltzmann Machines comes under unsupervised learning and works on a systematic network. The article gives a brief description of the types of Boltzmann Machines and the working of each of the types.
305+ registered