Introduction to Hopfield Neural Network

The Architecture of Hopfield Network

The architecture of the Hopfield network consists of the following elements:

• Individual nodes preserve their states until required an update.
• The node to be updated is selected randomly. 
• Each node is connected to all other nodes except itself. 
• The state of each node is either 0/1 or 1/-1. 
• The Hopfield network structure is symmetric, i.e., Wij = Wji for all i's and j's.
   

The representation of the sample architecture of the Hopfield network having three nodes is as follows:

In the above diagram, each symbol represents:

x1, x2, x3 - represents the input.

y1, y2, y3 - represents output obtained from each node.

Wij - represents the weight associated with the connection from i to j. 

Energy Function in Hopfield Network

In Hopfield networks, there are two different types of updations. 

• Synchronous: Updating all the nodes simultaneously each time. 
• Asynchronous: Updating only one node at a time. That node is selected randomly or based on specific rules. 
   

In asynchronous updation, each state of Hopfield networks is associated with an energy value. The value is obtained from a function, and that function is named an energy function. This function can decrease or remain unchanged during updation. This energy function of the Hopfield network is defined as:

Where Wij = weight of the connection between i to j

Xi = value of input i

Xj = value of input j
 

A network is considered to be in a stable state if the energy function tends to be minimum. . 

Training Model with Hopfield Network

Training a Hopfield network refers to lowering of energy of each state. The training model consists of a training and testing algorithm. Let's discuss each of them one by one.

Training Algorithm

It is based on the Hebbian principle, which Donald Hebb gave in 1949. In training, algorithm weights are updated using a specific rule. The rule is different for binary and bipolar. For storing a set of 'N' input patterns [ X(n) where n = 1 to N ], the rule followed is as follows:

• Binary Input:

• Bipolar Input:

Testing Algorithm

The testing algorithm involves several steps. These steps are:

• Initialize the weights using the above training algorithm rules. 
• Follow steps 3 to 7 for each input vector 'Xi'. 
• Assign the value of external input vector 'Xi' to initial activators Yi', for all i = 1 to N.
• Follow steps 5 to 7 for each initial activator 'Yi'. 
• Calculate the network's net input 'Yin' using the below rule:

• Based on the calculated net input and threshold value, calculate the output. There are three different conditions for the output. These are:

• Broadcast the obtained output 'Yi' to all other units. And update the activation vector. 
• Test the network. 

Check out this article - Padding In Convolutional Neural Network

Frequently Asked Questions

What is a Hopfield neural network?

The Hopfield network is a particular type of single-layered neuron network. Dr. John J. Hopfield invented it in 1982. These networks were introduced to collect and retrieve memory and store various patterns.

What are the types of Hopfield neural networks?

There are two types of Hopfield networks - Discrete and Continuous. The discrete network is further classified as Binary, and Bipolar networks based on the output received. 

What are the advantages of using Hopfield networks?

These networks are easy to design and implement. These networks are used in various applications like optimization, pattern recognition, etc.

What challenges are faced while using Hopfield networks?

These networks can recognize only one pattern and hence are considered very slow. Also, large patterns cannot be stored in these networks.

Mention some applications where we can use Hopfield networks.

These networks are widely used in research, data compression, image recognition, and optimization. Traveling salesman, knapsack problems, etc, are also some real-life problems in which the Hopfield network is helpful.

Conclusion

This article deeply discussed the definition of Hopfield networks. We also learned about the architecture and energy function of these networks. At last, we discussed how we could train Hopfield neural networks for different operations. 
Recommended Reading: 

Spring Boot Architecture

Artificial Intelligence in Education

Please refer to our guided paths on Coding Ninjas Studio to learn more about DSA, Competitive Programming, JavaScript, System Design, etc. And also, enroll in our courses and refer to the mock test and problems available. Have a look at the interview experiences and interview bundle for placement preparations.

Keep learning, and Keep Growing!

Happy Learning!