Code360 powered by Coding Ninjas X Naukri.com. Code360 powered by Coding Ninjas X Naukri.com
Table of contents
1.
Introduction
2.
Introduction to Hopfield Network
2.1.
Discrete Networks
2.2.
Continuous Networks
3.
The Architecture of Hopfield Network
4.
Energy Function in Hopfield Network
5.
Training Model with Hopfield Network
5.1.
Training Algorithm
5.2.
Testing Algorithm
6.
Frequently Asked Questions
6.1.
What is a Hopfield neural network?
6.2.
What are the types of Hopfield neural networks?
6.3.
What are the advantages of using Hopfield networks?
6.4.
What challenges are faced while using Hopfield networks?
6.5.
Mention some applications where we can use Hopfield networks.
7.
Conclusion
Last Updated: Mar 27, 2024

Introduction to Hopfield Neural Network

Author Ayushi Goyal
0 upvote
Master Python: Predicting weather forecasts
Speaker
Ashwin Goyal
Product Manager @

Introduction

During your daily schedule, various waves of networks propagate through your brain. The entire day is formed from the dynamic behavior of these networks. Neural Networks are Artificial Intelligence methods that teach computers to process the data like our brains. These networks use interconnected nodes that resemble neurons of the human brain. This method enables computers to make intelligent decisions by learning from their mistakes. The Hopfield network is one such type of recurrent neural network method. 

Introduction

The sequence of the article will be as follows:

We will begin with an introduction to the Hopfield neural network. Then, we will also discuss the architecture, energy function, and training model of the Hopfield network.  

Also Read, Resnet 50 Architecture and What are payloads

Introduction to Hopfield Network

A Hopfield network is a particular type of single-layered neuron network. Dr. John J. Hopfield invented it in 1982. These networks were introduced to collect and retrieve memory and store various patterns. Also, auto-association and optimization of the task can be done using these networks. In this network, each node is fully connected(recurrent) to other nodes. These nodes exist only in two states: ON (1) or OFF (0). These states can be restored based on the input received from other nodes. Unlike other neural networks, the output of the Hopfield network is finite. Also, the input and output sizes must be the same in these networks. .  

Hopfield networks

The Hopfield network consists of associative memory. This memory allows the system to retrieve the memory using an incomplete portion. The network can restore the closest pattern using the data captured in associative memory. This feature of Hopfield networks makes it a good candidate for pattern recognition. 

Associative memory is a content addressable memory that establishes a relation between the input vector and the output target vector. It enables the reallocation of data stored in the memory based on its similarity with the input vector. 

types of hopfield networks

The Hopfield networks are categorized into two categories. These are:

Discrete Networks

These networks give any of the two discrete outputs. Based on the output received, further two types:

  1. Binary: In this type, the output is either 0 or 1. 
  2. Bipolar: In bipolar networks, the output is either -1 (When output < 0) or 1 (When output > 0)
states of bipolar network

Continuous Networks

Instead of receiving binary or bipolar output, the output value lies between 0 and 1. 

Get the tech career you deserve, faster!
Connect with our expert counsellors to understand how to hack your way to success
User rating 4.7/5
1:1 doubt support
95% placement record
Akash Pal
Senior Software Engineer
326% Hike After Job Bootcamp
Himanshu Gusain
Programmer Analyst
32 LPA After Job Bootcamp
After Job
Bootcamp

The Architecture of Hopfield Network

The architecture of the Hopfield network consists of the following elements:

  • Individual nodes preserve their states until required an update.
  • The node to be updated is selected randomly. 
  • Each node is connected to all other nodes except itself. 
  • The state of each node is either 0/1 or 1/-1. 
  • The Hopfield network structure is symmetric, i.e., Wij = Wji for all i's and j's.
     

The representation of the sample architecture of the Hopfield network having three nodes is as follows:

Network structure

In the above diagram, each symbol represents:

x1, x2, x3 - represents the input.

y1, y2, y3 - represents output obtained from each node.

Wij - represents the weight associated with the connection from i to j. 

Energy Function in Hopfield Network

In Hopfield networks, there are two different types of updations. 

  • Synchronous: Updating all the nodes simultaneously each time. 
  • Asynchronous: Updating only one node at a time. That node is selected randomly or based on specific rules. 
     

In asynchronous updation, each state of Hopfield networks is associated with an energy value. The value is obtained from a function, and that function is named an energy function. This function can decrease or remain unchanged during updation. This energy function of the Hopfield network is defined as:

energy function

Where Wij = weight of the connection between i to j

Xi = value of input i

Xj = value of input j
 

A network is considered to be in a stable state if the energy function tends to be minimum. . 

stable state condition

Training Model with Hopfield Network

Training a Hopfield network refers to lowering of energy of each state. The training model consists of a training and testing algorithm. Let's discuss each of them one by one.

Training Algorithm

It is based on the Hebbian principle, which Donald Hebb gave in 1949. In training, algorithm weights are updated using a specific rule. The rule is different for binary and bipolar. For storing a set of 'N' input patterns [ X(n) where n = 1 to N ], the rule followed is as follows:

  • Binary Input:
Binary weight
  • Bipolar Input:
Bipolar weight

Testing Algorithm

The testing algorithm involves several steps. These steps are:

  • Initialize the weights using the above training algorithm rules. 
  • Follow steps 3 to 7 for each input vector 'Xi'. 
  • Assign the value of external input vector 'Xi' to initial activators Yi', for all i = 1 to N.
  • Follow steps 5 to 7 for each initial activator 'Yi'. 
  • Calculate the network's net input 'Yin' using the below rule:
Net input
  • Based on the calculated net input and threshold value, calculate the output. There are three different conditions for the output. These are:
conditions for output
  • Broadcast the obtained output 'Yi' to all other units. And update the activation vector. 
  • Test the network. 

Check out this article - Padding In Convolutional Neural Network

Frequently Asked Questions

What is a Hopfield neural network?

The Hopfield network is a particular type of single-layered neuron network. Dr. John J. Hopfield invented it in 1982. These networks were introduced to collect and retrieve memory and store various patterns.

What are the types of Hopfield neural networks?

There are two types of Hopfield networks - Discrete and Continuous. The discrete network is further classified as Binary, and Bipolar networks based on the output received. 

What are the advantages of using Hopfield networks?

These networks are easy to design and implement. These networks are used in various applications like optimization, pattern recognition, etc.

What challenges are faced while using Hopfield networks?

These networks can recognize only one pattern and hence are considered very slow. Also, large patterns cannot be stored in these networks.

Mention some applications where we can use Hopfield networks.

These networks are widely used in research, data compression, image recognition, and optimization. Traveling salesman, knapsack problems, etc, are also some real-life problems in which the Hopfield network is helpful.

Conclusion

This article deeply discussed the definition of Hopfield networks. We also learned about the architecture and energy function of these networks. At last, we discussed how we could train Hopfield neural networks for different operations. 
Recommended Reading: 

Spring Boot Architecture

Artificial Intelligence in Education

Please refer to our guided paths on Coding Ninjas Studio to learn more about DSA, Competitive Programming, JavaScript, System Design, etc. And also, enroll in our courses and refer to the mock test and problems available. Have a look at the interview experiences and interview bundle for placement preparations.

Keep learning, and Keep Growing!

Happy Learning!

Previous article
Quantization and Pruning
Next article
Applying CNN on MNIST Dataset
Live masterclass