1.
Introduction
2.
Introduction to Hopfield Network
2.1.
Discrete Networks
2.2.
Continuous Networks
3.
The Architecture of Hopfield Network
4.
Energy Function in Hopfield Network
5.
Training Model with Hopfield Network
5.1.
Training Algorithm
5.2.
Testing Algorithm
6.
6.1.
What is a Hopfield neural network?
6.2.
What are the types of Hopfield neural networks?
6.3.
What are the advantages of using Hopfield networks?
6.4.
What challenges are faced while using Hopfield networks?
6.5.
Mention some applications where we can use Hopfield networks.
7.
Conclusion
Last Updated: Mar 27, 2024

# Introduction to Hopfield Neural Network

Ayushi Goyal
0 upvote
Master Python: Predicting weather forecasts
Speaker
Ashwin Goyal
Product Manager @

## Introduction

During your daily schedule, various waves of networks propagate through your brain. The entire day is formed from the dynamic behavior of these networks. Neural Networks are Artificial Intelligence methods that teach computers to process the data like our brains. These networks use interconnected nodes that resemble neurons of the human brain. This method enables computers to make intelligent decisions by learning from their mistakes. The Hopfield network is one such type of recurrent neural network method.

The sequence of the article will be as follows:

We will begin with an introduction to the Hopfield neural network. Then, we will also discuss the architecture, energy function, and training model of the Hopfield network.

## Introduction to Hopfield Network

A Hopfield network is a particular type of single-layered neuron network. Dr. John J. Hopfield invented it in 1982. These networks were introduced to collect and retrieve memory and store various patterns. Also, auto-association and optimization of the task can be done using these networks. In this network, each node is fully connected(recurrent) to other nodes. These nodes exist only in two states: ON (1) or OFF (0). These states can be restored based on the input received from other nodes. Unlike other neural networks, the output of the Hopfield network is finite. Also, the input and output sizes must be the same in these networks. .

The Hopfield network consists of associative memory. This memory allows the system to retrieve the memory using an incomplete portion. The network can restore the closest pattern using the data captured in associative memory. This feature of Hopfield networks makes it a good candidate for pattern recognition.

Associative memory is a content addressable memory that establishes a relation between the input vector and the output target vector. It enables the reallocation of data stored in the memory based on its similarity with the input vector.

The Hopfield networks are categorized into two categories. These are:

### Discrete Networks

These networks give any of the two discrete outputs. Based on the output received, further two types:

1. Binary: In this type, the output is either 0 or 1.
2. Bipolar: In bipolar networks, the output is either -1 (When output < 0) or 1 (When output > 0)

### Continuous Networks

Instead of receiving binary or bipolar output, the output value lies between 0 and 1.

Get the tech career you deserve, faster!
Connect with our expert counsellors to understand how to hack your way to success
User rating 4.7/5
1:1 doubt support
95% placement record
Akash Pal
Senior Software Engineer
326% Hike After Job Bootcamp
Himanshu Gusain
Programmer Analyst
32 LPA After Job Bootcamp
After Job
Bootcamp

## The Architecture of Hopfield Network

The architecture of the Hopfield network consists of the following elements:

• Individual nodes preserve their states until required an update.
• The node to be updated is selected randomly.
• Each node is connected to all other nodes except itself.
• The state of each node is either 0/1 or 1/-1.
• The Hopfield network structure is symmetric, i.e., Wij = Wji for all i's and j's.

The representation of the sample architecture of the Hopfield network having three nodes is as follows:

In the above diagram, each symbol represents:

x1, x2, x3 - represents the input.

y1, y2, y3 - represents output obtained from each node.

Wij - represents the weight associated with the connection from i to j.

## Energy Function in Hopfield Network

In Hopfield networks, there are two different types of updations.

• Synchronous: Updating all the nodes simultaneously each time.
• Asynchronous: Updating only one node at a time. That node is selected randomly or based on specific rules.

In asynchronous updation, each state of Hopfield networks is associated with an energy value. The value is obtained from a function, and that function is named an energy function. This function can decrease or remain unchanged during updation. This energy function of the Hopfield network is defined as:

Where Wij = weight of the connection between i to j

Xi = value of input i

Xj = value of input j

A network is considered to be in a stable state if the energy function tends to be minimum. .

## Training Model with Hopfield Network

Training a Hopfield network refers to lowering of energy of each state. The training model consists of a training and testing algorithm. Let's discuss each of them one by one.

### Training Algorithm

It is based on the Hebbian principle, which Donald Hebb gave in 1949. In training, algorithm weights are updated using a specific rule. The rule is different for binary and bipolar. For storing a set of 'N' input patterns [ X(n) where n = 1 to N ], the rule followed is as follows:

• Binary Input:
• Bipolar Input:

### Testing Algorithm

The testing algorithm involves several steps. These steps are:

• Initialize the weights using the above training algorithm rules.
• Follow steps 3 to 7 for each input vector 'Xi'.
• Assign the value of external input vector 'Xi' to initial activators Yi', for all i = 1 to N.
• Follow steps 5 to 7 for each initial activator 'Yi'.
• Calculate the network's net input 'Yin' using the below rule:
• Based on the calculated net input and threshold value, calculate the output. There are three different conditions for the output. These are:
• Broadcast the obtained output 'Yi' to all other units. And update the activation vector.
• Test the network.

### What is a Hopfield neural network?

The Hopfield network is a particular type of single-layered neuron network. Dr. John J. Hopfield invented it in 1982. These networks were introduced to collect and retrieve memory and store various patterns.

### What are the types of Hopfield neural networks?

There are two types of Hopfield networks - Discrete and Continuous. The discrete network is further classified as Binary, and Bipolar networks based on the output received.

### What are the advantages of using Hopfield networks?

These networks are easy to design and implement. These networks are used in various applications like optimization, pattern recognition, etc.

### What challenges are faced while using Hopfield networks?

These networks can recognize only one pattern and hence are considered very slow. Also, large patterns cannot be stored in these networks.

### Mention some applications where we can use Hopfield networks.

These networks are widely used in research, data compression, image recognition, and optimization. Traveling salesman, knapsack problems, etc, are also some real-life problems in which the Hopfield network is helpful.

## Conclusion

This article deeply discussed the definition of Hopfield networks. We also learned about the architecture and energy function of these networks. At last, we discussed how we could train Hopfield neural networks for different operations.