Table of contents
1.
Introduction
1.1.
Prerequisites 
2.
What is Perceptron?
3.
Types of Perceptron
3.1.
1. Single-Layer Perceptron (SLP)
3.2.
2. Multi-Layer Perceptron (MLP)
4.
Perceptron in Machine Learning
5.
History of Perceptron
6.
What is the Perceptron Model in Machine Learning?
7.
How Does Perceptron Work?
8.
Characteristics of the Perceptron Model
9.
Limitation of Perceptron Model
10.
Perceptron Learning Rule
11.
Perceptron Function
12.
Inputs of a Perceptron
13.
Activation Functions of Perceptron
14.
Output of Perceptron
15.
Error in Perceptron
16.
Perceptron: Decision Function
17.
Perceptron at a Glance
18.
Implement Logic Gates with Perceptron
19.
What is Logic Gate?
20.
Implementing Basic Logic Gates With Perceptron
21.
Frequently Asked Questions
21.1.
What is the difference between neural network and perceptron?
21.2.
What are the 4 parts of perceptron?
21.3.
What is the objective of perceptron learning?
22.
Conclusion
Last Updated: Aug 27, 2024
Easy

What is Perceptron?

Author Arun Nawani
2 upvotes
Career growth poll
Do you think IIT Guwahati certified course can help you in your career?

Introduction

In artificial intelligence(AI) and machine learning, the perceptron is often regarded as one of the foundational building blocks of neural networks. It was introduced by Frank Rosenblatt in 1958. The perceptron is a simple yet powerful model that mimics the behavior of a biological neuron. It serves as a basic unit for more complex neural network architectures. The perceptron is designed to perform binary classification tasks by learning to separate data points into two distinct classes. In this blog, we will explore more about perceptron. 

What is Perceptron?

Prerequisites 

This blog assumes the readers to be well versed with Artificial Neural Networks, their architecture, and the important terminologies associated with them. You can check out our previous blogs on ANN if you want to brush up on your concepts. 

What is Perceptron?

As discussed, it’s the most simplistic form of an Artificial Neural Network. But what is meant by that? An ANN consists of three components- 

  • The input layer: It takes the input and preprocesses to feed it to the next subsequent layer. 
  • The hidden layer: It receives the input and this is where the majority of computation is done. The adjusting of weights is done by multiple iterations of forward and backward propagation. The hidden layer can contain several multiple layers. The results of this component are passed on to the next component of the architecture. 
  • The output layer: The final component of architecture. It receives the data from the hidden layers and then gives the output. It also sends feedback after every iteration of forward propagation. This is what initiates backpropagation. 

 

If you want to learn more about forward propagation and backpropagation, check out our previous articles by clicking on either of the two. 

This is an overview of what an ANN is like. Perceptron, being the simpler form of ANN, consists of a single neuron being fed with the input and returning the output. Perceptron is used for binary classification and hence is also referred to as a binary classifier. Take a look at the illustration below:

perceptron

Here, we have x as inputs and w as associated weights. 

For every input xi, we have an associated weight with that input wi. Look at the given equation.

Weighted sum = x1w1 + x2w+ x3w+ …. + xnwn + B (bias term)

We see that the weighted sum expression has a term ‘B’. It is known as the bias term. It can also be adjusted as per the feedback received, just like weights. The weighted sum expression is passed through the activation function. Let’s say the activation function employed is the Sigmoid function. Since the range of sigmoid function is (0,1), the value of the output generated by the neuron would be between 0 and 1. The final function is known as the step function. In this case, the perceptron works as a binary classifier. Therefore, it is a two-valued function. 

 

Step function = 1 , ysigmoid >=0.5

                         0 , otherwise

Here, 0.5 is our threshold value. x and w values can also be represented in matrix form. 

matrix form

Weighted sum = X.W + B, where X and N are matrices and B is the bias term. 

Types of Perceptron

There are two types of perceptron:

1. Single-Layer Perceptron (SLP)

  • Description: Consists of a single layer of output nodes connected directly to the input features. It performs binary classification tasks by linearly separating data into two classes.
  • Example: Basic binary classification problems like classifying data points as belonging to one of two categories.

2. Multi-Layer Perceptron (MLP)

  • Description: Contains one or more hidden layers between the input and output layers. It can solve more complex problems by learning non-linear decision boundaries.
  • Example: Image recognition or complex function approximation tasks.

Perceptron in Machine Learning

The Perceptron is a fundamental algorithm in machine learning that serves as a building block for more complex models. It is a type of linear classifier that makes predictions based on a linear predictor function. The Perceptron algorithm adjusts the weights of features based on the classification errors it makes, learning to make better predictions over time.

History of Perceptron

In 1957, Frank Rosenblatt introduced the Perceptron model at the Cornell Aeronautical Laboratory. He aimed to simulate a simplified version of the human brain's neuron. In the 1960s, the Perceptron was initially successful but faced limitations, leading to the "AI Winter," a period of reduced funding and interest in neural networks. In 1980, the development of multi-layer networks and backpropagation revived interest in neural networks, leading to modern deep learning.

What is the Perceptron Model in Machine Learning?

The Perceptron model is a simple type of artificial neural network used for binary classification. It consists of input nodes, weights, a bias term, and an activation function. The model computes a weighted sum of the inputs and applies an activation function to decide the output class.

How Does Perceptron Work?

  • Initialization: Start with random weights and bias.
  • Input Processing: Compute the weighted sum of the input features.
  • Activation Function: Apply a step function to determine the output class.
  • Error Calculation: Compare the predicted output with the actual label.
  • Weight Adjustment: Update the weights and bias based on the error using the Perceptron Learning Rule.
  • Iteration: Repeat the process for multiple epochs until convergence.

Characteristics of the Perceptron Model

  • Linear Separability: Effective only for linearly separable data.
  • Binary Classification: Primarily used for binary classification tasks.
  • Simple Structure: Consists of a single layer of weights.
  • Training Algorithm: Uses the Perceptron Learning Rule for weight updates.
  • Activation Function: Typically a step function.

Limitation of Perceptron Model

  • Linearly Separable Data: Cannot solve problems where data is not linearly separable.
  • Single Layer Limitation: Limited to problems that can be separated by a single hyperplane.
  • No Hidden Layers: Lacks the capability to model complex patterns without hidden layers.
  • Convergence Issues: May not converge on certain data sets if not properly tuned.

Perceptron Learning Rule

The Perceptron Learning Rule adjusts the weights and bias based on the classification error. It updates weights according to the formula: weightnew=weightold+learning rate×(actual−predicted)×input\text{weight}_{\text{new}} = \text{weight}_{\text{old}} + \text{learning rate} \times (\text{actual} - \text{predicted}) \times \text{input}weightnew​=weightold​+learning rate×(actual−predicted)×input The bias is updated similarly to account for the error.

Perceptron Function

The Perceptron function computes a weighted sum of the input features and applies an activation function to determine the output. The function is: output=activation(∑(weighti×inputi)+bias)

Inputs of a Perceptron

Inputs to a Perceptron are the features of the data. Each input is associated with a weight that represents its importance. The Perceptron uses these inputs and weights to compute the weighted sum for further processing.

Activation Functions of Perceptron

The activation function in a Perceptron is typically a step function, which outputs binary values (0 or 1) based on whether the weighted sum of inputs exceeds a threshold. Other functions like the sigmoid can also be used in variations of Perceptrons.

Output of Perceptron

The output of a Perceptron is a binary value (0 or 1) indicating the class of the input data. It is determined by applying the activation function to the weighted sum of inputs.

Error in Perceptron

Error in a Perceptron is the difference between the actual label and the predicted output. It is used to adjust the weights and bias during training to minimize classification errors.

Perceptron: Decision Function

The Perceptron decision function calculates the class of an input by computing a weighted sum of the input features and applying the activation function. If the weighted sum is above a certain threshold, it classifies the input into one class; otherwise, it classifies it into another.

Perceptron at a Glance

  • Basic Model: Single-layer neural network.
  • Linear Classifier: Effective for linearly separable data.
  • Training: Adjusts weights using the Perceptron Learning Rule.
  • Activation Function: Typically a step function.
  • Limitations: Struggles with non-linearly separable data.

Implement Logic Gates with Perceptron

Perceptrons can be used to model basic logic gates by setting appropriate weights and bias:

  • AND Gate: Outputs 1 only if both inputs are 1.
  • OR Gate: Outputs 1 if at least one input is 1.
  • NAND Gate: Inverts the output of the AND gate.
  • NOR Gate: Inverts the output of the OR gate.

What is Logic Gate?

Logic gates are fundamental components of digital circuits that perform basic logical operations on one or more binary inputs to produce a single binary output. They are used to build complex digital systems.

Implementing Basic Logic Gates With Perceptron

  1. AND Gate: A Perceptron with weights set to values that only activate if both inputs are high (1).
  2. OR Gate: A Perceptron with weights that activate if at least one input is high (1).
  3. NAND Gate: A Perceptron that outputs the opposite of the AND gate.
  4. NOR Gate: A Perceptron that outputs the opposite of the OR gate.

Frequently Asked Questions

What is the difference between neural network and perceptron?

A perceptron is a single-layer neural network used for binary classification, whereas a neural network can have multiple layers (hidden layers) and handle complex patterns and tasks beyond simple linear separability.

What are the 4 parts of perceptron?

The four parts of a perceptron are input features, weights, a bias term, and an activation function. Inputs are weighted and summed, adjusted by bias, and passed through the activation function to produce the output.

What is the objective of perceptron learning?

The objective of perceptron learning is to adjust weights and biases to minimize classification errors, thereby finding the optimal decision boundary that separates different classes in the input feature space.

Conclusion

In this blog, we have discussed what perceptron is. It is a foundational model in machine learning and neural networks, representing one of the simplest forms of a neural network. It serves as a building block for more complex models and algorithms. 

Check this out to know about Machine Learning.

Happy Learning

Live masterclass