Code360 powered by Coding Ninjas X Naukri.com. Code360 powered by Coding Ninjas X Naukri.com
Table of contents
1.
📜Introduction📜
2.
📜Dense Layers📜
2.1.
📜Matrix-Vector Multiplication📜
3.
📜Dense Layers With Keras📜
3.1.
🎯Syntax🎯
4.
💡Keras Dense Layer Parameters💡
5.
Keras Dense Layer Methods💡
6.
Basic Operations With Dense Layer
7.
Implementation
7.1.
🎯Sequential Model With Single Dense Layer
7.1.1.
🧑‍💻Code🧑‍💻
7.2.
🎯Sequential Model With Multiple Dense Layers
8.
Frequently Asked Questions
8.1.
Why do we use dense layers?
8.2.
Are dense layers fully connected?
8.3.
What is the use of dense Layer in CNN?
8.4.
What does dense do?
8.5.
Is a dense layer a hidden layer?
9.
Conclusion
Last Updated: Mar 27, 2024
Easy

Dense In Deep Learning

Author Mayank Goyal
0 upvote
Master Python: Predicting weather forecasts
Speaker
Ashwin Goyal
Product Manager @

📜Introduction📜

Layers are the basic building blocks in the deep learning model. The models can present various layers, like LSTM, convolutional, dense, etc. All of these different layers have their importance based on their features. 

dense

A dense layer is a fully connected layer used in the neural network's end stages to change the output's dimensionality from the preceding layer. Dense layers help define the relationship between the data values in which the model is working. 

Also Read About, Resnet 50 Architecture

📜Dense Layers📜

A dense layer is connected deeply with preceding layers in any neural network. Each neuron in the dense layer is connected to every neuron of its preceding layer. Dense layers are the most commonly used layers in Artificial Neural Networks models.
 The neurons in the dense layers in a model receive an outcome from every neuron of the preceding layer. That's where neurons of the dense layer perform matrix-vector multiplication. So in the background, the dense layer performs a matrix-vector multiplication. It is a procedure where the row vector of the outcome from its previous layers equals the column vector of the dense layer.

📜Matrix-Vector Multiplication📜

The general formula of matrix-vector multiplication can be represented as:

matrix

A is a (M x N) matrix, whereas x is a (1 x N) matrix. The values used in the matrix are parameters that can be trained and updated with the help of backpropagation. Backpropagation is an algorithm we use for training the feedforward neural networks. Backpropagation in neural networks computes the gradient of the loss function concerning the network's weights for single input or its associated outcome. Thus from the above theory, we can guarantee that the output from the dense layer is an N-dimensional vector. Thus it can be seen that it is reducing the dimension of the vectors. So basically, a dense layer is used to change the vectors' size by using every neuron. 

Get the tech career you deserve, faster!
Connect with our expert counsellors to understand how to hack your way to success
User rating 4.7/5
1:1 doubt support
95% placement record
Akash Pal
Senior Software Engineer
326% Hike After Job Bootcamp
Himanshu Gusain
Programmer Analyst
32 LPA After Job Bootcamp
After Job
Bootcamp

📜Dense Layers With Keras📜

🎯Syntax🎯

keras.layers.Dense(units, activation=None, use_bias=True, kernel_initializer='glorot_uniform', bias_initializer='zeros', kernel_regularizer=None, bias_regularizer=None, activity_regularizer=None, kernel_constraint=None, bias_constraint=None)


Now we will look at each of the hyperparameters quickly.

💡Keras Dense Layer Parameters💡

HyperParameteras Description
units It is the most primitive of all the parameters. It accepts only a positive integer and defines the size of the output vector from the dense layer,i.e., determines the size of the weight matrix along with the bias vector.
activation It helps define the activation function that can be applied per element of a dense layer. There are various types of activation functions provided by Keras, such as relu, sigmoid, softmax, Selu, Elu, Tanh, and many more. If not specified, it uses a linear activation function.
use_bias It takes a boolean value that helps decide whether we should include a bias vector for calculation purposes. By default, we set use_bias is set to true.
kernel_initializer It helps in initializing the kernel weight matrix of the dense layer.
bias_intializer It is used for initializing the bias vector. A bias vector is the additional weights requiring no input and corresponding to the output layer. By default, it is set as zeros.
kernel_regularizer It defines the regularizer or the loss function applied to the kernel weight matrix.
bias_regularizer It defines the regularizer for the bias vector if we have initialized any. By default, it is set to none.
activity_regularizer It defines the regularizer of the activation function we have described in the activation parameter. activtiy_regularizer is applied to the output of the layer. By default, it is set to none.
kernel_constraint It represents the constraint function applied to the kernel weight matrix.
bias_constraint It represents the constraint function that can be applied to the bias vector.

Further, Keras also provides some of the primary methods discussed below:

Keras Dense Layer Methods💡

Methods Description
get_weights It returns the complete list of the weights used in the layers.
set_weights It sets the weights of the layers.
summary() The model's summary() method displays all the layers, including each layer's name, output shape, and the number of parameters.
input_shape Get the input shape if the layer has only a single node.
input Get the input data if the layer has only a single node.
get_input_shape_at() The layer returns the input shape at the specified index if it has multiple nodes.
get_input() The layer returns the input data at the specified index if it has multiple nodes.
output_shape Returns the output shape if the layer has a single node.
output Returns the output data if the layer has a single node.
get_output_shape_at The layer returns the output shape at the specified index if it has multiple nodes.
get_output_at It returns the output data at the specified index if the layer has multiple nodes.

Also see, Introduction to Machine Learning

Basic Operations With Dense Layer

The three main parameters for a dense layer are an activation function, the weight matrix, and a bias vector. Using these parameters, a dense layer operation can be represented as:

👉output = activation(dot(input, kernel) + bias)

In the above equation, the activation function is used for performing element-wise activation, and the kernel is the weights matrix generated by the layer. Bias is a bias vector formed by the dense layer. Keras dense layer performs dot product of input tensor and weight kernel matrix.

Implementation

🎯Sequential Model With Single Dense Layer

We use the Sequential model, an in-built Keras model. First, we fed the input layer to the model and then added a dense layer along with ReLU activation.

🧑‍💻Code🧑‍💻

import tensorflow as tf
model_1 = tf.keras.models.Sequential()
model_1.add(tf.keras.Input(shape=(16,)))
model_1.add(tf.keras.layers.Dense(32, activation='relu'))
model_1.output_shape


👉Output

(None, 32)

🎯Sequential Model With Multiple Dense Layers

We look at a model where multiple hidden layers are used in deep neural networks. Here we are using the ReLu activation function in the neurons of the hidden dense layer.

🧑‍💻Code🧑‍💻
import tensorflow as tf
model1 = tf.keras.models.Sequential()
model1.add(tf.keras.Input(shape=(16,)))
model1.add(tf.keras.layers.Dense(32, activation='relu'))
model1.add(tf.keras.layers.Dense(32))
print(model1.output_shape)

 

👉Output

output

Check out this article - Padding In Convolutional Neural Network, and Difference between argument and parameter.

Frequently Asked Questions

Why do we use dense layers?

A Dense layer feeds all outputs from the previous layer to all its neurons, each providing one output to the next layer. It's the most basic layer in neural networks.

Are dense layers fully connected?

The dense layer, also called the fully-connected layer, refers to the layer whose inside neurons connect to every neuron in the preceding layer.

What is the use of dense Layer in CNN?

The dense layer is a simple Layer of neurons in which each neuron receives input from all the neurons of the previous layer, thus called as dense. The dense layer is used to classify images based on output from convolutional layers.

What does dense do?

Dense is the only actual network layer in that model. A Dense layer feeds all outputs from the previous layer to all its neurons, each providing one output to the next layer.

Is a dense layer a hidden layer?

The first Dense object is the first hidden layer. The input layer is specified as a parameter to the first Dense object's constructor.

Conclusion

Let us brief the article. Firstly, we saw the intuition behind the dense layer. Later, we saw how it could be implemented using Keras and saw different parameters and methods associated with it. Since it is the primary part of any neural network, we should know about the various primary and dense layers.  

You can also refer to Stochastic Gradient DescentFeature SelectionLogistic RegressionSigmoid neuron and many more to enhance your knowledge.

I hope you all like this article. 

Happy Learning, Ninjas!

Previous article
Fooling Convolutional Neural Network
Next article
Bias variance tradeoff
Live masterclass