Performing Masking and Padding using Keras

What is Masking?

Masking is a technique used for deep learning and neural network implementation. It is used when the concept of layers comes into existence. It is used to tell sequence-processing layers that certain timesteps are missing in the input. And thus, these layers are required to be skipped at the time of processing of data.

When considering each timestep in the input tensor, it is essential that the value of the tensor at a particular timestep is equal to mask_value; then the timestep will be considered masked or skipped in all the downstream layers.

Syntax:

Tfkeras. Layers.Masking(mask_value=0.0, **kwargs)

Other possibilities may occur when any downstream layer does not support Masking yet receives mask input, and an exception is raised.

In Masking, all the samples must have uniform lengths. The model needed to be informed that a few data input parts are padding and must be ignored. In keras, Masking can be implemented by providing masks to the layers that support it. The Embedding layer can also produce a mask. 

There are two ways to introduce input masks in Keras models:

• Adding a keras masking layer.
   
• Then configure the keras layer Embedding layer with mask_zero=True.

What is Padding?

Padding in Keras is special Masking in which the masked steps are at the beginning or end of the sequence. The Padding comes into the picture when it is required to encode the sequence data in contiguous batches to make the sequence fit in the batch fit of a fixed or given standard length. There is always a requirement or necessity for Padding or truncating some sequences. Padding also ensures that the given input data should have the same length.Types of Padding supported by Keras are:

• Valid Padding generally occurs during the model's building phase or in the stacking phase, where the individual layers are added to the model that has been built until this step.
   
• Same/Zero Padding is very similar to valid Padding. It is just that after the sequential API implementation, a Conv layer is added, and it is also termed as same/zero Padding.
   
• Causal Padding padding is used to apply causal to the Conv1D model to pad zeros to the front of the given input.
   

The Model Building with Keras involves the following steps:

Step 1: Creating the instance of the model, e.g., with the Sequential API

Step 2: Stacking layers on top of each other 

Step 3: Model compilation.

Let us consider an example of the starting stage of three convolution layers:

#Creating or model building 

model_1 = Sequential()

Performing Masking and Padding using Keras

The model for implementing the Keras using the google colab uses Masking and Padding.

Step 1: Importing the required libraries 

import numpy as np
import tensorflow as tf
from tensorflow import keras
from tensorflow.keras.preprocessing.text import Tokenizer
from tensorflow.keras.preprocessing.sequence import pad_sequences

You can also try this code with Online Python Compiler

Run Code

Step 2: Dataset creation 

data = [
    'I love my friend,'
    'I love my family,
    'My friend loves me.
    'Do you think my friend loves me?'
]

You can also try this code with Online Python Compiler

Run Code

Step 3: Model fitting and Tokenizing the text sequences 

from tensorflow.keras.preprocessing.text import Tokenizer
tokenize = Tokenizer(num_words = 100 , oov_token='<OOV>')
tokenize.fit_on_texts(data)
word_index = tokenize.word_index


sequences = tokenize.texts_to_sequences(data)


print('\nWord Index = {}'.format(word_index))
print('\nSequences = {}'.format(sequences))

You can also try this code with Online Python Compiler

Run Code

Output:

Step 4: Applying Padding and Masking using sequences. 

padded_len = pad_sequences(sequences, maxlen =5)
print("\nPadded_len Sequences = {}".format(padded_len))

You can also try this code with Online Python Compiler

Run Code

Output:
 

Step 5: Model fitting 

new_data = [ 
    'i really like dancing,
    'do I dance with good steps'
    
]

You can also try this code with Online Python Compiler

Run Code

Step 6: The Test Sequences 

new_seq = tokenize.texts_to_sequences(new_data)
print('\nTest Sequence = {}'.format(new_seq))

You can also try this code with Online Python Compiler

Run Code

Output:

Step 7: The Resulting Padding sequence 

Frequently Asked Questions 

What is masking in keras?

Masking is a technique used for deep learning and neural network implementation. It is used when the concept of layers comes into existence.

What is Padding?

Padding in Keras is special Masking in which the masked steps are at the beginning or end of the sequence.

Mention the type of Padding.

There are three types of Padding: Valid Padding (no padding), Same Padding (zero Padding), and Causal Padding.

Mention the syntax of Masking.

Tfkeras. Layers.Masking(mask_value=0.0, **kwargs)

How to import the Kears library in Python.

Pip install Keras is used to install the keras library from the python library. 

Conclusion 

This Article contained a brief introduction and implementation of Keras Padding and Masking and Also covered both comparisons between Padding and Masking. 

You can check coding ninjas, Padding, and Masking to learn more about them. Refer to our guided paths on Coding Ninjas Studio to learn more about DSA, Competitive Programming, JavaScript, System Design, etc. Enroll in our courses and refer to the mock test and problems available; take a look at the interview experiences and interview bundle for placement preparations.

Do upvote our blog to help other ninjas grow.

Merry Learning!