1.
Introduction
2.
Tensors
3.
TensorFlow
4.
Sparse and Ragged Tensors
5.
Introduction to tf.sparse and tf.ragged
6.
Working with tf.sparse Tensorflow
7.
Working with tf.ragged in Tensorflow
8.
8.1.
What is a tensor?
8.2.
What is the difference between a tensor and a matrix?
8.3.
What are the common libraries used for machine learning?
8.4.
Why is TensorFlow faster than PyTorch?
8.5.
What is PyTorch?
9.
Conclusion
Last Updated: Mar 27, 2024
Easy

# Tensorflow Advanced: Sparse and Ragged Tensors

Shubham Das
0 upvote
Leveraging ChatGPT - GenAI as a Microsoft Data Expert
Speaker
Prerita Agarwal
Data Specialist @
23 Jul, 2024 @ 01:30 PM

## Introduction

While working on machine learning and deep learning, it becomes necessary to work with tensors. In this article, we will discuss the Sparse and Ragged Tensors.

But we need to understand what tensors are before diving into the concepts.

## Tensors

A tensor is an algebraic object that describes a multilinear relationship between sets of algebraic objects related to a vector space. Tensors may map between objects, such as vectors, scalars, and even other tensors. We can implement various operations to tensors. After understanding the tensors, let's have a basic introduction to TensorFlow.

Get the tech career you deserve, faster!
Connect with our expert counsellors to understand how to hack your way to success
User rating 4.7/5
1:1 doubt support
95% placement record
Akash Pal
Senior Software Engineer
326% Hike After Job Bootcamp
Himanshu Gusain
Programmer Analyst
32 LPA After Job Bootcamp
After Job
Bootcamp

## TensorFlow

TensorFlow is a free library mainly used for artificial intelligence and machine learning. It is developed by Google. TensorFlow enables fast numerical computing. By using TensorFlow, we can implement model tracking, performance monitoring, data automation, and model retraining.

Alright! Now we are all set to learn about Sparse and Ragged tensors.

## Sparse and Ragged Tensors

sparse tensor is a dataset in which the majority of the elements have a value of zero. We can consider a diagonal matrix as an example of a sparse matrix. If there is a 2-D tensor of 7x8 (rows indexing from 0 to 6 and columns indexing from 0 to 7), then we need to store 56 values of all the elements. Thus, it can be seen that there is a lot of waste of memory if there are very few non-zero elements in it. To save memory, they can be represented in arrays or linked lists.

Ragged tensors are equivalent to nested variable-length lists. Ragged tensors allow the storage and processing of non-uniformly shaped data.

## Introduction to tf.sparse and tf.ragged

We can use tf.sparse for creating a sparse matrix. Similarly, we can use tf.ragged for creating a ragged matrix.

We can represent a sparse matrix in Tensorflow into three dense vectors, which are indices, values, and dense shape. In Python, the three vectors are collected into a SparseTensor class for easy use.
The sparse tensor tf.sparse comprises of the following components:

1. indices: It is a 2-D tensor of int64 and has shape [N, ndims], and they specify the indices of the non-zero elements.
2. values: It is a 1-D tensor of shape [N], and it supplies the values for each element.
3. dense_shape: It is a 1-D tensor of int64 data type and has shape [ndims]. It specifies the dense_shape of any given sparse tensor. It takes a list which indicates the number of elements present in each dimension.

tf.ragged is a package which defines ops for manipulating ragged tensors, which are tensors with shapes being irregular. Every ragged tensor may have one or more ragged dimensions. Ragged dimensions are the dimensions whose slices may have different lengths, i.e., the column slices have different lengths.

## Working with tf.sparse Tensorflow

Before working with sparse tensors, we need to import some libraries in Python.

Code

``````import tensorflow as tf
import numpy as np``````

Code

``````st = tf.SparseTensor(
indices=[[0, 3], [2, 4]], values=[10, 20], dense_shape=[3, 10])
st``````

Output

``<tensorflow.python.framework.sparse_tensor.SparseTensor at 0x7fabfa5b56d0>``

Code

``print(st)``

Output

``````SparseTensor(indices=tf.Tensor(
[[0 3]
[2 4]], shape=(2, 2), dtype=int64), values=tf.Tensor([10 20], shape=(2,), dtype=int32), dense_shape=tf.Tensor([ 3 10], shape=(2,), dtype=int64))``````

Code

``````np_array = np.array([[1,0,0,0],
[0,0,0,0],
[0,0,2,0],
[0,0,0,4]])
np_array``````

Output

``````array([[1, 0, 0, 0],
[0, 0, 0, 0],
[0, 0, 2, 0],
[0, 0, 0, 4]])``````

Code

``````st2_fd = tf.sparse.from_dense(np_array)
st2_fd``````

Output

``<tensorflow.python.framework.sparse_tensor.SparseTensor at 0x7fabfa59ec40>``

Code

``print(st2_fd)``

Output

``````SparseTensor(indices=tf.Tensor(
[[0 0]
[2 2]
[3 3]], shape=(3, 2), dtype=int64), values=tf.Tensor([1 2 4], shape=(3,), dtype=int64), dense_shape=tf.Tensor([4 4], shape=(2,), dtype=int64))``````

Code

``st2_fd.values.numpy()``

Output

``array([1, 2, 4])``

Code

``st2_fd.values.numpy().tolist()``

Output

``[1, 2, 4]``

Code

``st2_fd.indices.numpy().tolist()``

Output

``[[0, 0], [2, 2], [3, 3]]``

Code

``st2_fd.dense_shape.numpy().tolist()``

Output

``[4, 4]``

Code

``````dt_fst = tf.sparse.to_dense(st2_fd)
dt_fst``````

Output

``````<tf.Tensor: shape=(4, 4), dtype=int64, numpy=
array([[1, 0, 0, 0],
[0, 0, 0, 0],
[0, 0, 2, 0],
[0, 0, 0, 4]])>``````

Code

``dt_fst.numpy()``

Output

``````array([[1, 0, 0, 0],
[0, 0, 0, 0],
[0, 0, 2, 0],
[0, 0, 0, 4]])``````

Code

``````st_add = tf.sparse.add(st2_fd, st2_fd)

Output

``<tensorflow.python.framework.sparse_tensor.SparseTensor at 0x7fac1fce6a60>``

Code

``print(st_add)``

Output

``````SparseTensor(indices=tf.Tensor(
[[0 0]
[2 2]
[3 3]], shape=(3, 2), dtype=int64), values=tf.Tensor([2 4 8], shape=(3,), dtype=int64), dense_shape=tf.Tensor([4 4], shape=(2,), dtype=int64))``````

Code

``tf.sparse.to_dense(st_add).numpy()``

Output

``````array([[2, 0, 0, 0],
[0, 0, 0, 0],
[0, 0, 4, 0],
[0, 0, 0, 8]])``````

## Working with tf.ragged in Tensorflow

We need to do some setup to install and import some important libraries.

Setup

``````!pip install --pre -U tensorflow
import math
import tensorflow as tf``````

Ragged tensors support many TensorFlow operations, which include array operations, math operations, string manipulation ops, control flow operations, etc.

Let's create a ragged tensor consisting of some constants.

Code

``````digits = tf.ragged.constant([[2, 6, 1, 7], [], [9, 3, 2], [4], []])
print(digits)``````

Output

``<tf.RaggedTensor [[2, 6, 1, 7], [], [9, 3, 2], [4], []]>``

Now, let's add 5 to all the elements present in the ragged tensor.

Code

``print(tf.add(digits, 5)) # Adding 5 to all the elements``

Output

``<tf.RaggedTensor [[7, 11, 6, 12], [], [14, 8, 7], [9], []]>``

Now let's concatenate some elements to the ragged tensor.

Code

``print(tf.concat([digits, [[7, 8]]], axis=0)) # Concatenating (7, 8) to the ragged tensor``

Output

``<tf.RaggedTensor [[2, 6, 1, 7], [], [9, 3, 2], [4], [], [7, 8]]>``

Alright! Now we hope you understand the Tensorflow Advanced: Sparse and Ragged Tensors.

### What is a tensor?

A tensor is an algebraic object that describes a multilinear relationship between sets of algebraic objects related to a vector space.

### What is the difference between a tensor and a matrix?

A tensor is an n-dimensional array satisfying a particular transformation law. Unike a matrix, it shows an object placed in a specific coordinate system.

### What are the common libraries used for machine learning?

The common libraries used for machine learning are NumPy, Pandas, Scikit learn, NLTK, PyTorch, TensorFlow, etc.

### Why is TensorFlow faster than PyTorch?

TensorFlow offers better visualisation, which allows developers to debug and track the training process easily. However, PyTorch provides only limited visualisation.

### What is PyTorch?

PyTorch is a machine learning framework used for computer vision and natural language processing. It is free and easy to use.

## Conclusion

In this article, we discussed sparse tensors and ragged tensors. We learnt how to create a sparse and ragged tensor in TensorFlow.

We hope this blog on Tensorflow Advanced: Sparse and Ragged Tensors was helpful. You can also refer to other similar articles.

You may refer to our Guided Path on Code Studios for enhancing your skill set on DSACompetitive ProgrammingSystem Design, etc. Check out essential interview questions, practice our available mock tests, look at the interview bundle for interview preparations, and so much more!

Happy Learning Ninja!

Live masterclass