Tensorflow Advanced: Sparse and Ragged Tensors

TensorFlow

TensorFlow is a free library mainly used for artificial intelligence and machine learning. It is developed by Google. TensorFlow enables fast numerical computing. By using TensorFlow, we can implement model tracking, performance monitoring, data automation, and model retraining.

Alright! Now we are all set to learn about Sparse and Ragged tensors.

Sparse and Ragged Tensors

A sparse tensor is a dataset in which the majority of the elements have a value of zero. We can consider a diagonal matrix as an example of a sparse matrix. If there is a 2-D tensor of 7x8 (rows indexing from 0 to 6 and columns indexing from 0 to 7), then we need to store 56 values of all the elements. Thus, it can be seen that there is a lot of waste of memory if there are very few non-zero elements in it. To save memory, they can be represented in arrays or linked lists.

Ragged tensors are equivalent to nested variable-length lists. Ragged tensors allow the storage and processing of non-uniformly shaped data.

Must Read Difference between ArrayList and LinkedList

Introduction to tf.sparse and tf.ragged

We can use tf.sparse for creating a sparse matrix. Similarly, we can use tf.ragged for creating a ragged matrix.

We can represent a sparse matrix in Tensorflow into three dense vectors, which are indices, values, and dense shape. In Python, the three vectors are collected into a SparseTensor class for easy use.
The sparse tensor tf.sparse comprises of the following components:

1. indices: It is a 2-D tensor of int64 and has shape [N, ndims], and they specify the indices of the non-zero elements.
2. values: It is a 1-D tensor of shape [N], and it supplies the values for each element.
3. dense_shape: It is a 1-D tensor of int64 data type and has shape [ndims]. It specifies the dense_shape of any given sparse tensor. It takes a list which indicates the number of elements present in each dimension.
    

tf.ragged is a package which defines ops for manipulating ragged tensors, which are tensors with shapes being irregular. Every ragged tensor may have one or more ragged dimensions. Ragged dimensions are the dimensions whose slices may have different lengths, i.e., the column slices have different lengths.

Working with tf.sparse Tensorflow

Before working with sparse tensors, we need to import some libraries in Python.

Code

import tensorflow as tf
import numpy as np

 

Code

st = tf.SparseTensor(
 indices=[[0, 3], [2, 4]], values=[10, 20], dense_shape=[3, 10])
st

Output

<tensorflow.python.framework.sparse_tensor.SparseTensor at 0x7fabfa5b56d0>

 

Code

print(st)

Output

SparseTensor(indices=tf.Tensor(
[[0 3]
[2 4]], shape=(2, 2), dtype=int64), values=tf.Tensor([10 20], shape=(2,), dtype=int32), dense_shape=tf.Tensor([ 3 10], shape=(2,), dtype=int64))

 

Code

np_array = np.array([[1,0,0,0],
                    [0,0,0,0],
                    [0,0,2,0],
                    [0,0,0,4]])
np_array

Output

array([[1, 0, 0, 0],
      [0, 0, 0, 0],
      [0, 0, 2, 0],
      [0, 0, 0, 4]])

 

Code

st2_fd = tf.sparse.from_dense(np_array)
st2_fd

Output

<tensorflow.python.framework.sparse_tensor.SparseTensor at 0x7fabfa59ec40>

 

Code

print(st2_fd)

Output

SparseTensor(indices=tf.Tensor(
[[0 0]
[2 2]
[3 3]], shape=(3, 2), dtype=int64), values=tf.Tensor([1 2 4], shape=(3,), dtype=int64), dense_shape=tf.Tensor([4 4], shape=(2,), dtype=int64))

 

Code

st2_fd.values.numpy()

Output

array([1, 2, 4])

 

Code

st2_fd.values.numpy().tolist()

Output

[1, 2, 4]

 

Code

st2_fd.indices.numpy().tolist()

Output

[[0, 0], [2, 2], [3, 3]]

 

Code

st2_fd.dense_shape.numpy().tolist()

Output

[4, 4]

 

Code

dt_fst = tf.sparse.to_dense(st2_fd)
dt_fst

Output

<tf.Tensor: shape=(4, 4), dtype=int64, numpy=
array([[1, 0, 0, 0],
      [0, 0, 0, 0],
      [0, 0, 2, 0],
      [0, 0, 0, 4]])>

 

Code

dt_fst.numpy()

Output

array([[1, 0, 0, 0],
      [0, 0, 0, 0],
      [0, 0, 2, 0],
      [0, 0, 0, 4]])

 

Code

st_add = tf.sparse.add(st2_fd, st2_fd)
st_add

Output

<tensorflow.python.framework.sparse_tensor.SparseTensor at 0x7fac1fce6a60>

 

Code

print(st_add)

Output

SparseTensor(indices=tf.Tensor(
[[0 0]
[2 2]
[3 3]], shape=(3, 2), dtype=int64), values=tf.Tensor([2 4 8], shape=(3,), dtype=int64), dense_shape=tf.Tensor([4 4], shape=(2,), dtype=int64))

 

Code

tf.sparse.to_dense(st_add).numpy()

Output

array([[2, 0, 0, 0],
      [0, 0, 0, 0],
      [0, 0, 4, 0],
      [0, 0, 0, 8]])

Working with tf.ragged in Tensorflow

We need to do some setup to install and import some important libraries.

Setup

!pip install --pre -U tensorflow
import math
import tensorflow as tf

Ragged tensors support many TensorFlow operations, which include array operations, math operations, string manipulation ops, control flow operations, etc.

 

Let's create a ragged tensor consisting of some constants.

Code

digits = tf.ragged.constant([[2, 6, 1, 7], [], [9, 3, 2], [4], []])
print(digits)

Output

<tf.RaggedTensor [[2, 6, 1, 7], [], [9, 3, 2], [4], []]>

 

Now, let's add 5 to all the elements present in the ragged tensor.

Code

print(tf.add(digits, 5)) # Adding 5 to all the elements

Output

<tf.RaggedTensor [[7, 11, 6, 12], [], [14, 8, 7], [9], []]>

 

Now let's concatenate some elements to the ragged tensor.

Code

print(tf.concat([digits, [[7, 8]]], axis=0)) # Concatenating (7, 8) to the ragged tensor

Output

<tf.RaggedTensor [[2, 6, 1, 7], [], [9, 3, 2], [4], [], [7, 8]]>

 

Alright! Now we hope you understand the Tensorflow Advanced: Sparse and Ragged Tensors.

Frequently Asked Questions

What is a tensor?

A tensor is an algebraic object that describes a multilinear relationship between sets of algebraic objects related to a vector space. 

What is the difference between a tensor and a matrix?

A tensor is an n-dimensional array satisfying a particular transformation law. Unike a matrix, it shows an object placed in a specific coordinate system.

What are the common libraries used for machine learning?

The common libraries used for machine learning are NumPy, Pandas, Scikit learn, NLTK, PyTorch, TensorFlow, etc.

Why is TensorFlow faster than PyTorch?

TensorFlow offers better visualisation, which allows developers to debug and track the training process easily. However, PyTorch provides only limited visualisation.

What is PyTorch?

PyTorch is a machine learning framework used for computer vision and natural language processing. It is free and easy to use.

Conclusion

In this article, we discussed sparse tensors and ragged tensors. We learnt how to create a sparse and ragged tensor in TensorFlow.

We hope this blog on Tensorflow Advanced: Sparse and Ragged Tensors was helpful. You can also refer to other similar articles.

• TensorFlow
• Applications of TensorFlow
• TensorFlow vs PyTorch vs Keras
• TensorFlow Functions to make Deep Learning Easier

 

You may refer to our Guided Path on Code Studios for enhancing your skill set on DSA, Competitive Programming, System Design, etc. Check out essential interview questions, practice our available mock tests, look at the interview bundle for interview preparations, and so much more!

Happy Learning Ninja!