Singular Value Decomposition

Linear Transformations

A linear transformation is a matrix. Every matrix defines a linear transformation on the plane as follows. Let's say this matrix.

It takes every point in a plane and moves it to a different point. i.e, a point (p,q) is moved to another point (3p+0q, 4p+5q).  

       Coordinate 1      Coordinate 2

• (p,q)                  (3p+0q, 4p+5q)
• (1,0)                       (3,4)
• (0,1)                       (0,5)
• (-1,0)                      (-3,-4)
• (0,-1)                      (0,-5)

Coordinate 1

Coordinate 2 after Linear Transformation

In general, every  2x2 matrix will define a linear transformation that sends the unit circle in the first coordinate to some ellipse in the second coordinate. That ellipse represents the linear transformation. We can develop the ellipse from the ring by rotation, horizontal and vertical stretching, and another spin.

Singular Value Decomposition

We can write the matrix A as the product of these three matrices.

Formula of singular Value Decomposition

Amxn = Umxmmxn VnxnT

So matrix  A of order mxn can be factorized into three component matrices: a rotation matrix U called the Unitary matrix or  left singular matrix, a scaling or diagonal matrix  , and another rotation matrix VTcalled the conjugate transpose of the nxn unitary matrix.  In the scaling matrix, all the entries except diagonal are zero, σ1is the horizontal scaling factor, and σ2 is the vertical scaling factor. A positive value of σ means stretching, and the negative value of σmeans compression. If the value of σ is 1, then this implies neither stretching nor compression. 

Applications:

1) Image Compression: 

Singular Value Decomposition is used in Image Compression that is a process by which an Image file is encoded in such a way that consumes less space as compared to the original file.

                                    Source: codeproject.com

2) Recommendation Systems: 

Singular Value Decomposition is used in Recommendation Systems  that predicts user interest and recommends product items based on their choice.

                                       Source: semanticscholar.org

FAQs

1. Singular Value Decomposition is some sort of generalization of _______ decomposition.
   Eigenvector
    
2. Singular matrices are?
   non-invertible
    
3. What does an Eigenvector mean?
   Eigenvector of a matrix A is a vector represented by a matrix X such that when X is multiplied with matrix A, then the direction of the resultant matrix remains same as vector X.
    
4. What is a diagonal matrix?
   In a diagonal matrix, every entry except the principal diagonal is zero.
    
5. What is the transpose of a matrix?
   The transpose of a matrix is obtained by changing its rows into columns and its columns into rows

Key Takeaways:

This comes to the end of the discussion of Singular Value Decomposition. I hope you gained something helpful from this blog. For more information, you may visit  Singular Value Decomposition (SVD) and Image Compression

Have a Great Learning 😊😊