Code360 powered by Coding Ninjas X Naukri.com. Code360 powered by Coding Ninjas X Naukri.com
Table of contents
1.
Introduction
2.
What is Scale-Invariant Feature Transform (SIFT)?
3.
Four steps of Scale-Invariant Feature Transform (SIFT)
3.1.
Scale-space extrema selection
3.2.
Keypoint Localization
3.3.
Orientation Assignment
3.4.
Keypoint Descriptor
4.
Advantages of SIFT
5.
Frequently Asked Questions
5.1.
How can SIFT be implemented in our code?
5.2.
Are there any other algorithms like SIFT?
5.3.
Which is better, SIFT or SURF?
5.4.
What are the Steps of SIFT Features?
6.
Conclusion
Last Updated: May 20, 2024
Easy

Scale-Invariant Feature Transform (SIFT)

Introduction

Object recognition is a computer vision technology to identify objects in images and videos. Humans are nearly perfect in object recognition. We can easily recognize objects regardless of what angle we are looking at them. It doesn’t matter what their size or the scenery behind them is. Computers, however, struggle to do the same. It is very challenging for a computer to recognize an object if we change its features like angle or scale. This article will discuss an interesting technique known as scale-invariant feature transform (SIFT) that helps overcome this problem.

Scale-Invariant Feature Transform (SIFT)

What is Scale-Invariant Feature Transform (SIFT)?

SIFT is an algorithm that detects and matches the features of an image. The main idea behind SIFT is to extract key points (features) from a set of reference images and store them in a database. To identify an object in a new image, we compare the features of the new image from the database.

Get the tech career you deserve, faster!
Connect with our expert counsellors to understand how to hack your way to success
User rating 4.7/5
1:1 doubt support
95% placement record
Akash Pal
Senior Software Engineer
326% Hike After Job Bootcamp
Himanshu Gusain
Programmer Analyst
32 LPA After Job Bootcamp
After Job
Bootcamp

Four steps of Scale-Invariant Feature Transform (SIFT)

  1. Scale-space extrema selection: It is the first step of SIFT algorithm. The potential interest points are located using difference-of-gaussian.
  2. Keypoint localization: A model is fit to determine the location and scale at each potential location. Keypoints are selected based on their stability.
  3. Orientation assignment: orientations are assigned to keypoints locations based on local image gradient direction.
  4. Keypoint descriptor: It is the final step of SIFT algorithm. A coordinate system around the feature point is created that remains the same for the different views of the feature.
overall flow of sift computation

Scale-space extrema selection

Constructing the scaled space:

We first identify the candidate locations for the key points that are processed and examined in the succeeding steps. Detecting locations invariant to scale change can be done by searching the features that remain the same at different scales.

The scale-space of an image is defined as a function, L(x, y, σ), formed by performing a convolution operation of a variable-scale gaussian function, G(x, y, σ), on an input image, I(x, y). The variable-scale gaussian function is defined by:

And the scale space is defined by:

Where * is the convolution operation on x and y.

To detect the stable keypoint in scale space, we use the difference of gaussian, which is defined as:

   

Scale-space extrema selection

Finding keypoints:

After calculating the Difference of Gaussians, we calculate the Laplacian of Gaussian approximations that are scale-invariant. Each pixel is compared with its eight neighboring pixels and 9 pixels in the following scale and 9 pixels in the previous scale. It is selected if it is larger  or smaller than all these neighbors. If it’s a local extremum, it is potentially a key point.
 

Finding keypoints

Keypoint Localization

We get a lot of key points from the previous steps. Some of the key points are on the edges, and some of the key points have low contrast. These features are not as much important. 

To remove the low contrast features, we check the intensities of each feature and remove the features that have intensities lower than the threshold value.           

Keypoint Localization

Orientation Assignment

After getting useful key points, we assign an orientation to each key point to make it orientation invariance. We use histogram of oriented gradient to achieve this.

Orientation Assignment

Keypoint Descriptor

It is the final step of SIFT algorithm. After getting key points that are described according to their location, scale, and orientation, we create a coordinate system around the feature point that remains the same from the different views of the feature.

To do this, we take a 16X16 window around the key point. The window is divided into sixteen 4X4 blocks.

Keypoint Descriptor

For each 4X4 window, an 8 bin orientation histogram is created.

8 bin orientation histogram

Keypoint descriptor

 

Matching the keypoints:

After we get the keypoint descriptor, the key points between the two images are matched by identifying their neighboring key points from the database of key points that we get from training.

Also see, Sampling and Quantization

Advantages of SIFT

  • Scale Invariance: SIFT (Scale-Invariant Feature Transform) effectively detects and describes image features at different scales.
  • Robustness: Performs well under varying lighting conditions, rotations, and partial occlusions.
  • Distinctiveness: Generates highly distinctive features that enable accurate matching across different images.
  • Large Number of Features: Identifies a large number of key points, enhancing the reliability of image matching.
  • Efficient Matching: Facilitates efficient matching and recognition in large datasets due to its detailed feature descriptors.
  • Wide Applicability: Used extensively in applications like image stitching, object recognition, and 3D modeling due to its versatility.

Frequently Asked Questions

How can SIFT be implemented in our code?

We can use OpenCV to implement SIFT.

Are there any other algorithms like SIFT?

Yes, there are other algorithms like SIFT. For example, SURF.

Which is better, SIFT or SURF?

SURF is more accurate than SIFT.

What are the Steps of SIFT Features?

The steps of SIFT features include:

  1. Scale-space Extrema Detection: Identify key points at different scales.
  2. Keypoint Localization: Refine the location and scale of key points.
  3. Orientation Assignment: Assign orientations to key points.
  4. Keypoint Descriptor: Generate descriptors for key points.

Conclusion

In this article, we talked about the SIFT algorithm. We talked about its need and the overall process of how it is carried out. You can add your own image and get its key points. You can read the original paper here. To learn more about similar topics, check out our machine learning course at coding ninjas.

Previous article
Image Pyramids
Next article
Speeded Up Robust Features (SURF)
Live masterclass