Image Segmentation Techniques

What are the applications of Image Segmentation?

Image Segmentation has significant applications in computer vision. The most-commonly used applications of image segmentation are as follows:

1. Face Recognition
2. Medical Imaging
3. Image Based Search
4. Number Plate Identification

What are the various classifications for image segmentation?

Approach Based Classification

In this classification, we implement a direct approach of subdivisioning an image into various parts, which is essentially separating similar pixels or a collection of pixels from dissimilar pixels to differentiate the image objects. This classification is further divided into:

Region-Based Approach

This approach uses the technique of detecting similarity between pixels according to a set threshold value, region merging, region spreading and region growing.

                Source: Link

From this figure, we can understand that a seed pixel is selected as an initial pivot point and then similar as well as neighbouring pixels are detected using the region growing algorithm. After classifying the neighbouring pixels, we can differentiate between the regions efficiently.

Boundary-Based Approach

In this approach, the distinct pixels of an image are identified to differentiate the background and surrounding of an object from the real object itself. 

            Source: Link

In boundary-based image segmentation, the primary goal is to identify the boundaries using edge detection techniques and then ultimately differentiate the image object from the background.

Technique Based Classification

In this classification, the image data is used to separate the image objects. The techniques implemented for segmenting the image based on this approach are:

Structural Technique

This technique requires the structural data of the image for processing the pixels, distributions, pixel density and other relevant information. It is also a prerequisite that the structural data of the region from which the object is to be segmented from must be known. This technique has explicit use in the segmentation of medical images as the structural information such as texture, features, density of the medical data can be easily retrieved from these image data sets.

Stochastic Technique

In this method, the information of the discrete pixel values of the image are required. Generally, this technique is implemented in the clustering algorithm as discrete values of an image object can be clustered together to be identified as a whole.

image widget

Source: Link

Here, the discrete values of the spiral are clustered together to segment different portions of the spiral.

Hybrid Technique

In this technique, both the structural data of the image as well as the discrete pixel values are required for the segmentation.

What are the different types of Image Segmentation Techniques?

Thresholding

Thresholding is the process of segmenting objects from their background in an image by specifying a particular threshold value to distinguish the two groups of pixels in an image. Thresholding takes into account the pixel intensities, and then the threshold value can either be set manually or automatically. 

The selected threshold is represented using T. The pixels having values lower than the threshold value are assigned 0, which means they are in the background, and the pixels with values greater than the threshold are assigned 1, which means they represent the image object or the foreground pixel.

In global thresholding, the threshold value is assigned in the beginning and cannot be changed. 

Source: Link

The threshold value can also be assigned using the Otsu Method, developed by Nobuyuki Otsu. This method automatically assigns the threshold value according to the value of pixels in the image. For implementing this method, we can follow the following code

from skimage.filters import threshold_otsu
thresh = threshold_otsu(sample_g)
sample_ot = sample_g > thresh

You can also try this code with Online Python Compiler

Run Code

Let us see an example of segmentation using thresholding. The following grayscale image is segmented by setting the threshold value manually.

#experimented threshold values
sample_t = sample_g>0.70
sample_t1 = sample_g>0.50

fig, ax = plt.subplots(1,3,figsize = (15,5))
im = ax[0].imshow(sample_g, cmap='gray')
fig.colorbar(im,ax=ax[0])
ax[1].imshow(sample_t, cmap='gray')
ax[0].set_title('Grayscale image',fontsize=15)
ax[1].set_title('Threshold at 0.70',fontsize=15)

ax[2].imshow(sample_t1,cmap='gray')
ax[2].set_title('Threshold at 0.50',fontsize=15)
plt.show()

You can also try this code with Online Python Compiler

Run Code

                 Source: Link

After setting the value using Otsu Method, we have the following image segmentation

                    Source: Link

Region-Based Segmentation

In this type of segmentation, we divide the image into regions with similar attributes and properties. Initially, a seed point is located, and then the group of pixels around it are identified with matching attributes; hence, this region is segmented.

The region-based algorithm adds pixels with similar properties to the seed point and increases the region's size or shrinks the region by removing the dissimilar points. The algorithm also merges adjacent regions based on the seed points.

In the Region Growing method, we initially locate a pivot pixel known as the seed point and start with a comparatively small set of pixels and merge pixels according to the matches with neighbouring pixels. When a region cannot grow further, the algorithm jumps onto the next seed point and starts iterating the adjacent pixels. In order to avoid the growth of a single region to a large scale due to the presence of multiple similar attributes, the regions are grown simultaneously.

            Source: Link

Here, the region growing algorithm is implemented and after the selection of seed points for each region, the algorithm iteratively adds neighbouring pixels and the regions are grown simultaneously.

In the Region Splitting and Merging method, the entire image is considered at once, and the pixels are merged together into a single region or split with respect to the entire image. It follows the divide-and-conquer technique by first dividing the portions of an image and then merging them together if they share similar attributes.

      Source: Link

Edge-Based Segmentation

The Edge-Based Segmentation technique tracks the edges of the image objects and then differentiates the objects from the background based on these edges. First, we obtain the edge map and fill the holes in the detected edges using morphology, and then we separate the entire object. For detecting the edge maps, we make use of edge detection kernels, for example, the Canny Edge Detector. 

After detecting the edges, the objects are also filled in with the differentiating values of 0 and 1; thus, we obtain a segmented image. Let us see this visualisation using the following image

Source: Link

Clustering-Based Segmentation

This method falls under the category of segmentation using artificial intelligence. This method uses unsupervised learning, and clusters are formed in the image that share similar features. It separates the image elements from one cluster to another, and by doing it for the entire image, we obtain differentiated objects.

In image segmentation, the most commonly used algorithm is the K-means clustering algorithm. In K-means clustering, the image is divided into k pixels representing k group centroids. Then each object is assigned to the group on the basis of the distance it has from the centroid. When all the clusters are assigned with pixels, then the centroids can be moved and reassigned. 

Another method used is Mean-Shift Clustering. This method is also a centroid-based technique and is based on Kernel Density Estimation and additionally has a bandwidth parameter. KDE is the process of estimating the probability density function for a set of data points. In this method, we do not specify the number of clusters beforehand, they are generated by the algorithm itself based on the image data. 

Neural Network Segmentation

This method is also an implementation of artificial intelligence for image segmentation. Here we make use of Convolutional Neural Networks as they function very efficiently in identifying and processing images. In fact, they are generally the most suitable networks to work with medical images. 

The image is passed to a convolutional neural network, and a feature map for the particular image is generated. Then the region proposal network is applied to the feature maps, and object proposals along with objectness scores are generated. Following this step, pooling layers are applied to the proposals. Finally, the image is passed through the connected layer for classification, and the output with bounding boxes for each object is generated. In this way, the final obtained image is segmented into different components of objects it consists of.

Interactive Image Segmentation technique makes use of CNN for separating a target object (or foreground) from the background. A target object is annotated by a user in the type of bounding box or scribble and then is extracted as a binary mask. This process is followed by backpropogation which conveys the data through network layers backwardly. The user annotations are converted into interaction maps, by computing the distance of each pixel to the closest user-annotated foreground and background pixels. Following this, the network yields a probability map of the target object and the image is segmented into foreground and background.

Also read, Sampling and Quantization

Frequently Asked Questions

What is an image object?

The image object is a part/portion of the image that shares similar properties with other parts of the image. It essentially is a part of the object which needs to be identified using image segmentation.

Can thresholding be variable with continuous change in the value of T over an image?

Yes, the value of T can change over the image, depending on the neighbourhood of the pixel and if T is a function of (x,y), where x and y represent the pixel coordinates.

Which algorithm should be preferred if there is noise in the image?

Region Growing algorithm works efficiently if there is noise in the image because the noise makes it difficult to identify edges or use threshold values.

Conclusion

In this blog, we discussed Image Segmentation and its various techniques. We talked about the different approaches to image segmentation and understood the implementation of each technique in detail. We also learned the significance of image segmentation in applications like medical imaging and face detection. We hope you enjoyed this blog and will consider implementing some of these techniques yourself. 

Refer to our Guided Path on Coding Ninjas Studio to upskill yourself in Data Structures and Algorithms, Competitive Programming, JavaScript, System Design, Machine learning and many more! If you want to test your competency in coding, you may check out the mock test series and participate in the contests hosted on Coding Ninjas Studio! But if you have just started your learning process and are looking for questions asked by tech giants like Amazon, Microsoft, Uber, etc; you must look at the problems, interview experiences, and interview bundle for placement preparations.

Nevertheless, you may consider our paid courses to give your career an edge over others!

Do upvote our blogs if you find them helpful and engaging!

Happy Learning!!