Introduction
We know that as deep learning enthusiasts, we always try to improve or generate our models so as to solve real-world problems. Deep learning is an evolving concept, and there is so much to learn from it. One of the concepts that evolved from this research or learning is CycleGAN. We are already familiar with GANs. GANs are Generative Adversarial Networks.
These are the adversarial networks that are supposed to generate information such as images, music, etc. Well, this concept of GANs leads to more interesting insights to develop more similar kinds of models or concepts which perform more similar tasks. One of the concepts, CycleGANs, became very popular in this scenario. CycleGANs are the GANs that can convert the generated output back and forth to its original state. It is a kind of mapping of info from one state to another and vice versa. We will learn more about this in this article.
CycleGAN
CycleGANs are the Generative Adversarial Networks that are used to convert or generate output from one domain to another domain. For example, converting a horse to zebra, converting one season to another, etc. CycleGANs can do these things by using cycle consistency loss so as to enable training without the need for paired images/data.
Thus, we can say that this concept - CycleGANs can convert from one domain to another domain without directly mapping one-to-one from source to target domain.
Mathematically,\
Let’s say we are converting from source domain X to target domain Y. Then, the mapping can be shown as G: X - > Y.
Here the data of G(X) is indistinguishable from Y, this can be done using adversarial loss. As we said, this is a CycleGAN concept thus, the inverse can also be possible, can be shown as
F: Y - > X. Thus, we can conclude that F(G(X)) ≅ X.
The Architecture
The architecture of CycleGAN is simple and varies based on many values such as type of data, how accurate you are gonna translate from one domain to another, etc.
But, the overall architecture can be said as follows:
Here we can say that the AutoEncoder architecture involves a source domain data, a pool of convolutional networks, first a series of downsampling network layers that are used to pick only a few important features of domain X, and then upsampling layers used to encode the taken important info to convert to domain Y. This architecture is just for understanding purpose only, and this can be modified based on the use-case you are working on.
The Losses
We said, G is a Generator used to convert from domain X to domain Y, and F is a Generator to convert from domain Y to domain X. This can be represented as
Here the DX is a discriminator that gradually learns how to differentiate between X to generate image X F(Y). And similarly, DY.
The Loss that is calculated in between the generated image X F(Y) and original image Y is called Adversarial Loss. For example, from the architecture image, the difference between the image generated on the right side and the actual/original image similar to the right side can be termed Adversarial Loss.
This Adversarial Loss can be mathematically viewed as:
Here y, x is the real samples.
Similarly, Let’s say we have a sample x of domain X. Using a CycleGAN generator, we have converted it to y of domain Y. Then again, you have used another generator to convert y of domain Y back to domain Y X as x1. Now, the Loss that can be calculated between the original sample x of domain X, and the regenerated sample of the same domain X, i.e., x1, can be termed as Cycle- Consistency Loss.
This, the Cycle-Consistency Loss can be mathematically viewed as
Thus, we can conclude the overall loss can be calculated as,
Uses And Advantages
CycleGANs are mainly introduced to solve/clear the problem of image-image translation. As shown in the first figure. Thus, the ultimate use case of CycleGANs is image-image translation. Other applications of CycleGANs are as follows:
- Style Transfer: This application can be seen even before the development of this concept. The main intention of this application is to transfer one artistic style to another style. This can be simply said as transferring the style of one to another.
Example:
Here we are going to apply/transfer a style of Monet to our input image and the output as shown above.
- Object Transfiguration/Season Transfer: This application can be said as for a given image, we will convert this image to another domain image. For example, if orange is given, convert it to apple. And a seasoned picture to another season picture.
Example:
- PhotoGraph Generation/Photograph Enhancement: In this application, we will generate a photo from paintings or enhance the already existing photos by improving their ways.
For example:
34+ registered 
