What is a GAN?

A Generative Adversarial Network (GAN) is a generative model that learns how to create new data samples that look similar to the training data.

Instead of directly learning by reconstruction like an autoencoder, it learns through competition.

One network tries to generate fake samples.

Another network tries to detect whether those samples are real or fake.

Because both models are trained against each other, the generator gradually becomes better at producing realistic outputs.

Main Components

In GANs, you have two players with different goals:

• Discriminator: Wants to maximize its chance of correctly identifying real vs. fake data, this uses gradient ascent.
• Generator: Wants to minimize the discriminator’s ability to detect fakes this uses gradient descent.

This creates a minimax game, balanced by both updates during training.

$$ \min_{\theta_g} \max_{\theta_d} \frac{1}{2} \mathbb{E}{x \sim p{data}} [\log D(x)] + \frac{1}{2} \mathbb{E}_{z \sim p(z)} [\log (1 - D(G(z)))] $$

Discriminator : Using gradient ascent, it maximizes the loss with respect to discriminator parameters

$$ \theta_d \leftarrow \theta_d + \alpha \nabla_{\theta_d} V(\theta_d, \theta_g) $$

Generator : Using gradient descent, it minimizes the loss with respect to generator parameter

$$ \theta_g \leftarrow \theta_g - \alpha \nabla_{\theta_g} V(\theta_d, \theta_g) $$

Analogy

You can think of GANs like a forger vs a detective.