Artificial Intelligence

李宏毅GAN学习笔记（1）

2018-11-10 本文已影响0人 SmallRookie

GAN Algorithm

Initialize $\theta_d$ for D and $\theta_g$ for G

In each training iteration:
- Sample m examples $\{ x^1, x^2, \dots, x^m \}$ from database
- Sample m noise samples $\{ z^1, z^2, \dots, z^m \}$ from a distrubution
- Obtaining generated data $\{ \widetilde{x}^1, \widetilde{x}^2, \dots, \widetilde{x}^m \}, \, \widetilde{x}^i=G(z^i)$
- Update discriminator parameters to maximize
  - $\widetilde{V}=\frac{1}{m}\sum^m_{i=1}logD(x^i)+\frac{1}{m}\sum^m_{i=1}log(1-D(\widetilde{x}^i))$
  - $\theta_d \gets \theta_d+\eta\bigtriangledown_{\widetilde{V}(\theta_d)}$
- Sample m noise samples $\{z^1, z^2, \dots, z^m \}$ from a distribution
- Update generator parameters to maxmize
  - $\widetilde{V}=\frac{1}{m}\sum_{i=1}^mlog(D(G(z^i)))$
  - $\theta_g\gets\theta_g+\eta\bigtriangledown_{\widetilde{V}(\theta_g)}$

Structured Learning Approach

Bottom Up: Learn to generate the object at the component level
Top Down: Evaluating the whole object, and find the best one

$Bottom \; Up + Top \; Down \Rightarrow GAN$

Discriminator --- Training

General Algorithm
- Given a set of positive example, randomly generate a set of negative examples.
In each iteration
- Learn a discriminator D that can discriminate positive and negative examples.
- Generate negative examples by discriminator D: $\widetilde{x}=arg max_{x\in X}D(x)$

Generator v.s. Discriminator

Generator
- Pros:
  - Easy to generate even with deep model.
- Cons:
  - Imitate the appearance.
  - Hard to learn the correlation between components.

Discriminator
- Pros:
  - Considering the big picture.
- Cons:
  - Generation is not always feasible, especially when your model is deep.
  - How to do negative sampling?

Discriminator + Generator --- Training

General Algorithm
- Given a set of positive example, randomly generate a set of negative examples.
In each iteration
- Learn a discriminator D that can discriminate positive and negative examples.
- Generate negative examples by discriminator D: $\widetilde{x}=arg max_{x\in X}D(x) \,\Leftrightarrow\, G\to\widetilde{x}$

上一篇下一篇

猜你喜欢

热点阅读