VAE中KL散度设计

之前训练结果

Zzz.png

KL散度消失

Generating Sentences from a Continuous Space

loss function of vae image

def kl_anneal_function(ann_function, step, k, x0):
    if ann_function == 'logistic':
        return float(1/(1+np.exp(-k*(step-x0))))
    elif ann_function == 'linear':
        return min(1, step/x0)

When
    k = 0.0025
    x0 = 2500
    ann_function = 'logistic'

image.png

When
    x0 = 2500
    ann_function = 'linear'

image.png 论文原图