Wasserstein GAN

A variant of the generative adversarial networkgenerative adversarial network, n.: Two neural networks trained as rivals: one forges fake images, the other tries to spot the fakes. As both improve, the forger's output becomes realistic enough to fool people. that trains the generator to minimize an estimate of the Wasserstein distanceWasserstein distance, n.: A way of comparing two spreads of probability by imagining one as heaps of sand that must be shoveled into the shape of the other: the less shoveling, weighted by distance, the more alike they are. (earth mover's distance) between the data and model distributions instead of the Jensen–Shannon-type divergenceJensen–Shannon divergence, n.: A balanced, two-way measure of how different two sets of probabilities are, comparing each with their average. Unlike its better-known cousin, it gives the same answer in both directions. implicit in the original GAN. The discriminator becomes a 'critic' that outputs an unbounded real-valued score; by Kantorovich–Rubinstein duality, the gap between its mean scores on real and generated samples estimates the distance, provided the critic is 1-Lipschitz. Because this distance stays informative even when the two distributions barely overlap, the generator receives useful gradients throughout training, and the critic's loss tracks sample quality. The original paper enforced the Lipschitz constraint by clipping weights; WGAN-GP replaced clipping with a penalty that keeps the norm of the critic's input gradient near one, and spectral normalization later offered another route.