TL;DR: A Variational Autoencoder (VAE) encodes data as probability distributions in a continuous latent space rather than fixed points. This probabilistic approach enables smooth interpolation, meaningful semantic directions, and robust generative sampling—unlike standard autoencoders where random latent samples often produce noise.
What VAEs Do Differently
Standard autoencoders map each input to a single fixed latent vector, leaving the space between points undefined—sampling random points produces garbage. VAEs instead make the encoder output a probability distribution [1], specifically a mean (μ) and variance (σ) for each latent dimension. The decoder then samples from this distribution and reconstructs the input.
This probabilistic structure forces the decoder to handle variation, learning a smooth, well-structured latent space where nearby points decode to similar outputs [2].
The Architecture
The system has three parts: [1]
- Encoder: Maps input to latent parameters (μ, σ)
- Latent Space: The “playground for probabilities” where distributions exist [2]
- Decoder: Reconstructs or generates new data by sampling from the learned distribution
The reparameterization trick enables gradient flow through stochastic sampling: sample from a standard normal, then scale and shift by learned parameters. This makes the model differentiable [1].
Training: Two Loss Terms
The training objective combines two competing goals:
- Reconstruction Loss: Penalize differences between original and reconstructed data
- KL Divergence: Push the encoder’s learned distribution toward a unit Gaussian prior [1]
This balance ensures the latent space is both useful (faithful reconstruction) and regular (smooth, generative) [2].
Practical Advantages
- Smooth interpolation: Moving through latent space produces realistic intermediate samples [1]
- Generative sampling: New data points can be created by sampling latent vectors
- Interpretable structure: Similar data points cluster together; specific latent directions often encode meaningful attributes (e.g., rotation, color) [3]
- Anomaly detection: Points far from the learned distribution stand out as anomalies [3]
Applications
Image generation and editing [4], drug discovery [4], and representation learning for downstream tasks. Hierarchical VAEs can compress images to semantic features via a standard autoencoder, then layer a VAE on top to model structural variation efficiently [5].