Autoencoder

An autoencoder is a type of neural network trained to copy its input to its output — but with a crucial bottleneck in the middle. This bottleneck forces the network to learn a compressed representation of the data, capturing only the most important features.

How autoencoders work

An autoencoder has two parts:

• Encoder — compresses the input into a smaller, dense representation called the latent space
• Decoder — reconstructs the original input from this compressed representation

The network is trained by measuring how well the reconstruction matches the original. Over time, it learns which features to keep and which to discard.

Types of autoencoders

• Vanilla autoencoders learn basic compression and reconstruction
• Variational autoencoders (VAEs) add randomness to the latent space, making them useful for generating new data similar to the training examples
• Denoising autoencoders are trained on corrupted inputs and learn to reconstruct clean versions, making them effective at removing noise from data
• Sparse autoencoders add constraints that force only a few neurons to activate at once, producing more interpretable features

Practical applications

• Anomaly detection — train on normal data, then flag inputs that reconstruct poorly as potential anomalies (useful in fraud detection and manufacturing quality control)
• Dimensionality reduction — compress high-dimensional data for visualisation or faster downstream processing
• Image denoising — clean up noisy images in medical imaging or photography
• Feature learning — extract meaningful features from raw data for use in other models

Autoencoders vs. generative models

While variational autoencoders can generate new data, they have largely been superseded by diffusion models and GANs for generation tasks. However, autoencoders remain valuable for compression, anomaly detection, and representation learning.