Deep Learning
Deep learning is a subset of machine learning that uses neural networks with many hidden layers (hence "deep") to automatically extract hierarchical representations from raw data. Lower layers learn low-level features (edges, phonemes), while deeper layers combine them into increasingly abstract concepts (faces, words). Deep learning has revolutionized computer vision, natural language processing, and speech recognition, achieving human-level or superhuman performance on many benchmarks.
Major Deep Learning Architectures and Applications
| Architecture | Abbreviation | Key Innovation | Primary Application |
|---|---|---|---|
| Convolutional Neural Network | CNN | Shared weight filters | Image classification, detection |
| Recurrent Neural Network | RNN | Sequential hidden state | Time series, early NLP |
| Long Short-Term Memory | LSTM | Gated memory cells | Speech recognition, translation |
| Transformer | Transformer | Self-attention mechanism | Large language models (GPT, BERT) |
| Generative Adversarial Network | GAN | Generator vs discriminator | Image synthesis, deepfakes |
| Diffusion Model | DM | Iterative denoising | Stable Diffusion, DALL-E |
Interactive Tools
Wikimedia Commons, CC BY-SA
Related Terms
Neural Network
A neural network is a computational model loosely inspired by the structure of biological brains, consisting of layers of interconnected nodes (neurons) that process and transform data. Each neuron computes a weighted sum of its inputs, applies a non-linear activation function, and passes the result to the next layer. Neural networks are the foundation of modern AI and are capable of learning highly complex patterns in images, text, audio, and tabular data.
Backpropagation
Backpropagation (backward propagation of errors) is the algorithm used to train neural networks by efficiently computing the gradient of the loss function with respect to every weight in the network. It applies the chain rule of calculus in a reverse pass through the network — from the output layer back to the input layer — so that each weight can be updated in the direction that reduces the loss. Without backpropagation, training deep neural networks with millions of parameters would be computationally infeasible.
Machine Learning
Machine learning is a branch of artificial intelligence in which systems learn from data to improve their performance on tasks without being explicitly programmed for each task. It works by identifying statistical patterns in training data and using those patterns to make predictions or decisions on new, unseen data. Machine learning powers applications ranging from spam filters and recommendation engines to medical diagnosis and autonomous vehicles.
The adjective "deep" refers to the many layers (depth) in the network. The term "deep learning" was popularized by Geoffrey Hinton, Yann LeCun, and Yoshua Bengio from around 2006 onward, as they demonstrated that networks with many layers could be trained effectively using improved techniques.