Hebbian Learning
“Cells that fire together wire together” — synapses strengthen when pre- and post-synaptic activity coincide.
Introduced by Donald Hebb in The Organization of Behavior (1949). Underlies the naming of memory tools like Hebbian Vault.
Biological mechanism
- LTP (Long-Term Potentiation) — sustained synaptic strengthening after high-frequency stimulation; first reported by Bliss & Lømo (1973) in rabbit hippocampus.
- STDP (Spike-Timing-Dependent Plasticity) — a presynaptic spike ~10 ms before a postsynaptic spike produces potentiation; reverse order produces depression. Requires NMDA-type glutamate receptors and a postsynaptic Ca²⁺ rise.
- Synaptic tagging & capture — transient “tags” at activated synapses capture plasticity-related proteins, converting early to late LTP.
- Neuromodulation — dopamine, acetylcholine, norepinephrine gate plasticity by shifting LTP/LTD thresholds (context-dependent learning).
- Homeostasis — synaptic scaling acts as negative feedback to prevent runaway potentiation.
Formal rules
| Rule | Form | Property |
|---|---|---|
| Basic Hebb | Δw ∝ η·x·y | Correlational, but unstable without normalization |
| Oja’s rule | ẇ = ⟨yx⟩ − ⟨y²⟩w | Weight normalization → principal-component extraction |
| BCM rule | sliding threshold θ_M | Potentiate if y > θ_M, else depress; stable metaplasticity |
| STDP | exponential timing windows | Spiking-network online implementation via pre/post traces |
Computational models
- Associative memory — Hopfield networks via outer-product storage
W = Σ xᵘ(xᵘ)ᵀ(pattern completion) - ICA / source separation — (anti-)Hebbian rules maximize non-Gaussianity
- Sparse coding — anti-Hebbian decorrelation produces localized receptive fields
- Self-organizing maps — Hebbian-like updates with neighborhood functions preserve topography
AI applications
- Engram Neural Network (ENN) — stores an outer-product Hebbian memory trace; reports comparable performance to RNN baselines on MNIST/CIFAR-10 with faster training.
- Neuromorphic hardware — BrainScaleS implements Hebbian/STDP in analog/digital hardware with large speedups over biological real-time.
- Open-source — PyTorch-Hebbian libraries enable reproducible Hebbian training of conv layers.
vs. gradient-based learning
Hebbian/local plasticity is biologically plausible and intrinsically local, but struggles to propagate supervised teacher signals through deep hierarchies (deep credit assignment). Hybrid schemes — e.g. Random Feedback Local Online (RFLO), meta-learned plasticity — combine local eligibility traces, feedback projections, and modulatory signals to approximate structured credit assignment while retaining locality.
Open questions
- No systematic large-scale benchmarks of Hebbian vs. gradient-based deep learning on modern tasks.
- How complex neuromodulatory states implement task-level gating across cortical hierarchies remains uncharacterized.
Researched via Tavily tvly research, 2026-05-30.