From single neurons to the architecture of thought
Speed: 1–120 m/s along myelinated axons
Duration: ~1–2 milliseconds
Refractory period: ~2ms (can't fire again)
Brain neurons: ~86 billion
Synapses: ~100 trillion connections
Output: 0–1. Used in output layer for probability. Problem: vanishing gradients in deep networks.
Output: 0 to ∞. Default for hidden layers. Fast, sparse activations. Dead neuron problem below 0.
Click a region to learn about its functions.
Click a brain region on the diagram to see its functions.
Miller's Law: humans can hold 7 ± 2 items in working memory (chunking helps). Working memory is limited but is the "workspace" of conscious thought.
The brain reorganizes neural connections throughout life. Learning creates new synapses; disuse weakens them. "Neurons that fire together, wire together."
During sleep, the hippocampus replays memories and transfers them to cortical long-term storage. REM sleep consolidates procedural and emotional memories.