What is an Activation Function?
An activation function introduces non‑linearity into a neural network, allowing it to learn complex patterns. Different functions have different properties and are suited to various tasks.
Explore Functions
Formulas
- Sigmoid: σ(x) = 1 / (1 + e⁻ˣ)
- Tanh: tanh(x) = (eˣ – e⁻ˣ) / (eˣ + e⁻ˣ)
- ReLU: f(x) = max(0, x)
- Leaky ReLU: f(x) = max(0.01·x, x)
- ELU: f(x) = x (x ≥ 0) else α(eˣ – 1) with α=1.0