Deep Learning Terminology
Neural Network
A computational model inspired by the structure and function of the human brain, consisting of interconnected nodes (neurons) organized in layers. It learns by adjusting the weights of these connections.
Also known as: Artificial Neural Network (ANN)Neuron
The basic processing unit of a neural network. It receives input signals, processes them, and transmits an output signal, often after applying an activation function.
Key component of ANNs.Activation Function
A function that determines the output of a neuron based on its input. It introduces non-linearity into the network, enabling it to learn complex patterns.
Examples: ReLU, Sigmoid, TanhBackpropagation
An algorithm used to train neural networks by iteratively adjusting the weights of connections based on the error gradient calculated through the network.
Core training mechanism.Convolutional Neural Network (CNN)
A type of neural network particularly effective for image and video recognition tasks. It uses convolutional layers to automatically learn spatial hierarchies of features.
Specialized for visual data.Recurrent Neural Network (RNN)
A type of neural network designed for sequential data, such as text or time series. It has internal memory that allows it to process sequences of inputs.
Handles sequential data.Overfitting
A phenomenon where a model learns the training data too well, including its noise and outliers, leading to poor performance on unseen data.
A common training challenge.Underfitting
Occurs when a model is too simple to capture the underlying trends in the data, resulting in poor performance on both training and test sets.
Model lacks complexity.Loss Function
A function that quantifies the error between the predicted output of a model and the actual target value. The goal of training is to minimize this function.
Measures model error.Gradient Descent
An optimization algorithm used to find the minimum of a function. In deep learning, it's used to adjust model parameters to minimize the loss function.
Optimization technique.Epoch
One complete pass through the entire training dataset during the training of a neural network.
Unit of training iteration.Batch Size
The number of training examples used in one iteration of the training process. The dataset is divided into batches.
Controls training updates.Hyperparameter
A parameter whose value is set before the learning process begins. It controls the learning process itself (e.g., learning rate, number of layers).
Configuration settings.Feature Extraction
The process of transforming raw data into features that better represent the underlying problem to predictive models, often done automatically by deep learning models.
Learning data representations.Deep Learning
A subfield of machine learning that uses artificial neural networks with multiple layers (deep architectures) to learn complex patterns from data.
Multi-layered learning.