Gradient Descent

Definition

Gradient descent is the most fundamental optimization algorithm in machine learning. It iteratively updates parameters along the negative gradient of the loss function to minimize the loss value. It is the core method for training neural networks.

Usage Example

When training an image classification model, use stochastic gradient descent (SGD) to iteratively optimize the network weights.

Related Terms

BackpropagationLearning rateSGD

760View

Related terminology

Reinforcement Learning

Reinforcement Learning (RL) is one of the three core machine learning paradigms. It learns optimal policies through agent-environment interaction, where agents continuously adjust their strategies based on reward feedback to maximize cumulative rewards.