What is Gradient Descent?

Gradient descent is an iterative optimization algorithm that finds the minimum of a function by repeatedly moving in the direction of steepest descent (negative gradient). It is the fundamental algorithm behind training machine learning models.