What is Learning Rate Schedules?

Decay, warmup, and cyclical rates. Learning Rate Schedules is a key concept in 4 that forms the mathematical foundation for many machine learning algorithms. Prerequisites include: 33, 36. This builds on foundational concepts and requires some background knowledge.