Interactively visualizing timm learning rate schedulers

#pytorch #code #dashboards #learning-rate #timm #my-oss

Are you a PyTorch Image Models (timm) user? Want an easier way to explore learning rate scheduler parameters? I've created a lightweight interactive dashboard to help you visualize LR schedules clearly.

Here's an example using SGDR with warm restarts: 20250708-Interactively visualizing timm learning rate schedulers.png
It can be hard to understand how some parameters affect the schedule without visualizing them. For instance, see how decay_rate and k_decay affect the Poly scheduler:
20250708-Interactively visualizing timm learning rate schedulers-1.png
20250708-Interactively visualizing timm learning rate schedulers-2.png
20250708-Interactively visualizing timm learning rate schedulers-3.png

Or the non-obvious ways warm-up can affect an LR schedule:

(1) Vanilla cosine scheduler:
20250708-Interactively visualizing timm learning rate schedulers-4.png
(2) Adding warm-up:
20250708-Interactively visualizing timm learning rate schedulers-5.png
(3) Adding step_on_epochs=False:
20250708-Interactively visualizing timm learning rate schedulers-6.png
(4) Adding warmup_prefix=True:
20250708-Interactively visualizing timm learning rate schedulers-7.png

Check it out on GitHub. Stars appreciated! timm-lr-scheduler-explorer

Special thanks to PyTorch vision legend Ross Wightman for tirelessly maintaining timm.

Copyright Ricardo Decal. richarddecal.com