blob: 723ec22bfa88438f53c18f5fef5110a5c0499494 (
plain) (
blame)
1
2
3
4
5
6
7
8
9
10
11
|
Schedulefree is a Schedule-Free optimizer in PyTorch.
We provide several Schedule-Free optimizer implementations:
* SGDScheduleFree and SGDScheduleFreeReference: Schedule-free variants of SGD
* AdamWScheduleFree and AdamWScheduleFreeReference: Schedule-free variants
of AdamW
* RAdamScheduleFree: Schedule-free variant of RAdam, which eliminates the need
for both learning rate scheduling and warmup (implementation community
contributed)
* Experimental ScheduleFreeWrapper to combine with other optimizers
|
