Implementation for FP8/INT8 Rollout for RL training without performence drop.
Stars
299
Forks
22
Watchers
299
Open Issues
8
Overall repository health assessment
No package.json found
This might not be a Node.js project
68
commits
5
commits
2
commits
1
commits
1
commits
Backup weight_scale in cpu memory and restore them after reloading, because all weights will be set as zero by sglang if enable_memory_saver is set as True.
433a0e0View on GitHubClone existing params instead of creating empty tensors to fix the bug of qkv_weight caused by loading qkv respectively.
be4c72cView on GitHubClean python and torch cache only once per updating weights.
ebbb260View on GitHubFinish patched load_weights_and_postprocess, the program can run now.
12fc6dbView on GitHubPatch load_weights_and_postprocess of sglang; Bugs exist.
fcb0ce3View on GitHubadd load_column_parallel_weight to recorded_loader_keys
4eefac8View on GitHub