deepspeed
https://github.com/microsoft/deepspeed
Python
DeepSpeed is a deep learning optimization library that makes distributed training easy, efficient, and effective.
Triage Issues!
When you volunteer to triage issues, you'll receive an email each day with a link to an open issue that needs help in this project. You'll also receive instructions on how to triage issues.
Triage Docs!
Receive a documented method or class from your favorite GitHub repos in your inbox every day. If you're really pro, receive undocumented methods or classes and supercharge your commit history.
Python not yet supported6 Subscribers
Add a CodeTriage badge to deepspeed
Help out
- Issues
- enable phi3_mini autotp
- [BUG] Jamba (Mamba+MoE) + ZeRO3 + LoRA training hangs
- [BUG] 3 GPUs is not as good as expectation compare with 2 GPUs; NV vs AMD performace; flash attention not support for AMD GPUs
- Fused adam for HPU
- [REQUEST] Any arguments for disabling saving global steps?
- [BUG] Training crashes with "'Tensor' object has no attribute 'ds_id'"
- [BUG] Memory Leak in Stage 2 Optimizer
- [BUG] import deepspeed, MissingCUDAException
- [REQUEST] Add documentation on how to run fast inference of `transformers` models with ZeRO-3
- Update to ROCm6
- Docs
- Python not yet supported