DeepSpeed

What's Changed

• Update version post release by @loadams in https://github.com/deepspeedai/DeepSpeed/pull/8070
• Validate fp16 dynamic loss scaling parameters are positive by @aryanputta in https://github.com/deepspeedai/DeepSpeed/pull/8050
• Add merge queue DCO workflow by @tohtana in https://github.com/deepspeedai/DeepSpeed/pull/8086
• Stop obsolete CI jobs on workflow cancellation by @tohtana in https://github.com/deepspeedai/DeepSpeed/pull/8034
• Fix ZeRO-3: Use per-param dtype for output buffers in _allgather_params_coalesced by @albertvillanova in https://github.com/deepspeedai/DeepSpeed/pull/8073
• fix: close file descriptor in deepspeed_io_handle_t::wait() to prevent fd leak by @MarkCLChang in https://github.com/deepspeedai/DeepSpeed/pull/8075
• Add type hints to top-level public API functions by @arunshar in https://github.com/deepspeedai/DeepSpeed/pull/8079
• Default gradient_clipping to 1.0 by @sfc-gh-truwase in https://github.com/deepspeedai/DeepSpeed/pull/8068
• activation_checkpointing: default num_layers to None so configure() assert fires by @Kymi808 in https://github.com/deepspeedai/DeepSpeed/pull/8041
• Add configurable engine log level by @sfc-gh-truwase in https://github.com/deepspeedai/DeepSpeed/pull/8067
• [CI] diff driven test selection by @stas00 in https://github.com/deepspeedai/DeepSpeed/pull/8077
• Enable bf16 check_grad_overflow by default (matching fp16) by @yongzhe-wang in https://github.com/deepspeedai/DeepSpeed/pull/8035
• Support AutoEP with ZeRO-3 zero.Init source modules by @tohtana in https://github.com/deepspeedai/DeepSpeed/pull/8060
• Update publication list in README.md by @PKUWZP in https://github.com/deepspeedai/DeepSpeed/pull/8095
• fix: AutoTP partition_config uses full hierarchical module path by @delock in https://github.com/deepspeedai/DeepSpeed/pull/8088
• Reject Muon optimizer with reduce_scatter in ZeRO-1/2 by @whycoming in https://github.com/deepspeedai/DeepSpeed/pull/8090
• Fix DeepCompile profile metadata backfill by @tohtana in https://github.com/deepspeedai/DeepSpeed/pull/8094
• Add numerical-correctness test for Muon under ZeRO-1/2 by @whycoming in https://github.com/deepspeedai/DeepSpeed/pull/8091
• Don't swallow KeyboardInterrupt/SystemExit in NPUOpBuilder by @ajinkyajawale14499 in https://github.com/deepspeedai/DeepSpeed/pull/8096
• Avoid CUDA context initialization during op compatibility checks at import by @Achyuthan-S in https://github.com/deepspeedai/DeepSpeed/pull/8078
• [DeepCompile] fix gather params in dynamo skipped frames for ZeRO3 by @XAheli in https://github.com/deepspeedai/DeepSpeed/pull/8059
• Feat: zero3 deprecate elastic checkpoint by @nathon-lee in https://github.com/deepspeedai/DeepSpeed/pull/8099
• Fix eigenvalue parsing for compression-only quantize configs by @sowndappan5 in https://github.com/deepspeedai/DeepSpeed/pull/8057
• fix: log eigenvalue monitor values by @he-yufeng in https://github.com/deepspeedai/DeepSpeed/pull/8049
• Fix DeepCompile profiling memory cleanup by @tohtana in https://github.com/deepspeedai/DeepSpeed/pull/8106
• Add optional torchembed RoPE backend to apply_rotary_pos_emb by @py-ai-dev in https://github.com/deepspeedai/DeepSpeed/pull/8052
• Fix minor comment/docstring typos in runtime and inference modules by @nathon-lee in https://github.com/deepspeedai/DeepSpeed/pull/8046
• feat(zenflow): run the overlapped CPU optimizer in a native process by @Antlera in https://github.com/deepspeedai/DeepSpeed/pull/8058
• fix: add buffer-length check in shm.cpp by @orbisai0security in https://github.com/deepspeedai/DeepSpeed/pull/8082
• Fix FlopsProfiler crash when dp_world_size is None under sequence parallelism by @vineethsaivs in https://github.com/deepspeedai/DeepSpeed/pull/8122
• Fix Python 3.9 import-time TypeError in AutoEP ep_router by @vineethsaivs in https://github.com/deepspeedai/DeepSpeed/pull/8119
• Add input validation for save_checkpoint and warmup LR schedulers by @jahnavi-yelamanchi in

License Apache 2.0

PyPI version

Downloads

Build

OpenSSF Best Practices

Twitter

Japanese Twitter

Chinese Zhihu

Slack

<div align="center">

<img src="docs/assets/images/DeepSpeed_light.svg#gh-light-mode-only" width="400px">

<img src="docs/assets/images/DeepSpeed_dark_transparent.svg#gh-dark-mode-only" width="400px">

</div>

Office Hours

DeepSpeed hosts regular office hours on the last Tuesday of each month at 12:00 America/New_York to discuss development plans, features, etc. This meeting is public for anyone to join and ask questions.

The meeting is hosted on Zoom and can be joined here.

Latest News

• [2026/05] Using Muon Optimizer with DeepSpeed

• [2026/05] System DMA (SDMA) for ZeRO-3: offload collectives off compute units on AMD GPUs for better overlap

• [2026/03] DeepSpeed Team gave a tutorial at ASPLOS 2026 titled "Building Efficient Large-Scale Model Systems with DeepSpeed: From Open-Source Foundations to Emerging Research"

• [2026/03] Our SuperOffload work received an Honorable Mention for the ASPLOS 2026 Best Paper Award

• [2025/12] DeepSpeed Core API updates: PyTorch-style backward and low-precision master states

• [2025/11] DeepSpeed ZeRO++ powers large-scale distillation training of LLMs for Recommendation Systems at LinkedIn

• [2025/10] We hosted the Ray x DeepSpeed Meetup at Anyscale. We shared our most recent work on SuperOffload, ZenFlow, Muon Optimizer Support, Arctic Long Sequence Training and DeepCompile. Please find the meetup slides here.

• [2025/10] SuperOffload: Unleashing the Power of Large-Scale LLM Training on Superchips

• [2025/10] Study of ZenFlow and ZeRO offload performance with DeepSpeed CPU core binding

• [2025/08] ZenFlow: Stall-Free Offloading Engine for LLM Training

• [2025/06] Arctic Long Sequence Training (ALST) with DeepSpeed: Scalable And Efficient Training For Multi-Million Token Sequences

• [2025/06] DeepNVMe: Affordable I/O scaling for Deep Learning Applications

<!-- NOTE: we must use html for news items otherwise links will be broken in the 'more news' section -->

<details>

<!-- NOTE: Maintain only three items in 'more news' section -->

<summary>More news</summary>

<ul>

<li>[2025/04] <a href="https://github.com/deepspeedai/DeepSpeed/blob/master/blogs/deepcompile/README.md">DeepCompile: Unlocking Compiler Optimization for Distributed Training</a></li>

<li>[2025/03] <a href="https://github.com/deepspeedai/DeepSpeed/blob/master/blogs/huggingface-tp/README.md">DeepSpeed AutoTP: Automatic Tensor Parallel Training of Hugging Face models</a></li>

<li>[2024/12] <a href="https://github.com/deepspeedai/DeepSpeed/blob/master/blogs/ulysses-offload/README.md">Ulysses-Offload: Democratizing Long Context LLM Training</a></li>

</ul>

</details>

---

Extreme Speed and Scale for DL Training

DeepSpeed enabled the world's most powerful language models (at the time of this writing) such as MT-530B and BLOOM. DeepSpeed offers a confluence of system innovations, that has made large scale DL training effective, and efficient, greatly improved ease of use, and redefined the DL training landscape in terms of scale that is possible. These innovations include ZeRO, ZeRO-Infinity, 3D-Parallelism, Ulysses Sequence Parallelism, DeepSpeed-MoE, etc.

---

DeepSpeed Adoption

DeepSpeed was an important part of Microsoft’s

AI at Scale

initiative to enable next-generation AI capabilities at scale, where you can find more

information here.

DeepSpeed has been used to train many different large-scale models, below is a list of several examples that we are aware of (if you'd like to include your model please submit a PR):

• Megatron-Turing NLG (530B)
• Jurassic-1 (178B)
• BLOOM (176B)
• GLM (130B)
• xTrimoPGLM (100B)
• YaLM (100B)
• GPT-NeoX (20B)
• AlexaTM (20B)
• Turing NLG (17B)
• METRO-LM (5.4B)

DeepSpeed has been integrated with several different popular open-source DL frameworks such as:

| | Documentation |

| ---------------------------------------------------------------------------------------------- | -------------------------------------------- |

<img src="docs/assets/images/transformers-light.png#gh-light-mode-only" width="250px"><img src="docs/assets/images/transformers-dark.png#gh-dark-mode-only" width="250px"> | Transformers with DeepSpeed |

| <img src="docs/assets/images/accelerate-light.png#gh-light-mode-only" width="250px"><img src="docs/assets/images/accelerate-dark.png#gh-dark-mode-only" width="250px"> | Accelerate with DeepSpeed |

| <img src="docs/assets/images/lightning-light.svg#gh-light-mode-only" width="200px"><img src="docs/assets/images/lightning-dark.svg#gh-dark-mode-only" width="200px"> | Lightning with DeepSpeed |

| <img src="docs/assets/images/mosaicml.svg" width="200px"> | MosaicML with DeepSpeed |

| <img src="docs/assets/images/determined.svg" width="225px"> | Determined with DeepSpeed |

| <img src="https://user-images.githubusercontent.com/58739961/187154444-fce76639-ac8d-429b-9354-c6fac64b7ef8.jpg" width=150> | MMEngine with DeepSpeed |

---

Build Pipeline Status

| Description | Status |

| ----------- | ------ |

| NVIDIA | nv-pre-compile-ops modal-torch-latest |

| AMD | amd-mi200 |

| CPU | torch-latest-cpu |

| Intel Gaudi | hpu-gaudi2 |

| Intel XPU | xpu-max1100 |

| Integrations | aws-accelerate |

| Misc | Formatting pages-build-deployment Documentation Statuspython |

| Huawei Ascend NPU | Huawei Ascend NPU |

Installation

The quickest way to get started with DeepSpeed is via pip, this will install

the latest release of DeepSpeed which is not tied to specific PyTorch or CUDA

versions. DeepSpeed includes several C++/CUDA extensions that we commonly refer

to as our 'ops'. By default, all of these extensions/ops will be built

just-in-time (JIT) using [torch's JIT C++ extension loader that relies on

ninja](https://pytorch.org/docs/stable/cpp_extension.html) to build and

dynamically link them at runtime.

Requirements

• PyTorch must be installed _before_ installing DeepSpeed.
• For full feature support we recommend a version of PyTorch that is >= 2.0 and ideally the latest PyTorch stable release.
• A CUDA or ROCm compiler such as nvcc or hipcc used to compile C++/CUDA/HIP extensions.
• Specific GPUs we develop and test against are listed below, this doesn't mean your GPU will not work if it doesn't fall into this category it's just DeepSpeed is most well tested on the following:
• NVIDIA: Pascal, Volta, Ampere, and Hopper architectures
• AMD: MI100 and MI200

Contributed HW support

• DeepSpeed now support various HW accelerators.

| Contributor | Hardware | Accelerator Name | Contributor validated | Upstream validated |

|-------------|-------------------------------------|------------------| --------------------- |--------------------|

| Huawei | Huawei Ascend NPU | npu | Yes | No |

| Intel | Intel(R) Gaudi(R) 2 AI accelerator | hpu | Yes | Yes |

| Intel | Intel(R) Xeon(R) Processors | cpu | Yes | Yes |

| Intel | Intel(R) Data Center GPU Max series | xpu | Yes | Yes |

| Tecorigin | Scalable Data Analytics Accelerator | sdaa | Yes | No |

PyPI

We regularly push releases to PyPI and encourage users to install from there in most cases.

pip install deepspeed

After installation, you can validate your install and see which extensions/ops

your machine is compatible with via the DeepSpeed environment report.

ds_report

If you would l