> For the complete documentation index, see [llms.txt](https://sliu583.gitbook.io/blog/llms.txt). Markdown versions of documentation pages are available by appending `.md` to page URLs; this page is available as [Markdown](https://sliu583.gitbook.io/blog/specific-work/shivarams-group/group-papers/lists/zero-infinity-breaking-the-gpu-memory-wall-for-extreme-scale-deep-learning.md).

# ZeRO-Infinity: Breaking the GPU Memory Wall for Extreme Scale Deep Learning