KEY POINTS
- PyTorch 2.15 includes a rewritten backend for Nvidia's NCCL library, aimed at better fault diagnosis and recovery.
- A new flight recorder captures snapshots of operations so admins can inspect a stalled GPU cluster after a failure.
- The legacy backend stays in 2.15 as a rollback and is planned for removal in 2.16, expected early next year.
The 2.15 release of PyTorch includes a rewritten backend for the Nvidia Collective Communications Library (NCCL), which is meant to make large GPU training jobs easier to diagnose and recover when a single device fails. The Register reported the details on Thursday, citing a livestream by a PyTorch core developer.
