A PyTorch extension library for high performance and large scale training
FairScale FairScale is a PyTorch extension library for high performance and large scale training.This library extends basic PyTorch capabilities while adding new SOTA scaling techniques.FairScale makes available the latest distributed training techniques in the form of composablemodules and easy to use APIs. These APIs are a fundamental part of a researcher’s toolbox asthey attempt to scale models with limited resources. FairScale was designed with the following values in mind: Usability – Users should be able to understand and use FairScale […]
Read more