huggingface

Introducing the Hugging Face LLM Inference Container for Amazon SageMaker

Philipp Schmid's avatar

This is an example on how to deploy the open-source LLMs, like BLOOM to Amazon SageMaker for inference using the new Hugging Face LLM Inference Container.
We will deploy the 12B Pythia Open Assistant Model, an open-source Chat LLM trained with the Open Assistant dataset.

The example covers:

  1. Setup development environment
  2. Retrieve the new Hugging Face

     

     

     

    To finish reading, please visit source site