Improving Parquet Dedupe on Hugging Face Hub
The Xet team at Hugging Face is working on improving the efficiency of the Hub’s
storage architecture to make it easier and quicker for users to
store and update data and models. As Hugging Face hosts nearly 11PB of