Rearchitecting Hugging Face Uploads and Downloads
As part of Hugging Face’s Xet team’s work to improve Hugging Face Hub’s storage backend, we analyzed a 24 hour window of Hugging Face upload requests to better understand access patterns. On October 11th, 2024, we saw:
- Uploads from 88 countries
- 8.2 million upload requests
- 130.8 TB of data transferred
The map below visualizes this activity, with countries colored by bytes uploaded per hour.
Currently, uploads are stored in an S3 bucket in us-east-1 and optimized using S3 Transfer Acceleration. Downloads are cached and served