Falcon-H1: A Family of Hybrid-Head Language Models Redefining Efficiency and Performance
Introduction
Check out also our official blogpost
Today, we are proud to introduce the Falcon-H1 series, a collection of six open-source models ranging from 0.5B to 34B parameters, each available in both base and instruction-tuned variants. At the core of these models lies a hybrid architecture that combines the strengths of the classical Transformer-based attention mechanism with the State Space Model (SSM), known