About

IOTA (Incentivised Orchestrated Training Architecture) is Bittensor's Subnet 9, a permissionless network for pretraining large language models across geographically distributed, heterogeneous GPUs. It uses data- and pipeline-parallel training so that each participant holds only a slice of the model rather than a full replica, removing the per-node VRAM ceiling that constrains other decentralized training designs. The subnet is built and operated by Macrocosmos, which rewards miners in proportion to measured contribution rather than winner-takes-all.

History

IOTA is the second architecture to run on Bittensor's Subnet 9. In its original form SN9 was a pretraining competition in which miners each trained a complete model and only the single best result was rewarded. That design proved decentralized pretraining was viable but capped model size at whatever one machine could hold and encouraged miners to hoard weights. Macrocosmos, founded in April 2024 by Will Squires and Steffen Cruz, rebuilt the subnet around a swarm: an orchestrator splits a single model across miners in pipeline-parallel stages and streams activations between them, so capacity scales with participant count instead of with any one GPU. Contribution is measured continuously and emissions are paid proportionally.

• August 2024: SN9 demonstrates permissionless pretraining of models from 700 million to 14 billion parameters under a winner-takes-all reward design.
• May 2025: Macrocosmos publishes the IOTA technical primer, introducing pipeline-parallel swarm training, 128x activation compression, Butterfly All-Reduce weight merging, and the CLASP contribution-attribution method.
• June 2025: IOTA goes live on Bittensor mainnet, opening with a 15-billion-parameter target before scaling back to a 1.5-billion-parameter testbed.
• July 2025: The primer is posted to arXiv (2507.17766).
• February 2026: Train at Home ships for macOS, letting consumer hardware contribute to the training pipeline without command-line setup.
• June 2026: Macrocosmos announces Orion-100B, a 100-billion-parameter run across 16 pipeline stages and three replicas on globally distributed A100s, reporting 30.8% average model FLOP utilisation and roughly 65% of the training speed of a comparable datacenter deployment.

Project Orion continues with Orion-16B, a 16-billion-parameter model training live across three continents. Documentation and the source code are public.