The open-source project Deltafin has made it possible to run the massive 2.8 trillion parameter Kimi K3 AI model on a standard Apple MacBook without needing a dedicated GPU. By leveraging the model's distinct architecture and selective cloud fetching, the project successfully brings data-center-scale AI to consumer hardware.
The Mixture of Experts Architecture
play_arrow0:25 Kimi K3 contains 2.8 trillion parameters, resulting in roughly 1.56 terabytes of data, which easily exceeds standard local hardware capacities
play_arrow0:41 Instead of activating the entire network for every task, Kimi K3 operates as a mixture of experts model, physically split into 82,432 small expert brains
play_arrow0:56 For every single word the model writes, it only needs to activate 16 of those experts, meaning only a tiny fraction of the 2.8 trillion parameters is ever used at any given moment
How Deltafin Manages Data
play_arrow1:46 The Deltafin project sidesteps downloading the full 1.56 terabytes by keeping a core foundational piece of about 114 gigabytes permanently saved on the Mac
play_arrow1:58 The remaining 82,000+ experts stay hosted online on Hugging Face servers, and Deltafin sends a quick request over the internet to grab just the specific 17-megabyte piece it needs
play_arrow2:10 As specific experts are downloaded and temporarily saved to the local machine, the system gradually gets faster the more it is actively used
Performance Optimizations and Requirements
play_arrow2:17 Without optimization, the very first run of the model was incredibly slow, taking about 20 minutes to write a single word
play_arrow2:27 By halving memory use and introducing a predictive trick called speculation (where the model guesses ahead and checks itself), the speed improved 16 times, bringing generation down to one word every 60 to 76 seconds
play_arrow2:21 To actually run this setup locally, users will need an Apple Silicon Mac and approximately 350 gigabytes of free storage
Summary
While generation times remain much slower than commercial cloud alternatives, this specific project successfully proves that massive, enterprise-scale AI models can be decentralized and reliably run on consumer machines, paving the way for faster home AI optimizations in the future.