Introduction

WiCi, a systems research lab known for its wireless GPU hardware, has released a free desktop application called WiCi Infer. The software is designed to run extremely large language models on ordinary Mac and Windows laptops by keeping the bulk of the model on solid-state storage and loading only the pieces needed for each step of text generation. Until now, models with hundreds of billions of parameters have generally required data-center hardware or far more memory than a typical laptop contains.

What Happened

WiCi Infer is built on InferSSD, an open-source inference engine based on llama.cpp. Rather than loading an entire model into RAM or GPU memory before generating a response, the app stores model weights on the computer's SSD and streams active portions into memory as tokens are produced. This approach targets mixture-of-experts models, which route each word fragment through only a subset of their total parameters.

WiCi highlighted two models in its launch materials: DeepSeek V4.1 Flash at 763 billion parameters and Muse Spark Open at 816 billion parameters. According to the company, Muse Spark Open keeps about 42 billion parameters active at any given time. The model files occupy roughly 520GB of storage.

Published minimum memory requirements start at 16GB for DeepSeek V4.1 Flash and 24GB for Muse Spark Open, with 48GB recommended for either. Supported platforms include macOS on Apple Silicon and Windows x64, while Linux users can run the InferSSD command-line engine directly.

Why This Matters

Running frontier AI locally has usually been impractical because model memory demands exceed what consumer devices offer. WiCi Infer shifts that burden from memory to storage, making it possible to run an 800B-class model on hardware many people already own. The app also serves models through a local OpenAI-compatible API, requires no account or login, and processes prompts on the device rather than sending them to a cloud provider.

For developers, teams handling sensitive documents, or privacy-conscious users, local execution removes the need to upload prompts or business data to external servers. The open-source InferSSD engine also allows developers to inspect the code and build their own tools on top of it.

Key Takeaways

  • WiCi Infer runs 763B- and 816B-parameter models on consumer laptops by streaming weights from SSD storage.
  • Minimum memory requirements range from 16GB to 24GB, with 48GB recommended, plus about 520GB of free SSD space.
  • WiCi's own benchmarks show roughly 2.7 tokens per second for DeepSeek V4.1 Flash on a 48GB M5 Max MacBook Pro, about three times faster than standard llama.cpp in the same test.
  • Generation speeds remain slow compared with cloud services, and a 300-word response can take around two and a half minutes.
  • Prompts are processed locally, no account is required, and the underlying InferSSD engine is open source.

Conclusion

WiCi Infer does not make 800B-parameter models fast or lightweight, but it demonstrates that they can run on a personal computer at all. The trade-off is straightforward: users must dedicate roughly half a terabyte of SSD storage and accept generation speeds measured in single-digit tokens per second. In exchange, they gain private, local access to frontier-scale language models without purchasing data-center hardware. Whether this approach becomes practical for everyday use will depend on future storage speeds, memory efficiency, and independent performance testing beyond WiCi's own published figures.