A developer has built Colibri, a lightweight inference engine that can run Z.ai’s GLM-5.2, a 744-billion-parameter Mixture-of-Experts model, on consumer hardware without an Nvidia GPU.
The project is designed to run the model on a machine with around 25GB of RAM, using CPU and local storage instead of CUDA, PyTorch, or cloud infrastructure.
Colibri does not load the full model into memory.
Instead, it keeps only the active Mixture-of-Experts components in RAM and streams the remaining model weights from local SSD storage when needed. This allows the system to run a model that would normally require far more memory.
The idea is similar to streaming video. You do not download the whole movie before watching it. You stream the part you need at that moment.
The GLM-5.2 INT4 model package is designed to work with Colibri.
Because the model is streamed from storage, users still need enough disk space and a fast SSD. This is not the same as running a small chatbot model on a basic laptop, and performance will not match GPU-based inference.
However, the project shows that very large MoE models can be made more accessible through smarter memory and storage management.
The biggest technical point is that Colibri does not require the usual GPU software stack.
It runs without CUDA, PyTorch, or cloud infrastructure. This makes it useful for developers who want to experiment with very large open models locally without renting expensive GPU servers.
The project is available on GitHub under the public JustVugg/colibri repository.
Setup involves cloning the repository, downloading the GLM-5.2 INT4 model, pointing Colibri to the model files, and starting a local chat session.
Colibri can also run as an OpenAI-compatible API endpoint. This lets developers connect existing apps by changing the Base URL instead of rebuilding their workflow.
Colibri does not make Nvidia GPUs irrelevant.
Large GPU clusters will still be much faster for production inference, heavy workloads, and multi-user serving. Colibri is better understood as a proof of concept showing that very large MoE models do not always need to sit entirely inside GPU memory before they can run.
That is still an important shift. Instead of depending only on massive VRAM, the system moves some of the burden to cheaper RAM and SSD storage.
Colibri lowers the barrier for developers who want to test large open models without using cloud GPUs.
It also shows where local AI may be heading. As model architectures, quantization, and inference engines improve, more capable systems may become usable on ordinary hardware.
For now, Colibri’s main achievement is not replacing Nvidia. It is proving that a 744B-parameter model can be made usable on commodity hardware with the right inference approach.
Get the latest tech news, telecom insights, and product launches wherever you prefer.
Add ProPakistani to Preferred Sources and see more of our stories in Google Search and Top Stories.
Shares
