The trick: share the work across the whole PC
Models like these normally live on servers with hundreds of gigabytes of graphics memory; a gaming card has 8-32 GB. PolyStrata makes them fit by spreading the model across the machine – the parts you use constantly stay on the graphics card, the rest wait in system RAM and on the SSD. It is free, open source, and nothing leaves your PC.
Two engines, twelve models
One model, Qwen3.8-Flash-Next, runs on the Strata project’s engine unchanged – on an NVIDIA or AMD card with 12 GB or more, on Windows or Linux, and it reads pictures. The other eleven run on an engine written for PolyStrata, on one NVIDIA card or two: GLM 5.3 Flash (a 79-114 GB file, the largest), DeepSeek V4 Flash (91 GB), LFM2.5-24B-A2B, Flint 40B-A2.5B (built for coding agents), Qwen3.6-35B-A3B, Clover-35B-A3B, Ornith 1.5, Gemma 4 26B-A4B, Tiel Coder, MiniCPM-V and Kolibri 1. These are experimental – they have run on one PC with a 32 GB card, and the smaller ones on an 8 GB laptop.
What it asks for
Qwen3.8-Flash-Next wants 12 GB of VRAM and 32 GB of RAM (64 GB runs every size); the biggest MoE models want a 32 GB card with 72-96 GB of system RAM beside it. The project’s own figures: Qwen3.6-35B-A3B writes at 94-100 tok/s on an 8 GB laptop card and 394-395 on an RTX 5090, and GLM 5.3 Flash only runs on a 32 GB card.
Keeping the RAM honest
The engine reads experts from the system file cache rather than pinning them in place, and an opt-in --ram-experts-mib budget cut what it held after a 32K-token prompt from about 17.9 GiB to 13.1 GiB on a laptop – while still writing at nearly the same speed. It is an honest account of what a big model really costs you.
We size and run these builds on your own hardware across Perpinyà, Girona and Barcelona. Ask: privatelocalai@proton.me. Source: https://github.com/simplexai-labs/PolyStrata

Leave a Reply