Project Maya (MIT, github.com/mw00/project-maya) is a free project whose pitch is the same one this whole blog keeps landing on: the biggest open models should run on hardware people already have. What makes it interesting to me is not that pitch but the way it’s built, one engine per model, four flagship-scale mixture-of-experts models supported on purpose rather than generalized, and a self-tuning memory system that keeps the experts your conversation actually uses close to the GPU and everything else on RAM and NVMe.
Created and led by Peasant Smith (@mw00), co-maintained with @needmorevram. It is built from a pinned commit of ggml / llama.cpp (the quant formats, CPU dot products, prompt path), follows exllamav3’s EXL3 format for the third-party quant options, and uses beellama.cpp’s DFlash2 drafter, all with notices kept in third_party/.
The four models, and what "one engine per model" means here
Maya supports four flagship-class MoE models, each with its own engine, its own docs page and its own options:
| Model | Size / active | License | Context | |—|—|—|—| | GLM-5.3 | 744 B, 40 B active | zai’s own license | up to 1M, text only | | GLM-5.3-Flash | 321 B, 18 B active | MIT | up to 1M, pictures included | | DeepSeek-V4.1-Flash-1017 | 748 B (incl. Engram tables) | MIT | long-context, pictures included | | MiniMax-M3.1-Flash | 428 B, 23 B active | MiniMax Community License | up to 1M, pictures included |
Each model gets Maya’s own quant tiers, XS, S, M or L, by how many bits the experts keep, sized against a memory pool (VRAM + RAM together) rather than against a single card:
- GLM-5.3-Flash: Maya-XS at 73.4 GB ("64 GB of RAM and a 24 GB card"), Maya-S the default at 96.5 GB, Maya-S24 tuned for 24-GB cards (decode +14%), Maya-M at 116 GB calibrated toward tool calls, Maya-L at 156.3 GB.
- GLM-5.3: Maya-XS at 170 GB runs the whole model in a 192 GB pool, on NVIDIA from V100 onward and on AMD.
- DeepSeek-V4.1-Flash: Maya-S at 340 GB ("what VRAM and RAM don’t hold is read from the SSD"), Maya-XS v2 at 277 GB sized for a 128 GB PC, its Engram tables deliberately stay on the drive.
- MiniMax-M3.1-Flash: Maya-XS at 93 GB, Maya-S at 122 GB.
Third-party options are on purpose too: unsloth’s GGUFs and Q8 for DeepSeek, turboderp’s and LibertAIDAI’s EXL3, orcarouter’s abliterated DeepSeek, Mia-AiLab’s MiniMax quant, the setup recommends one for your machine, and says which claims each option makes.
That "one engine per model, with tiers named after memory pools" is exactly the lesson my Avalon and Xolotl notes came back to: a mixture-of-experts model does not need a bigger GPU, it needs a memory plan. Maya is that lesson taken seriously across four models at once.
What it actually does with your memory
The engine keeps the experts your conversation uses most on the GPU, the next ones in RAM, the rest on NVMe, and moves them as you chat, the same three-tier design I described in "Hot experts, cold RAM", with two honest wrinkles stated up front:
- The engine pins almost all free RAM (everything but ~6 GB) for its expert tier. Other programs get little RAM while Maya runs; the docs say so instead of pretending otherwise.
- It measures your machine and tunes itself.
./maya.sh --calibratetunes the engine to your specific GPUs, CPU, RAM and SSD in 10, 15 minutes, and the numbers in its benchmark tables show what that found on real boxes.
Hardware floor: one NVIDIA card at compute capability 7.0 or newer (V100, RTX 20 onward), up to sixteen sharing the model; 32 GB of RAM to start; a fast NVMe with room for the model (75, 470 GB by quant). AMD is supported on RX 7900 XT/XTX, Radeon AI PRO R9700, RX 9070 and Strix/Gorgon Halo parts, with a validation list stated per card. Windows is native (a START-MAYA.bat, not WSL2). Apple Silicon is experimental, Metal for GLM-5.3-Flash only, "we have no Mac" admitted in the open.
Close to the original, and measuring it
The house rule of this blog, every speed claim needs an agreement metric beside it, is met here better than almost anywhere:
- Maya’s own quants of GLM-5.3-Flash keep 97.7, 99.2% of the FP8 model’s zero-shot accuracy and pick its next token 83, 90% of the time (Maya-L: 99.2%, 90%).
- DeepSeek V4.1 · Maya-S picks the same next token as the original 92.6% of the time; MiniMax M3.1 · Maya-S, 87.6%.
Those percentages are the difference between "a big model that sort of works" and a quant you can put in front of an agent. The tier ladder exists precisely so the quality/sizes choice is explicit rather than hidden: L is closest to the original, XS makes the pool fit.
The benchmark culture is the other reason to read it
Maya’s README has three long speed tables, GLM-5.3-Flash, DeepSeek-V4.1-Flash, MiniMax, and every line names a machine, a version and a GitHub issue number, submitted by users through --bench and --report. The breadth is the point:
- 148.6 tok/s decode (and ~2,600 prefill) on two RTX 5090s plus a 4090, a gaming box.
- 184 tok/s with parallel sessions on eight RTX 5090s for DeepSeek; 122 on two H100s at 1M context.
- A single RTX 5090: 30+ tok/s. A single RTX PRO 6000 Blackwell: 81. A modded 32 GB RTX 4080 (SATA SSD!): 26.7 at 262K context.
- Decade-old datacenter cards still run it: four Tesla V100s hit ~70 tok/s on GLM-5.3-Flash; two P40s reach ~28; two Tesla T4s get 22.6. A DGX Spark gets 16.5.
- AMD everywhere, including Strix Halo machines with 128, 192 GB of unified memory and Radeon 8060S graphics.
The "first answers after a start are the slowest while caches fill" note is in there too, the honest part of numbers quoted from cold starts.
Everything else in the box
- A browser dashboard: chat kept locally with folders and per-chat settings, a live monitor down to what fills each GPU’s memory, a Models page that downloads/loads/unloads models without a command line, a Turbo button for fastest-settings-at-any-quality-cost.
- APIs your tools already use: OpenAI-compatible at
/v1, Anthropic-compatible at/v1/messages(Claude Code: set the base URL), a/v1/responseswith function calls and background tasks, plus an Ollama-style model API (/api/tags,/api/chat,/api/pull). Tool calls stream. ASavedledger sums what the hosted APIs would have charged for the same tokens, kept per model. - Agent features: thinking levels with a token budget, an on-by-default loop guard that steers an agent off repeated identical tool calls, MCP server support, picture and file attachments (PDFs as text or scanned pages as images) through both chat and the APIs.
- Up to 8 concurrent requests on GLM-5.3-Flash and friends, several people or several agent sessions on one PC.
git clone, ./setup.sh, answer a few questions, and it starts: 20, 40 minutes of build plus downloads, then the model runs at http://127.0.0.1:8080. It is free, MIT-licensed, and its sponsor request is blunt: 50 monthly sponsors buys test hardware and a real Mac, which is what keeping this kind of work validated actually costs.
Where it fits alongside the others I have written about
| Project | Bet | Machine it optimizes for | |—|—|—| | Maya | four flagship MoE models, each with its own engine, AMD/Windows included | big memory pool + NVMe, any recent card | | Strata / PolyStrata (notes) | one well-served flagship / a dozen backends in an app | one high-end NVIDIA card | | Xolotl (notes) | DeepSeek-V4.1 + GLM-Flash depth, own kernels | big NVIDIA card + huge pinned-RAM arena | | Avalon (notes) | one model, old AMD cards | 12, 24 GB AMD cards + lots of cheap RAM | | llama.cpp (notes) | general runtime, broadest hardware reach | everything, with tuning work on top |
There is real overlap between Maya and Xolotl, both serve DeepSeek-V4.1-class and GLM-5.3-Flash-class models with tiered expert caches and MTP-style drafting, but the projects point in different directions. Xolotl optimizes one gaming PC as deep as it can go; Maya aims at every class of machine from a single 5090 to old Teslas and Strix Halo laptops, with Windows and AMD validated lists attached, and a polished desktop-app surface around it. If I had to describe the difference in one line: Maya’s unit of support is the model; everyone else’s is usually the engine.
Two license notes for people using this commercially: GLM-5.3 itself carries zai’s own license, MiniMax-M3.1-Flash the MiniMax Community License, and the optional DFlash2 drafter is CC BY-NC-ND (non-commercial), the core project is MIT and the engine parts are upstream llama.cpp/EXL3, so the clean-room path is to read each model card before building a product on it.
My take
Maya is the most complete "big MoE at home" package I’ve seen: per-model engines, quant tiers sized by memory pool rather than marketing numbers, next-token agreement percentages instead of vibes, and a community benchmark culture that keeps issue numbers attached to every claim. The trade-offs it admits are the right ones, pinning your RAM, cold-start slowness, experimental Metal, a Windows path that is native rather than convenient.
For anyone running one of my ten-engines-for-one-box rigs: if you have 128 GB of RAM and a recent NVIDIA or Radeon card, Maya is the shortest path to a 300, 750-billion-parameter model answering from your own machine. And if you have something weirder, four V100s, a Strix Halo box, a P40 for history’s sake, its benchmark tables are already proof it runs there too.

Leave a Reply