Repository navigation
Releases
v1.1.65
A stability release: fixes for start-up and loading failures on AMD and Windows, a steadier AMD + NVIDIA pairing, clearer errors when the engine fails, and a test suite where a red test always means something.
- Fix: GLM on two 16 GB AMD cards found no room for the experts (#899, @bob-u): on Linux with AMD GPUs, the free VRAM read right after Maya's budget check could still count the checked memory as used, so the expert pool got 0 slots and the engine stopped at start. Maya now waits until the freed memory reads as free again (
MAYA_GLM_POOL_CHECK_WAIT_MS, Linux HIP only). The automatic layer split works again; the manual split is no longer needed. - Fix: DeepSeek on Windows with several NVIDIA GPUs failed with "0 MiB free" (#794, @wolfwoof88, @benjaminfphillips and @KuroNekoMaid): pinning the RAM copy of the experts could leave every GPU without usable VRAM on Windows. Maya now reads each GPU's free VRAM before and after the pin, and undoes the pin by itself when it leaves a GPU nearly empty, so
DS4_COPY_PIN=0is no longer needed. When the free VRAM cannot be read, the load line now says why instead of "0 MiB free". - Steadier AMD + NVIDIA pairing: the APU side now imports the shared expert memory in a way the AMD driver does not track, so Linux moving memory around no longer pauses it for seconds at a time. GLM-5.3 (744B) still starts on the APU alone unless you use
--mixed-gpus on. - The same answers on every install of the AMD + NVIDIA pairing: how the pair splits prompt work between the CPU and the GPUs no longer depends on a few percent of difference in each build's measured CPU timing, so two installs of the same model on the same GPUs give the same answers (
MAYA_GLM_LANE_STEPS=0restores the old behaviour). EXL3 answers on the pair can differ slightly from v1.1.64. - Clearer errors when the engine fails (#827, @anbow1): an answer the engine stops because of a GPU fault is now reported as an error, not as "length". Streamed answers keep the text already sent and end with the API's own error event; other requests get HTTP 422 with type
engine_error(not retried automatically by the OpenAI and Anthropic SDKs), the reason, the position and the partial text. The console line now names the failed step, the position and the context size. - Finding a GLM drafting stall (#860, @githug2024): if one drafting step takes longer than 20 s, the log now prints a
glm stall:line saying where each GPU is stuck (MAYA_GLM_STALL_S). It only reports. - Smaller fixes:
- The "use a smaller
--vram-reserve-mib" hint now only appears when it can help, and says that setting the reserve by hand turns off its automatic shrinking. - The help says which flags a start of an existing config ignores.
- A speed calibration you ask for by name now runs when a GPU only holds a display's memory, with a warning; it still refuses while another Maya model is loaded.
- AMD install docs: what to do when Ubuntu's rocWMMA package lacks headers.
strata-deviceaccepts GPUs from compute capability 6.0, like the rest of Maya.
- The "use a smaller
- Tests: every known failing or flaky test now has its cause fixed: tests that need model files skip with the reason instead of failing, slow GPU tests are 5-8 times faster, tests that clashed in parallel runs no longer do, and the Windows and
nohupruns pass.
Full list: CHANGELOG.md.
Supported by @wjHunter, @os-groot, @davedgd, @jeanluchayes, @sugattor, @vikcious, @ThatSimpleTech, @router0mail and @davidfernengel.
Maya is free; if it helps you, you can sponsor it: https://github.com/sponsors/mw00
v1.1.64
EXL3 quants now run on an AMD Strix Halo or Gorgon Halo APU paired with an NVIDIA GPU. In v1.1.62 the pairing took GGUF quants only.
-
New: EXL3 on an AMD Halo APU + NVIDIA GPU (Linux, experimental): GLM-5.3-Flash's EXL3 quants now run on both GPUs at once, as the GGUF quants have since v1.1.62. The NVIDIA card runs attention, the cache, drafting and the most-used experts; the APU computes every other expert in place from the memory both share. Maya tunes itself: the first start measures the APU alone and the pair, and keeps the pair only where it is faster.
--mixed-gpus offturns it off. -
Steadier AMD + NVIDIA pairing: the APU side now shares the model's memory with the NVIDIA card in 1 GB pieces. The AMD driver's pauses of the APU side (seconds to minutes when Linux moved memory) became much rarer. GLM-5.3 (744B) still stays on the APU alone unless you start it with
--mixed-gpus on. -
Where the model lives on the pair: the experts the NVIDIA card does not hold stay in ordinary system RAM, shared by both GPUs, not in the APU's BIOS VRAM carve-out. There is one copy of the model, only activations cross between the GPUs, and no large BIOS VRAM reserve is needed (now in docs/SETTINGS.md).
-
Fix: DeepSeek decode on several GPUs (#956, @speja969 and @mgabor3141): since v1.1.58, DeepSeek on a layer split or
--tpcould keep its PCIe expert lane on when it gained nothing, which cost about 4-5% decode on the reporters' PCs. On several GPUs the lane now turns off again unless it measures faster, as in v1.1.57; one GPU is unchanged.DS4_STREAM_TIE=keep|offforces either rule. The first start after the update measures again. -
Qwen GGUFs with mixed expert formats: Maya now runs Qwen GGUF quants whose expert gate and up weights use different formats in a layer (for example unsloth's UD-IQ1_S of Qwen3.8-Flash-Next). Every other quant packs and answers exactly as before.
-
MiniMax: damaged model files are caught (#672, @tanutanu56): the setup now checks MiniMax GGUF files already in your model folder by their checksum, once, and says which file to download again if one does not match. Before, it checked their size only. If the router still meets broken numbers, the error now says where they come from instead of naming an impossible expert.
Full list: CHANGELOG.md.
Supported by @wjHunter, @os-groot, @davedgd, @jeanluchayes, @sugattor, @vikcious, @ThatSimpleTech, @router0mail and @davidfernengel.
Maya is free; if it helps you, you can sponsor it: https://github.com/sponsors/mw00
v1.1.63
Fixes for v1.1.62: the DeepSeek engine builds on Windows again, and GLM on an AMD APU paired with an NVIDIA GPU no longer restarts its engine before every request.
- Fix: DeepSeek on Windows with an NVIDIA GPU (#983, @pietroMastro92): v1.1.62's DeepSeek engine did not compile on Windows, so the setup could not update it. The new two-GPU code used Linux-only calls; it compiles on Windows again, and nothing else changes there.
- Fix: GLM on an AMD APU + NVIDIA GPU (Linux): after a context change, an engine restart or an idle unload, Maya started the engine again before every request, about a minute each. It now starts once and stays running.
Full list: CHANGELOG.md.
Supported by @wjHunter, @os-groot, @davedgd, @jeanluchayes, @sugattor, @vikcious, @ThatSimpleTech, @router0mail and @davidfernengel.
Maya is free; if it helps you, you can sponsor it: https://github.com/sponsors/mw00
v1.1.62
AMD Strix Halo and Gorgon Halo APUs (Ryzen AI Max) can now be paired with an NVIDIA GPU: Maya runs one model on both at once, with GLM-5.3-Flash prefilling up to 3 times faster and DeepSeek-V4.1-Flash decoding up to 18% faster than on the APU alone.
-
New: an AMD Halo APU and an NVIDIA GPU sharing one model (Linux, experimental): the AMD side runs on ROCm (HIP) and the NVIDIA side on CUDA, both drivers loaded side by side in one PC. The NVIDIA card can sit in a USB4/PCIe dock. The two GPUs share one copy of the model in the APU's system memory: each computes its own part of every layer, and while it runs only activations cross between them, not the model's weights. For GLM-5.3-Flash the NVIDIA card runs attention, the KV cache, drafting and the most-used experts, and the APU computes every other expert in place from its own memory. For DeepSeek-V4.1-Flash the APU runs the whole model and the NVIDIA card computes its most-used experts for it.
- Maya tunes itself: when it finds both GPUs, the setup builds both engines and the first start measures each model on the APU alone and on both. It keeps both only where that is more than 3% faster; the Models page then shows "AMD + NVIDIA".
--mixed-gpus off(or"mixed_gpus": "off") turns it off. - Answers repeat run to run, and can differ slightly from the APU alone.
- GLM-5.3 (744B) stays on the APU alone unless you start it with
--mixed-gpus on: on some starts it stalls for a while on this setup. - Tested on a Gorgon Halo APU (Radeon 8065S) with a Tesla V100 32 GB in a USB4 dock, ROCm 7.14 and CUDA 12.4. Strix Halo uses the same AMD code.
Biggest gains, A/B on the same PC (tokens per second; A = the APU alone, B = the APU and the NVIDIA GPU together)
A B Change GLM-5.3-Flash prefill, 32K tokens 293 901 +208% GLM-5.3-Flash prefill, 8K tokens 315 837 +166% GLM-5.3-Flash prefill, 2K tokens 310 591 +91% GLM-5.3-Flash decode 26.6 32.1 +21% DeepSeek-V4.1-Flash decode 36.8 43.5 +18% DeepSeek-V4.1-Flash prefill, 32K tokens, --mixed-gpus split615 1,062 +73% All measurements
GLM-5.3-Flash (Maya-S)
A B Change Decode, question 1 30.2 34.4 +14% Decode, question 2 26.6 32.1 +21% Prefill, 2K tokens 310 591 +91% Prefill, 8K tokens 315 837 +166% Prefill, 32K tokens 293 901 +208% DeepSeek-V4.1-Flash (Maya-XS)
A B Change Decode, question 1 36.8 43.5 +18% Decode, question 2 29.6 31.4 +6% Decode, question 3 44.7 49.6 +11% Prefill, 2K tokens 283 322 +14% Prefill, 8K tokens 475 528 +11% Prefill, 32K tokens 615 696 +13% DeepSeek-V4.1-Flash (Maya-XS) with
--mixed-gpus split, for long documents: the NVIDIA card runs the first layers instead (decode is 1-4% slower than the APU alone)A B Change Prefill, 2K tokens 283 348 +23% Prefill, 8K tokens 475 674 +42% Prefill, 32K tokens 615 1,062 +73% With DeepSeek, requests with pictures, log-probabilities or refusal ablation run on the APU alone.
- Maya tunes itself: when it finds both GPUs, the setup builds both engines and the first start measures each model on the APU alone and on both. It keeps both only where that is more than 3% faster; the Models page then shows "AMD + NVIDIA".
-
Faster: GLM decode after a prefill on AMD APUs under Linux: GLM-5.3-Flash (Maya-S) on the APU alone, v1.1.61 against v1.1.62. The answers now repeat run to run. Prefill and decode in a fresh chat are unchanged.
Decode right after v1.1.61 v1.1.62 Change a 2K-token prefill 20.9 29.2 +40% an 8K-token prefill 19.7 20.7 +5% a 32K-token prefill 16.6 21.0 +27%
Full list: CHANGELOG.md.
Supported by @wjHunter, @os-groot, @davedgd, @jeanluchayes, @sugattor, @vikcious, @ThatSimpleTech, @router0mail and @davidfernengel.
Maya is free; if it helps you, you can sponsor it: https://github.com/sponsors/mw00
v1.1.61
DeepSeek reads prompts up to 17% faster on AMD Ryzen AI Max APUs with the same answers, a fast prompt reading switch, faster EXL3 decode on AMD, and prompt lookup for MiniMax on one GPU.
- Faster: DeepSeek prompt reading on AMD Ryzen AI Max APUs: Maya-XS reads prompts 14-17% faster than in v1.1.60 at every length from 2K to 128K tokens, and Maya-S 8-9% faster at 8K-32K, with identical answers. Each expert's weights are now unpacked once for more prompt rows, and the sparse attention handles all head groups of a token together without memory bank clashes. Measured on a Ryzen AI Max APU; other RDNA3 GPUs (Radeon RX 7000) use the same code.
- Faster: DeepSeek EXL3 decode on AMD: EXL3 quants decode 5-8% faster than in v1.1.60 on a Ryzen AI Max APU, with identical answers (thanks @pepuscz for #945).
- DeepSeek: fast prompt reading (opt-in): Models page > Speed > Fast prompt reading (or
"fast_prefill": true,--fast-prefill on). For shorter prompts, the later layers also skip the rows that cannot affect the answer: on a Ryzen AI Max APU Maya-XS reads prompts about 14% faster at 2K and 12% faster at 8K than with it off. The answers can shift slightly, so it is off until you switch it on. - MiniMax: prompt lookup on one GPU: where an answer repeats earlier text (code edits, tool-call arguments, quoted text), the tokens that followed it earlier are checked in one step. On one GPU it is on: code and tool calls decoded about 6% faster than in v1.1.60 on our test PC, with the DSpark draft on. On several GPUs it is your choice (
--suffix-draft 3). - Setup: the speed calibration now measures code and tool calls too: DeepSeek's and MiniMax's draft (DSpark) is kept or turned off by its speed on chat, code and tool calls together, not chat alone, so it stays on where it speeds up coding work. It applies the next time you run the setup.
Full list: CHANGELOG.md.
Supported by @wjHunter, @os-groot, @davedgd, @jeanluchayes, @sugattor, @vikcious, @ThatSimpleTech, @router0mail and @davidfernengel.
Maya is free; if it helps you, you can sponsor it: https://github.com/sponsors/mw00
v1.1.60
GLM-5.3-Flash decodes faster again on AMD Ryzen AI Max APUs (up to ~0.8% vs v1.1.59, back to v1.1.56's speed), turboderp's 2.25 bpw EXL3 quant of DeepSeek in the setup, DFlash2 drafting for GLM-5.3 (744B) as an opt-in, and fixes.
- Faster: GLM-5.3-Flash on AMD Ryzen AI Max APUs: drafting decode is up to ~0.8% faster than in v1.1.59, back to v1.1.56's speed, with identical answers (how much depends on the system's state; sometimes there is no difference). Since v1.1.57 a routing buffer sized for GLM-5.3 (744B) slowed it; AMD builds now size it for the model. NVIDIA is unchanged.
- turboderp's DeepSeek-V4.1-Flash EXL3 at 2.25 bpw in the setup: turboderp's 2.25 bpw EXL3 quant is now in the setup's DeepSeek list (
--ds4-quant exl3-2.25), the smallest of its EXL3 options: 158.6 GB of weights, about 160 GB in total, VRAM and RAM added together (192 GB recommended); 362 GB on disk with its Engram tables. Every file is pinned and checked. Pictures work, and DeepSeek's sampling and effort, DSpark drafting and the refusal ablation switch work as with the other EXL3 quants. It is listed, never recommended over Maya's own quants. Thanks to turboderp for the quant. - GLM-5.3 (744B): DFlash2 drafting as an opt-in:
--dflash GGUFin a GLM-5.3 config's"args"now works, with Anbeeld's GLM-5.3-DFlash2-GGUF drafter (CC BY-NC-ND 4.0: non-commercial use only) in place of the model's MTP block. The setup does not offer it: GLM-5.3 keeps drafting with its MTP block by default, which was faster at the default sampling everywhere we measured. - README: new speed rows from community reports (#890 thanks @imaginitivename, #895 thanks @digitalhomesteader, #905 and #908 thanks @pepuscz, #934 thanks @bob-u, #939 thanks @scranst, #942 thanks @rim-jaynor), in all four languages.
- Fixes:
- GLM-5.3 (744B) on AMD Ryzen AI Max APUs: the setup's context recommendation leaves a little more memory free, so with GLM-5.3-Maya-XS on a 192 GB APU it recommends 32K instead of 64K, where a few experts were still read from the SSD. A longer context you chose is kept, with the warning.
Full list: CHANGELOG.md.
Supported by @wjHunter, @os-groot, @davedgd, @jeanluchayes, @sugattor, @vikcious, @ThatSimpleTech, @router0mail and @davidfernengel.
Maya is free; if it helps you, you can sponsor it: https://github.com/sponsors/mw00
v1.1.59
Refusal ablation for DeepSeek, MiniMax and GLM-5.3-Flash, Turbo for GLM-5.3 (744B), a setup that recommends the quant and context your PC's memory holds, and fixes.
- The setup recommends the quant your PC holds: each model's list (GLM-5.3-Flash, GLM-5.3 (744B), DeepSeek and MiniMax) now recommends the largest quant, up to the model's usual pick, whose files and default cache fit in your VRAM and RAM together, so no expert has to be read from the SSD. AMD Ryzen AI Max APUs and Macs count their shared memory once. Every quant stays in the list. For GLM-5.3 (744B) the recommendation is at most GLM-5.3-Maya-XS.
- GLM-5.3 (744B) on AMD Ryzen AI Max APUs: the setup recommends the largest context that still keeps the whole model in memory. A longer context you choose is kept, with a warning that some experts will then come from the SSD.
- Turbo for GLM-5.3 (744B): the Turbo button now works for GLM-5.3 too, with its own setting: it prefers the experts already in VRAM (
MAYA_GLM_ROUTE_AFFINITY=0.2) and uses a 32K context. With GLM-5.3-Maya-XS, part of it outside VRAM, decoding was about 22-25% faster on our test PC. Like all of Turbo it changes the answers, it is off until you switch it on, and switching it off restores your settings exactly. - Dashboard: Settings > Chat defaults shows a small "Saved" tick after every change.
- Refusal ablation for DeepSeek-V4.1-Flash, MiniMax-M3.1-Flash and GLM-5.3-Flash: Maya's own direction files are now in each model's repo, and the setup downloads them. It is off until you switch it on (the Models page, or a request's
"ablate"); with it off the answers are unchanged. On held-out test prompts, refused requests went from 98% to 1% (DeepSeek), 100% to 7% (MiniMax) and 100% to 4% (GLM-5.3-Flash), with no extra refusals of ordinary requests. Fitting your own withtools/ablate_fit.pynow works on MiniMax and GLM-5.3. - Fixes:
- GLM-5.3 (744B) and GLM-5.3-Flash keep their own chat defaults on the dashboard; changing one no longer changes the other.
--parallelwith fair slices (MAYA_FAIR_SLICE_S): one waiting request pauses one running answer, not all of them (thanks @HTsulfuric for #883).- DeepSeek with
DS4_LENT_RAM_GB: a failed copy no longer makes every request fail with HTTP 400; prompts run as they do without the setting (#884). - DeepSeek on NVIDIA tensor-parallel pairs:
MAYA_DS4_HIP_PEER_AR=auto/1uses the direct GPU-to-GPU prompt all-reduce again, as in v1.1.54; the answers are the same.
Full list: CHANGELOG.md.
Supported by @wjHunter, @os-groot, @davedgd, @jeanluchayes, @sugattor, @vikcious, @ThatSimpleTech, @router0mail and @davidfernengel.
Maya is free; if it helps you, you can sponsor it: https://github.com/sponsors/mw00
v1.1.58
GLM-5.3-Maya-XS, Maya's own quant of Z.ai's GLM-5.3 (744B) at 170 GB; faster GLM-5.3 decode and drafting; DeepSeek's one-GPU decode back to speed; and fixes.
- GLM-5.3-Maya-XS: Maya's own quant of Z.ai's GLM-5.3 (744B), with the routed experts in about 1.6 bits. It is 170.0 GB in 4 files and as close to the original as unsloth's UD-IQ1_S (the same mean KL divergence, 0.506, and the same top token 81.8% of the time) at 78% of its size.
- The files: peasantsmith/GLM-5.3-Maya-GGUF, pinned and checked shard by shard. Set it up with
./maya.sh --setup --model glm-5.3-maya-xs(Windows:START-MAYA.bat --setup --model glm-5.3-maya-xs), or pick it in the setup's list or on the dashboard's Models page. It is shown as "GLM-5.3 (744B) Β· Maya-XS". - Memory: about 176 GB of VRAM and RAM together keeps every expert in memory, so 192 GB is recommended (see docs/MEMORY.md). With less, Maya reads the rest from the SSD while it answers.
- Where it runs: NVIDIA from the Tesla V100 on and AMD (the Strix and Gorgon Halo included), Linux and Windows. Text only, like GLM-5.3 itself, with drafting from its own NextN block.
- The files: peasantsmith/GLM-5.3-Maya-GGUF, pinned and checked shard by shard. Set it up with
- Faster GLM-5.3 decode:
- The pack keeps the GGUF's Q8_0 attention weights as they are instead of converting them to BF16: fewer bytes read per token, which helps most on a Gorgon Halo. Answers can differ slightly from v1.1.57's; the next setup rebuilds an existing GLM-5.3 pack once.
- The part of the decode that runs on the CPU computes four rows at once, with the same answers bit for bit.
- GLM-5.3 on an AMD Ryzen AI Max APU (Strix / Gorgon Halo): drafting on one GPU is now chosen by measurement: the setup's speed calibration tries decode with and without drafting on a few short chats and keeps the faster (it was always off before).
- Faster, smarter drafting for GLM-5.3 and GLM-5.3-Flash: Q5_K weights now take the fast multi-token path (the same answers). On several GPUs, GLM-5.3 hands the prompt between GPUs with a quarter of the data,
"spec": 1turns drafting off per request, and answers in Chinese, Japanese, Korean and other scripts are drafted again. - DeepSeek on one NVIDIA GPU (#920): decode is back to v1.1.54's speed on PCs where the CPU handles experts in RAM faster than the GPU's link can bring them over. On our test PC it was about 75% faster than v1.1.57 from the second start after the update on (the first start measures once).
- Names: GLM-5.3-Flash's quants now show the model first ("GLM-5.3-Flash Β· Maya-S" ...); the setup ids are unchanged. GLM-5.3 (744B) is named in every model list.
- Opt-in:
MAYA_GLM_PREFILL_STABLE=1makes GLM prompt reads reproducible whatever the expert cache holds (for A/B checks; about 2-5% slower prompts). - Fixes:
- Codex and other Responses API clients: pictures with any
detailvalue are accepted (#914). - Windows with AMD graphics: GLM's RAM tier leaves room below the memory Windows lets a program pin, so the start no longer fails with "the warmed expert tables did not upload" (#924).
- DeepSeek EXL3 quants with their own F16 vision tower: pictures turn on by themselves, using DeepSeek's BF16 tower (#886).
- Loop guard: a finished short answer is no longer kept going, and a kept agent answer continues on a new paragraph (#921, thanks @TacoTakumi for #925).
- DeepSeek on one NVIDIA GPU: a tie in the PCIe lane's check no longer turns streaming off (#697).
- Setup: a very small extra card (such as a 2 GB display card) is left out of the automatic GPU choice instead of stopping the engine;
--gpusstill uses any card.
- Codex and other Responses API clients: pictures with any
Full list: CHANGELOG.md.
Supported by @vikcious, @ThatSimpleTech, @wjHunter and @davidfernengel.
Maya is free; if it helps you, you can sponsor it: https://github.com/sponsors/mw00
v1.1.57
GLM-5.3 (744B) joins Maya: Z.ai's full GLM-5.3 on Maya's GLM engine, with a new look for the project and fixes.
- GLM-5.3 (744B): Z.ai's GLM-5.3 (744 B parameters, about 40 B active per token) runs on the same engine, server and dashboard as GLM-5.3-Flash, shown as "GLM-5.3 (744B) Β· UD-IQ2_M".
- The files: unsloth's UD-IQ2_M GGUF (238.6 GB), pinned and checked shard by shard. Set it up with
./maya.sh --setup --model GLM-5.3-UD-IQ2_M(Windows:START-MAYA.bat --setup --model GLM-5.3-UD-IQ2_M), or with--gguf-dir <folder>for GGUF files you already have. The dashboard's model list offers it too. - Where it runs: NVIDIA from the Tesla V100 on (CUDA, Linux and Windows) and AMD (HIP, the Strix and Gorgon Halo included). Not on Mac or Pascal cards. About 256 GB of VRAM and RAM together keeps every expert in memory; with less, Maya reads the rest from the SSD while it answers, so keep the model on an NVMe SSD.
- Text only (the model has no vision tower; a picture gets a clear message), up to 1M context: the setup offers the contexts that fit your GPUs.
- Its own defaults from the model card: temperature 1.0, top-p 0.95, thinking at Max with a 32K-token thinking budget, plus Low, High and thinking off. Z.ai's own chat template.
- Everything GLM-5.3-Flash has: drafting with the model's own MTP block, the memory tiers and the CPU lane, speed calibration, tool calls, reasoning in every API (OpenAI, Responses, Anthropic and Maya's model API) and the loop guard. Details: docs/GLM53.md.
- The files: unsloth's UD-IQ2_M GGUF (238.6 GB), pinned and checked shard by shard. Set it up with
- A new look (#881): the new Maya logo in the README, the browser tab, the home-screen icon and the sidebar; issue forms, a docs index and SECURITY.md.
- Which model fits your PC: docs/MEMORY.md lists each model's recommended memory (VRAM and RAM together), with examples.
- Fixes:
- Server: a SIGTERM or Ctrl+C right as the server was waiting could hang it for good. It now stops.
- Maya's model API (
/api/chat,/api/generate):"think": falsenow turns thinking off. Before, thinking stayed on for every model. - DeepSeek: a video in a request now gets a clear error ("reads pictures, not videos") instead of an encoder error, or being dropped next to a picture.
- Setup: the line under the logo now names the model being set up. It said GLM-5.3-Flash whichever model was picked.
Thanks to unsloth for the GLM-5.3 GGUF.
Maya is free; if it helps you, you can sponsor it: https://github.com/sponsors/mw00
v1.1.56
Attach any file in the chat - PDFs, Word, Excel, PowerPoint and more - with every model, and community pull requests: fixes, and opt-in speed-ups for AMD and multi-GPU PCs.
- Files in the chat (#837): the attach button (and drag and drop) now takes any file, for every model:
- PDFs come in as their text. A scanned PDF with no text in it comes in as its first pages as pictures, for a model with images on. This works on every PC: Maya reads PDFs itself, and uses poppler first when it is installed.
- Word, Excel, PowerPoint, OpenDocument, EPUB and RTF files come in as their text (a spreadsheet as its sheets' rows).
- Text and code in any encoding come in as they are, whatever the file's name.
- Videos attach for the models that read them.
- A file Maya can't read (an old .doc, an archive, audio, a program) is not attached, and a message says what it is and what to send instead. A file over the model's context limit is cut, and the message says so.
- APIs: the same files work through OpenAI
file/ Responsesinput_fileparts and Anthropicdocumentblocks, sent inline. - Built on the PDF work in #842 (thanks @org-tekeli-borisp).
- Fixes:
- GLM on two GPUs: since v1.1.52 the drafting decode could crash ("illegal memory access") or slow to a crawl on some two-GPU PCs. A two-GPU split now drafts exactly as in v1.1.51, and a draft with no valid token can no longer read past its table on any setup (#860).
MAYA_GLM_NO_SPEC,MAYA_GLM_NO_MTPandMAYA_GLM_NO_PREFILLset to0,false,offornonow mean off; before, any value turned them on (#860).- GLM: a sampled request without a
seednow gets a fresh random seed, so asking again gives a different answer; aseedin the request (or--seed) still repeats exactly (#845). - GLM EXL3 packs:
MAYA_GLM_SPEC_WINno longer stops the load (#857). - The pack tool keeps GLM-5.3-Flash's expert selection bias in full precision (F32) instead of rounding it (packs built from now on) (#861).
- DeepSeek on AMD: the VRAM BAR size is read correctly for the GPU-to-GPU all-reduce check (#809).
- The Engram RAM lock error now names the current memory-lock limit and how to raise it (#836).
- GLM: a saved conversation whose slot failed to load now tries the next saved slot and then the checkpoint, instead of reading the whole conversation again (#840, fixes #838, thanks @tragicwinds).
- DeepSeek: the loop guard now catches DeepSeek's repeated tool calls too; before, it never fired on DeepSeek (#844, thanks @oscarlius).
- DeepSeek EXL3 quants that name the drafter's head differently (turboderp's) now pack and draft with DSpark on (#835, thanks @pepuscz).
MAYA_DS4_LANE_PARITYis honoured everywhere it applies (#852, thanks @pattalkslaw-del).- The
DS4_HELPER_CHECKdiagnostic no longer fails at random on AMD (it read back through pageable memory) (#868, thanks @pepuscz).
- Speed-ups, AMD, EXL3 quants (the same answers, bit for bit): on RDNA 3 / 3.5 (Strix / Gorgon Halo included) the prompt's expert kernels pick their block size by how many tokens an expert takes - about 39% less kernel time on the contributor's RX 7900 XTX (#864); and the decode kernels' inner product is inlined on every AMD GPU (#834) (thanks @pepuscz). Together, on a Gorgon Halo with DeepSeek-V4.1-Flash EXL3 2.25 bpw, against v1.1.55: decode +4% (32.3 -> 33.6 tok/s), prompt reading (prefill) +3% at 2K and 8K tokens, the same answers.
- Opt-in speed-ups (off by default; each one in docs/SETTINGS.md):
- DeepSeek EXL3 read from the model's files, with a helper GPU: prompt experts the helper already holds are not read from the disk again - prompts about 2x faster on the contributor's two RX 7900 XTX plus an iGPU helper; the decode's look-ahead reads EXL3 experts too (
MAYA_DS4_FLOW_HELPER=1,MAYA_DS4_AHEAD_EXL3=1; the same answers; #865, #866, thanks @pepuscz). - DeepSeek on RDNA 2 (RX 6000) and on AMD with Maya-XS: two prompt settings, +18-33% prompt speed on the contributor's two RX 6900 XT (
MAYA_DS4_PROMPT_F16=1,MAYA_DS4_EXPERT_MMQ=1; they change the answers slightly; #869, #870, thanks @danexpo). - GLM's expert helper: EXL3 packs too, and the helper's work overlapped with the CPU's (+11% decode on the contributor's PC, the same bits); EXL3 decode on Pascal cards (
MAYA_GLM_HELPER_EXL3=1,MAYA_GLM_HELPER_OVERLAP=1,MAYA_GLM_EXL3_DECODE=legacy-f32; #849, #850, #851, thanks @arpleasa). - DeepSeek: lane parity for MXFP4 experts and tensor-parallel groups (#858, thanks @markrussi).
- GLM across several GPUs: direct GPU-to-GPU copies on any pair that supports it, not only the MI50 (
MAYA_GLM_P2P=1, the same answers; #853, thanks @izabera).
- DeepSeek EXL3 read from the model's files, with a helper GPU: prompt experts the helper already holds are not read from the disk again - prompts about 2x faster on the contributor's two RX 7900 XTX plus an iGPU helper; the decode's look-ahead reads EXL3 experts too (
Maya is free; if it helps you, you can sponsor it: https://github.com/sponsors/mw00