GGUF quant picker · Rust CLI
The quant that fits your machine, first try.
Snug probes your VRAM and RAM, reads the GGUF file list for any HuggingFace model, and downloads the largest quant that actually fits. For anyone running local LLMs who is tired of guessing and re-downloading.
One command, three moves
No menus, no config file, no spreadsheet of bits-per-weight tradeoffs. Snug does the sizing math for you and shows its reasoning in one line.
Read your machine
Detects VRAM (NVIDIA, Apple unified memory, or DXGI adapters) and system RAM in well under a second, with no vendor SDK to install.
List the quants
Pulls the live .gguf file list for the repo from the HuggingFace API (names and sizes only, no weights), grouping multi-part splits into one option.
Pick the winner
Scores each quant against your budget, reserves headroom for context and KV cache, and downloads the largest one that fits, resuming if the transfer drops.
What it actually does for you
Real capabilities, not slogans.
Stops the download-OOM-repeat loop
Instead of guessing Q4, downloading 4 GB, watching it swap, and trying again, you get the right build named up front with the headroom it leaves.
Sizes headroom to your real context
Pass --context 8192 and Snug resolves the model architecture and reserves the exact KV-cache bytes that context needs, not a flat guess.
Scriptable from day one
--json emits one machine-readable object, --yes skips the prompt, and every failure class has its own exit code to branch on.
Tunable when you know better
--reserve-vram pads the margin, --prefer speed steps down one size, and -o <dir> picks where the file lands.
Install and run
One static binary, no runtime. Works on Linux, macOS, and Windows.
Exit codes are stable per failure class: 0 success, 2 network, 3 repo not found, 4 no GGUF files, 5 download failed.
Questions people actually ask
The reasoning Snug automates, spelled out.
Which GGUF quant should I use?
The largest one that fits entirely in your VRAM (or system RAM if you have no GPU) after leaving room for the context window and KV cache. Bigger quants keep more of the model's quality; the trick is finding the biggest one your machine can hold without spilling to disk and crawling. That is the single decision Snug makes for you: it measures your budget, checks every quant's real file size, and names the best fit with the headroom it leaves. If nothing fits, it says so and points at the smallest option rather than pretending.
How much VRAM do I need to run a 7B model?
A 7B model at Q4_K_M is roughly 4 GB on disk, so an 8 GB card usually runs it comfortably with room for a normal context window. Q5_K_M and Q6_K climb toward 5 to 6 GB and want 10 to 12 GB of VRAM once you account for KV cache. Q8_0 is close to 7 GB and is happier on 16 GB. Snug does this arithmetic against your exact card instead of these rules of thumb, and it factors in the context length you plan to use.
What do quant names like Q4_K_M mean?
They describe how aggressively the model weights are compressed. The number is the approximate bits per weight (Q4 is 4-bit, Q8 is 8-bit), and the suffix marks the variant: _K is the k-quant format, and _S, _M, _L are small, medium, and large versions of it. Lower bits mean a smaller file and less quality; higher bits mean the opposite. Snug reads these sizes straight from the repo, so you compare real file sizes rather than decoding the labels by hand.
Does Snug download the whole model just to check?
No. The recommendation step only calls the HuggingFace API for the file tree, which returns quant names and byte sizes with no weights attached. It downloads the actual multi-gigabyte file only after you confirm, and if that transfer is interrupted it resumes from where it stopped with an HTTP range request instead of starting over.
Which platforms and tools does it work with?
Snug runs on Linux, macOS, and Windows, with real hardware probing on each. It works with any GGUF model repo on HuggingFace, so the file it downloads loads directly in llama.cpp, Ollama, LM Studio, koboldcpp, and anything else in that family. It picks the file; your usual runner loads it.