- Published on
LLM Model Formats Explained (2026)
- Authors
- Name
- DeepHorus
The landscape of large language model formats in 2026 is defined by a persistent confusion between containers and quantization methods. Understanding that distinction is essential for anyone deploying open-weight models. A container defines how tensors are stored on disk, while a quantization method defines how weights are squeezed into fewer bits. Safetensors and GGUF are containers; GPTQ, AWQ, bitsandbytes NF4, and llama.cpp K-quants are methods. EXL2 and EXL3 combine a method with a storage layout tied to one inference library.
Safetensors, created at Hugging Face, addresses a critical security flaw in older PyTorch pickle files. Loading a pickle file can execute arbitrary code, making untrusted checkpoints dangerous. Safetensors uses a small JSON header plus raw tensor buffers, with nothing executable inside. It also supports memory-mapped loading and is now a PyTorch Foundation project. Notably, most GPTQ, AWQ, EXL2, and MLX models are still stored in safetensors files, with quantization living in the tensor contents and config.
GGUF, the binary format for llama.cpp, was created by Georgi Gerganov and introduced on August 21, 2023, replacing the older GGML format. GGML could not identify a model's architecture, and adding hyperparameters broke existing files. GGUF switched to typed key-value metadata, enabling extensibility without breaking compatibility. Its design goals include single-file deployment, mmap compatibility, and complete information inside the file, including tokenizer and chat template.
Quantization size follows simple arithmetic: weight memory equals parameters times bits-per-weight divided by eight. For example, a Q4_K_M super-block holds 256 weights at 4 bits, plus scales and minimums, averaging 4.5 bits per weight. The _S, _M, and _L suffixes indicate mixes of different quantization schemes, not new types. Newer types like TQ1_0, TQ2_0, and MXFP4 are emerging for extreme compression.
These choices matter because quality and size trade off directly. Hugging Face's reference table for a Llama-2-7B-class model illustrates the range. Calibration data via llama.cpp's imatrix can improve low-bit quantizations, and Q8_0 remains the standard near-lossless GGUF choice. As formats evolve, EXL3 and MLX show a trend toward library-specific layouts, but the underlying container-versus-method distinction will remain central to model deployment.
