GGML Types¶

Reference table for all GGML tensor quantization types supported by the GGUF format. These types determine how tensor data is stored and how much space each tensor requires.

Quantization Type Table¶

Type	ID	Block Size	Type Size (bytes)	Description
`F32`	0	1	4	32-bit IEEE 754 float
`F16`	1	1	2	16-bit IEEE 754 float
`Q4_0`	2	32	18	4-bit quantization, symmetric
`Q4_1`	3	32	20	4-bit quantization, asymmetric
`Q5_0`	6	32	22	5-bit quantization, symmetric
`Q5_1`	7	32	24	5-bit quantization, asymmetric
`Q8_0`	8	32	34	8-bit quantization, symmetric
`Q8_1`	9	32	36	8-bit quantization, asymmetric
`Q2_K`	10	256	84	K-quant 2-bit
`Q3_K`	11	256	110	K-quant 3-bit
`Q4_K`	12	256	144	K-quant 4-bit
`Q5_K`	13	256	176	K-quant 5-bit
`Q6_K`	14	256	210	K-quant 6-bit
`Q8_K`	15	256	292	Internal 8-bit intermediate quantization
`IQ2_XXS`	16	256	66	Importance quant 2-bit, extra-extra-small
`IQ2_XS`	17	256	74	Importance quant 2-bit, extra-small
`IQ3_XXS`	18	256	98	Importance quant 3-bit, extra-extra-small
`IQ1_S`	19	256	50	Importance quant 1-bit, small
`IQ4_NL`	20	32	18	Importance quant 4-bit, non-linear
`IQ3_S`	21	256	110	Importance quant 3-bit, small
`IQ2_S`	22	256	82	Importance quant 2-bit, small
`IQ4_XS`	23	256	136	Importance quant 4-bit, extra-small
`I8`	24	1	1	8-bit integer
`I16`	25	1	2	16-bit integer
`I32`	26	1	4	32-bit integer
`I64`	27	1	8	64-bit integer
`F64`	28	1	8	64-bit IEEE 754 float
`IQ1_M`	29	256	56	Importance quant 1-bit, medium
`BF16`	30	1	2	Brain float 16
`TQ1_0`	34	256	54	Ternary quant 1-bit
`TQ2_0`	35	256	66	Ternary quant 2-bit
`MXFP4`	39	32	17	Microscaling FP4
`NVFP4`	40	64	36	NVIDIA FP4
`Q1_0`	41	128	18	1-bit block quantization
`Q2_0`	42	64	18	2-bit block quantization

Deprecated and removed types (IDs 4, 5, 31–33, 36–38)

Eight type IDs exist in the ggml_type enum but have blck_size = 0 and type_size = 0. Passing any of these to ggml_new_tensor() causes a division-by-zero (SIGFPE) in ggml_row_size().

ID	Former Name	Status
4	(unnamed)	DEPRECATED
5	(unnamed)	DEPRECATED
31	`Q4_0_4_4`	REMOVED; use `Q4_0` with runtime repacking
32	`Q4_0_4_8`	REMOVED
33	`Q4_0_8_8`	REMOVED
36	`IQ4_NL_4_4`	REMOVED
37	`IQ4_NL_4_8`	REMOVED
38	`IQ4_NL_8_8`	REMOVED

All 8 are within the valid enum range (0 ≤ type < GGML_TYPE_COUNT), so range-only checks do not catch them. The GGUF parser validates blck_size == 0 at parse time and rejects files with these types. The ggml-rpc backend validates them since PR #20712 (March 2026). However, the ggml C API (ggml_new_tensor()) performs no validation; callers must check themselves.

The table is Crucible's supported sizing map

GGMLTypeInfoMap is the package contract used by TensorDataSize; it is not a promise that every upstream backend accepts every type in every model context. GGMLTypeCount is 43, so values at or above 43 are outside the mirrored enum range.

Tensor Data Size Formula¶

The byte size of a tensor's data is calculated as:

data_size = ceil(n_elements / block_size) * type_size

Where:

n_elements is the product of all tensor dimensions
block_size and type_size come from the table above

In Go, using Crucible's API:

size, err := gguf.TensorDataSize(tensor.NElements(), tensor.Type)

Examples¶

Tensor Shape	Type	Elements	Blocks	Data Size
`[4096]`	`F32`	4,096	4,096	16,384 bytes
`[4096]`	`F16`	4,096	4,096	8,192 bytes
`[4096, 4096]`	`Q4_0`	16,777,216	524,288	9,437,184 bytes
`[4096, 4096]`	`Q2_K`	16,777,216	65,536	5,505,024 bytes
`[32000, 4096]`	`Q6_K`	131,072,000	512,000	107,520,000 bytes

Type Categories¶

Floating Point¶

F16, BF16, F32, F64 -- unquantized floating-point types with block size 1.

Legacy Quantization (Q4/Q5/Q8)¶

Q4_0, Q4_1, Q5_0, Q5_1, Q8_0, Q8_1 -- original quantization schemes with 32-element blocks. Each block stores a scale factor plus packed low-bit weights.

K-Quantization¶

Q2_K through Q6_K -- improved quantization with 256-element blocks and per-block super-scales. Better quality-to-size ratio than legacy types.

Importance Quantization (IQ)¶

IQ1_S, IQ1_M, IQ2_XXS, IQ2_XS, IQ2_S, IQ3_XXS, IQ3_S, IQ4_NL, IQ4_XS -- state-of-the-art quantization using importance weights. Achieves the highest compression ratios with minimal quality loss.

Integer¶

I8, I16, I32, I64 -- raw integer storage with block size 1. Used for embedding indices, token IDs, and other non-weight data.

Ternary and FP4¶

TQ1_0, TQ2_0, MXFP4, NVFP4, Q1_0, and Q2_0 are newer compact block formats. The block and byte sizes above come from Crucible's tested GGMLTypeInfoMap; keep this page in sync with that map rather than inferring sizes from the type names.