GGML Types¶

Reference table for all GGML tensor quantization types supported by the GGUF format. These types determine how tensor data is stored and how much space each tensor requires.

Quantization Type Table¶

Type	ID	Block Size	Type Size (bytes)	Description
`F32`	0	1	4	32-bit IEEE 754 float
`F16`	1	1	2	16-bit IEEE 754 float
`Q4_0`	2	32	18	4-bit quantization, symmetric
`Q4_1`	3	32	20	4-bit quantization, asymmetric
`Q5_0`	6	32	22	5-bit quantization, symmetric
`Q5_1`	7	32	24	5-bit quantization, asymmetric
`Q8_0`	8	32	34	8-bit quantization, symmetric
`Q8_1`	9	32	36	8-bit quantization, asymmetric
`Q2_K`	10	256	84	K-quant 2-bit
`Q3_K`	11	256	110	K-quant 3-bit
`Q4_K`	12	256	144	K-quant 4-bit
`Q5_K`	13	256	176	K-quant 5-bit
`Q6_K`	14	256	210	K-quant 6-bit
`IQ2_XXS`	16	256	66	Importance quant 2-bit, extra-extra-small
`IQ2_XS`	17	256	74	Importance quant 2-bit, extra-small
`IQ3_XXS`	18	256	98	Importance quant 3-bit, extra-extra-small
`IQ1_S`	19	256	50	Importance quant 1-bit, small
`IQ4_NL`	20	32	18	Importance quant 4-bit, non-linear
`IQ3_S`	21	256	110	Importance quant 3-bit, small
`IQ2_S`	22	256	82	Importance quant 2-bit, small
`IQ4_XS`	23	256	136	Importance quant 4-bit, extra-small
`I8`	24	1	1	8-bit integer
`I16`	25	1	2	16-bit integer
`I32`	26	1	4	32-bit integer
`I64`	27	1	8	64-bit integer
`F64`	28	1	8	64-bit IEEE 754 float
`IQ1_M`	29	256	56	Importance quant 1-bit, medium
`BF16`	30	1	2	Brain float 16
`TQ1_0`	34	256	42	Ternary quant 1-bit
`TQ2_0`	35	256	66	Ternary quant 2-bit

Deprecated and removed types (IDs 4, 5, 31–33, 36–38)

Eight type IDs exist in the ggml_type enum but have blck_size = 0 and type_size = 0. Passing any of these to ggml_new_tensor() causes a division-by-zero (SIGFPE) in ggml_row_size() — see CRUCIBLE-2026-004.

ID	Former Name	Status
4	(unnamed)	DEPRECATED
5	(unnamed)	DEPRECATED
31	`Q4_0_4_4`	REMOVED — use `Q4_0` with runtime repacking
32	`Q4_0_4_8`	REMOVED
33	`Q4_0_8_8`	REMOVED
36	`IQ4_NL_4_4`	REMOVED
37	`IQ4_NL_4_8`	REMOVED
38	`IQ4_NL_8_8`	REMOVED

All 8 are within the valid enum range (0 ≤ type < GGML_TYPE_COUNT), so range-only checks do not catch them. The GGUF parser validates blck_size == 0 at parse time and rejects files with these types. The ggml-rpc backend validates them since PR #20712 (March 2026). However, the ggml C API (ggml_new_tensor()) performs no validation — callers must check themselves.

ID 15 (Q8_K)

ID 15 is assigned to Q8_K (blck_size=256, type_size=292), used internally for intermediate quantization results. It is not typically seen in GGUF files.

Tensor Data Size Formula¶

The byte size of a tensor's data is calculated as:

data_size = ceil(n_elements / block_size) * type_size

Where:

n_elements is the product of all tensor dimensions
block_size and type_size come from the table above

In Go, using Crucible's API:

size, err := gguf.TensorDataSize(tensor.NElements(), tensor.Type)

Examples¶

Tensor Shape	Type	Elements	Blocks	Data Size
`[4096]`	`F32`	4,096	4,096	16,384 bytes
`[4096]`	`F16`	4,096	4,096	8,192 bytes
`[4096, 4096]`	`Q4_0`	16,777,216	524,288	9,437,184 bytes
`[4096, 4096]`	`Q2_K`	16,777,216	65,536	5,505,024 bytes
`[32000, 4096]`	`Q6_K`	131,072,000	512,000	107,520,000 bytes

Type Categories¶

Floating Point¶

F16, F32, F64 -- unquantized types with block size 1. Every element is stored at full precision.

Legacy Quantization (Q4/Q5/Q8)¶

Q4_0, Q4_1, Q5_0, Q5_1, Q8_0, Q8_1 -- original quantization schemes with 32-element blocks. Each block stores a scale factor plus packed low-bit weights.

K-Quantization¶

Q2_K through Q6_K -- improved quantization with 256-element blocks and per-block super-scales. Better quality-to-size ratio than legacy types.

Importance Quantization (IQ)¶

IQ1_S, IQ1_M, IQ2_XXS, IQ2_XS, IQ2_S, IQ3_XXS, IQ3_S, IQ4_NL, IQ4_XS -- state-of-the-art quantization using importance weights. Achieves the highest compression ratios with minimal quality loss.

Integer¶

I8, I16, I32, I64 -- raw integer storage with block size 1. Used for embedding indices, token IDs, and other non-weight data.