AI & Computer Architecture

Packing Ternary Numbers Into 8-Bit Bytes

A small number-theory trick lets five ternary digits — "trits" — squeeze into one ordinary 8-bit byte, and the scheme is fast enough that AI model weights can be unpacked with a single multiplication rather than a slow division.

• Five trits fit into 8 bits at 1.6 bits per trit, reaching 99.06% of the theoretical packing limit.
• Unpacking uses multiplication and a bit-shift instead of modulo/division, making it friendly to SIMD vector units.
• The technique is now used in llama.cpp for ternary model types such as BitNet b1.58, with optimized AVX2 and ARM NEON implementations.

Ternary logic uses three possible values per digit — commonly −1, 0, and 1 — instead of binary's two. A single ternary digit carries log₂(3) ≈ 1.585 bits of information, so the ideal density is to pack as many trits as possible into the fixed 8-bit bytes that real hardware stores. The math picks the block size: 3⁵ = 243 distinct five-trit numbers, and the next power of two is 256. Because 243 fits almost exactly into a byte, five trits become the natural unit. A five-trit value can be written as one number between 0 and 242, then mapped into the 0–255 byte range.

The map is the heart of the trick. After forming the base-3 number from five trits, the packer multiplies by 256 and performs a ceiling division by 243: b = ((n × 256) + 242) ÷ 243. The result is a byte that is slightly larger than the perfect fractional value 256·n/243. That deliberate rounding-up is not an accident — it cancels the off-by-one that would otherwise appear when digits are later extracted.

To unpack, the byte is multiplied by 3 and the top two bits are shifted out to read the leading trit; the rest of the byte stays for the next digit. Repeating the multiply-and-shift five times recovers all five original values, which are then shifted back from {0,1,2} to {−1,0,1}. A short C program verifying all 243 possible five-trit numbers confirms the round trip is lossless. Crucially, every step is multiplication and bit-shift — operations that vector processors execute in parallel — while division and modulo, which the naive unpacking would require, are not available in SIMD integer units.

That hardware match is why the trick matters for AI. Ternary-weight models like BitNet b1.58 store billions of weights in the {−1,0,1} set, so tighter packing means less memory bandwidth to move the same model. With five trits per byte and SIMD-friendly unpacking, the model fits closer to its information-theoretic minimum and decodes faster. The approach has landed in llama.cpp's ternary types, with pull requests covering both x86 AVX2 and ARM NEON, and inspired follow-on work such as byte-level packing of five ternary weights achieving an effective 1.6 bits per weight.

Knowledge takeaway: five trits pack into one byte because 3⁵ = 243 fits just inside 256, hitting 99% of theoretical density; packing multiplies by 256 and ceiling-divides by 243 so unpacking reads digits via multiply-and-shift; the division-free path makes the scheme a natural fit for SIMD-accelerated ternary AI models.