FP16 & bfloat16 Bit Pattern Visualizer
Visualize the bit patterns of half-precision floating point (FP16, IEEE 754 binary16) and bfloat16. Convert decimal to bits and back, with color-coded sign, exponent, and mantissa fields. Useful for learning about ML quantization and inference speedup.
FP16 vs. bfloat16 structure
Both formats total 16 bits, but they split those bits between the exponent and mantissa very differently — giving each a very different representable range and precision.
| Item | FP16 (IEEE 754 半精度) | bfloat16 |
|---|---|---|
| Total bits | 16 | 16 |
| Exponent bits | 5 | 8 |
| Mantissa bits | 10 | 7 |
| Representable range | About ±6.1×10⁻⁵ to ±65504 (narrower range, but more mantissa bits for precision) | About ±1.2×10⁻³⁸ to ±3.4×10³⁸ (same wide range as float32, but lower precision) |
| Main use case | Image processing, graphics, and memory-efficient inference (half-precision training) | Deep learning training (TPUs and modern GPUs) — no need to worry about exponent overflow when converting from float32 |
bfloat16 shares float32's 8-bit exponent width and bias (127), which means it can be produced from a float32 value by simply truncating the mantissa — a handy implementation shortcut.
Usage tips
- Even for the same value like 0.1, FP16 and bfloat16 round it very differently. Switch formats and compare the mantissa bit patterns.
- Because bfloat16 shares float32's exponent width and bias, it resists overflow even for large values above 65504. FP16, by contrast, overflows to infinity as soon as you exceed 65504.
- Converting "70000" to FP16 overflows to infinity, while bfloat16 represents it fine (with reduced precision).
- When quantizing models in frameworks like PyTorch or TensorFlow, bfloat16 converts easily from float32, while FP16's narrower range sometimes requires pre-scaling.
- To convert bits back to decimal, enter a 4-digit hex value with a `0x` prefix, or a 16-digit binary value (the `0b` prefix is optional).
Frequently asked questions
Side Note — Why machine learning wanted a "less precise" number format
Around 2018, Google devised a new 16-bit floating-point format called bfloat16 for its Tensor Processing Units (TPUs). FP16 (IEEE 754 half precision) already existed and was in practical use in image processing and graphics, but its mere 5 exponent bits gave it a narrow representable range — a poor fit for the vanishing and exploding gradients that show up during deep learning training.
bfloat16's design idea is straightforward: sacrifice mantissa precision, but keep the exponent exactly as wide as float32's 8 bits, so the extremely large and small values that crop up during training can be handled safely. The name "Brain Floating Point" comes from Google Brain, the machine learning research team that created it.
In conventional computer science, reducing floating-point precision has long been treated as a breeding ground for bugs. But deep learning models have an unusual property: modest numerical error rarely hurts the final prediction accuracy much. Turning that "somewhat inaccurate but still works overall" property to its advantage — and cutting memory and compute cost in half with a 16-bit format — represents an optimization distinctly at home in machine learning, quite apart from traditional computer science wisdom.