Draft:HiFloat8

  • Comment: The three don't have in-depth coverage of the topic, and the last source isn't usable per WP:ARXIV. JustARandomSquid (talk) 15:21, 8 May 2026 (UTC)


The HiFloat8 (HiF8) is an 8‑bit tapered‑precision floating‑point format proposed for the training and inference of deep learning models.[1][2][3] Under an 8‑bit encoding budget, HiF8 introduces an additional prefix‑coded dot field that selects the exponent field width (and a denormal mode), which allows the available mantissa bits to vary with the encoded exponent range and better balance numerical precision and dynamic range.[4]

Definition

HiF8 is an 8-bit floating-point format designed for deep learning workloads. As illustrated in Fig. 1, in addition to the conventional sign, exponent, and mantissa fields (with the exponent using an implicit leading magnitude bit, shown in red), HiF8 introduces a variable-length dot field encoded with prefix codes. The dot field directly indicates the storage width of the exponent field and carries a denormal flag, thereby enabling dynamic allocation of exponent and mantissa bit widths within the 8-bit encoding budget.[4]

Normal encoding mode

Fig. 1 HiFloat8 Format Definition

In the normal mode, the dot field encodes five values (0–4), corresponding to stored exponent widths from 0 to 4 bits. Given the sign bit, the dot field, and the selected exponent width, the mantissa width is determined by the 8-bit budget. Let Dot denote the decoded value of the dot field. When Dot = 0, no exponent bits are stored and the exponent is defined as E = 0. When Dot = 1–4, HiF8 encodes the exponent in sign-magnitude form and treats the most significant magnitude bit as an implicit 1 (shown in red in Fig. 1) and therefore does not store it. Specifically, with Dot = 1, the only possible exponents are E ∈ {−1, +1}, since E = 0 is represented by Dot = 0; when Dot = 2, the only possible exponents are E ∈ {−3, −2, +2, +3} (excluding {−1, +1} already covered by Dot = 1). By extension, the five cases Dot = 0–4 jointly represent all exponents in [-15, 15] without duplication.[4]

Denormal encoding mode

In the denormal mode, HiF8 further extends its dynamic range. As shown in Fig. 1, when the dot field indicates the denormal mode, no exponent field is stored; instead, the remaining mantissa bits are used to encode additional low-magnitude exponent values. Specifically, the denormal mode extends the 31 exponents [-15, 15] available in normal mode by seven additional exponents [-22, -16], with one additional pattern reserved for special values, yielding 38 distinct exponents in total over [-22, 15].[4]

Special values

Fig. 2 Special Value Support in HiF8

In addition to regular numeric encodings, HiF8 defines four special values: Zero, NaN, positive and negative infinity. HiF8 does not distinguish between +0 and −0, using a single bit pattern to represent zero. Together, these special values cover common exceptional and boundary cases encountered in floating-point computation.[4]

Properties

Tapered precision

Fig. 3 Tapered Precision

As illustrated in Fig. 3, fixed-field 8-bit floating-point formats such as FP8-E4M3 and FP8-E5M2 use constant exponent and mantissa widths, which can be difficult to simultaneously meet the dynamic-range and numerical precision requirements of deep learning training and inference within an 8-bit budget. In contrast, HiF8 exhibits a tapered-precision profile in which the effective precision varies with magnitude: it allocates more mantissa bits for values with magnitude near 1 while allocating more bits to the exponent toward the tails.[4]

Non-redundant encoding

HiF8 represents the exponent using sign-magnitude encoding and fixes the most significant magnitude bit to a known implicit value that is not stored. This construction ensures that the exponent ranges associated with different stored bit-widths do not overlap, resulting in a non-redundant encoding.[4]

Wide dynamic range

With the denormal mode, HiF8 extends the representable exponent range from [-15, 15] to [-22, 15], yielding 38 distinct exponent values in total—close to FP16’s 40 exponents.[4]

References

  1. ^ Pedro, Juan. "Huawei outlines roadmap for Ascend AI chips". RCR Wireless News. Retrieved 6 February 2026.
  2. ^ Udinmwen, Efosa (4 October 2025). "Huawei Ascend 950 vs Nvidia H200 vs AMD MI300 Instinct: How do they compare?". TechRadar. Retrieved 6 February 2026.
  3. ^ "Huawei unveils next-gen Ascend chips, SuperPods, and UnifiedBus interconnect". Converge Digest. 18 September 2025. Retrieved 6 February 2026.
  4. ^ a b c d e f g h Luo, Yuanyong; et al. "Ascend HiFloat8 Format for Deep Learning". arXiv. Retrieved 5 February 2026.

Content Disclaimer

Informasi ini disarikan dari Wikipedia dan disajikan kembali untuk tujuan edukasi. Konten tersedia di bawah lisensi CC BY-SA 3.0. Kami tidak bertanggung jawab atas ketidakakuratan data yang bersumber dari kontribusi publik tersebut.

  1. The information displayed on this website is sourced in part or in whole from Wikipedia and has been adapted for the purpose of restating it. We strive to provide accurate and relevant information, however:
  2. There is no guarantee of absolute accuracy. Wikipedia is an open, collaborative project that can be edited by anyone, so information is subject to change.
  3. It is not intended to constitute professional advice. The content displayed is for informational and educational purposes only. For important decisions (e.g., medical, legal, or financial), please consult a professional.
  4. Content copyright. Wikipedia is licensed under the Creative Commons Attribution-ShareAlike License (CC BY-SA). This means that content may be reused with appropriate attribution and shared under a similar license.
  5. Responsible use. Any risk arising from the use of information from this website is entirely the responsibility of the user.