研究者Jay Berry提出了UTF-8000(又称UTF-8K),一种能够编码任意大整数的UTF-8扩展方案1。该方案通过将UTF-8的自同步前缀与自标点符号分离,允许标点符号位溢出到续字节中,在不引入新特殊情况的前提下实现了无限扩展1。
UTF-8000保留了UTF-8的核心特性,包括自同步性、自标点符号性、strcmp排序和无覆长编码等1。编码效率上,n字节代码单元包含5n+1个内容位,例如3字节UTF-8存储16位,而10字节UTF-8000则能存储51位1。该方案仅继承UTF-8的两个特殊情况——ASCII和2字节单元的覆长编码检查,未引入任何新的特殊情况1。
Berry已发布了参考实现,可通过GitHub上的UTF-8000-Python项目访问,并支持通过pipx安装1。UTF-8创造者Ken Thompson对该方案进行了回复,指出"你的前两个扩展(5和6字节)已经被明确预期。7字节版本是唯一新的。这就像用IPv50替代IPv6"1。此外,Berry还收到了Zachary Weinberg(Owl提案作者)和Tom Bishop(UCS-X作者)等人的反馈,讨论了与UTF-∞-8、UTF-16K等方案的权衡1。该方案已被投稿至Summer of Math Exposition 20261。
Researcher Jay Berry has proposed UTF-8000, also known as UTF-8K, an extension of the UTF-8 encoding scheme that enables the representation of arbitrarily large integers without introducing new special cases 1. The scheme decouples UTF-8's self-synchronizing prefix mechanism from its self-delimiting property, allowing delimiter bits to overflow into continuation bytes 1.
The new encoding preserves key UTF-8 characteristics including self-synchronization, self-delimitation, strcmp-compatible ordering, and the absence of overlong encodings 1. A code unit of n bytes contains 5n+1 content bits; for example, a 3-byte UTF-8 sequence stores 16 bits of data, while a 10-byte UTF-8000 sequence stores 51 bits 1. Only two special cases from UTF-8 are retained—ASCII handling and overlong encoding checks for 2-byte units—with no new special cases introduced 1.
Ken Thompson, who created the original UTF-8 standard, responded to Berry's work with: "Your first two extensions (5 and 6 bytes) were already explicitly anticipated. The 7-byte version is the only new one. This is like replacing IPv6 with IPv50" 1. A reference implementation has been released on GitHub under the name UTF-8000-Python and is available for installation via pipx 1. Berry has submitted the proposal to the Summer of Math Exposition 2026 competition and has received feedback from contributors including Zachary Weinberg, author of the Owl proposal, and Tom Bishop, creator of the UCS-X scheme 1.
评论
还没有评论,欢迎留下第一条。