Within Unicode is defined a DSL used internally by the library implementers to define some business logic, like most DSLs it is turing complete. Anyone with the ability to make you run their rules file already has the ability to make you run arbitrary code (it's a software vendor for software you use).
It's still always fun to find Weird Machines, but as they go, this one is not very weird (it's one of the known families of programming languages, the Mathematica language being the most well known example. The person who specified this most likely was aware that this is turing complete and it's the rules author's responsibility not to write infinite loops).
I find it interesting/weird (that the spec is written in such unrestricted DSL) for pretty much that reason. They could send you input for rules that are in the spec, and hope you translated them to your programming language of choice in a fairly straightforward manner. Which may then have perfectly acceptable average runtime properties, whether you do it in UTF-8 or UTF-32 (fixed-width) space .. but a worst-case that can reliably be triggered with chosen input!
recent example: https://github.com/python/cpython/issues/149079
Software developer’s job description: prove that this instance of the halting problem you’re being paid to work on today is decidable
Injecting new rule is into Uinicode transliterator is remote possiblility, but even if that happens, it does very little.
And so it is (https://en.wikipedia.org/wiki/Semi-Thue_system#Undecidabilit...).
It's a scary and wonderful part of our world that you can bootstrap so much complexity out of a little iterated self-reference.
My only quibble with the article is in this line...
> The surprise is that it lives in a data format for locale files, shipped in every OS, whose specification doesn't mention the possibility.
... I'm not surprised. After all, the processor that interprets the data format is Turing-complete not only in its instructions, but in the page table! See https://github.com/jbangert/trapcc
If anything, when you build a system and it starts to get complex, you have to go out of your way to ensure it's decidable and can't accidentally bootstrap the universe.
In practice, the ICU implementation does limit the number of rewrites, even though the specification doesn't.
Unless you install special rules you won't be doing extra computation.
Lynx does[1], but it does that using iconv, which I think doesn't use UTS #35 transliteration rules.
[1] https://lynx.invisible-island.net/lynx2.9.3/CHANGES.html#:~:...
Yes, ICU is ubiquitous. But, some NLP projects use various other libraries, such as uroman (just for romanization - to Latin script).
Llama.ttf: A font which is also an LLM
https://fuglede.github.io/llama.ttf/
https://news.ycombinator.com/item?id=40766791 (June 23, 2024)