Tesla Patent Don't multiply, add. It saves time and energy Tesla has filed a patent "High Precision Complex Number Based Rotary Positional Encoding Calculation On 8-Bit Compute Hardware"
(US20260017019) There are several ideas in the patent but it appears that this work will be available in the latest AI5 chip. A good introduction is https://www.youtube.com/watch?v=GG9yOsPEGek One of the key ideas is to use logarithms and addition rather than multiplication. Faster and less power needed. NVIDIA is going to hurt from this. |