Trap – Transformers in APL(github.com) |
Trap – Transformers in APL(github.com) |
This is really cool. At about 150 lines, terse indeed. And it makes sense that of course APL could work well with gpus, but I’m kind of surprised there’s enough of it still out in the wild so that there’s already a reliable tool chain for doing this.
I've seen at least an APL implementation running on top of Julia, thanks to macros.
Julia has good GPU support, and it makes it easy to compose that support with any library.
However, kdb+ and q, which are APL descendants, have good GPU support already: https://code.kx.com/q/interfaces/gpus. But licenses are not cheap...
I hope one day its normal like the 1000s of CPU languages. Would be nice to have more than 10 gpu languages.
(I got the t-shirt)
After looking at the code, I find this claim questionable.
APL was originally a rewrite and normalisation of traditional math notation for use on blackboards. Before it was anything to do with computers it was linear algebra without all the bizarre precedence rules and with some common useful operations.
I've actually spent the better part of last year wondering why we _haven't_ been using APL for deep learning. And actually I've been wondering why we don't just use APL for everything that operates over arrays, like data lakes and such.
Honestly, APL is probably a good fit for compilers. I seem to remember a guy who had some tree-wrangling APL scheme, and could execute his compiler on a GPU. But I can't find it now.
Here are some videos related to his work: https://www.youtube.com/playlist?list=PLDU0iEj6f8duXzmgnlGX4...
Co-dfns was most recently discussed on Hacker News 3 months ago: https://news.ycombinator.com/item?id=40928450
JAX?
I am the author of this project. If anyone has any questions concerning trap, I'd be more than happy to address them.
Well done.
k-torch llm(61) 14M 2 14 6 288 288 x+l7{l8x{x%1+E-x}l6x}rl5x+:l4@,/(hvi,:l3w)Ss@S''h(ki,:ql2w)mql1w:rl0x (18M 2 32000 288)
which apparently can run on the gpu someone told me on discord (but i'm not sure if it's true or not).
It sure did to me, even as someone who has written (a trivial amount of) J. But the argument that follows is more than convincing.
I agree with a sibling comment, also heavily downvoted, that the real blackboard notation is linear algebra notation. Either that, or pseudocode. Python and Haskell look like pseudocode. This doesn’t, and it doesn’t matter what the developer was targeting, he didn’t hit the target.
"I'm sceptical that one can write +.× on a blackboard to indicate matrix cross product"
"well, one can"
"it doesn't look like one could. This is not a low effort reply. Yours is a low effort reply. The 'real' way to write a matrix cross product by hand in chalk on a blackboard is with nested loops and Python".
The APL creator Ken Iverson's paper Notation as a Tool of Thought[1] talks through an introduction to why it was designed the way it was; to do things Python and Haskell and pseudocode and traditional math notation don't do. Early on he writes:
> "APL, a general-purpose language which originated in an attempt to provide clear and precise expression in writing and teaching, and which was implemented as a programming language only after several years of use and development"
To use a thing for several years for expressing and teaching mathematics, is evidence that it can be used for that. It was designed for things code and traditional math notation don't do, e.g. code doesn't hide irrelevant details, math notation has inconsistent precedence, both have instances of wildly different syntax and symbols for closely related concepts which hinder seeing the connections. Code isn't amenable to formal proofs, pseudocode isn't good at expressing a problem, only expressing instructions for solving a problem.
The APL of modern times has a lot added since 1972, but the core is still there. See also anecdote in [2] a demonstration of K. Iverson casually writing APL on a napkin at dinner to solve a problem.
[1] https://www.eecg.utoronto.ca/~jzhu/csc326/readings/iverson.p...