Explaining Llm.c in Layman Terms(nitter.poast.org) |
Explaining Llm.c in Layman Terms(nitter.poast.org) |
They probably know what C is in the very general sense, but there are lots and lots of people people who know all the deep learning stuff but have never written a line of C or really understands the when/why you might want to use C and the benefits and limitations of the language. You have to remember that a lot of people currently working in this space don't come from a computer science and programming background and don't have that general background knowledge of computer science.
It will be very interesting to implement this small llm in D language and use the new tiny autograd library in D [1]. Since now D compiler does support C, this should results in less line of codes excluding the small autograd library in D [2]. Given that the D code will be able to inline C code, and even CTFE it, if done correctly the implementation will be faster as well.
[1] A tiny autograd library:
https://code.dlang.org/packages/tiny-autodiff
[2] Adding ANSI C11 C compiler to D so it can import and compile C files directly:
It's not a thread, no nitter needed.
Llm.c – LLM training in simple, pure C/CUDA