Hi HN, I built a machine learning library entirely from scratch using only Python and NumPy. I then used it to train a range of models—from classical CNNs, ResNets, RNNs, and LSTMs to modern Transformers and even a toy GPT-2. The motivation came from my curiosity about how to build deep learning models from scratch, like literally from mathematical formulas. I built this project not to replace production-ready libraries like PyTorch or TensorFlow, but to strip away the abstractions and reveal the underlying mathematics of machine learning.

Key points:

- Everything is derived in code — no opaque black boxes.

- API mirrors PyTorch so you can pick it up quickly.

- You can train CNNs, RNNs, Transformers, and even GPT models.

- Designed more for learning/debugging than raw performance.

What’s different here?

While there are many powerful ML libraries available (TensorFlow, PyTorch, Scikit-learn, etc.), they often hide the underlying math behind layers of abstraction. I believe that to truly master these tools, you first need to understand how they work from the ground up. This project explicitly derives all the mathematical and calculus operations in the code, making it a hands-on resource for deepening the understanding of neural networks and library building :)

Check it out:

- Github Repository: https://github.com/workofart/ml-by-hand

- API Documentation: https://ml-by-hand.readthedocs.io/en/latest/

- Examples: Explore models like GPT-2, CNNs, Transformers, and LSTMs in the examples/ folder: https://github.com/workofart/ml-by-hand/tree/main/examples

- Blog Post: Read about the project’s motivation, design, and challenges at https://www.henrypan.com/blog/2025-02-06-ml-by-hand/

I’d love to hear any thoughts, questions, or suggestions — thanks for checking it out!