Almost a year ago I shared an autodiff I wrote from scratch and an implementation of LeNet in it (https://news.ycombinator.com/item?id=41875358). Another HNer, tightbookkeeper, offered some helpful suggestions.

The actual code is here: https://github.com/mebassett/quixotic-autodiff/blob/master/d...