The annotated PyTorch training loop

The annotated PyTorch training loop(idlemachines.co.uk)

36 points by smaddrellmander 2 days ago | 9 comments

GL26 58 minutes ago |

PyTorch already does a huge job at keeping DL not verbose. It captures the complexity to allow you to make a whole GPT model with it, running on your GPU, and keep the code readable. Yes, there are a lot of breaking points, but at some point, DL is supposed to be non trivial, it handles math concepts most people don’t understand. That is why Data Scientists have a job ;)

sva_ 21 minutes ago | |

I mean partial derivatives aren't that complicated if you know normal derivatives, which most people do. And backpropagation isn't too difficult either.

The value of PyTorch lies more in utilizing accelerators like GPUs while offering a nice abstraction. But you can build your own (inefficient) tensor library without too much effort as e.g. Andrej Karpathy has shown in his "NN zero to hero" youtube series.

vovavili 1 hour ago |

The host website seems extremely interesting in general. Anyone here used it before?

f3408fh 1 hour ago | |

The design is similar to what Claude generates. Makes me skeptical that all the content is AI-generated.

vovavili 1 hour ago | | |

This looks better than 90% of websites I visited. The palette is pleasant, content has the right density and spacing is very relaxed. If this is AI-generated without some clever prompt tuning, frontend as a profession is done.

Synthetic7346 1 hour ago |

https://idlemachines.co.uk/courses/foundations doesn't render correctly on mobile

moelf 1 hour ago | |

even the submitted page doesn't, huge margin if you zoom out on mobile (Firefox)

LoganDark 57 minutes ago | | |

I've seen that huge margin when zooming out issue before. I'm using Safari on iOS, but have also experienced this issue in Chrome on Android. It's probably some sort of weird compatibility thing for desktop websites. You start zoomed correctly, but for some reason, there's horizontal scroll and you can also zoom out. For seemingly no reason, as I doubt any of the websites where this happens intended it.

Note that I can't reproduce it myself on this submission, but I've seen it so many times and in so many different places that I don't think it's a website issue.