30papers.com – Ilya's 30 essential ML papers, in a beginner friendly format

30papers.com – Ilya's 30 essential ML papers, in a beginner friendly format(30papers.com)

200 points by notmcrowley 4 hours ago | 36 comments

HAL3000 54 minutes ago |

Someone posts on X, "These are Ilya’s 30 papers", gives no source, doesn't say where he got it from, and isn't connected to either Ilya or Carmack (Ilya gave him the list).

Then someone vibe codes a barely usable website based on that, and it lands on the HN front page? Is this correct?

youniverse 21 minutes ago | |

Compiled resources for nerds are catnip. Hit that bookmark/upvote button to never get to it :)

dominotw 24 minutes ago | |

they kind of mention the source on their website though

" rumoured list of papers that Ilya Sutskever gave to John Carmack. "

there is aslo manning book called illya list

https://www.manning.com/books/sutskevers-list

jackp96 10 minutes ago |

So the styling and animation work looks really cool (when isolated), but they distract from the content itself, IMO.

I think it'd work better if you featured the animated background effect toward the top of the page and shifted toward static graphics (or much subtler animations) as the user scrolls.

And I don't think the zoom-out effect on the listing cards has the intended effect; I found myself wanting to get a better look at the papers and was a little disappointed/annoyed when they got smaller and harder to see as I pulled them into view.

The colors/shadows/layout all looks really nice, but I feel like the animations (as-is) ultimately detract from the experience rather than add to it. Thanks for sharing, though!

clintonc 2 hours ago |

I wish this were organized according to suggested/logical reading order. For example, the paper introducing the attention mechanism probably ought to precede "attention is all you need".

janpmz 39 minutes ago |

After seeing this for the first time, I've build PdfToMp3 to listen to these papers. It has now evolved into ListenDock. Fun fact: PdfToMp3 existed before NotebookLM and I already had "overviews", but I called them teacher explanations.

Here is an example of a "Teacher Explanation" of the paper "Quantifying the Rise and Fall of Complexity in Closed Systems: The Coffee Automaton"

https://listendock.com/e/quantifying_the_rise_and_fall_of_co...

notmcrowley 4 hours ago |

Author here. First year CS student at Trinity College Dublin. I Built this because when I was getting into reading research papers I ended up burning a ton of my Claude usage asking questions other people have probably already asked. The website is just a side project and definitely a WIP. Happy to answer questions or take PRs on GitHub.

gowld 1 hour ago | |

An option to disable animation and show the paper links in a simple list would be helpful.

anomaloustho 17 minutes ago | | |

The problem is the background is often times doing a wave motion across the screen.

Then the foreground content is doing an in/out undulation on top. So you’re seeing an undulating in/out in every possible direction + the background. And the foreground animations are all at the same time. So it’s not that we’re emphasizing any one thing. We’re emphasizing all of it.

The key with animations is in what they’re trying to draw attention to, the character of the movement, and the timing of it. You usually don’t want everything to equally animate at once.

I would: • Use background movement that also isn’t a “wave” • Stagger the timing of foreground animations so the main content is emphasized, followed by a pause, followed by the sidebars • Change the nature of the animations so they’re not doing the same essentially thing “zoom and pan” - so have the center zoom and pan, but do something different for the sides.

jodacola 1 hour ago | | |

Agree on the animation.

As an aside, I've seen folks mention respecting reduced animation hints and such in the past and was always curious about this because I've never had any negative experiences with animations... until now!

Something about the animations on this site did my brain in while scrolling through the papers, and now I "get it."

groby_b 1 hour ago | |

I think it'd be interesting to hear what you think the goal of the site is.

Is it just rehosting the list, plus a reformatted copy of the papers? I was hoping you'd have at least annotated them with what you'd learned?

jawarner 1 hour ago |

Noting the theory papers on Kolmorogov complexity. For those not familiar, Ilya argues that the reason why neural networks generalize -- why they work at all -- is because they are effectively finding a simple description of their training data, converging down onto the limit of the Kolmorogov complexity. [1]

[1] https://www.youtube.com/watch?v=AKMuA_TVz3A

eachro 6 minutes ago |

Anyone got a list for the agentic LLM age?

quibono 2 hours ago |

I was confused for a minute, I thought this was "top 30 papers by Ilya" and was then wondering why "Quantifying the Rise and Fall of Complexity in Closed Systems: The Coffee Automaton" is on the list.

> In additition, even though I have read the vast majority of the papers featured on the website, I have not read through each of the website's versions end to end.

Website's versions, as in - the actual text or the "explanations"? Either way this is a big red flag.

imenani 2 hours ago |

Nice presentation of the list!

I'd recommend watching a few of his talks/podcasts before during reading these to get the overview and how all the bits in these works tie together.

https://www.dwarkesh.com/p/ilya-sutskever

https://simons.berkeley.edu/talks/ilya-sutskever-openai-2023...

https://www.dwarkesh.com/p/ilya-sutskever-2

cute_boi 1 hour ago |

No need stupid moving texts.

CS231n: Convolutional Neural Networks for Visual Recognition - https://cs231n.github.io/

The Unreasonable Effectiveness of Recurrent Neural Networks - https://karpathy.github.io/2015/05/21/rnn-effectiveness/

Understanding LSTM Networks - https://colah.github.io/posts/2015-08-Understanding-LSTMs/

ImageNet Classification with Deep Convolutional Neural Networks - https://papers.nips.cc/paper/2012/hash/c399862d3b9d6b76c8436...

Deep Residual Learning for Image Recognition - https://arxiv.org/abs/1512.03385

Multi-Scale Context Aggregation by Dilated Convolutions - https://arxiv.org/abs/1511.07122

Identity Mappings in Deep Residual Networks - https://arxiv.org/abs/1603.05027

Recurrent Neural Network Regularization - https://arxiv.org/abs/1409.2329

Deep Speech 2: End-to-End Speech Recognition in English and Mandarin - https://arxiv.org/abs/1512.02595

Order Matters: Sequence to Sequence for Sets - https://arxiv.org/abs/1511.06391

Neural Machine Translation by Jointly Learning to Align and Translate - https://arxiv.org/abs/1409.0473

Pointer Networks - https://arxiv.org/abs/1506.03134

Attention Is All You Need - https://arxiv.org/abs/1706.03762

The Annotated Transformer - https://nlp.seas.harvard.edu/annotated-transformer/

Neural Turing Machines - https://arxiv.org/abs/1410.5401

A Simple Neural Network Module for Relational Reasoning - https://arxiv.org/abs/1706.01427

Relational Recurrent Neural Networks - https://arxiv.org/abs/1806.01822

Neural Message Passing for Quantum Chemistry - https://arxiv.org/abs/1704.01212

Scaling Laws for Neural Language Models - https://arxiv.org/abs/2001.08361

GPipe: Efficient Training of Giant Neural Networks using Pipeline Parallelism - https://arxiv.org/abs/1811.06965

Keeping Neural Networks Simple by Minimizing the Description Length of the Weights - https://www.cs.toronto.edu/~hinton/absps/colt93.pdf

A Tutorial Introduction to the Minimum Description Length Principle - https://arxiv.org/abs/math/0406077

The First Law of Complexodynamics - https://scottaaronson.blog/?p=762

Quantifying the Rise and Fall of Complexity in Closed Systems: The Coffee Automaton - https://arxiv.org/abs/1405.6903

Kolmogorov Complexity - https://onlinelibrary.wiley.com/doi/book/10.1002/047174882X

Variational Lossy Autoencoder - https://arxiv.org/abs/1611.02731

Machine Super Intelligence - https://www.vetta.org/documents/Machine_Super_Intelligence.p...

omneity 2 hours ago |

I thought the actual 30 papers have never been disclosed. Do you have a source tying the recommendations back to Ilya, or did you come up with this list?

ayhanfuat 2 hours ago | |

I think someone on Twitter made it up. It was also 40 papers, not 30. https://dallasinnovates.com/exclusive-qa-john-carmacks-diffe...

renyicircle 2 hours ago | |

This list was made by some guy on twitter. https://x.com/keshavchan/status/1787861946173186062

It's unknown whether it has anything to do with Ilya Sutskever.

prideout 2 hours ago |

Kolmogorov Complexity looks interesting. It seems to formalize Occam’s Razor and the notion that intelligence = compression.

Destructotor 1 hour ago | |

If you find this interesting, you should look into Solomonoff induction. It combines Kolmogorov complexity with Bayes rule to provide a general framework for inductive inference, and naturally formalizes Occam's razor.

Lerc 1 hour ago | |

I wouldn't say so about Occam's Razor which is a heuristic.

The relationship between compression and intelligence, while not equal is definitely there. It looks like 3Blue1Brown is going to be doing some videos on this aspect.

nextaccountic 7 minutes ago | | |

there's a way to connect kolmogorov complexity and occam's razor, which is solomonoff induction

david_shi 1 hour ago |

Is this meant to be read in order?

throwaw12 1 hour ago |

Where did you get the list? AFAIK, list was never shared

aperrien 2 hours ago |

Is there a way to download them all in one go?

renyicircle 2 hours ago |

The formatting of the articles on this website is bad. I've opened the first one and all the LaTeX formulas are messed up. The subscripts and superscripts are all flattened rendering the math hard to comprehend. Did the author actually try to read any of the articles?

>∏ plocal(x|z) = i p(xi|z,xWindowAround(i))

Images and tables are not rendered at all. What is the point of this? Just keep the links to arxiv and leave it at that, otherwise render the articles properly

IceDane 1 hour ago |

Why on earth would you deliberately choose to do whatever the fuck it is you did with the scroll and the animations for each paper when scrolling through the landing page? What are those animations supposed to be? I use firefox but I also visited on chrome, and the page is even more broken there. Scroll doesn't "take" unless I scroll hard enough, otherwise it bounces back. But on chrome, at least, it seems like the animation for each paper is clearer - it's supposed to be animating the scale of the paper as you scroll to it.. but it seems that your background animation is lagging everything so much it just doesn't work.

lostmsu 4 hours ago |

Main page UX is terrible. If you go for quirky, fine, but I would not want to use it.

mattmatheus 12 minutes ago | |

Indeed. It's very bad.

soperj 2 hours ago | |

Yes, normally wouldn't ever say anything, but I could even read the text things were just flying around. (On firefox)

solarengineer 2 hours ago | |

Indeed. I scoffed at your comment and went to the website. After scrolling a bit, I find myself having a mild headache and slight dizziness.

I would request the author to consider something that does not distract us from this educational and informative website ( I have bookmarked it ).

brachkow 1 hour ago |

> "beginner friendly format" > looks inside > math