NYU DS-GA 1008 – Deep Learning

NYU DS-GA 1008 – Deep Learning(atcold.github.io)

121 points by eugenhotaj 5 years ago | 25 comments

nurettin 5 years ago |

This is the researcher who published the original convolutional neural network papers providing a basis for modern deep neural networks. http://yann.lecun.com/exdb/publis/pdf/lecun-99.pdf

galuggus 5 years ago |

I'm interested in deep learning. I'm worried that my lack of higher level math skills will limit my progress

How much math is needed for studying deep learning?

Any suggestions for good resources on math for deep learning much appreciated

blackbear_ 5 years ago | |

Lack of math knowledge won't limit much as a practitioner, but will definitely make it harder for you to understand what is going on.

Luckily, the math involved is not so difficult. For deep learning specifically you should be comfortable with linear algebra and multivariate calculus, and for machine learning in general you should be familiar with probabilistic thinking.

"Mathematics for machine learning" [1] is a good introduction to these topics.

[1] https://mml-book.github.io/

person_of_color 5 years ago | | |

Any solutions manual for this?

dagw 5 years ago | |

What is your interest in deep learning? Developing new and novel algorithms, or applying already existing algorithms to new and novel problems, domains and datasets? If it's the latter then high level math skills aren't that necessary.

A solid understanding of the domain and data under investigation is much more important, as is being a decent programmer, knowing your way around data cleaning and data management and having a solid understanding the strengths and weaknesses of the different algorithms out there.

If you're interested the absolute best way to get started is just to start. Download a well studied data set like this one: http://yann.lecun.com/exdb/mnist/ Grab one of the dozens of tutorials that talk about how to approach this data set and follow along in something like scikit-learn or Flux.jl. You'll soon enough have gone from zero to having developed a tool that can recognize handwritten digits. From there you can just keep going.

enriquto 5 years ago | |

I'm not sure I understand your concern. Why are you interested, specifically, in deep learning (as opposed to more general statistics and optimization techniques)? What makes you un-interested in the big picture of statistics and optimization?

I'd say: start studying linear regression and when you master it end to end move into more complex topics.

raidicy 5 years ago | | |

There are very cool things you can do with DL. And the things you need to learn are not too difficult for basic NNs. Please don't dissuade someone's interests by saying they need the knowledge "from the ground up". Of course you will always have a "deeper" understanding any maths subjects if you start from the very beginning. But for me there was always so little context as to their use and application it was disheartening.

It will undoubtedly sound naive to some, but I've been preferring the ability to "drive the car" over how to build one from scratch. Using fastai and their book/videos I've been able to go from dropping calculus and quitting a web dev bootcamp to building an ML product in about a a year. (and would really be much fast if I didn't have repetitive strain injury)

GP, keep searching for your starting point.

Drive the car, learn to change the oil and brakes as you go. And if you're super interested, mod it and then make one.

throwaway6734 5 years ago | |

https://course.fast.ai/ and the associated book are very helpfulf if you're coming from a software engineering perspective

amatic 5 years ago | |

One option is starting to learn about DL from the big picture, then filling in the details as you go along, eventually going to higher math. I prefer this approach to the opposite where you first learn math foundation and then the implementation and use-cases. I liked the Fast.ai course.

archit3cture 5 years ago |

I wonder what is a good hardware setup for following this kind of course and seriously play with deep learning in general (outside of an enterprise context where the enterprise would provide the hardware). What kind of budget should be considered ?

WrtCdEvrydy 5 years ago | |

I did a lot of my learning on a GTX 1060 about 2 years ago with 16gb of RAM and an i5-6400.

The issue with machine learning is that you need enough GPU VRAM to load your dataset and then have to wait for a result being trying something else.

If you have too little VRAM, you get nothing done, but if your GPU is slow (GTX 1070 is about 2x faster than a GTX 1060) you will have to wait before learning something after trying something. The feedback loop for learning is better if you're able to iterate quickly. This is why you sometimes see GPU rigs with up to 4 GPUs that are not being used on the same task (so you can do more than 1 thing at a time)

moyix 5 years ago | |

You can do very well with just Google Colab. If you prefer local compute, the higher-end NVIDIA gaming cards (GeForce series) will do pretty well.

chin7an 5 years ago | |

This post[0] by Tim Dettmers should help. Reading through the article, it seems like he covers a lot of ground and lays out choices quite well. tl;dr is that only Nvidia is your best bet for local computing power, stay away from AMD & Intel for now.

[0] - https://timdettmers.com/2020/09/07/which-gpu-for-deep-learni...

Grimm1 5 years ago |

Gah, graduated a few years too early would have loved to take this directly.

ricklamers 5 years ago |

Typo in the title: it's Yann not Yaan

dakdak 5 years ago |

This is unrelated to the actual link, but I am always surprised by the English language tendency to - for the lack of a better word - smash names together.

Yann LeCun is called Yann Le Cun [0] in French, but "Le" and "Cun" are smashed together in English. Lafayette [2] is of course called La Fayette [3]. Du Pont becomes DuPont, and so on.

[0] https://fr.wikipedia.org/wiki/Yann_Le_Cun

[1] https://en.wikipedia.org/wiki/Yann_LeCun

[2] https://en.wikipedia.org/wiki/Gilbert_du_Motier,_Marquis_de_...

[3] https://fr.wikipedia.org/wiki/Gilbert_du_Motier_de_La_Fayett...