Greg Brockman (Founder of Open AI) has written this amazing answer:
If you want to read one main resource... the Goodfellow, Bengio, Courville book (available for free from http://www.deeplearningbook.org/) is an extremely comprehensive survey of the field. It contains essentially all the concepts and intuition needed for deep learning engineering (except reinforcement learning).
If you'd like to take courses... Pieter Abbeel and Wojciech Zaremba suggest the following course sequence:
- Linear Algebra — Stephen Boyd’s EE263 (Stanford)
- Neural Networks for Machine Learning — Geoff Hinton (Coursera)
- Neural Nets — Andrej Karpathy’s CS231N (Stanford)
- Advanced Robotics (the MDP / optimal control lectures) — Pieter Abbeel’s CS287 (Berkeley)
- Deep RL — John Schulman’s CS294-112 (Berkeley)
(Pieter also recommends the Cover & Thomas information theory and Nocedal & Wright nonlinear optimization books).
If you'd like to get your hands dirty... Ilya Sutskever recommends implementing simple MNIST classifiers, small convnets, reimplementing char-rnn, and then playing with a big convnet. Personally, I started out by picking Kaggle competitions (especially the "Knowledge" ones) and using those as a source of problems. Implementing agents for OpenAI Gym (or algorithms for the set of research problems we’ll be releasing soon) could also be a good starting place.
Thats awesome, I wasn't aware of this. I slightly disagree with Greg though about the "one main resource". While the DL book is amazing, its not really aimed at newbies, and so I think CS231n is a much better starting point. For example, for a beginner I think the way backprop is explained in Stanford's course is better than the explanation in the book.
"One main resource" as in, it goes through all the underlying math required in details etc. It's usually assumed that people entering into DL have some experience with Machine Learning. Of course, for someone staring with ML, CS229 is the first thing she should pick up.
I just wanted to say this is a great recommendation for anyone who is serious about learning deep learning / machine learning. I also want to note, there are other resources available for people who are less serious about it and just want to get a general overview.
If you want to read one main resource... the Goodfellow, Bengio, Courville book (available for free from http://www.deeplearningbook.org/) is an extremely comprehensive survey of the field. It contains essentially all the concepts and intuition needed for deep learning engineering (except reinforcement learning).
If you'd like to take courses... Pieter Abbeel and Wojciech Zaremba suggest the following course sequence:
- Linear Algebra — Stephen Boyd’s EE263 (Stanford) - Neural Networks for Machine Learning — Geoff Hinton (Coursera) - Neural Nets — Andrej Karpathy’s CS231N (Stanford) - Advanced Robotics (the MDP / optimal control lectures) — Pieter Abbeel’s CS287 (Berkeley) - Deep RL — John Schulman’s CS294-112 (Berkeley)
(Pieter also recommends the Cover & Thomas information theory and Nocedal & Wright nonlinear optimization books).
If you'd like to get your hands dirty... Ilya Sutskever recommends implementing simple MNIST classifiers, small convnets, reimplementing char-rnn, and then playing with a big convnet. Personally, I started out by picking Kaggle competitions (especially the "Knowledge" ones) and using those as a source of problems. Implementing agents for OpenAI Gym (or algorithms for the set of research problems we’ll be releasing soon) could also be a good starting place.
Quora link: https://www.quora.com/What-are-the-best-ways-to-pick-up-Deep...