Dive into Deep Learning (D2L Book)
Dive into Deep Learning: an interactive deep learning book with code, math, and discussions astonzhang released this
Framework Adaptation
We have added PyTorch implementations up to Chapter 11 (Optimization Algorithms). Chapter 1--7 and Chapter 11 have also been adapted to TensorFlow.
Towards v1.0
The following chapters have been significantly improved for v1.0:
- Linear Neural Networks
- Multilayer Perceptrons
- Deep Learning Computation
- Convolutional Neural Networks
- Modern Convolutional Neural Networks
- Recurrent Neural Networks
Finalized chapters are being translated into Chinese (d2l-zh v2)
Other Improvements
- Fixed issues of not showing all the equation numbers in the HTML and PDF
- Consistently used f-string
- Revised overfitting experiments
- Fixed implementation errors for weight decay experiments
- Improved layer index style
- Revised "breaking the symmetry"
- Revised descriptions of covariate and label shift
- Fixed mathematical errors in covariate shift correction
- Added true risk, empirical risk, and (weighted) empirical risk minimization
- Improved variable naming style for matrices and tensors
- Improved consistency of mathematical notation for tensors of order two or higher
- Improved mathematical descriptions of convolution
- Revised descriptions of cross-correlation
- Added feature maps and receptive fields
- Revised mathematical descriptions of batch normalization
- Added more details to Markov models
- Fixed implementations of k-step-ahead predictions in sequence modeling
- Fixed mathematical descriptions in language modeling
- Improved the
d2l.Vocab
API - Fixed mathematical descriptions and figure illustrations for deep RNNs
- Added BLEU
- Improved machine translation application results
- Improved the animation plot function in the all the training loops
Assets
3
Highlights
We have added both PyTorch and TensorFlow implementations up to Chapter 7 (Modern CNNs).
Improvements
- We updated the text to be framework neutral, such as now we call
ndarray
as tensor. - Readers can click the tab in the HTML version to switch between frameworks, both colab button and discussion thread will change properly.
- We changed the release process, d2l.ai will host the latest release (i.e. the release branch), instead of the contents from the master branch. We unified the version number of both text and the
d2l
package. That's why we jumped from v0.8 to v0.14.0 - The notebook zip contains three folders,
mxnet
,pytorch
andtensorflow
(though we only build the PDF for mxnet yet).
Assets
4
astonzhang released this
Highlights
D2L is now runnable on Amazon SageMaker and Google Colab.
New Contents
The following chapters are re-organized:
- Natural Language Processing: Pretraining
- Natural Language Processing: Applications
The following sections are added:
- Subword Embedding (Byte-pair encoding)
- Bidirectional Encoder Representations from Transformers (BERT)
- The Dataset for Pretraining BERT
- Pretraining BERT
- Natural Language Inference and the Dataset
- Natural Language Inference: Using Attention
- Fine-Tuning BERT for Sequence-Level and Token-Level Applications
- Natural Language Inference: Fine-Tuning BERT
Improvements
There have been many light revisions and improvements throughout the book.
Assets
3
astonzhang released this
Highlights
- D2L is now based on the NumPy interface. All the code samples are rewritten.
New Contents
-
Recommender Systems
- Overview of Recommender Systems
- The MovieLens Dataset
- Matrix Factorization
- AutoRec: Rating Prediction with Autoencoders
- Personalized Ranking for Recommender Systems
- Neural Collaborative Filtering for Personalized Ranking
- Sequence-Aware Recommender Systems
- Feature-Rich Recommender Systems
- Factorization Machines
- Deep Factorization Machines
-
Appendix: Mathematics for Deep Learning
- Geometry and Linear Algebraic Operations
- Eigendecompositions
- Single Variable Calculus
- Multivariable Calculus
- Integral Calculus
- Random Variables
- Maximum Likelihood
- Distributions
- Naive Bayes
- Statistics
- Information Theory
-
Attention Mechanisms
- Attention Mechanism
- Sequence to Sequence with Attention Mechanism
- Transformer
-
Generative Adversarial Networks
- Generative Adversarial Networks
- Deep Convolutional Generative Adversarial Networks
-
Preliminaries
- Data Preprocessing
- Calculus
Improvements
- The Preliminaries chapter is improved.
- More theoretical analysis is added to the Optimization chapter.
Preview Version
Hard copies of a D2L preview version based on this release (excluding chapters of Recommender Systems and Generative Adversarial Networks) are distributed at AWS re:Invent 2019 and NeurIPS 2019.
Assets
3
astonzhang released this
Fixes
- Fixed issues in sections of Machine Translation and the Dataset, Encoder-Decoder Architecture, Sequence to Sequence, Beam Search, Attention Mechanism, Sequence to Sequence with Attention Mechanism, and Transformer.
Assets
3
astonzhang released this
Textbook/Reference Book Adoption at Universities
We thank instructors and students for providing feedbacks during use.
USA
- University of California, Berkeley
- Georgia Institute of Technology
- University of North Carolina at Chapel Hill
- University of California, Los Angeles
- University of California, Santa Barbara
- University of Illinois at Urbana-Champaign
- University of Maryland
- University of Washington
- University of New Hampshire
China
- University of Science and Technology of China
- Shanghai University of Finance and Economics
- Deep Learning
- Zhejiang University
- Graduate course: IoT and Information Processing
Australia
- University of Technology Sydney
Spain
- Universitat Politècnica de Catalunya
Change of Contents
We heavily revised the following chapters, especially during teaching STAT 157 at Berkeley.
- Preface
- Installation
- Introduction
- The Preliminaries: A Crashcourse
- Linear Neural Networks
- Multilayer Perceptrons
- Recurrent Neural Networks
The Community Are Translating D2L into Korean and Japanese
d2l-ko in Korean (website: ko.d2l.ai) joins d2l.ai! Thank Muhyun Kim, Kyoungsu Lee, Ji hye Seo, Jiyang Kang and many other contributors!
d2l-ja in Japanese (website: ja.d2l.ai) joins d2l.ai! Thank Masaki Samejima!
Thanks to Our Contributors
@alxnorden, @avinashingit, @bowen0701, @brettkoonce, Chaitanya Prakash Bapat, @cryptonaut, Davide Fiocco, @edgarroman, @gkutiel, John Mitro, Liang Pu, Rahul Agarwal, @mohamed-ali, @mstewart141, Mike Müller, @NRauschmayr, @prakhar Srivastav, @sad-, @sfermigier, Sheng Zha, @sundeepteki, @topecongiro, @tpdi, @vermicelli, Vishaal Kapoor, @vishwesh5, @YaYaB, Yuhong Chen, Evgeniy Smirnov, @lgov, Simon Corston-Oliver, @IgorDzreyev, @trungha-ngx, @pmuens, @alukovenko, @senorcinco, @vfdev-5, @dsweet, Mohammad Mahdi Rahimi, Abhishek Gupta, @uwsd, @DomKM, Lisa Oakley, @vfdev-5, @bowen0701, @arush15june, @prasanth5reddy.
Assets
3
Watchers:301 |
Star:8894 |
Fork:2072 |
创建时间: 2018-10-09 09:04:37 |
最后Commits: 5天前 |
d98c886
Compare
Verified
Brand-New Attention Chapter
We have added the brand-new Chapter: Attention Mechanisms:
Attention Cues
Attention Pooling: Nadaraya-Watson Kernel Regression
Attention Scoring Functions
Bahdanau Attention
Multi-Head Attention
Self-Attention and Positional Encoding
Transformer
PyTorch Adaptation Completed
We have completed PyTorch implementations for Vol.1 (Chapter 1--15).
Towards v1.0
The following chapters have been significantly improved for v1.0:
Chinese Translation
The following chapters have been translated into Chinese (d2l-zh v2 Git repo, Web preview):
Turkish Translation
The community are translating the book into Turkish (d2l-tr Git repo, Web preview). The first draft of Chapter 1--7 is complete.