Highlights

D2L is now runnable on Amazon SageMaker and Google Colab.

New Contents

The following chapters are re-organized:

• Natural Language Processing: Pretraining
• Natural Language Processing: Applications

The following sections are added:

• Subword Embedding (Byte-pair encoding)
• Bidirectional Encoder Representations from Transformers (BERT)
• The Dataset for Pretraining BERT
• Pretraining BERT
• Natural Language Inference and the Dataset
• Natural Language Inference: Using Attention
• Fine-Tuning BERT for Sequence-Level and Token-Level Applications
• Natural Language Inference: Fine-Tuning BERT

Improvements

There have been many light revisions and improvements throughout the book.

Highlights

• D2L is now based on the NumPy interface. All the code samples are rewritten.

New Contents

• Recommender Systems

  • Overview of Recommender Systems
  • The MovieLens Dataset
  • Matrix Factorization
  • AutoRec: Rating Prediction with Autoencoders
  • Personalized Ranking for Recommender Systems
  • Neural Collaborative Filtering for Personalized Ranking
  • Sequence-Aware Recommender Systems
  • Feature-Rich Recommender Systems
  • Factorization Machines
  • Deep Factorization Machines

• Appendix: Mathematics for Deep Learning

  • Geometry and Linear Algebraic Operations
  • Eigendecompositions
  • Single Variable Calculus
  • Multivariable Calculus
  • Integral Calculus
  • Random Variables
  • Maximum Likelihood
  • Distributions
  • Naive Bayes
  • Statistics
  • Information Theory

• Attention Mechanisms

  • Attention Mechanism
  • Sequence to Sequence with Attention Mechanism
  • Transformer

• Generative Adversarial Networks

  • Generative Adversarial Networks
  • Deep Convolutional Generative Adversarial Networks

• Preliminaries

  • Data Preprocessing
  • Calculus

Improvements

• The Preliminaries chapter is improved.
• More theoretical analysis is added to the Optimization chapter.

Preview Version

Hard copies of a D2L preview version based on this release (excluding chapters of Recommender Systems and Generative Adversarial Networks) are distributed at AWS re:Invent 2019 and NeurIPS 2019.

Fixes

• Fixed issues in sections of Machine Translation and the Dataset, Encoder-Decoder Architecture, Sequence to Sequence, Beam Search, Attention Mechanism, Sequence to Sequence with Attention Mechanism, and Transformer.

Textbook/Reference Book Adoption at Universities

We thank instructors and students for providing feedbacks during use.

USA

• University of California, Berkeley
  • STAT 157 Introduction to Deep Learning
• Georgia Institute of Technology
  • CX4240: Introduction to Computational Data Analysis
• University of North Carolina at Chapel Hill
  • INLS 690-270 - Data Mining: Methods and Applications
• University of California, Los Angeles
  • CS269 Foundations of Deep Learning
• University of California, Santa Barbara
  • CS165A: Artificial Intelligence
• University of Illinois at Urbana-Champaign
  • CS 498 Introduction to Deep Learning
• University of Maryland
  • Introduction to Deep Learning
• University of Washington
  • CSE 599W Systems for ML
• University of New Hampshire
  • CS 953 Adv Top / Data Science for Knowledge Graphs and Text

China

• University of Science and Technology of China
  • Deep Learning
• Shanghai University of Finance and Economics
  • Deep Learning
• Zhejiang University
  • Graduate course: IoT and Information Processing

Australia

• University of Technology Sydney
  • 42028 Deep Learning and Convolutional Neural Network

Spain

• Universitat Politècnica de Catalunya
  • Master Course: Deep Learning for Artificial Intelligence

Change of Contents

We heavily revised the following chapters, especially during teaching STAT 157 at Berkeley.

• Preface
• Installation
• Introduction
• The Preliminaries: A Crashcourse
• Linear Neural Networks
• Multilayer Perceptrons
• Recurrent Neural Networks

The Community Are Translating D2L into Korean and Japanese

d2l-ko in Korean (website: ko.d2l.ai) joins d2l.ai! Thank Muhyun Kim, Kyoungsu Lee, Ji hye Seo, Jiyang Kang and many other contributors!

d2l-ja in Japanese (website: ja.d2l.ai) joins d2l.ai! Thank Masaki Samejima!

Thanks to Our Contributors

@alxnorden, @avinashingit, @bowen0701, @brettkoonce, Chaitanya Prakash Bapat, @cryptonaut, Davide Fiocco, @edgarroman, @gkutiel, John Mitro, Liang Pu, Rahul Agarwal, @mohamed-ali, @mstewart141, Mike Müller, @NRauschmayr, @prakhar Srivastav, @sad-, @sfermigier, Sheng Zha, @sundeepteki, @topecongiro, @tpdi, @vermicelli, Vishaal Kapoor, @vishwesh5, @YaYaB, Yuhong Chen, Evgeniy Smirnov, @lgov, Simon Corston-Oliver, @IgorDzreyev, @trungha-ngx, @pmuens, @alukovenko, @senorcinco, @vfdev-5, @dsweet, Mohammad Mahdi Rahimi, Abhishek Gupta, @uwsd, @DomKM, Lisa Oakley, @vfdev-5, @bowen0701, @arush15june, @prasanth5reddy.