Winda

Stanford CS224N: Natural Language Processing with Deep Learning

Contents · 36

  1. 01-Introduction to NLP and Deep LearningIntroduces NLP applications, human language as a symbolic system, deep learning and representation learning, and why deep learning is effective for language tasks.
  2. 01-Introduction to NLP and Deep LearningIntroduces natural language processing, its goals and applications, and why language understanding is difficult.
  3. 02-Word Vector Representations: word2vecWord meaning representations, distributional similarity, and the word2vec skip-gram and CBOW models, including their objectives, gradients, and optimization.
  4. 02-Word Vector Representations: word2vecIntroduces word meaning and representation, contrasting dictionary definitions and denotation with taxonomic resources such as WordNet, before previewing Word2vec.
  5. 03-Advanced Word Vector RepresentationsCovers word2vec and negative sampling, CBOW, co-occurrence matrices, and dimensionality reduction with SVD.
  6. 03-Advanced Word Vector RepresentationsReviews the skip-gram word2vec model, its training updates, and upcoming analysis and evaluation of word vectors.
  7. 04-Word Window Classification and Neural NetworksIntroduces classification objectives, softmax and cross-entropy, regularization, word-vector fine-tuning, and neural networks for classifying words in context windows.
  8. 04-Word Window Classification and Neural NetworksIntroduces classification with softmax and cross-entropy, supervised updates to word vectors, window classifiers, neural networks, max-margin loss, and backpropagation.
  9. 05-Backpropagation and Project AdviceDerives backpropagation for multilayer neural networks using chain-rule and computational-circuit perspectives, and offers tips for debugging math and dimensions in the first problem set.
  10. 05-Backpropagation and Project AdviceBackpropagation through multilayer neural networks, with gradient calculations and activations, plus project and debugging guidance.Backpropagation through multilayer neural networks, including gradient calculations and activations, with project and debugging guidance.
  11. 06-Dependency ParsingIntroduces dependency grammar and parsing, including attachment ambiguity, treebank resources, dependency relations, and cues used to identify heads and dependents.
  12. 06-Dependency ParsingIntroduces syntax, dependency grammar, transition-based parsing, and neural dependency parsers.
  13. 07-Introduction to TensorFlowIntroduces TensorFlow’s graph-based programming model, sessions, placeholders, variables, loss functions, gradient descent, training, and variable sharing.
  14. 07-Introduction to TensorFlowIntroduces TensorFlow’s computational graphs and deep learning features, with course announcements about assignments, GPU access, and final projects.
  15. 08-Recurrent Neural Networks and Language ModelsRecurrent neural networks for language modeling, including sequence training, vanishing and exploding gradients, and bidirectional and deep architectures; also covers SyntaxNet’s structured parsing methods.
  16. 08-Recurrent Neural Networks and Language ModelsLanguage modeling with n-gram counts, the Markov assumption, and the memory costs of traditional models.
  17. 09-Machine Translation and Advanced Recurrent LSTMs and GRUsReviews core NLP concepts, statistical machine translation and alignment, then introduces RNN encoder–decoder models and GRU/LSTM architectures for translation.
  18. 09-Machine Translation and Advanced Recurrent LSTMs and GRUsRecurrent neural networks, including GRUs and LSTMs, and their use in machine translation.
  19. 10-Neural Machine Translation and Models with AttentionNeural machine translation, encoder-decoder sequence models, attention, decoder design, and the progress and strengths of NMT.
  20. 10-Neural Machine Translation and Models with AttentionCourse announcements cover midterm grading, Assignment 3 and GPU access, and final project office hours; upcoming lectures address attention, machine translation, and gated recurrent models.
  21. 11-Gated Recurrent Units and Further Topics in NMTExplains GRU and LSTM gating, recurrent-model training and ensembles, lip-reading research, and machine-translation evaluation.
  22. 11-Gated Recurrent Units and Further Topics in NMTExplains the motivations and workings of GRUs and LSTMs, then introduces machine translation evaluation and practical system-building considerations.
  23. 12-End-to-End Models for Speech ProcessingIntroduces end-to-end speech recognition, covering classical ASR, CTC alignment and decoding, and sequence-to-sequence models such as Listen Attend and Spell.
  24. 12-End-to-End Models for Speech ProcessingTraditional speech recognition pipelines and end-to-end ASR, including CTC, Listen, Attend and Spell, language models, and decoding.
  25. 13-Convolutional Neural NetworksExplains CNN architectures for NLP, including sentence classification, convolution and pooling, character-aware language models, evaluation, and comparisons with RNNs and bag-of-words models.
  26. 13-Convolutional Neural NetworksCovers CNN architectures, GPU use with Azure, course project guidance, and character-aware language models.
  27. 14-Tree Recursive Neural Networks and Constituency ParsingExplains compositional meaning, tree-structured recursive neural networks, and constituency parsing, including model training, structure search, and backpropagation through trees.
  28. 14-Tree Recursive Neural Networks and Constituency ParsingIntroduces tree-structured recursive neural networks, their use in constituency parsing, and architectures for compositional language modeling.
  29. 15-Coreference ResolutionExplains coreference and anaphora, reference types, applications, evaluation metrics, and neural coreference resolution.
  30. 15-Coreference ResolutionIntroduces coreference resolution: identifying text mentions and grouping those that refer to the same entity, including names, pronouns, and other noun phrases.
  31. 16-Dynamic Neural Networks for Question AnsweringCovers Dynamic Memory Networks for question answering, program embeddings for student code, and related experiments in attention, sentiment analysis, and knowledge tracing.
  32. 16-Dynamic Neural Networks for Question AnsweringHow dynamic neural networks frame question answering and diverse NLP tasks as input-question-answer problems, and the challenges of unified architectures and joint multitask learning.
  33. 17-Issues in NLP and Possible Architectures for NLPCovers NLP’s remaining challenges, inference and knowledge representations, tree-structured neural models such as SPINN for natural language inference, and neural approaches to question answering, copying, and character-level processing.
  34. 17-Issues in NLP and Possible Architectures for NLPDiscusses broader goals for solving language, the limitations of a single breakthrough, and a return to tree-structured models.
  35. 18-Tackling the Limits of Deep Learning for NLPDynamic memory networks, joint multitask learning, and layered NLP models for question answering, tagging, parsing, and semantic relatedness.
  36. 18-Tackling the Limits of Deep Learning for NLPDiscusses PA4 and final project deadlines, grading, poster-session expectations, and a baseline LSTM question-answering model.