LING530: Deep Learning for Natural Language Processing

Muhammad Abdul-Mageed

UBC LING530 Deep Learning for Natural Language Processing

1. Course Rationale & Goal:

Rationale/Background: Natural language processing (NLP) is the field focused at teaching computers to understand and generate human language. Dialog systems where the computer interacts with humans, such as the Amazon Echo, constitute an instance of both language understanding and generation, as the machine attempts to identify the meaning of questions and generate meaningful answers. Recent advances in machine learning, especially in Deep learning, a class of machine learning methods inspired by information processing in the human brain, have boosted performance on several NLP tasks.

Deep learning of natural language is in its infancy, with expected breakthroughs ahead. Solving NLP problems directly contributes to the development of pervasive technologies with significant social and economic impacts and the potential to enhance the lives of millions of people. Given the central role that language plays in our lives, advances in deep learning of natural language have implications across almost all fields of science and technology, as well as many other disciplines like linguistics, as NLP and deep learning are instrumental for making sense of the ever-growing data collected in these fields.

Goal: This course provides a graduate-level introduction to Natural Language Processing With Deep Learning. The goal of the course is to familiarize students with the major NLP problems and the primary deep learning methods being developed to solve them. This includes problems at various linguistic levels (e.g., word and sub-word, phrase, clause, and discourse). Methodologically, this involves unsupervised, distributed representations and supervised deep learning methods across these linguistic levels. The course also includes providing an introductory level of hands-on experience in using deep learning software as well as opportunities to develop advanced solutions for NLP problems in the context of a final project.

Potential audiences for this course are:

  • Students with a linguistics and/or a computer science background interested in learning novel NLP methods, with a focus on deep learning.
  • People interested in NLP, text analytics, information retrieval, information visualization, human-information interaction, machine learning and deep learning, etc., who want to prepare for, or advance, carrying out research in these fields exploiting deep learning of natural language methods.

2. Course Objectives:

Upon completion of this course students will be able to:

  • identify the inherent ambiguity in natural language, and appreciate challenges associated with teaching machines to understand and generate it

  • become aware of a core of NLP problems, and demonstrate how these are relevant to the lives of diverse individuals, communities and organizations.

  • become aware of the major deep learning methods being developed for solving NLP problems, and be in a position to apply this deepened understanding in critical, creative, and novel ways

  • collaborate effectively with peers through course assignments

  • identify an NLP problem (existing or novel) and apply deep learning methods to develop a solution for it

3. Course Topics:

  • ambiguity in human language (across different levels) % week 01
  • learning words & phrases (word vectors)% weeks 2, 3%, and sentences
  • recurrent neural networks
  • convolutional neural networks

Applications

  • POS tagging and morphological disambiguation
  • learning in low-resource languages
  • language/dialect identification
  • emotion and sentiment analysis
  • dialog systems
  • machine translation

4. Prerequisites:

  • have some familiarity with at least one linguistics field
  • have some familiarity with basic linear algebra, basic calculus, \and basic probability (basic $=$ high school level)
  • have programming experience in Python
  • have access to a computer (preferably with a GPU) on a regular basis

Students lacking any of the above pre-requisites must be open to learn outside their comfort zone to make up, including investing time outside class learning these pre-requisites on their own. Some relevant material across these pre-requisites will be provided as part of the syllabus. Should you have questions about pre-requisites, please email the instructor.

5. Format of the course:

• This course will involve lectures, class hands-on activities, individual and group work, and instructor-, peer-, and self-assessment.