Readings, Syllabus, Assignments, Software&Data

We will communicate through Canvas: to log in use your CWL

Intro

- ACL
- NLP demos/videos
- Ambiguity

NLP toolkits: NLTK (Python), Stanford CoreNLP (java)

ProbInfoTheory Handout

• 2004 book and software
• Xerox: FiniteState Technology
• Finite State Utilities  (Van Noord)
• OpenFST from Google

• The Porter Stemmer (includes perl implementation)
• ProbInfoTheory Handout
• min-edit-dist demo
•  A spelling correction program based on a noisy channel model Kerninghan et al. COLING ,1990.
• minimal Python implementation of spelling correction (by P. Norvig)

Google ngrams model

Google books Ngrams viewer

An empirical study of smoothing techniques for NLP S.F. Chen, J. Goodman - TR CS Harvard Univ - 1998

• Stanford CoreNLP
• CMU-Cambridge language model toolkit
• SRI LM Toolkit

Markov Sequence Labelling Models - Part-of-speech Tagging

- state of the art POS tagging

why tagging can be challenging for humans: Penn tagging scheme

Part-of-Speech Tagging from 97% to 100% C. Manning 2011

Assignment2 on Canvas (due Feb 11)
Corpora: wsj-p.txt  wsj-ps.txt  atis3.pos.tags.txt cmpt-hw2-3.txt

 Interactive tutorials on the English grammar 
English Dept. University of Calgary.

- NLTK (demos) - look at *Getting Started*
 - Some public parsers (inlcuding Stanford and MINIPAR visualization  tools)

-Penn Treebank - Universal dependency Treebanks

Time ML

Semantic Parser (Cornell - Yoav Artzi)

- (Domain specific thesaurus) Medical Subject Headings (MeSH)
- FrameNet
- ProbBank (adding semantic annotations to the Penn Treebank)

Assignment3 on Canvas (due March 2nd)  needed files

- A systematic comparison of context-counting vs. context-predicting semantic vectors ! (predicting is clearly better)

- generalization of skip-grams to sentences (skip-thought vectors) 2015

- SENSEVAL(Evaluation for WSD)

- WSD online public systems

- WSD with Deep Belief Networks
- Dependency-based word similarity demo
- TREC (Text REtrieval Conference)
- Semantic Labeling (ASSERT)

-Illinois Semantic Role Labeler

see project work plan

Generic Topic Modeling (background reading Comm. ACM) and Topic Modeling in Asynchronous Conversations

• Steyvers, M. & Griffiths, T. (). Probabilistic topic models. In T. Landauer, D McNamara, S. Dennis, and W. Kintsch (eds), Latent Semantic Analysis: A Road to Meaning. Laurence Erlbaum 2006 [ pdf ] or [ pdf ]   [Roger Lo  ]  

• GS. Joty, G. Carenini and R. T. Ng (2013) Topic Segmentation and Labeling in Asynchronous Conversations JAIR, Volume 47, pages 521-573 (2013) (only intro, conclusions and sections of topic segmentation (not labeling) [pdf]   [Patrick Boutet ]  

Visual Text Analytics and Interactive Topic Modeling

• Enamul Hoque and Giuseppe Carenini MultiConVis: A Visual Text Analytics System for Exploring a Collection of Online Conversations  Proceedings of ACM Intelligent User Interfaces (IUI), 2016 ] VIDEO  [Steve Kasica  ]  

• Interactive topic hierarchy revision for exploring a collection of online conversations Enamul Hoque and Giuseppe Carenini Journal of Information Visualization (accepted), 2018 PDF | VIDEO   [ Nazli Ozum Kafaee ]

Natural Language Generation (data2text)

• F Portet, E Reiter, A Gatt, J Hunter, S Sripada, Y Freer, C Sykes  Automatic Generation of Textual Summaries from Neonatal Intensive Care Data. Artificial Intelligence 173:789-816. 2009 (pdf)    [  Martin Wang  ] 

• (Pedagogical) Conditioned Neural Generation (Encoder-Decoder framework) pag. 195-211 - Y. Goldberg book 2017- Chp. 17  [ Farnoosh Javadi  ]

Distributed Representations for Sentence + Summarization (1)

•  BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding J. Devlin, MW Chang, K. Lee, K. Toutanova   2018 [pdf] [Chiyu Zhang   ]

• (background reading:  Summarization Evaluation notes)
  Lidong Bing, Piji Li, Yi Liao, Wai Lam, Weiwei Guo, Rebecca J. Passonneau,  Abstractive Multi-Document Summarization via Phrase Selection and Merging. Proceedings of the 53rd Annual Meeting of the Association for Computational Linguistics  (ACL 2015) pdf  [ Wen Xiao   ]   

 Summarization (2)

• Janara Christensen, Stephen Soderland, Gagan Bansal, and Mausam "Hierarchical Summarization: Scaling Up Multi-Document Summarization" Proceedings of the 52nd Annual Meeting of the Association for Computational Linguistics (ACL 2014) [pdf ]  [Muhammad Shayan  ]

• Jianpeng Cheng Mirella Lapata, Neural Summarization by Extracting Sentences and Words   ACL-2016 [pdf] [  Yuxi Peter Feng ]

• Richard Socher, Alex Perelygin, Jean Wu, Jason Chuang, Christopher Manning, Andrew Ng and Christopher Potts, Recursive Deep Models for Semantic Compositionality Over a Sentiment Treebank Conference on Empirical Methods in Natural Language Processing (EMNLP 2013) [pdf] [ Peyman Bateni ]

• Navigli, R. and Lapata, M. (2010) An experimental study of graph connectivity for unsupervised word sense disambiguation.  IEEE Transactions on Pattern Analysis and Machine Intelligence, 32(4), 678–692. [  John-Jose Nunez   ]

Neural Text Classification + health application

- Yang, Z., Yang, D., Dyer, C., He, X., Smola, A., Hovy, E.: Hierarchical attention networks for document classication. In: Proceedings of the 2016 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies. pp. 1480{1489 NAACL (2016) [pdf]   [ Bicheng Xu ]

ONTOHAN: An Ontology-based Neural Network Model for Patient Need Detection (draft paper PLEASE DO NOT DISTRIBUTE) pdf "announced" on Canvas 2019 [  Hyeju  Jang ]

• portions of CL paper CODRA: A Novel Discriminative Framework
   for Rhetorical Analysis.   Computational Linguistics (2015) Vol. 41, No. 3: 385–435, MIT press only sections 1 - 4 are mandatory
  More Info [ Saha Rudra]
• pre-reading (not mandatory) for paper1: Chaper 20 (sec 20.1 and 20.2) of Y. Goldberg (only 9 pages, lots of figures ;-)
• Yang Liu and Sujian Li. Implicit discourse relation classification via multi-task neural networks. In Proceedings of AAAI Conference, (2016). [pdf] [Grigorii Guz] 

Final Project Report Hand in