CPSC 522 - Artificial Intelligence 2
Acting in Uncertain Environments

This is an advanced AI course on what should an agent do based on its ability, its beliefs, its perceptions and its values/goals in an uncertain environment. We will assume basic logic, probability, and search, such as covered in an introductory AI course.

One of the primary goals of AI is the design, control and analysis of agents or systems that behave appropriately in various circumstances. Such intelligent agents require not only the ability to act but also to decide how to act as circumstances vary. In turn, good decision making requires that the agent have knowledge or beliefs about its environment and its dynamics, which are updated through perception, about its own abilities, and its goals and preferences.

This course builds on the foundations of logic, dynamic systems, decision theory, knowledge representation and machine learning all of which will be presented in enough detail to understand the rest of the course.

We will begin by studying representations of belief and reasoning methods. In the first half of the couce, we will cover enough to understand current research in statistical relational AI, which is about probabilistic knowledge about things and relations amongst things. In the second half of the course, we will cover decision-theoretic planning and reinforcement learning which are about how to act in dynamic stochastic environments.

Design Space of Intelligent Agents Covered

This course will cover the design space of intelligent computational agents, so that you con understand the frontier of research. We cover the following dimensions of the design space:

• modular and hierarchical representations.
• stochastic dynamics and partial observability
• preference models and utility
• finite stage and infinite stage planning horizons
• feature based and relational representations
• integrating prior knowledge and data (learning)
• multiple agents
• bounded rationality

Unfortunately these cannot be studied independently, as they interact in complex ways.

Structure of the Course

Instructor: David Poole, poole@cs.ubc.ca. Office hours: After class any day or by appointment (room ICCS 109).

There will be 3 hours of in-class interaction per week. The classes will be a mix of lectures on the foundations, student presentations, discussion of research papers and problem solving. This is a participatory class; everyone will be expected to participate fully, to have read the reading material before class, and come ready to discuss and critically analyze it.

The classes will be held:

• Mondays and Wednesdays: 10:30-12:00, DMP 101. The first class will be on Monday, January 4, 2016.

Grades are on Blackboard Connect. Discussion is on Piazza.

Topics

• Agents, knowledge representation and AI - overview of the big picture.
• Probabilistic models (directed and undirected models): semantics, inference, learning, causality
• First-order logic: semantics, inference, learning.
• Relational Probabilistic models: representations, inference, learning
• Ontologies, relational data, and hypotheses
• Identity, number and existence uncertainty
• Preferences and utilities. Multi-attribute utility theory.
• Multi-stage decision making; decision networks, Markov decision processes.
• Value of information, control and computation (Bounded Rationality)
• Learning how to act; reinforcement learning.
• Relational reinforcement learning
• Partial observability, sensing the environment. (POMDPs)
• Multi-agent systems; applications of game theory to cooperative and competitive decision making.

Assessment

The course assessment will be based on 3 assignments/projects. The participants will write a textbok on UBC Wiki. Each wiki page will have one principal author and two secondary authors. Each assignment of for you to:

• Be principal author of one page
• Be a secondary author of two pages
• Be critique of four pages, which involves critiquing and giving feedback for the
• Be an evaluator for the remaining pages

You can be involved in multiple authorship/critiques, but you get to be evaluated by the (average of the) one(s) you choose to be evaluated for. (You can think of the critique / evaluator like the validation set / test set in machine learning.)

Each month's pages will be different theme:

• January. Foundations: the basic building blocks from which richer representations are built
• February. Existing combinations: interesting combinations that have been shown to work
• March. Future combinations: "The best way to predict the future is to invent it" (Alan Kay, 1971)

Each wiki page should include both the prior argument and the evidence that it works.

Last updated: 2012-12-04, David Poole