This is an advanced AI course on what should an agent do based on its ability, its beliefs, its perceptions and its values/goals in an uncertain environment. We will assume basic logic, probability, and search, such as covered in an introductory AI course.

One of the primary goals of AI is the design, control and analysis of agents
or systems that behave appropriately in various circumstances. Such
intelligent agents require not only the ability to **act** but also to
**decide** how to act as circumstances vary. In turn, good decision making
requires that the agent have knowledge or **beliefs** about its
environment and its dynamics, about its own **abilities** to observe and
change the environment, and its **goals** and **preferences**.

This course builds on the foundations of **logic**, **dynamic systems**,
**decision theory**, **knowledge representation** and **machine
learning** all of which will be presented in enough detail to understand
the rest of the course.

We will begin by studying representations of belief and reasoning
methods. In the first half of the couce, we will cover enough to understand current research in
**statistical relational AI**, which is about probabilistic
knowledge about things and relations amongst things. In the second
half of the course, we will cover **decision-theoretic planning**
and **reinforcement learning** which are about how to act in
dynamic stochastic environments.

This course will cover the design space of intelligent computational agents, so that you con understand the frontier of research. We cover the following dimensions of the design space:

- modular and hierarchical representations.
- stochastic dynamics and partial observability
- preference models and utility
- finite stage and infinite stage planning horizons
- feature based and relational representations
- integrating prior knowledge and data (learning)
- multiple agents
- bounded rationality

Unfortunately these cannot be studied independently, as they interact in complex ways.

Instructor: David Poole, poole@cs.ubc.ca. Office hours: Wednesdays 11:30-12:20, Fridays 2:00-2:30 or by appointment (room ICCS 127).

There will be 3 hours of in-class interaction per week. The classes will be a mix of lectures on the foundations, student presentations, discussion of research papers and problem solving. This is a participatory class; everyone will be expected to participate fully, to have read the reading material before class, and come ready to discuss and critically analyze it.

The classes will be held:

- Wednesdays and Fridays: 12:30-14:00, ICCS 206. The first class will be on Wednesday, January 4.

Discussions and grades are on Blackboard Vista.

- Agents, knowledge representation and AI - overview of the big picture.
- Foundations of Statistical Relational AI
- First-order logic: semantics, inference, learning.
- Ontologies, relational data, and hypotheses
- Representations. belief (Bayesian) networks. undirected models.
- Inference in graphical models: exact and approximate (variable elimination, search-based methods, stochastic simulation, particle filters, variational methods)
- Learning belief networks: learning probabilities, unobserved variables, learning structure, causality
- Relational Probabilistic models: representations, inference, learning
- Identity, number and existence uncertainty

- Acting and deciding
- Preferences and utilities. Multi-attribute utility theory.
- Multi-stage decision making; decision networks, Markov decision processes.
- Solving decision networks (dynamic programming, search)
- Value of information, control and computation (Bounded Rationality)
- Decision-theoretic and relational planning
- Learning how to act; reinforcement learning.
- Relational reinforcement learning
- Partial observability, sensing the environment. (POMDPs)
- Multi-agent systems; applications of game theory to cooperative and competitive decision making.

Everyone will be expected to choose a "vertical domain", with 4 properties: (a) you know something about it, (b) someone may be interested in predictions in this domain, (c) there is potentially rich data about the domain and (d) there are complex interactions about the concepts in the domain. Each participant needs to be ready to explain how the concepts covered in class can apply to their domain.

The course assessment will be based on assignments, in-class participation, a midterm exam, and a final research paper. The evaluation of the final project will be based on peer review. Assignments will not be traditional assignments, but will be to apply what is being covered in class to your vertical domain, and to present it to the class.

Participants will be expected to complete a small course project (details to be announced). The project will be decided in consultation with the instructor and may consist of an implementation of a particular reasoning, decision-making or planning theory (or theories), a short research paper on problem related to a course topic, or a critical literature survey. You are encouraged to collaborate with other students to find complementary synergistic projects (e.g., sharing the same code to test different hypotheses), but your written project must be the your own. You will be required to hand in a short proposal in the middle of the term, and then 4 copies of your project in the last week of classes. You will be expected to peer review three other projects and give a conference-style presentation of your project. You will have a chance to revise your project based on the reviews and feedback from your presentation.

The breakdown of marks is tentatively:

- 10%: paper presentations + participation.
- 40%: project (paper + presentations)
- 20%: assignments
- 5%: reviewing
- 25%: exam