**CPSC 533V Project - Sept-Dec 2024** The goal of the project is to investigate a particular topic or application in further depth. The topic should have connections with RL or learning for control and decision making. If in doubt, so seek our feedback. You are free to work in groups of 2-3 if you like. The project, worth 35% of your final grade, consists of: 1. A one or two page project proposal (PDF), due by midnight on Tue Nov 12, 2024, submitted via Canvas. This will be worth 20% of the project grade. Please feel free to discuss your ideas with the instructor or TA in advance of the proposal submission. 2. A project report (PDF) is due on Thu Dec 12, 2024, submitted via Canvas. Your project proposal should include: - motivation and goals for the project - a list of intermediate steps - most relevant work or papers - an assessment of the unknowns or risks associated with the project - if working in a group, a breakdown of who is likely to work on what. Structure your project in a way that will allow you to declare success even if you run into a problem or you run out of time -- this is the purpose of listing the intermediate steps. Keep the scope of your project reasonable, given the limited time available. Possible types of projects include: - Applying RL methods on a problem of interest to you. We recommend beginning with simplified versions of the problem in order to speed design-and-debug iteration. You can build on your assignment code, or use other existing reference implementations. - Exploring algorithms or algorithmic issues, including hyperparameters and learning stability, reproducibility, task and reward structure, environment/creature structure, and more. You can build on your assignment code, or using other code bases. - Writing a review summary of recent work in a particular research direction. This may typically involve reading 3 papers and understanding their connections with each other and with common RL methods. Provide a balanced evaluation and critique of the work in the area. The project report can be in a format of your choosing. It should not exceed the length of a typical conference research paper. For an implementation-related paper, document the problem you are solving, the method, the results, the key insights you have gained, and possible future directions. -->