NeurIPS'22: Dr. Clune and co-researchers land at the top of their game in Machine Learning
Part 3 in a series about some of the department’s accepted papers at NeurIPS 2022 (conference being held Nov. 28 - Dec. 9)
Dr. Jeff Clune, associate professor of UBC Computer Science and his co-authors from OpenAI have a paper accepted at the upcoming NeurIPS conference, which also qualified for a prestigious oral presentation spot, to be given by Peter Zhokhov.
Only a select two per cent of the accepted papers at this premiere conference in AI and Machine Learning (ML) are asked to provide an oral presentation.
The paper: Learning to Play Minecraft with Video PreTraining (VPT) Bowen Baker, Ilge Akkaya, Peter Zhokhov, Joost Huizinga, Jie Tang, Adrien Ecoffet, Brandon Houghton, Raul Sampedro, Jeff Clune
Minecraft: a goldmine of inspiration
Dr. Clune and his fellow researchers have put artificial intelligence to the test on one of the world’s most famous video games: Minecraft. While relatively easy for humans, Minecraft is extremely difficult for AI to learn to play. That’s because of the vast number of things to do in the game, and because doing one particular thing often involves first doing many other things. To solve this challenge, Clune and his colleagues had AI watch eight years’ worth of internet videos of people playing Minecraft, after which it proved to be excellent at the game.
The idea originated from a recognition that the internet contains an enormous amount of untapped information in the form of internet videos that show you how to solve a task (in this case, Minecraft). The challenge, however, is that those videos do not tell you what actions the people in the video are taking (in the Minecraft example, you see the character moving, but do not know what buttons the player is pressing, or how the mouse is being moved).
To solve this problem, Clune and colleagues came up with a simple solution: to train an AI model to guess which action must have been taken at each moment in time. With this model, they could ‘label’ all Minecraft videos on the internet, from which another AI model could watch and learn.
The result is a novel, yet simple, semi-supervised imitation learning method called Video PreTraining (VPT), which serves to train a neural network to play Minecraft on a massive video dataset taken from human Minecraft play. The key factor is that the model only uses a small amount of labeled data in the training. That means the model is learning from mostly unlabeled data.
With fine-tuning, their model can learn to craft diamond tools, a complex and labour-intensive task that takes proficient humans over 20 minutes on average, through 24,000 actions to complete. The result represents a giant step toward general computer-using agents.
Quick learner
Trained on 70,000 hours of labeled online video, their VPT foundation model accomplishes tasks in Minecraft that are nearly impossible to achieve with reinforcement learning from scratch. The model learns to chop down virtual trees to collect logs, craft those virtual logs into planks, and then craft the planks into a crafting table. This sequence takes a human that is proficient in Minecraft approximately 50 seconds or 1,000 consecutive game actions.
With AI becoming so adept beyond human capability, there is serious thought about the pace of AI. Dr. Clune said, “With AI, progress is tremendously fast. What we need to do alongside AI research is to ensure that AI benefits all of humanity. That includes everything from handling the inevitable economic disruptions that come as jobs are automated, to securing the safety of humanity alongside AI, that may in fact be smarter than humans.”
Jeff explains that there is a responsibility with AI researchers to think hard about how we can ensure AI shares our values. “In fact,” says Jeff, “I recently received a grant request for conducting research into AI safety and value alignment, because I increasingly think it's time for researchers to spend a larger fraction of their time thinking about the concerns of AI safety. We also need to make sure that policymakers and the public are aware of what's coming and we need to spend increasingly greater amounts of time doing so.”
Through models like VPT, researchers like Jeff have the opportunity to find out not only what AI is capable of, but can pivot from this research to ensure these capabilities are applied wisely for mankind.
More about Dr. Jeff Clune’s research.
In total, the department has 13 accepted papers by 9 professors at the NeurIPS conference. Read more about the accepted papers and their authors.