AIJ award

UBC CS research team wins Journal of AI Prominent Paper Award

The significance of artificial intelligence research from a UBC Computer Science team of researchers led by Professor Kevin Leyton-Brown and Affiliate Professor Holger H. Hoos, was acknowledged this summer with an Artificial Intelligence Journal (AIJ) Prominent Paper Award.

The award-winning paper is  Algorithm runtime prediction:  Methods & evaluation   Hutter, Xu, Hoos, and Leyton-Brown. Artificial Intelligence, Volume 206, January 2014, Pages 79-111.

“The paper represents a significant milestone in the field of algorithmic runtime prediction."  ~  AIJ citation

Frank Hutter
Dr. Frank Hutter

The lead author, Frank Hutter, was then a postdoc at UBC CS working with Dr. Leyton-Brown and Dr. Hoos. Frank is now a professor and head of the machine learning lab at the University of Freiburg. This paper arose from the doctoral dissertation work of UBC CS alum Lin Xu, who earned a PhD in 2015 and went on to co-found the Vancouver AI start-up GenerationsE. Dr. Leyton-Brown, and UBC CS Affiliate Professor Dr. Holger H. Hoos who is now at Leiden University, co-supervised the project.

Lin Xu
Dr. Lin Xu

Exceptional impact 

The Artificial Intelligence Journal (AIJ) Prominent Paper Award recognizes outstanding papers, exceptional in both significance and impact, that were published within the past seven years in the AI Journal. The AIJ citation states, “The paper represents a significant milestone in the field of algorithmic runtime prediction. It provides a unifying technical overview, novel technical contributions involving improvements and extensions of existing methods, and a comprehensive empirical analysis of algorithm run-time prediction across three fundamental problems in AI and Algorithms: propositional satisfiability, travelling salesperson, and mixed integer programming. This paper not only serves as an important and highly cited reference on algorithmic runtime prediction for the fields of AI and Algorithms, but it has also influenced work in High Performance and Distributed Computing as evidence by a diverse array of citations from these fields.”

Holger Hoos
Dr. Holger H. Hoos

The authors are delighted, “Especially considering the AIJ is one of the top publication venues in AI,” said Holger. “The impact has been huge from this paper. Google Scholar shows 409 citations for the article, and as the AIJ notes, its impact reaches far beyond the AI community.”

Kevin’s group continues to build on the ideas advanced in the AIJ paper. “In 2020, three UBC students and I published a paper on a similar task:  showing how to use deep learning to predict the output of SAT solvers,” Kevin said. “This work is exciting because, unlike the AIJ approach, it makes end-to-end predictions from raw inputs rather than relying on human-crafted features.”

Kevin LB
Dr. Kevin Leyton-Brown

The paper Leyton-Brown and students published in 2020:
Predicting Propositional Satisfiability via End-to-End Learning. C. Cameron, R. Chen, J. Hartford, K. Leyton-Brown. Thirty-Fourth AAAI Conference on Artificial Intelligence, 2020.

The collaboration continues

“Since the publication of the algorithm runtime prediction paper, we've become geographically separated—Holger is a professor in the Netherlands, and Frank is a professor in Germany,” Kevin said. “We nevertheless continue to work together on algorithm configuration, performance prediction, and algorithm selection, with ongoing work on research papers, book chapters, tutorials, and more.”    

Learn more: 
Kevin Leyton-Brown 

Holger H. Hoos
Frank Hutter
Lin Xu