2014 onwards

Please see my new website for all publications starting with 2014.

2013

• Katharina Eggensperger and Matthias Feurer and Frank Hutter and James Bergstra and Jasper Snoek and Holger Hoos and Kevin Leyton-Brown.
  Towards an Empirical Foundation for Assessing Bayesian Optimization of Hyperparameters [pdf] [bib] [poster]
  In: NIPS Workshop on Bayesian Optimization in Theory and Practice (BayesOpt '13). Software and benchmarks are available from our HPOlib website.

• Kevin Swersky and David Duvenaud and Jasper Snoek and Frank Hutter and Michael Osborne.
  Raiders of the Lost Architecture: Kernels for Bayesian Optimization in Conditional Parameter Spaces [pdf] [bib] [poster]
  In: NIPS Workshop on Bayesian Optimization in Theory and Practice (BayesOpt '13).
  
  
• Frank Hutter, Holger Hoos, and Kevin Leyton-Brown.
  An Efficient Approach for Assessing Parameter Importance in Bayesian Optimization [pdf] [bib] [poster]
  In: NIPS Workshop on Bayesian Optimization in Theory and Practice (BayesOpt '13).

• Chris Thornton, Frank Hutter, Holger Hoos, and Kevin Leyton-Brown.
  Auto-WEKA: Combined Selection and Hyperparameter Optimization of Classifiaction Algorithms [pdf][bib][arXiv]
  In: Proceedings of the 19th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining (KDD'13). The software is available from our Auto-WEKA page.

• Frank Hutter, Holger Hoos, and Kevin Leyton-Brown.
  An Evaluation of Sequential Model-Based Optimization for Expensive Blackbox Functions [pdf][bib]
  GECCO Workshop on Blackbox Optimization Benchmarking (BBOB'13). Software and data are available from the SMAC page.

• Ziyu Wang, Masrour Zoghi, Frank Hutter, David Matheson and Nando de Freitas.
  Bayesian Optimization in High Dimensions via Random Embeddings [pdf][bib][extended arXiv version][code]
  In: Proceedings of the 23rd International Joint Conference on Artificial Intelligence (IJCAI-13). 
  Distinguished paper award.

• Frank Hutter, Holger Hoos, and Kevin Leyton-Brown.
  Identifying Key Algorithm Parameters and Instance Features using Forward Selection [pdf][bib]
  In: Learning and Intelligent Optimization (LION 7).
  The data and source code for this paper are available from our Empirical Performance Models project page.
  

2012

• Lin Xu, Frank Hutter, Holger Hoos, and Kevin Leyton-Brown.
  Evaluating Component Solver Contributions to Portfolio-Based Algorithm Selectors [pdf][bib]
  In: 15th International Conference on Theory and Applications of Satisfiability Testing (SAT'12).

• Frank Hutter, Holger Hoos, and Kevin Leyton-Brown.
  Parallel Algorithm Configuration
  In: Learning and Intelligent Optimization (LION 6). [pdf][pptx slides][pdf slides][bib]
  

• Lin Xu, Frank Hutter, Jonathan Shen, Kevin Leyton-Brown, and Holger H. Hoos.
  SATzilla2012: Improved Algorithm Selection Based on Cost-sensitive Classification Models [pdf][bib]
  Solver description for the 2012 SAT challenge. SATzilla2012 won 3 out of the 4 categories for which it was eligible, and placed 2nd in the remaining one. Details: it won the sequential portfolio track, was the best solver for 2 of the 3 main sequential categories (Application and Hard Combinatorial), and 2nd in the sequential Random Category (beaten only by a new non-portfolio solver, CCASAT). See the SATzilla project page for details on SATzilla and source code.

2011

• Frank Hutter, Holger Hoos, and Kevin Leyton-Brown.
  Bayesian Optimization With Censored Response Data [pdf] [poster] [bib] [extended arXiv version from 2013] [bib for extended arXiv version from 2013]
  NIPS workshop on Bayesian Optimization, Experimental Design, and Bandits. 

• Lin Xu, Frank Hutter, Holger Hoos, and Kevin Leyton-Brown.
  

  Hydra-MIP: Automated Algorithm Configuration and Selection for Mixed Integer Programming [pdf] [bib]
  In: RCRA workshop on Experimental Evaluation of Algorithms for Solving Problems with Combinatorial Explosion
  at IJCAI-11. 

• Frank Hutter, Holger Hoos, and Kevin Leyton-Brown.
  Sequential Model-Based Optimization for General Algorithm Configuration [pdf] [ppt slides] [pdf slides] [bib]
  In: Learning and Intelligent Optimization (LION 5), 2011.
  Best paper award (second prize)
  An extended version with additional details is available as UBC tech report TR-2010-10. [pdf] [bib]
  SMAC, ROAR, and the instances used are available from the Automated Algorithm Configuration project page.  

• Lin Xu, Frank Hutter, Holger Hoos, Kevin Leyton-Brown.
  Detailed SATzilla Results from the Data Analysis Track of the 2011 SAT Competition. In Fourteenth International Conference on Theory and Applications of Satisfiability Testing, SAT 2011 Competition: Data Analysis Track [pdf]

2010

• Frank Hutter, Holger Hoos, and Kevin Leyton-Brown.
  Tradeoffs in the Empirical Evaluation of Competing Algorithm Designs [pdf] [bib]
  In: Annals of Mathematics and Artificial Intelligence's Special Issue on Learning and Intelligent Optimization (AMAI) 60 (1), 65-89
  The data from this paper, as well as the empirical analysis tools we introduced are available from the Automated Algorithm Configuration project page.
  The original publication is available at www.springerlink.com.

• Frank Hutter, Holger Hoos, and Kevin Leyton-Brown
  Automated Configuration of Mixed Integer Programming Solvers [pdf] [bib] [slides]
  In CP-AI-OR 2010.
  The original publication is available at www.springerlink.com.
  Our webpage on Automated Configuration of MIP solvers also gives the parameter files for CPLEX, Gurobi, and lpsolve.




• Frank Hutter, Holger Hoos, Kevin Leyton-Brown, and Kevin Murphy.
  Time-Bounded Sequential Parameter Optimization [pdf ] [bib] [slides] 
  In Learning and Intelligent Optimization (LION 4), 2010
  Runner-up for the best paper award
  The original publication is available at www.springerlink.com.




• Frank Hutter, Thomas Bartz-Beielstein, Holger H. Hoos, Kevin Leyton-Brown, and Kevin Murphy.
  Sequential Model-Based Parameter Optimisation: an Experimental Investigation of Automated and Interactive Approaches [pdf] [bib]
  Chapter 15 in Empirical Methods for the Analysis of Optimization Algorithms, T. Bartz-Beielstein, M. Chiarandini, L. Paquete, M. Preuss (editors), pages 361–411, Springer, 2010. Preprint
  Publisher's web page: www.springer.com 

2009

• Frank Hutter. Automated Configuration of Algorithms for Solving Hard Computational Problems.
  PhD thesis, Department of Computer Science, University of British Columbia. 
  2010 CAIAC Doctoral Dissertation Award for the best thesis in Artificial Intelligence at a Canadian University completed in 2009.
  October 2009 [pdf] [bib] [slides from the defense] [slides from invited presentation at Canadian AI grad student symposium]
  See the Automated Algorithm Configuration project page for a lot of experimental data (target algorithms, parameters, benchmark instances, and configuration proceduers).
  




• Frank Hutter and Marco A. Montes de Oca (Eds.)
  SLS-DS 2009: Doctoral Symposium on Engineering Stochastic Local Search Algorithms
  August 2009 [pdf] [bib]




• Frank Hutter, Holger H. Hoos, Kevin Leyton-Brown, and Thomas Stützle.
  ParamILS: An Automatic Algorithm Configuration Framework [pdf] [bib] 
  In Journal of Artificial Intelligence Research (JAIR), volume 36, pp. 267-306, October 2009.
  See the ParamILS project page  for a lot of experimental data for this paper (target algorithms, parameters, resulting parameter configurations).
  There's also a quick start guide available to help you apply ParamILS for tuning your own algorithms.
  There's also an older tech report about ParamILS, including additional material. [pdf] [bib]




• Frank Hutter, Holger H. Hoos, Kevin Leyton-Brown, and Kevin Murphy.
  An Experimental Investigation of Model-Based Parameter Optimisation: SPO and Beyond [pdf] [bib] [slides]
  In ACM Genetic and Evolutionary Computation Conference (GECCO-09). 



• Lin Xu, Frank Hutter, Kevin Leyton-Brown, and Holger H. Hoos. January 2009
  SATzilla2009: an Automatic Algorithm Portfolio for SAT [pdf] [bib]
  Solver description for the 2009 SAT competition. SATzilla2009 won 3 gold and 2 silver medals in that competition. See the SATzilla project page for details and source code.

2008

• Lin Xu, Frank Hutter, Holger Hoos, and Kevin Leyton-Brown.
  SATzilla: Portfolio-based Algorithm Selection for SAT [pdf] [bib]
  In Journal of Artificial Intelligence Research (JAIR), volume 32, pp. 565–606, June 2008.
  2010 IJCAI/JAIR Best Paper Prize for the period 2005-2009.
  See the SATzilla project page for details and source code.

2007

• Frank Hutter.
  On the Potential of Automatic Algorithm Configuration [pdf] [poster]
  In Proceedings of the Doctoral Symposium on Engineering Stochastic Local Search Algorithms (SLS-DS).
  Best poster award (voted by the attendees of SLS 07).




• Frank Hutter, Domagoj Babic, Holger Hoos, and Alan Hu.
  Boosting Verification by Automatic Tuning of Decision Procedures [pdf] [bib]
  In Formal Methods in Computer Aided Design (FMCAD-07).
  With the tuning discussed in this paper Domagoj's solver Spear won the QF_BV (Quantifier-Free Bit Vector) category of the 2007 Satisfiability Modulo Theories Competition.




• Lin Xu, Frank Hutter, Holger Hoos, and Kevin Leyton-Brown.
  SATzilla-07: The Design and Analysis of an Algorithm Portfolio for SAT [pdf] [bib]
  In Proceedings of the 13th International Conference on Principles and Practice of Constraint Programming (CP-07).
  SATzilla won 3 gold medals, 1 silver and 1 bronze in the 2007 SAT competition! It is available for download from the SATzilla website.




• Frank Hutter, Holger Hoos, and Thomas Stützle.
  Automatic Algorithm Configuration based on Local Search [pdf] [slides] [bib]
  In Proceedings of the Twenty-First Conference on Artificial Intelligence (AAAI-07).
  The ParamILS algorithm introduced in this paper is available for download from the ParamILS website. There's also a quick start guide available to help you apply it for tuning your own algorithms.

I'm involved with 3 submissions to the SAT competition 2007. For each of these, there is an unrefereed 2-page solver description:

• Lin Xu, Frank Hutter, Kevin Leyton-Brown, and Holger H. Hoos. January 2007
  SATzilla2007: a New and Improved Algorithm Portfolio for SAT [pdf]
  In a nutshell, SATzilla predicts the runtime of each solver in the portfolio and picks the most promising one.
  SATzilla2007 won 3 gold medals, 1 silver and 1 bronze! See the SAT competition webpage for details.




• Dave Tompkins, Frank Hutter, and Holger H. Hoos. January 2007
  Scaling and Probabilistic Smoothing (SAPS) [pdf]
  SAPS is unchanged from last year, but I got a tenfold speedup by automated parameter tuning (using the techniques from the AAAI-07 paper above).




• Domagoj Babic (Theorem prover architect), and Frank Hutter (Search parameter optimization). January 2007
  SPEAR Theorem Prover [pdf] [bib]
  SPEAR is a new tree search algorithm with 25 free parameters. I tuned it (using the techniques from the AAAI-07 paper above), getting a 30% speedup; for software verification, my parameter settings beat the default by a factor of 50!

2006

• Frank Hutter.
  Automated Algorithm Configuration Based on Machine Learning
  PhD proposal, Department of Computer Science, University of British Columbia.




• Frank Hutter, Youssef Hamadi, Holger H. Hoos, and Kevin Leyton-Brown.
  Performance Prediction and Automated Tuning of Randomized and Parametric Algorithms [pdf] [slides] [bib]
  In Proceedings of the 12th International Conference on Principles and Practice of Constraint Programming (CP-06), pp. 213--228.
  All our experimental data for this paper, as well as our Matlab code, is available on the Empirical Hardness Models project page.
  A short version of this paper also appeared at the AAAI 06 workshop on Learning for Search. [pdf] [slides]




• Wheeler Ruml, and Frank Hutter, editors.
  Proceedings of the AAAI 06 workshop on Learning for Search. [Available through the AAAI digital library]

2005

• Frank Hutter and Youssef Hamadi.
  Parameter Adjustment Based on Performance Prediction: Towards an Instance-Aware Problem Solver [pdf] [bib]
  Technical Report MSR-TR-2005-125, Microsoft Research Cambridge, UK, 2005.
  Slides from a talk I gave at the Cork Constraint Computation Centre (4C)
  Slides from a talk I gave in the Lab for Computational Intelligence at UBC
  



• Frank Hutter, Holger H. Hoos, and Thomas Stützle.
  Efficient Stochastic Local Search for MPE Solving. [pdf] [slides] [bib]
  In Proceedings of the Nineteenth International Joint Conference on Artificial Intelligence (IJCAI-05), pp. 169--174.
  My solver GLS+ and the test instances we used are available on our MPE page.
  The solver can read general factor graphs, i.e. Bayes nets (in BNT format), MRFs, CRFs, etc. There's also a nice Matlab interface.

2004

• Frank Hutter.
  Stochastic Local Search for Solving the Most Probable Explanation Problem in Bayesian Networks. [pdf]
  M.Sc. thesis, Intellectics Group, Darmstadt University of Technology, 2004.
  Supervisor: Thomas Stützle, Cosupervisor: Holger Hoos
  My solver GLS+ and most of the test instances I used are available on our MPE page.




• Frank Hutter, Brenda Ng, and Richard Dearden.
  Incremental Thin Junction Trees for Dynamic Bayesian Networks. [pdf]
  Technical Report TR-AIDA-04-01, Intellectics Group, Darmstadt University of Technology, 2004.




• Nando de Freitas, Richard Dearden, Frank Hutter, Ruben Morales-Menendez, Jim Mutch, and David Poole.
  Diagnosis by a Waiter and a Mars Explorer. [pdf]
  Invited paper for Proceedings of the IEEE, Special Issue on Sequential State Estimation, 92(3):455–468, 2004.
  Check out my GPF webpage for the particle filtering code used for the rover examples.




• Richard Dearden, Frank Hutter, Reid Simmons, Sebastian Thrun, Vandi Verma, and Thomas Willeke.
  Real-time Fault Detection and Situational Awareness for Rovers: Report on the Mars Technology Program Task. [pdf]
  Proceedings of IEEE Aerospace Conference, March 2004.
  Check out my GPF webpage for the Gaussian particle filtering code.




• Mirela Andronescu, Anthony P. Fejes, Frank Hutter, Anne Condon, and Holger H. Hoos.
  A New Algorithm for RNA Secondary Structure Design. [pdf]
  Journal of Molecular Biology, 336(3):607– 624, 2004.
  Check out the free RNA Designer Software at http://www.rnasoft.ca/

2003

• Frank Hutter and Richard Dearden:
  The Gaussian Particle Filter for Diagnosis of Non-Linear Systems. [pdf]
  In Proceedings of the 14th International Conference on Principles of Diagnosis(DX-03), Washington, DC, USA, pp. 65--70, June 2003.
  Check out my GPF webpage for the Gaussian particle filtering code.
  




• Frank Hutter and Richard Dearden:
  Efficient On-line Fault Diagnosis for Non-Linear Systems. [pdf]
  In Seventh International Symposium on Artificial Intelligence and Robotics in Space (i-SAIRAS-03), May, 2003.
  Check out my GPF webpage for the Gaussian particle filtering code.
  

2002

• Frank Hutter, Dave A.D. Tompkins and Holger H. Hoos:
  Scaling and Probabilistic Smoothing: Efficient Dynamic Local Search for SAT. [pdf] [bib]
  In LNCS 2470: Proceedings of the 8th International Conference on Principles and Practice of Constraint Programming (CP-02), pp.233-248, Springer Verlag, 2002.
  Check out the DLS for SAT webpage, maintained by Dave

• Mirela Andronescu, Anthony P. Fejes, Firas Hamze, Frank Hutter, Holger H. Hoos, and Anne Condon:
  A New SLS Algorithm for RNA Secondary Structure Design. [ps.tar.gz]
  Technical Report, Department of Computer Science, University of British Columbia, TR-2002-10, April 2002
  Check out the free RNA Designer Software at http://www.rnasoft.ca/

Pre-PhD projects

• Not papers per se, but here are some of my pre-PhD projects.

Non academic papers (old)

• Report on my experiences studying abroad (at the University of British Columbia, Canada). Submitted to the German Academic Exchange Service.
  Frank Hutter: Studieren einmal anders (in German).
  Erfahrungsbericht August 2001 - Mai 2002, Integriertes Auslandsstudium an der University of British Columbia in Vancouver, Kanada im Department of Computer Science
  
  


• Whitepaper on a software system developed by our company BIIT (Business Intelligence Information Technology).
  Frank Hutter and Markus Breitenbach: Harvesta - A new Approach in Information Management Suited for Corporate Intelligence (.pdf)