ParamILS

News

Abstract

ParamILS is a versatile tool for parameter optimization / algorithm configuration.
It has helped us speed up both local search and tree search algorithms by orders of magnitude on certain instance distributions, and we hope you find it useful, too.
Ultimately, we hope that ParamILS helps algorithm designers focus on tasks that are more scientifically valuable than parameter tuning.

People

• Frank Hutter, Postdoctoral research fellow (UBC)
• Holger Hoos, Professor (UBC)
• Thomas Stützle, Research Associate (Université Libre de Bruxelles, ULB)
• Kevin Leyton-Brown, Associate Professor (UBC)
• Dave Tompkins, Postdoctoral research fellow (UBC): designer of UBCSAT, a framework for local search algorithms; we used the UBCSAT implementation of SAPS in our experiments
• Domagoj Babic, Ph.D. from UBC in 2008: designer of the tree search algorithm Spear (for which we get great speedups)
• Chris Fawcett, Ph.D. student (UBC): Responsible for maintenance and future feature development for ParamILS as of Fall 2008.

Papers

• Frank Hutter, Holger Hoos, and Thomas Stützle.
  Automatic Algorithm Configuration based on Local Search [pdf] [slides] [bib]
  In AAAI-07.
• Frank Hutter, Holger H. Hoos, Kevin Leyton-Brown, and Thomas Stützle.
  ParamILS: An Automatic Algorithm Configuration Framework [pdf] [bib] 
  In Journal of Artificial Intelligence Research (JAIR), volume 36, pp. 267-306, October 2009.

Data

• Benchmark instances used in our AAAI-07 paper, and parameter files for SAT4J and GLS+
• Algorithms used in our JAIR paper, their parameters and the values we considered for each parameter
• Benchmark instances used in our JAIR paper, and parameter settings found with FocusedILS for each of them

Software

• ParamILS version 2.3.5 [source, executables and examples] [quick start guide]. This is the newest version, and the one you should be using to tune your algorithms. Minor bugfix extending the 2.3.2 bugfix to nondeterministic cases when the user has specified an instance-seed file but failed to specify N correctly. A warning is now printed, and the appropriate values are clamped.
• ParamILS version 2.3.4 [source, executables and examples] [quick start guide]. Minor bugfix removing the need for a "-I" parameter when calling ParamILS from source. Users should now be able to call ParamILS from whatever working directory they like.
• ParamILS version 2.3.3 [source, executables and examples] [quick start guide]. This is another bugfix release, in order to rectify a problem experienced when trying to run ParamILS under Windows. This is a joint release of the source along with binaries for Linux and Windows.
• ParamILS version 2.3.2 [executable and examples] [complete source] [quick start guide]. This is another bugfix release, fixing a bug when tuning deterministic solvers when the -N parameter is unspecified. Tuning runs should be unchanged for other scenarios. Also contains some merged and cleaned up code from Frank Hutter.
• ParamILS version 2.3.1 [executable and examples] [complete source] [quick start guide]. This is another bugfix release, in order to fix a problem where timed out runs were penalised by a factor of 10 when tuning for runtime, even when overall_obj was set to 'mean' and not 'mean10'. Scenarios not tuning for runtime should be unaffected.
• ParamILS version 2.3 [executable and examples] [complete source] [quick start guide]. Version 2.3 is a bugfix release, correcting a problem that would cause ParamILS to try and access a database regardless of the value passed with the -db flag.
• ParamILS version 2.2 [executable and examples] [complete source]. There is a new new quick start guide (updated Aug 1, 2008) that covers the much improved user interface.
  The zip file also contains examples enabling you to tune all the algorithms we cover in our journal article on ParamILS submitted to JAIR.
  Version 2.2  is idential to 2.1 except that the user-interface has been simplified: users no longer have to generate instance-seed files, and also have the option to define their tuning scenario in an input file (for later reuse)
• ParamILS version 2.1 (zip, 105KB). This release is to ensure repeatability of experiments: it is the modified version used for the self-tuning experiments in our journal article on ParamILS submitted to JAIR
  In version 2.1, the inefficient internal caching mechanism inside the ParamILS scripts has been addressed (for tuning scenarios where single algorithm runs are well below a second, the time
  spent for ParamILS's internal bookkeeping became substantial compared to the time spent running algorithms).
  Unfortunately, since Hashes are indexed differently now, version 2.1 will not return the exact same result as version 2.0, even with identical seeds.
  (Means and standard deviations of 25 runs are, however, virtually the same as for version 2.0)
• ParamILS version 2.0 (zip, 109KB). This release is to ensure repeatability of experiments: it is the version used for the experiments in our journal article on ParamILS submitted to JAIR.
  Version 2.0 introduced major changes to enable Bound & Prune, our technique for early pruning of unsuccesful runs.
• ParamILS version 1.0 (tar.bz2, 440KB). The tar.gz file also contains a quick start guide that should help you apply it to your algorithms.

Licensing

Please send any questions, concerns or comments to Frank Hutter or Chris Fawcett