|
news
| abstract
| people
| papers
| software
News
| February 16, 2013 |
New release (version 2.04.01), including some bug fixes,
usability improvements and minor feature improvements |
| October 25th, 2012 |
After a series of internal releases, SMAC version 2 has been
publically released! In short, this is a complete
rewrite
of SMAC in Java that features many improvements, is well
documented, and is portable & easy to use. |
| February 1, 2012 |
A substantially
improved version of SMAC will be available soon; if you want to start
using SMAC in the meantime, please send a quick email to Frank.
We also plan to provide a quickstart guide similar to the one
for ParamILS, as well as a Java implementation of SMAC. |
| September 9, 2011 |
First
version of this
page set up. Before this, SMAC was only available upon request.
|
Abstract
SMAC (sequential model-based
algorithm configuration) is a versatile
tool for optimizing algorithm parameters (or the parameters of
some
other process we can run automatically, or a function we can evaluate,
such as a simulation).
SMAC has helped us speed up both local search and tree search
algorithms
by orders of magnitude on certain instance distributions. Recently, we
have also found it to be very effective for the
hyperparameter optimization of machine learning algorithms, scaling
better to high dimensions and discrete input dimensions than other
algorithms. Finally, the predictive models SMAC is based on can
also capture and exploit important information
about the model domain, such as which input variables are most
important.
We hope
you find SMAC similarly useful. Ultimately, we hope that it helps
algorithm designers focus on
tasks that are more scientifically valuable than parameter tuning.
People
Papers
- Frank Hutter, Holger Hoos, and Kevin Leyton-Brown.
An evaluation of sequential
model-based optimization for expensive blackbox functions
In GECCO 2013 Blackbox Optimization
Benchmarking workshop (BBOB'13). To appear. [pdf][bib]
-
Frank
Hutter, Holger Hoos, and Kevin
Leyton-Brown.
Parallel
Algorithm Configuration
In: Learning
and Intelligent
Optimization (LION 6). To
appear [pdf][pptx
slides][pdf
slides][bib]
-
Frank
Hutter, Holger
Hoos, and Kevin
Leyton-Brown.
Bayesian Optimization With Censored Response Data
2011 NIPS workshop on Bayesian
Optimization,
Experimental Design, and Bandits. [pdf]
[poster]
[bib]
- Frank
Hutter, Holger Hoos, and Kevin
Leyton-Brown.
Sequential
Model-Based
Optimization for General Algorithm Configuration
In LION-5, 2011. [pdf]
[ppt slides]
[pdf
slides]
[bib]
Second best paper prize
An
extended version with additional details is available as
UBC tech
report TR-2010-10. [pdf]
[bib]
- Frank
Hutter, Holger Hoos, Kevin Leyton-Brown, and
Kevin Murphy.
Time-Bounded
Sequential Parameter
Optimization
In LION4,
2010 [pdf ]
[bib] [slides]
Runner-up
for the best paper
award
The original publication
is available at www.springerlink.com.
Data
We maintain algorithm configuration benchmarks on the Automated
Algorithm Configuration
project page.
Software
- SMAC version 2.04.01 (February 2013) [source, executables,
documentation,
examples] (tar.gz, 10818KB).
Note: If using the Sun JVM, version 1.6.0_23 or later is
highly recommended.
Minor improvements to SMAC including:
- [BUG FIX] Previous versions could ocassionally select
a worse configuration if the target algorithm ran out of memory
- SMAC can now verify that target algorithm reports of SAT or
UNSAT match the expected value (given as instance-specific information)
- When starting SMAC via the shell script, you can set the
SMAC_MEMORY environment variable to limit the amount of memory SMAC uses
- Scenario context is now saved with state files to make the
state folder mostly self contained
- Improved memory usage in state serialization and when
building model to reduce OutOfMemoryErrors
- FAQ reorganized
- Bash autocompletion
The zip file includes a Quickstart
guide, an extensive Manual, and a FAQ.
Previous Versions
- SMAC version 2.02 (October 2012) [source,executables,documentation,
examples] (tar.gz, 9715KB).
A complete rewrite of SMAC in Java. This features numerous new
features, is well documented, and is portable & easy to use, only
relying on Java.
The zip file includes a Quickstart guide, an extensive manual, and a
FAQ.
- SMAC-BBOB [source,
executables and examples]
(tar.gz, 10.1MB). A SMAC version for expensive continuous blackbox
optimization; it uses GPs with a Matern kernel, no initial design, and
optimizes EI by a combination of local search, DIRECT, and CMA-ES (see
BBOB-2013 workshop paper above). Here is the data underlying that
paper: [data] (tar.gz, 136MB);
[data only for SMAC-BBOB and
CMA-ES, 100D] (tar.gz, 11MB)
- d-SMAC version 0.8 [source,
executables and examples]
(tar.gz,
3.4MB). A distributed version of SMAC (see LION 6 paper above).
- SMAC version 1.0.1 [source,
executables and examples]
(tar.gz, 5.5MB). Minor bugfix release.
- SMAC version 1.0 [source,
executables and examples]
(tar.gz,
4.8MB).
This is the publically released version of SMAC. SMAC will be
free to use for non-commercial purposes.
There is not yet a quick start guide, but the input format is very
similar to that of ParamILS (in fact, SMAC should be able to read
ParamILS scenarios files directly, although performance will improve if
you also specify features for your problem instances). You can call
smbo_runner without any arguments to see some usage examples (the same
as we had for ParamILS).
To reproduce the SMAC version from the LION-5 paper, use the following
callstring:
smbo_runner <numRun> model rf scenario_file
<scenario_file>
To reproduce the ROAR version from the LION-5 paper, use the following
callstring:
smbo_runner <numRun> method pure-random scenario_file
<scenario_file>
Please send any questions,
concerns or comments to Frank
Hutter