Feature Milestones

HAL 1.0

• Ability to monitor the trajectories of all output variables of an executed algorithm run, in real time done
• Ability to query database of previous runs directly done in redesign
• Ability to access instance features in prog. in refactor

• Pre-defined metrics for aggregating performance across runs done in redesign

Backend functionality exposed in above

Meta-Algorithms Included

• Configuration procedure: ParamILS (external) done; will need minor updates to work with backend redesign
• Configuration procedure: ROAR (internal) done; will need minor updates to work with backend redesign

• Analysis procedure: Paired algorithm comparison done, updated
• Analysis procedure: Single-algorithm analysis done; will need minor updates to work with backend redesign

• Ability to "chain" experiments (eg. design procs. followed by analysis proc comparing incumbents)

Functionality for meta-algorithm developers

• support for feature extraction procedures

Backend functionality

• Support for "bag-of-machines" execution manager

Meta-Algorithms Included

• Multi-algorithm comparison
• SATzilla-like portfolio builder
• Parallelized AC