Pre-CRC/release tasks

UI tasks

Release-critical

• CF N-way performance comparison first-cut for Frank.

• CF algorithm specification screen: implement
• CF left side of landing page: task selection/presentation according to pattern concept
• CF experiment specification and monitor screens from a pattern template, and procedure-specific requirements (may require DB changes)
• CF instance specification screen: implement (requires DB change)
• CF Execution environment specification (incl. R, Gnuplot, java locations; requires DB change)
• CN RTDs/per-target-algorithm-run monitoring and navigation

Important

• Data management interface:
  • deleting runs/expts/etc.
  • data export
• Error logging/handling/browsing
• Plotting ex-gnuplot

Database tasks

Release-critical

• CN Named instance set table
• CN Named configuration table
• CN Execution environment table
• CN Configuration spaces vs. algorithms; appropriate unique ID hashing

Important

• Connection pooling
• Caching analysis results
• Database schema -- speed-related redesign
• Query optimization
• Selective limitation of run-level logging (dynamic based on runtime?)

Nice-to-have

• DataManager API refinement
• Stale connection issue; incl. robustness to general network issues
• Read-only DataManager connection for use by individual MA procedures

Code/Robustness/Misc. tasks

Important

• user-facing documentation (help)
• Better logging/error-reporting (to console/within HAL). eg: log4j
• Better handling of overhead runtime vs. target algorithm runtime
• WAY more unittests; also functional/integration tests

Nice-to-have

• developer-facing documentation (javadocs)

Medium-term Plans

• Packaging complete experiments
• Windows support
• libraries of:
  • search/optimization procedures
  • machine learning tools
• multi-algorithm comparisons
• scaling analyses
• bootstrapped analyses
• robustness analyses
• parameter response analyses
• SATzilla in HAL
• ParamILS in HAL
• Parallel portfolios in HAL
• ActiveConfigurator in HAL
• Iterated F-Race in HAL
• chained-procedure experiments
• support for optimization/Monte-Carlo experiments
• support instance generators
• Git, not CVS
• Support text-file inputs and outputs for external algorithms
• Instance features
• Experiments calling experiments, not just external target algs
• array jobs in SGE

Feature Requests

• (FH) submitting runs from a machine that is itelf a cluster submit host should not need to go through SSH

Known Bugs:

• (CN) JSC test reliability issue (compared to R)
• (CN) end-of-experiment hanging bug (GGA, multinode cluster runs)
• (JS) InnoDB SQL errors
• (LX) missing current-time point in solution quality trace, so don't see the final "flat line"
• (CN) accuracy of mid-run overhead accounting for PILS/GGA