|
<
< | Pre-CRC/release tasks
UI tasks |
>
> | Short-term priorities (Pre-CRC/release)
Frontend |
| | Release-critical |
|
<
< |
- CF N-way performance comparison first-cut for Frank.
|
| | functionality promised in paper |
|
<
< |
- CF algorithm specification screen: implement
|
>
> |
- CF algorithm specification screen: implement (includes initial design space specification)
|
| |
- CF left side of landing page: task selection/presentation according to pattern concept
|
|
<
< |
- CF experiment specification and monitor screens from a pattern template, and procedure-specific requirements (may require DB changes)
- CF instance specification screen: implement (requires DB change)
- CF Execution environment specification (incl. R, Gnuplot, java locations; requires DB change)
- CN RTDs/per-target-algorithm-run monitoring and navigation
|
>
> |
- CF experiment specification and monitor screens from a pattern template, and procedure-specific requirements, including experiment and inclubment naming
- CF instance specification screen: implement
- CF Execution environment specification (incl. R, Gnuplot, java locations)
- RTDs/per-target-algorithm-run monitoring and navigation
- design space specification by revision of existing spaces
|
| |
Important
works as-is but end-user experience significantly impacted |
| |
- Named configurations, ability to specify name for final incumbent a priori and reference it in subsequent comparison experiments.
|
|
<
< | Database tasks |
>
> | Backend |
| | Release-critical
mostly to enable critical UI tasks
- CN Named instance set table done
- CN Named configuration table done
- CN Execution environment table done
|
|
<
< |
- CN Configuration spaces vs. algorithms; appropriate unique ID hashing; algorithm versions?
|
>
> |
- CN Split algorithms and configuration spaces, allowing run reuse for common-binary configuration spaces. Both DB and Java object model.
- CN Explicit representation of problems/encodings, compatability of algs and instances via problem (encodings)
- CN add incumbentname semantic input to (design) procedures
- CN rename objects to match paper terminology
|
| |
Important
mostly to (substantially) improve UI responsiveness |
|
>
> |
- Database schema -- speed-related refactor
|
| |
- Connection pooling
- Caching analysis results
|
|
<
< |
- Database schema -- speed-related redesign
|
| | |
|
<
< |
- Selective limitation of run-level logging (dynamic based on runtime?)
|
>
> |
- Selective limitation of run-level archiving (dynamic based on runtime?)
|
| |
Nice-to-have
noticeable mostly to developer-users |
|
>
> |
- CF N-way performance comparison first-cut for Frank.
|
| |
- Stale connection issue; incl. robustness to general network issues
- Read-only DataManager connection for use by individual MA procedures
|
|
>
> |
- Allowing relationships (incl. possible run-reuse) between different-binary "builds" of algorithms, including due to bugfixes, additional exposed parameters, etc. Also for different "versions" (without reuse) corresponding to added funcitonality.
- Ability to quantify membership of configurations to different design spaces
|
| |
Code/Robustness/Misc. tasks |
| |
- Experiments calling experiments, not just external target algs
- array jobs in SGE
- Hashing everything, including instances, instance sets and configurations.
|
|
>
> | |
| | |
|
<
< | Feature Requests |
>
> | Feature Requests (unprioritized/long-term) |
| |
- (FH) Support for complete configuration experiment, front to back: run configurator N times on a training set, report the N training and test set performances
- (FH) Developers of configurators should be able to swap in new versions of a configurator
- (FH) Configuration scenarios, specifying a complete configuration task including the test set; only missing part being the configurator
- (FH) Saveable sets of configuration scenarios to perform (use case: I change the configurator and want to evaluate it)
- (FH) Taking this a step further: support for optimizing a parameterized configurator (configurator is an algorithm, and the above set of experiments is the set of "instances")
- (FH) Submitting runs from a machine that is itelf a cluster submit host should not need to go through SSH
|
|
<
< |
- (CF) Distinction made in HAL between parameterised and parameter-less algorithms, including the ability to see the "parent" parameterised algorithm of a given parameter-less algorithm. Other similar queries would be equally useful (all SPEAR configurations, the parameterised algorithm corresponding to a partial instantiation of another parameterised algorithm, etc.) (CN) need to flesh out how this interacts with named configurations/configuration spaces also under discussion
|
| |
- (CF) Ability to browse algorithms, instances, instance sets, configurations, etc. This includes the ability to see things related to the item being browsed. Performance of different algorithms/configurations on a given instance, performance of algorithms across an instance set, performance of a given configuration.
- (CF) Form validation.
- (CF) Wider support for working directory requirements of individual algorithm runs, i.e. Concorde's creation of 20 files with fixed names.
|
Copyright © 2008-2025 by the contributing authors. All material on this collaboration platform is the property of the contributing authors.