| Tuning scenario | Training set of benchmark instances | Test set of benchmark instances | Citation | Short description |
| SAPS-QWH | QWHmedinst | Same single instance | @inproceedings{GomSel97, author = "C.~P. Gomes and B. Selman", title = "Problem Structure in the Presence of Perturbations", booktitle = aaai97, year = "1997" } |
Single quasigroups with holes (QWH) instance of median hardness for SAPS |
| SAPS-QCPmed | QCPmedinst | Same single instance | @inproceedings{GomSel97, author = "C.~P. Gomes and B. Selman", title = "Problem Structure in the Presence of Perturbations", booktitle = aaai97, year = "1997" } |
Single quasigroup completion instance (QCP) of median hardness for SAPS |
| SAPS-QCP 75% quantile | QCPq075inst | Same single instance | @inproceedings{GomSel97, author = "C.~P. Gomes and B. Selman", title = "Problem Structure in the Presence of Perturbations", booktitle = aaai97, year = "1997" } |
Single quasigroup completion instance (QCP), 75% quantile of hardness distribution for SAPS |
| SAPS-QCP 95% quantile | QCPq095inst | Same single instance | @inproceedings{GomSel97, author = "C.~P. Gomes and B. Selman", title = "Problem Structure in the Presence of Perturbations", booktitle = aaai97, year = "1997" } |
Single quasigroup completion instance (QCP), 95% quantile of hardness distribution for SAPS |
| SAPS-SWGCPmed | SWGCPmedinst | Same single instance | @inproceedings{GenHooProWal99, author = "I.~P. Gent and H.~H.~Hoos and P.~Prosser and T.~Walsh", title = "Morphing: Combining Structure and Randomness", booktitle = aaai99, pages = "654--660", year = "1999" } |
Single graph colouring based on small world graph (SWGCP) instance, median hardness for SAPS |
| SAPS-SWGCPq075 | SWGCPq075inst | Same single instance | @inproceedings{GenHooProWal99, author = "I.~P. Gent and H.~H.~Hoos and P.~Prosser and T.~Walsh", title = "Morphing: Combining Structure and Randomness", booktitle = aaai99, pages = "654--660", year = "1999" } |
Single graph colouring based on small world graph (SWGCP) instance, 75% quantile of hardness distribution for SAPS |
| SAPS-SWGCPq095 | SWGCPq095inst | Same single instance | @inproceedings{GenHooProWal99, author = "I.~P. Gent and H.~H.~Hoos and P.~Prosser and T.~Walsh", title = "Morphing: Combining Structure and Randomness", booktitle = aaai99, pages = "654--660", year = "1999" } |
Single graph colouring based on small world graph (SWGCP) instance, 95% quantile of hardness distribution for SAPS |
| SAPS-SWGCP | Training
set (1000 instances) |
Test
set (1000 instances) |
@inproceedings{GenHooProWal99, author = "I.~P. Gent and H.~H.~Hoos and P.~Prosser and T.~Walsh", title = "Morphing: Combining Structure and Randomness", booktitle = aaai99, pages = "654--660", year = "1999" } |
SAT-encoded
graph colouring based on small world graphs. all instances satisfiable |
| Spear-SWGCP | Training
set (1000 instances) |
Test
set (1000 instances) |
@inproceedings{GenHooProWal99, author = "I.~P. Gent and H.~H.~Hoos and P.~Prosser and T.~Walsh", title = "Morphing: Combining Structure and Randomness", booktitle = aaai99, pages = "654--660", year = "1999" } |
SAT-encoded
graph colouring based on small world graphs. sat/unsat instances |
| Old
version of SAPS-QCP, used in ParamILS 2009 Tech report Do not use (see short description) |
Training
set (1000 instances) |
Test
set (1000 instances) |
@inproceedings{GomSel97, author = "C.~P. Gomes and B. Selman", title = "Problem Structure in the Presence of Perturbations", booktitle = aaai97, year = "1997" } |
SAT-encoded
quasigroup completion. all instances satisfiable This scenario has a problem with the split into training and test set: the training set was systematically easier than the test set; I only keep it up here for completeness, but would advise against using it |
| New version of SAPS-QCP, used in JAIR-09 paper and all successive publications | Training
set (1000 instances) |
Test
set (1000 instances) |
@inproceedings{GomSel97, author = "C.~P. Gomes and B. Selman", title = "Problem Structure in the Presence of Perturbations", booktitle = aaai97, year = "1997" } |
SAT-encoded
quasigroup completion. all instances satisfiable |
| Spear-QCP | Training
set (1000 instances) |
Test
set (1000 instances) |
@inproceedings{GomSel97, author = "C.~P. Gomes and B. Selman", title = "Problem Structure in the Presence of Perturbations", booktitle = aaai97, year = "1997" } |
SAT-encoded
quasigroup completion sat/unsat instances |
| Spear-swv | Training
set (302 instances) |
Test
set (302 instances) |
@inproceedings{babic07structural-abs, author = {Domagoj Babi\'c and Alan J. Hu}, title = {{Structural Abstraction of Software Verification Conditions}}, booktitle = {Computer Aided Verification: 19th International Conference, CAV 2007}, year = {2007}, pages={366--378} } |
SAT-encoded software verification |
| Spear-ibm | We cannot provide these instances online due to copyright issues. You can acquire them from the IBM Formal Verification Benchmarks Library. We used 40 random subsets of these instances. Here are the names of our 382 training instances and names of our 383 test instances. | @inproceedings{zarpas05benchmarking, author = {Emanuel Zarpas}, title = {{Benchmarking SAT Solvers for Bounded Model Checking}}, booktitle = {SAT '05: Proc.~of the 8th International Conference on Theory and Applications of Satisfiability Testing}, year = {2005}, pages = {340--354} } |
SAT-encoded hardware verification (BMC) | |
| SAPS-random | Training
set (363 instances) |
Test
set (363 instances) |
See,
e.g. @inproceedings{LeBSim04, author = {{Le~Berre}, D. and Simon, L. }, title = {Fifty-five solvers in {Vancouver}: The {SAT} 2004 competition}, booktitle = sat04, year = {2004}, pages = {321--344} } |
All satisfiable instances in the RANDOM category from all SAT competitions until 2007 |
| SAPS-crafted | Training
set (189 instances) |
Test
set (188 instances) |
See,
e.g. @inproceedings{LeBSim04, author = {{Le~Berre}, D. and Simon, L. }, title = {Fifty-five solvers in {Vancouver}: The {SAT} 2004 competition}, booktitle = sat04, year = {2004}, pages = {321--344} } |
All satisfiable instances in the CRAFTED category from all SAT competitions until 2007 |
Please send any questions, concerns or comments to Frank Hutter