autoweka
Class Experiment

java.lang.Object
  extended by autoweka.XmlSerializable
      extended by autoweka.Experiment

public class Experiment
extends XmlSerializable


Nested Class Summary
static class Experiment.TrajectoryPointExtra
          Class capturing any of the extra dataset partitions that should be run on all the trajectory points for analysis
 
Field Summary
 java.util.List<java.lang.String> allowedClassifiers
          Forces Auto-WEKA to only use the list of classifiers here - if it is empty, then Auto-WEKA will try to use everything that it can
 boolean attributeSelection
          Boolean indicating if Attribute/Feature selection should happen
 float attributeSelectionTimeout
          How many seconds should be spent performing attribute selection?
 java.util.List<java.lang.String> callString
          The list of command line arguments that are used to invoke the SMBO method - this is set by the ExperimentConstructor
 java.lang.String datasetString
          The string that indicates what dataset is going to be used
 java.util.List<java.lang.String> envVariables
          A list of var=values that are set in environment variables before the SMBO method is executed - these are set by the ExperimentConstructor
 java.lang.String extraPropsString
          Any extra properties that are associated with the experiment, these are generally for the constructor or your wrappers
 java.lang.String instanceGenerator
          The class name of the instance generator
 java.lang.String instanceGeneratorArgs
          The property string that contains the construction arguments for the instance generator
 java.lang.String memory
          The string passed to the Xmx argument of a sub process limiting the RAM that WEKA will have
 java.lang.String name
          The name of the experiment, designed to be human readable and contain useful ID information
 java.lang.String resultMetric
          The name of the result metric to use - see ClassiferResult.Metric
 float trainTimeout
          How many seconds should be spent training a set of hyperparameters on a specific partition of the training and test data?.
 java.lang.String trajectoryParserClassName
          The name of the class that's going to be used to parse the trajectory - this is set by the ExperimentConstructor
 java.util.List<Experiment.TrajectoryPointExtra> trajectoryPointExtras
          For analysis experiements, these extra classifier evaluations will be done for every point along the trajectory.
 float tunerTimeout
          How many seconds should be spent for this experiment overall? (IE the SMBO method's budget)
 java.lang.String type
          A helper variable that indicates the type of the experiment - this is set by the ExperimentConstructor
 
Constructor Summary
Experiment()
           
 
Method Summary
static Experiment createFromFolder(java.io.File folder)
           
static Experiment fromXML(java.io.InputStream xml)
          Static method to load up an experiment from XML.
static Experiment fromXML(java.lang.String filename)
          Static method to load up an experiment from XML.
static void main(java.lang.String[] args)
          Main method to actually run an experiment with a given seed.
 void validate()
          Throws a runtime exception if the experiment contains some sort of crazy values.
 
Methods inherited from class autoweka.XmlSerializable
fromXML, fromXML, populateObjectFromCMDParams, populateObjectFromCMDParams, toXML, toXML
 
Methods inherited from class java.lang.Object
clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait
 

Field Detail

name

public java.lang.String name
The name of the experiment, designed to be human readable and contain useful ID information


type

public java.lang.String type
A helper variable that indicates the type of the experiment - this is set by the ExperimentConstructor


resultMetric

public java.lang.String resultMetric
The name of the result metric to use - see ClassiferResult.Metric


instanceGenerator

public java.lang.String instanceGenerator
The class name of the instance generator


instanceGeneratorArgs

public java.lang.String instanceGeneratorArgs
The property string that contains the construction arguments for the instance generator


datasetString

public java.lang.String datasetString
The string that indicates what dataset is going to be used


attributeSelection

public boolean attributeSelection
Boolean indicating if Attribute/Feature selection should happen


attributeSelectionTimeout

public float attributeSelectionTimeout
How many seconds should be spent performing attribute selection?


trajectoryParserClassName

public java.lang.String trajectoryParserClassName
The name of the class that's going to be used to parse the trajectory - this is set by the ExperimentConstructor


callString

public java.util.List<java.lang.String> callString
The list of command line arguments that are used to invoke the SMBO method - this is set by the ExperimentConstructor


envVariables

public java.util.List<java.lang.String> envVariables
A list of var=values that are set in environment variables before the SMBO method is executed - these are set by the ExperimentConstructor


tunerTimeout

public float tunerTimeout
How many seconds should be spent for this experiment overall? (IE the SMBO method's budget)


trainTimeout

public float trainTimeout
How many seconds should be spent training a set of hyperparameters on a specific partition of the training and test data?. Note that this is more of a guideline, Auto-WEKA has quite a bit of slack built into to accomodate classifiers that can return partial results, or ones that decide to offload training onto their evaluation phase.


memory

public java.lang.String memory
The string passed to the Xmx argument of a sub process limiting the RAM that WEKA will have


extraPropsString

public java.lang.String extraPropsString
Any extra properties that are associated with the experiment, these are generally for the constructor or your wrappers


allowedClassifiers

public java.util.List<java.lang.String> allowedClassifiers
Forces Auto-WEKA to only use the list of classifiers here - if it is empty, then Auto-WEKA will try to use everything that it can


trajectoryPointExtras

public java.util.List<Experiment.TrajectoryPointExtra> trajectoryPointExtras
For analysis experiements, these extra classifier evaluations will be done for every point along the trajectory. For example, if we want to look at the Testing performance all the way along, we would add a new TrajectoryPointExtra that contains an instance string of 'default', since that causes Auto-WEKA to preserve the provided training/test split

Constructor Detail

Experiment

public Experiment()
Method Detail

validate

public void validate()
Throws a runtime exception if the experiment contains some sort of crazy values.


fromXML

public static Experiment fromXML(java.lang.String filename)
Static method to load up an experiment from XML.


fromXML

public static Experiment fromXML(java.io.InputStream xml)
Static method to load up an experiment from XML.


main

public static void main(java.lang.String[] args)
Main method to actually run an experiment with a given seed. This requires two command line arguments, the path to the experiment folder (but not the actuall .experiment itself) and the seed on which you want to run this experiment. The seed is passed to the SMBO method to allow for easy paralellization of each SMBO method. Takes an optional argument '-silent' that doesn't print everything out to stdout


createFromFolder

public static Experiment createFromFolder(java.io.File folder)