public class Evaluation
extends java.lang.Object
implements weka.core.Summarizable, weka.core.RevisionHandler, java.io.Serializable
public static void main(String [] args) {
runClassifier(new FunkyClassifier(), args);
}
------------------------------------------------------------------
Example usage from within an application:
Instances trainInstances = ... instances got from somewhere
Instances testInstances = ... instances got from somewhere
Classifier scheme = ... scheme got from somewhere
Evaluation evaluation = new Evaluation(trainInstances);
evaluation.evaluateModel(scheme, testInstances);
System.out.println(evaluation.toSummaryString());
Modifier and Type | Field and Description |
---|---|
static java.lang.String[] |
BUILT_IN_EVAL_METRICS |
protected static int |
k_MarginResolution
Resolution of the margin histogram.
|
protected boolean |
m_ClassIsNominal
Is the class nominal or numeric?
|
protected java.lang.String[] |
m_ClassNames
The names of the classes.
|
protected double[] |
m_ClassPriors
The prior probabilities of the classes.
|
protected double |
m_ClassPriorsSum
The sum of counts for priors.
|
protected boolean |
m_ComplexityStatisticsAvailable
Whether complexity statistics are available.
|
protected double |
m_ConfLevel
The confidence level used for coverage statistics.
|
protected double[][] |
m_ConfusionMatrix
Array for storing the confusion matrix.
|
protected double |
m_Correct
The weight of all correctly classified instances.
|
protected weka.classifiers.CostMatrix |
m_CostMatrix
The cost matrix (if given).
|
protected boolean |
m_CoverageStatisticsAvailable
Whether coverage statistics are available.
|
protected boolean |
m_DiscardPredictions
whether to discard predictions (and save memory).
|
protected Instances |
m_Header
The header of the training set.
|
protected double |
m_Incorrect
The weight of all incorrectly classified instances.
|
protected double[] |
m_MarginCounts
Cumulative margin distribution.
|
protected double |
m_MaxTarget
Maximum target value.
|
protected java.util.List<java.lang.String> |
m_metricsToDisplay
The list of metrics to display in the output
|
protected double |
m_MinTarget
Minimum target value.
|
protected double |
m_MissingClass
The weight of all instances that had no class assigned to them.
|
protected boolean |
m_NoPriors
enables/disables the use of priors, e.g., if no training set is present in
case of de-serialized schemes.
|
protected int |
m_NumClasses
The number of classes.
|
protected int |
m_NumFolds
The number of folds for a cross-validation.
|
protected int |
m_NumTrainClassVals
Number of non-missing class training instances seen.
|
protected java.util.List<weka.classifiers.evaluation.AbstractEvaluationMetric> |
m_pluginMetrics
Holds plugin evaluation metrics
|
protected java.util.ArrayList<weka.classifiers.evaluation.Prediction> |
m_Predictions
The list of predictions that have been generated (for computing AUC).
|
protected weka.estimators.UnivariateKernelEstimator |
m_PriorEstimator
Numeric class estimator for prior.
|
protected double |
m_SumAbsErr
Sum of absolute errors.
|
protected double |
m_SumClass
Sum of class values.
|
protected double |
m_SumClassPredicted
Sum of predicted * class values.
|
protected double |
m_SumErr
Sum of errors.
|
protected double |
m_SumKBInfo
Total Kononenko & Bratko Information.
|
protected double |
m_SumPredicted
Sum of predicted values.
|
protected double |
m_SumPriorAbsErr
Sum of absolute errors of the prior.
|
protected double |
m_SumPriorEntropy
Total entropy of prior predictions.
|
protected double |
m_SumPriorSqrErr
Sum of absolute errors of the prior.
|
protected double |
m_SumSchemeEntropy
Total entropy of scheme predictions.
|
protected double |
m_SumSqrClass
Sum of squared class values.
|
protected double |
m_SumSqrErr
Sum of squared errors.
|
protected double |
m_SumSqrPredicted
Sum of squared predicted values.
|
protected double |
m_TotalCost
The total cost of predictions (includes instance weights).
|
protected double |
m_TotalCoverage
Total coverage of test cases at the given confidence level.
|
protected double |
m_TotalSizeOfRegions
Total size of predicted regions at the given confidence level.
|
protected double[] |
m_TrainClassVals
Array containing all numeric training class values seen.
|
protected double[] |
m_TrainClassWeights
Array containing all numeric training class weights.
|
protected double |
m_Unclassified
The weight of all unclassified instances.
|
protected double |
m_WithClass
The weight of all instances that had a class assigned to them.
|
protected static double |
MIN_SF_PROB
The minimum probablility accepted from an estimator to avoid taking log(0)
in Sf calculations.
|
Constructor and Description |
---|
Evaluation(Instances data)
Initializes all the counters for the evaluation.
|
Evaluation(Instances data,
weka.classifiers.CostMatrix costMatrix)
Initializes all the counters for the evaluation and also takes a cost
matrix as parameter.
|
Modifier and Type | Method and Description |
---|---|
protected void |
addNumericTrainClass(double classValue,
double weight)
Adds a numeric (non-missing) training class value and weight to the buffer
of stored values.
|
double |
areaUnderPRC(int classIndex)
Returns the area under precision-recall curve (AUPRC) for those predictions
that have been collected in the evaluateClassifier(Classifier, Instances)
method.
|
double |
areaUnderROC(int classIndex)
Returns the area under ROC for those predictions that have been collected
in the evaluateClassifier(Classifier, Instances) method.
|
double |
avgCost()
Gets the average cost, that is, total cost of misclassifications (incorrect
plus unclassified) over the total number of instances.
|
double[][] |
confusionMatrix()
Returns a copy of the confusion matrix.
|
double |
correct()
Gets the number of instances correctly classified (that is, for which a
correct prediction was made).
|
double |
correlationCoefficient()
Returns the correlation coefficient if the class is numeric.
|
double |
coverageOfTestCasesByPredictedRegions()
Gets the coverage of the test cases by the predicted regions at the
confidence level specified when evaluation was performed.
|
void |
crossValidateModel(weka.classifiers.Classifier classifier,
Instances data,
int numFolds,
java.util.Random random,
java.lang.Object... forPredictionsPrinting)
Performs a (stratified if class is nominal) cross-validation for a
classifier on a set of instances.
|
void |
crossValidateModel(java.lang.String classifierString,
Instances data,
int numFolds,
java.lang.String[] options,
java.util.Random random)
Performs a (stratified if class is nominal) cross-validation for a
classifier on a set of instances.
|
boolean |
equals(java.lang.Object obj)
Tests whether the current evaluation object is equal to another evaluation
object.
|
double |
errorRate()
Returns the estimated error rate or the root mean squared error (if the
class is numeric).
|
double[] |
evaluateModel(weka.classifiers.Classifier classifier,
Instances data,
java.lang.Object... forPredictionsPrinting)
Evaluates the classifier on a given set of instances.
|
static java.lang.String |
evaluateModel(weka.classifiers.Classifier classifier,
java.lang.String[] options)
Evaluates a classifier with the options given in an array of strings.
|
static java.lang.String |
evaluateModel(java.lang.String classifierString,
java.lang.String[] options)
Evaluates a classifier with the options given in an array of strings.
|
double |
evaluateModelOnce(weka.classifiers.Classifier classifier,
weka.core.Instance instance)
Evaluates the classifier on a single instance.
|
double |
evaluateModelOnce(double[] dist,
weka.core.Instance instance)
Evaluates the supplied distribution on a single instance.
|
void |
evaluateModelOnce(double prediction,
weka.core.Instance instance)
Evaluates the supplied prediction on a single instance.
|
double |
evaluateModelOnceAndRecordPrediction(weka.classifiers.Classifier classifier,
weka.core.Instance instance)
Evaluates the classifier on a single instance and records the prediction.
|
double |
evaluateModelOnceAndRecordPrediction(double[] dist,
weka.core.Instance instance)
Evaluates the supplied distribution on a single instance.
|
protected double |
evaluationForSingleInstance(weka.classifiers.Classifier classifier,
weka.core.Instance instance,
boolean storePredictions)
Evaluates the classifier on a single instance and records the prediction.
|
double |
evaluationForSingleInstance(double[] dist,
weka.core.Instance instance,
boolean storePredictions)
Evaluates the supplied distribution on a single instance.
|
double |
falseNegativeRate(int classIndex)
Calculate the false negative rate with respect to a particular class.
|
double |
falsePositiveRate(int classIndex)
Calculate the false positive rate with respect to a particular class.
|
double |
fMeasure(int classIndex)
Calculate the F-Measure with respect to a particular class.
|
static java.util.List<java.lang.String> |
getAllEvaluationMetricNames()
Utility method to get a list of the names of all built-in and plugin
evaluation metrics
|
double[] |
getClassPriors()
Get the current weighted class counts.
|
boolean |
getDiscardPredictions()
Returns whether predictions are not recorded at all, in order to conserve
memory.
|
protected static java.lang.String |
getGlobalInfo(weka.classifiers.Classifier classifier)
Return the global info (if it exists) for the supplied classifier.
|
Instances |
getHeader()
Returns the header of the underlying dataset.
|
java.util.List<java.lang.String> |
getMetricsToDisplay()
Get a list of the names of metrics to have appear in the output The default
is to display all built in metrics and plugin metrics that haven't been
globally disabled.
|
weka.classifiers.evaluation.AbstractEvaluationMetric |
getPluginMetric(java.lang.String name)
Get the named plugin evaluation metric
|
java.util.List<weka.classifiers.evaluation.AbstractEvaluationMetric> |
getPluginMetrics()
Returns the list of plugin metrics in use (or null if there are none)
|
java.lang.String |
getRevision()
Returns the revision string.
|
protected static weka.classifiers.CostMatrix |
handleCostOption(java.lang.String costFileName,
int numClasses)
Attempts to load a cost matrix.
|
double |
incorrect()
Gets the number of instances incorrectly classified (that is, for which an
incorrect prediction was made).
|
double |
kappa()
Returns value of kappa statistic if class is nominal.
|
double |
KBInformation()
Return the total Kononenko & Bratko Information score in bits.
|
double |
KBMeanInformation()
Return the Kononenko & Bratko Information score in bits per instance.
|
double |
KBRelativeInformation()
Return the Kononenko & Bratko Relative Information score.
|
static void |
main(java.lang.String[] args)
A test method for this class.
|
protected double[] |
makeDistribution(double predictedClass)
Convert a single prediction into a probability distribution with all zero
probabilities except the predicted value which has probability 1.0.
|
protected static java.lang.String |
makeOptionString(weka.classifiers.Classifier classifier,
boolean globalInfo)
Make up the help string giving all the command line options.
|
double |
matthewsCorrelationCoefficient(int classIndex)
Calculates the matthews correlation coefficient (sometimes called phi
coefficient) for the supplied class
|
double |
meanAbsoluteError()
Returns the mean absolute error.
|
double |
meanPriorAbsoluteError()
Returns the mean absolute error of the prior.
|
protected java.lang.String |
num2ShortID(int num,
char[] IDChars,
int IDWidth)
Method for generating indices for the confusion matrix.
|
double |
numFalseNegatives(int classIndex)
Calculate number of false negatives with respect to a particular class.
|
double |
numFalsePositives(int classIndex)
Calculate number of false positives with respect to a particular class.
|
double |
numInstances()
Gets the number of test instances that had a known class value (actually
the sum of the weights of test instances with known class value).
|
double |
numTrueNegatives(int classIndex)
Calculate the number of true negatives with respect to a particular class.
|
double |
numTruePositives(int classIndex)
Calculate the number of true positives with respect to a particular class.
|
double |
pctCorrect()
Gets the percentage of instances correctly classified (that is, for which a
correct prediction was made).
|
double |
pctIncorrect()
Gets the percentage of instances incorrectly classified (that is, for which
an incorrect prediction was made).
|
double |
pctUnclassified()
Gets the percentage of instances not classified (that is, for which no
prediction was made by the classifier).
|
double |
precision(int classIndex)
Calculate the precision with respect to a particular class.
|
java.util.ArrayList<weka.classifiers.evaluation.Prediction> |
predictions()
Returns the predictions that have been collected.
|
double |
priorEntropy()
Calculate the entropy of the prior distribution.
|
double |
recall(int classIndex)
Calculate the recall with respect to a particular class.
|
double |
relativeAbsoluteError()
Returns the relative absolute error.
|
double |
rootMeanPriorSquaredError()
Returns the root mean prior squared error.
|
double |
rootMeanSquaredError()
Returns the root mean squared error.
|
double |
rootRelativeSquaredError()
Returns the root relative squared error if the class is numeric.
|
void |
setDiscardPredictions(boolean value)
Sets whether to discard predictions, ie, not storing them for future
reference via predictions() method in order to conserve memory.
|
void |
setMetricsToDisplay(java.util.List<java.lang.String> display)
Set a list of the names of metrics to have appear in the output.
|
protected void |
setNumericPriorsFromBuffer()
Sets up the priors for numeric class attributes from the training class
values that have been seen so far.
|
void |
setPriors(Instances train)
Sets the class prior probabilities.
|
double |
SFEntropyGain()
Returns the total SF, which is the null model entropy minus the scheme
entropy.
|
double |
SFMeanEntropyGain()
Returns the SF per instance, which is the null model entropy minus the
scheme entropy, per instance.
|
double |
SFMeanPriorEntropy()
Returns the entropy per instance for the null model.
|
double |
SFMeanSchemeEntropy()
Returns the entropy per instance for the scheme.
|
double |
SFPriorEntropy()
Returns the total entropy for the null model.
|
double |
SFSchemeEntropy()
Returns the total entropy for the scheme.
|
double |
sizeOfPredictedRegions()
Gets the average size of the predicted regions, relative to the range of
the target in the training data, at the confidence level specified when
evaluation was performed.
|
java.lang.String |
toClassDetailsString()
Generates a breakdown of the accuracy for each class (with default title),
incorporating various information-retrieval statistics, such as true/false
positive rate, precision/recall/F-Measure.
|
java.lang.String |
toClassDetailsString(java.lang.String title)
Generates a breakdown of the accuracy for each class, incorporating various
information-retrieval statistics, such as true/false positive rate,
precision/recall/F-Measure.
|
java.lang.String |
toCumulativeMarginDistributionString()
Output the cumulative margin distribution as a string suitable for input
for gnuplot or similar package.
|
void |
toggleEvalMetrics(java.util.List<java.lang.String> metricsToToggle)
Toggle the output of the metrics specified in the supplied list.
|
java.lang.String |
toMatrixString()
Calls toMatrixString() with a default title.
|
java.lang.String |
toMatrixString(java.lang.String title)
Outputs the performance statistics as a classification confusion matrix.
|
java.lang.String |
toSummaryString()
Calls toSummaryString() with no title and no complexity stats.
|
java.lang.String |
toSummaryString(boolean printComplexityStatistics)
Calls toSummaryString() with a default title.
|
java.lang.String |
toSummaryString(java.lang.String title,
boolean printComplexityStatistics)
Outputs the performance statistics in summary form.
|
double |
totalCost()
Gets the total cost, that is, the cost of each prediction times the weight
of the instance, summed over all instances.
|
double |
trueNegativeRate(int classIndex)
Calculate the true negative rate with respect to a particular class.
|
double |
truePositiveRate(int classIndex)
Calculate the true positive rate with respect to a particular class.
|
double |
unclassified()
Gets the number of instances not classified (that is, for which no
prediction was made by the classifier).
|
double |
unweightedMacroFmeasure()
Unweighted macro-averaged F-measure.
|
double |
unweightedMicroFmeasure()
Unweighted micro-averaged F-measure.
|
protected void |
updateMargins(double[] predictedDistribution,
int actualClass,
double weight)
Update the cumulative record of classification margins.
|
protected void |
updateNumericScores(double[] predicted,
double[] actual,
double weight)
Update the numeric accuracy measures.
|
void |
updatePriors(weka.core.Instance instance)
Updates the class prior probabilities or the mean respectively (when
incrementally training).
|
protected void |
updateStatsForClassifier(double[] predictedDistribution,
weka.core.Instance instance)
Updates all the statistics about a classifiers performance for the current
test instance.
|
protected void |
updateStatsForConditionalDensityEstimator(weka.classifiers.ConditionalDensityEstimator classifier,
weka.core.Instance classMissing,
double classValue)
Updates stats for conditional density estimator based on current test
instance.
|
protected void |
updateStatsForIntervalEstimator(weka.classifiers.IntervalEstimator classifier,
weka.core.Instance classMissing,
double classValue)
Updates stats for interval estimator based on current test instance.
|
protected void |
updateStatsForPredictor(double predictedValue,
weka.core.Instance instance)
Updates all the statistics about a predictors performance for the current
test instance.
|
void |
useNoPriors()
disables the use of priors, e.g., in case of de-serialized schemes that
have no access to the original training set, but are evaluated on a set
set.
|
double |
weightedAreaUnderPRC()
Calculates the weighted (by class size) AUPRC.
|
double |
weightedAreaUnderROC()
Calculates the weighted (by class size) AUC.
|
double |
weightedFalseNegativeRate()
Calculates the weighted (by class size) false negative rate.
|
double |
weightedFalsePositiveRate()
Calculates the weighted (by class size) false positive rate.
|
double |
weightedFMeasure()
Calculates the macro weighted (by class size) average F-Measure.
|
double |
weightedMatthewsCorrelation()
Calculates the weighted (by class size) matthews correlation coefficient.
|
double |
weightedPrecision()
Calculates the weighted (by class size) precision.
|
double |
weightedRecall()
Calculates the weighted (by class size) recall.
|
double |
weightedTrueNegativeRate()
Calculates the weighted (by class size) true negative rate.
|
double |
weightedTruePositiveRate()
Calculates the weighted (by class size) true positive rate.
|
static java.lang.String |
wekaStaticWrapper(weka.classifiers.Sourcable classifier,
java.lang.String className)
Wraps a static classifier in enough source to test using the weka class
libraries.
|
protected int m_NumClasses
protected int m_NumFolds
protected double m_Incorrect
protected double m_Correct
protected double m_Unclassified
protected double m_MissingClass
protected double m_WithClass
protected double[][] m_ConfusionMatrix
protected java.lang.String[] m_ClassNames
protected boolean m_ClassIsNominal
protected double[] m_ClassPriors
protected double m_ClassPriorsSum
protected weka.classifiers.CostMatrix m_CostMatrix
protected double m_TotalCost
protected double m_SumErr
protected double m_SumAbsErr
protected double m_SumSqrErr
protected double m_SumClass
protected double m_SumSqrClass
protected double m_SumPredicted
protected double m_SumSqrPredicted
protected double m_SumClassPredicted
protected double m_SumPriorAbsErr
protected double m_SumPriorSqrErr
protected double m_SumKBInfo
protected static int k_MarginResolution
protected double[] m_MarginCounts
protected int m_NumTrainClassVals
protected double[] m_TrainClassVals
protected double[] m_TrainClassWeights
protected weka.estimators.UnivariateKernelEstimator m_PriorEstimator
protected boolean m_ComplexityStatisticsAvailable
protected static final double MIN_SF_PROB
protected double m_SumPriorEntropy
protected double m_SumSchemeEntropy
protected boolean m_CoverageStatisticsAvailable
protected double m_ConfLevel
protected double m_TotalSizeOfRegions
protected double m_TotalCoverage
protected double m_MinTarget
protected double m_MaxTarget
protected java.util.ArrayList<weka.classifiers.evaluation.Prediction> m_Predictions
protected boolean m_NoPriors
protected Instances m_Header
protected boolean m_DiscardPredictions
protected java.util.List<weka.classifiers.evaluation.AbstractEvaluationMetric> m_pluginMetrics
protected java.util.List<java.lang.String> m_metricsToDisplay
public static final java.lang.String[] BUILT_IN_EVAL_METRICS
public Evaluation(Instances data) throws java.lang.Exception
useNoPriors()
if the dataset is the test set and you can't
initialize with the priors from the training set via
setPriors(Instances)
.data
- set of training instances, to get some header information and
prior class distribution informationjava.lang.Exception
- if the class is not defineduseNoPriors()
,
setPriors(Instances)
public Evaluation(Instances data, weka.classifiers.CostMatrix costMatrix) throws java.lang.Exception
useNoPriors()
if the dataset is the
test set and you can't initialize with the priors from the training set via
setPriors(Instances)
.data
- set of training instances, to get some header information and
prior class distribution informationcostMatrix
- the cost matrix---if null, default costs will be usedjava.lang.Exception
- if cost matrix is not compatible with data, the class is
not defined or the class is numericuseNoPriors()
,
setPriors(Instances)
public static java.util.List<java.lang.String> getAllEvaluationMetricNames()
public Instances getHeader()
public void setDiscardPredictions(boolean value)
value
- true if to discard the predictionspredictions()
public boolean getDiscardPredictions()
predictions()
public java.util.List<weka.classifiers.evaluation.AbstractEvaluationMetric> getPluginMetrics()
public void setMetricsToDisplay(java.util.List<java.lang.String> display)
display
- a list of metric names to have appear in the outputpublic java.util.List<java.lang.String> getMetricsToDisplay()
public void toggleEvalMetrics(java.util.List<java.lang.String> metricsToToggle)
metricsToToggle
- a list of metrics to togglepublic weka.classifiers.evaluation.AbstractEvaluationMetric getPluginMetric(java.lang.String name)
name
- the name of the metric (as returned by
AbstractEvaluationMetric.getName()) or the fully qualified class
name of the metric to findpublic double areaUnderROC(int classIndex)
classIndex
- the index of the class to consider as "positive"public double weightedAreaUnderROC()
public double areaUnderPRC(int classIndex)
classIndex
- the index of the class to consider as "positive"public double weightedAreaUnderPRC()
public double[][] confusionMatrix()
public void crossValidateModel(weka.classifiers.Classifier classifier, Instances data, int numFolds, java.util.Random random, java.lang.Object... forPredictionsPrinting) throws java.lang.Exception
classifier
- the classifier with any options set.data
- the data on which the cross-validation is to be performednumFolds
- the number of folds for the cross-validationrandom
- random number generator for randomizationforPredictionsPrinting
- varargs parameter that, if supplied, is
expected to hold a
weka.classifiers.evaluation.output.prediction.AbstractOutput
objectjava.lang.Exception
- if a classifier could not be generated successfully or
the class is not definedpublic void crossValidateModel(java.lang.String classifierString, Instances data, int numFolds, java.lang.String[] options, java.util.Random random) throws java.lang.Exception
classifierString
- a string naming the class of the classifierdata
- the data on which the cross-validation is to be performednumFolds
- the number of folds for the cross-validationoptions
- the options to the classifier. Any optionsrandom
- the random number generator for randomizing the data accepted
by the classifier will be removed from this array.java.lang.Exception
- if a classifier could not be generated successfully or
the class is not definedpublic static java.lang.String evaluateModel(java.lang.String classifierString, java.lang.String[] options) throws java.lang.Exception
classifierString
- class of machine learning classifier as a stringoptions
- the array of string containing the optionsjava.lang.Exception
- if model could not be evaluated successfullypublic static void main(java.lang.String[] args)
args
- an array of command line arguments, the first of which must be
the class name of a classifier.public static java.lang.String evaluateModel(weka.classifiers.Classifier classifier, java.lang.String[] options) throws java.lang.Exception
classifier
- machine learning classifieroptions
- the array of string containing the optionsjava.lang.Exception
- if model could not be evaluated successfullyprotected static weka.classifiers.CostMatrix handleCostOption(java.lang.String costFileName, int numClasses) throws java.lang.Exception
costFileName
- the filename of the cost matrixnumClasses
- the number of classes that should be in the cost matrix
(only used if the cost file is in old format).CostMatrix
value, or null if costFileName is emptyjava.lang.Exception
- if an error occurs.public double[] evaluateModel(weka.classifiers.Classifier classifier, Instances data, java.lang.Object... forPredictionsPrinting) throws java.lang.Exception
classifier
- machine learning classifierdata
- set of test instances for evaluationforPredictionsPrinting
- varargs parameter that, if supplied, is
expected to hold a
weka.classifiers.evaluation.output.prediction.AbstractOutput
objectjava.lang.Exception
- if model could not be evaluated successfullypublic double evaluationForSingleInstance(double[] dist, weka.core.Instance instance, boolean storePredictions) throws java.lang.Exception
dist
- the supplied distributioninstance
- the test instance to be classifiedstorePredictions
- whether to store predictions for nominal classifierjava.lang.Exception
- if model could not be evaluated successfullyprotected double evaluationForSingleInstance(weka.classifiers.Classifier classifier, weka.core.Instance instance, boolean storePredictions) throws java.lang.Exception
classifier
- machine learning classifierinstance
- the test instance to be classifiedstorePredictions
- whether to store predictions for nominal classifierjava.lang.Exception
- if model could not be evaluated successfully or the data
contains string attributespublic double evaluateModelOnceAndRecordPrediction(weka.classifiers.Classifier classifier, weka.core.Instance instance) throws java.lang.Exception
classifier
- machine learning classifierinstance
- the test instance to be classifiedjava.lang.Exception
- if model could not be evaluated successfully or the data
contains string attributespublic double evaluateModelOnce(weka.classifiers.Classifier classifier, weka.core.Instance instance) throws java.lang.Exception
classifier
- machine learning classifierinstance
- the test instance to be classifiedjava.lang.Exception
- if model could not be evaluated successfully or the data
contains string attributespublic double evaluateModelOnce(double[] dist, weka.core.Instance instance) throws java.lang.Exception
dist
- the supplied distributioninstance
- the test instance to be classifiedjava.lang.Exception
- if model could not be evaluated successfullypublic double evaluateModelOnceAndRecordPrediction(double[] dist, weka.core.Instance instance) throws java.lang.Exception
dist
- the supplied distributioninstance
- the test instance to be classifiedjava.lang.Exception
- if model could not be evaluated successfullypublic void evaluateModelOnce(double prediction, weka.core.Instance instance) throws java.lang.Exception
prediction
- the supplied predictioninstance
- the test instance to be classifiedjava.lang.Exception
- if model could not be evaluated successfullypublic java.util.ArrayList<weka.classifiers.evaluation.Prediction> predictions()
public static java.lang.String wekaStaticWrapper(weka.classifiers.Sourcable classifier, java.lang.String className) throws java.lang.Exception
classifier
- a Sourcable ClassifierclassName
- the name to give to the source code classjava.lang.Exception
- if code-generation failspublic final double numInstances()
public final double coverageOfTestCasesByPredictedRegions()
public final double sizeOfPredictedRegions()
public final double incorrect()
public final double pctIncorrect()
public final double totalCost()
public final double avgCost()
public final double correct()
public final double pctCorrect()
public final double unclassified()
public final double pctUnclassified()
public final double errorRate()
public final double kappa()
public final double correlationCoefficient() throws java.lang.Exception
java.lang.Exception
- if class is not numericpublic final double meanAbsoluteError()
public final double meanPriorAbsoluteError()
public final double relativeAbsoluteError() throws java.lang.Exception
java.lang.Exception
- if it can't be computedpublic final double rootMeanSquaredError()
public final double rootMeanPriorSquaredError()
public final double rootRelativeSquaredError()
public final double priorEntropy() throws java.lang.Exception
java.lang.Exception
- if the class is not nominalpublic final double KBInformation() throws java.lang.Exception
java.lang.Exception
- if the class is not nominalpublic final double KBMeanInformation() throws java.lang.Exception
java.lang.Exception
- if the class is not nominalpublic final double KBRelativeInformation() throws java.lang.Exception
java.lang.Exception
- if the class is not nominalpublic final double SFPriorEntropy()
public final double SFMeanPriorEntropy()
public final double SFSchemeEntropy()
public final double SFMeanSchemeEntropy()
public final double SFEntropyGain()
public final double SFMeanEntropyGain()
public java.lang.String toCumulativeMarginDistributionString() throws java.lang.Exception
java.lang.Exception
- if the class attribute is nominalpublic java.lang.String toSummaryString()
toSummaryString
in interface weka.core.Summarizable
public java.lang.String toSummaryString(boolean printComplexityStatistics)
printComplexityStatistics
- if true, complexity statistics are
returned as wellpublic java.lang.String toSummaryString(java.lang.String title, boolean printComplexityStatistics)
title
- the title for the statisticsprintComplexityStatistics
- if true, complexity statistics are
returned as wellpublic java.lang.String toMatrixString() throws java.lang.Exception
java.lang.Exception
- if the class is numericpublic java.lang.String toMatrixString(java.lang.String title) throws java.lang.Exception
title
- the title for the confusion matrixjava.lang.Exception
- if the class is numericpublic java.lang.String toClassDetailsString() throws java.lang.Exception
java.lang.Exception
- if class is not nominalpublic java.lang.String toClassDetailsString(java.lang.String title) throws java.lang.Exception
title
- the title to prepend the stats string withjava.lang.Exception
- if class is not nominalpublic double numTruePositives(int classIndex)
correctly classified positives
classIndex
- the index of the class to consider as "positive"public double truePositiveRate(int classIndex)
correctly classified positives ------------------------------ total positives
classIndex
- the index of the class to consider as "positive"public double weightedTruePositiveRate()
public double numTrueNegatives(int classIndex)
correctly classified negatives
classIndex
- the index of the class to consider as "positive"public double trueNegativeRate(int classIndex)
correctly classified negatives ------------------------------ total negatives
classIndex
- the index of the class to consider as "positive"public double weightedTrueNegativeRate()
public double numFalsePositives(int classIndex)
incorrectly classified negatives
classIndex
- the index of the class to consider as "positive"public double falsePositiveRate(int classIndex)
incorrectly classified negatives -------------------------------- total negatives
classIndex
- the index of the class to consider as "positive"public double weightedFalsePositiveRate()
public double numFalseNegatives(int classIndex)
incorrectly classified positives
classIndex
- the index of the class to consider as "positive"public double falseNegativeRate(int classIndex)
incorrectly classified positives -------------------------------- total positives
classIndex
- the index of the class to consider as "positive"public double weightedFalseNegativeRate()
public double matthewsCorrelationCoefficient(int classIndex)
classIndex
- the index of the class to compute the matthews
correlation coefficient forpublic double weightedMatthewsCorrelation()
public double recall(int classIndex)
correctly classified positives ------------------------------ total positives(Which is also the same as the truePositiveRate.)
classIndex
- the index of the class to consider as "positive"public double weightedRecall()
public double precision(int classIndex)
correctly classified positives ------------------------------ total predicted as positive
classIndex
- the index of the class to consider as "positive"public double weightedPrecision()
public double fMeasure(int classIndex)
2 * recall * precision ---------------------- recall + precision
classIndex
- the index of the class to consider as "positive"public double weightedFMeasure()
public double unweightedMacroFmeasure()
public double unweightedMicroFmeasure()
public void setPriors(Instances train) throws java.lang.Exception
train
- the training instances used to determine the prior
probabilitiesjava.lang.Exception
- if the class attribute of the instances is not setpublic double[] getClassPriors()
public void updatePriors(weka.core.Instance instance) throws java.lang.Exception
instance
- the new training instance seenjava.lang.Exception
- if the class of the instance is not setpublic void useNoPriors()
public boolean equals(java.lang.Object obj)
equals
in class java.lang.Object
obj
- the object to compare againstprotected static java.lang.String makeOptionString(weka.classifiers.Classifier classifier, boolean globalInfo)
classifier
- the classifier to include options forglobalInfo
- include the global information string for the classifier
(if available).protected static java.lang.String getGlobalInfo(weka.classifiers.Classifier classifier) throws java.lang.Exception
classifier
- the classifier to get the global info forjava.lang.Exception
- if there is a problem reflecting on the classifierprotected java.lang.String num2ShortID(int num, char[] IDChars, int IDWidth)
num
- integer to formatIDChars
- the characters to useIDWidth
- the width of the entryprotected double[] makeDistribution(double predictedClass)
predictedClass
- the index of the predicted classprotected void updateStatsForClassifier(double[] predictedDistribution, weka.core.Instance instance) throws java.lang.Exception
predictedDistribution
- the probabilities assigned to each classinstance
- the instance to be classifiedjava.lang.Exception
- if the class of the instance is not setprotected void updateStatsForIntervalEstimator(weka.classifiers.IntervalEstimator classifier, weka.core.Instance classMissing, double classValue) throws java.lang.Exception
classifier
- the interval estimatorclassMissing
- the instance for which the intervals are computed,
without a class valueclassValue
- the class value of this instancejava.lang.Exception
- if intervals could not be computed successfullyprotected void updateStatsForConditionalDensityEstimator(weka.classifiers.ConditionalDensityEstimator classifier, weka.core.Instance classMissing, double classValue) throws java.lang.Exception
classifier
- the conditional density estimatorclassMissing
- the instance for which density is to be computed,
without a class valueclassValue
- the class value of this instancejava.lang.Exception
- if density could not be computed successfullyprotected void updateStatsForPredictor(double predictedValue, weka.core.Instance instance) throws java.lang.Exception
predictedValue
- the numeric value the classifier predictsinstance
- the instance to be classifiedjava.lang.Exception
- if the class of the instance is not setprotected void updateMargins(double[] predictedDistribution, int actualClass, double weight)
predictedDistribution
- the probability distribution predicted for the
current instanceactualClass
- the index of the actual instance classweight
- the weight assigned to the instanceprotected void updateNumericScores(double[] predicted, double[] actual, double weight)
predicted
- the predicted valuesactual
- the actual valueweight
- the weight associated with this predictionprotected void addNumericTrainClass(double classValue, double weight)
classValue
- the class valueweight
- the instance weightprotected void setNumericPriorsFromBuffer()
public java.lang.String getRevision()
getRevision
in interface weka.core.RevisionHandler