Package org.terrier.evaluation

Class AdhocEvaluation

  • All Implemented Interfaces:
    Evaluation
    Direct Known Subclasses:
    AdhocFullQueryEvaluation
    public class AdhocEvaluation
extends TerrierEvaluation
    Performs the evaluation for TREC's tasks, except the named page task. The evaluation measures include the mean average precision and other measures such as precision at 10, precision at 10%, and so on....
    Author:
    Gianni Amati, Ben He

    • Field Detail

      • logger

        protected static final org.slf4j.Logger logger

      • PRECISION_RANKS

        protected static final int[] PRECISION_RANKS

      • PRECISION_PERCENTAGES

        protected static final int[] PRECISION_PERCENTAGES

      • maxNumberRetrieved

        protected int maxNumberRetrieved
        The maximum number of documents retrieved for a query.

      • numberOfEffQuery

        protected int numberOfEffQuery
        The number of effective queries. An effective query is a query that has corresponding relevant documents in the qrels file.

      • totalNumberOfRetrieved

        protected int totalNumberOfRetrieved
        The total number of documents retrieved in the task.

      • totalNumberOfRelevant

        protected int totalNumberOfRelevant
        The total number of relevant documents in the qrels file for the queries processed in the task.

      • totalNumberOfRelevantRetrieved

        protected int totalNumberOfRelevantRetrieved
        The total number of relevant documents retrieved in the task.

      • precisionAtRank

        protected gnu.trove.TIntDoubleHashMap precisionAtRank
        Precision at rank number of documents

      • precisionAtRecall

        protected gnu.trove.TIntDoubleHashMap precisionAtRecall

      • meanAveragePrecision

        protected double meanAveragePrecision
        Average Precision.

      • meanRelevantPrecision

        protected double meanRelevantPrecision
        Relevant Precision.

      • averagePrecisionOfEachQuery

        protected double[] averagePrecisionOfEachQuery
        The average precision of each query.

      • queryNo

        protected java.lang.String[] queryNo
        The query number of each query.

    • Constructor Detail

      • AdhocEvaluation

        public AdhocEvaluation()
        Create adhoc evaluation

      • AdhocEvaluation

        public AdhocEvaluation​(java.lang.String qrelsFile)
        Create adhoc evaluation
        Parameters:
        qrelsFile -

      • AdhocEvaluation

        public AdhocEvaluation​(java.lang.String[] qrelsFiles)
        Create adhoc evaluation
        Parameters:
        qrelsFiles -

    • Method Detail

      • initialise

        public void initialise()
        Initialise variables.

      • evaluate

        public void evaluate​(java.lang.String resultFilename)
        Evaluates the given result file.
        Specified by:
        evaluate in interface Evaluation
        Specified by:
        evaluate in class TerrierEvaluation
        Parameters:
        resultFilename - String the filename of the result file to evaluate.

      • writeEvaluationResult

        public void writeEvaluationResult​(java.io.PrintWriter out)
        Output the evaluation result to the specific file.
        Specified by:
        writeEvaluationResult in interface Evaluation
        Specified by:
        writeEvaluationResult in class TerrierEvaluation
        Parameters:
        out - java.io.PrintWriter the stream to which the results are printed.