Terrier Core

Full support for direct file generation in Hadoop mode indexing

Details

  • Type: Improvement Improvement
  • Status: Resolved Resolved
  • Priority: Major Major
  • Resolution: Fixed
  • Affects Version/s: 3.0
  • Fix Version/s: 3.0
  • Component/s: .structures
  1. InvertedIndexInputFormat.java
    (4 kB)
    Richard McCreadie
    23/Mar/09 5:23 PM
  2. InvertedIndexRecordReader.java
    (2 kB)
    Richard McCreadie
    23/Mar/09 5:23 PM
  3. InvertedIndexSplit.java
    (4 kB)
    Richard McCreadie
    23/Mar/09 5:23 PM

Activity

Hide
Richard McCreadie added a comment - 23/Mar/09 5:21 PM

MapReduce Index reading of Terrier Inverted Indicies is now ready for testing. Output Takes the form of 'n' lists of Term-PostingList pairs. Where 'n' is set by the user.

Classes:
InvertedIndexSplit : Defines an ordered subset of documents from an Inverted Index and provides opperations on it.
InvertedIndexInputFormat : Splits the Inverted Index into 'n' InvertedIndexSplits by term.
InvertedIndexRecordReader : Defines Iteration through an InvertedIndexSplit

Show
Richard McCreadie added a comment - 23/Mar/09 5:21 PM MapReduce Index reading of Terrier Inverted Indicies is now ready for testing. Output Takes the form of 'n' lists of Term-PostingList pairs. Where 'n' is set by the user. Classes: InvertedIndexSplit : Defines an ordered subset of documents from an Inverted Index and provides opperations on it. InvertedIndexInputFormat : Splits the Inverted Index into 'n' InvertedIndexSplits by term. InvertedIndexRecordReader : Defines Iteration through an InvertedIndexSplit
Hide
Craig Macdonald added a comment - 16/Jul/09 7:12 PM

Committed my own version to trunk.

Show
Craig Macdonald added a comment - 16/Jul/09 7:12 PM Committed my own version to trunk.

People

Dates

  • Created:
    19/Feb/09 2:42 PM
    Updated:
    05/Mar/10 4:46 PM
    Resolved:
    16/Jul/09 7:12 PM