Terrier Core

Delay index path checking during indexing till HDFS is loaded

Details

  • Type: Improvement Improvement
  • Status: Closed Closed
  • Priority: Minor Minor
  • Resolution: Fixed
  • Affects Version/s: None
  • Fix Version/s: 2.2.1
  • Component/s: None
  • Description:
    Hide
    TR-7 gave the Hadoop file system plugin to Terrier the ability to check that files exist. However, when using HOD and HadoopIndexing, indexing would fail in TrecTerrier stating that the index did not exit. This is because, in TrecTerrier, HOD has not been called, and hence the HDFS is not yet available.
    Show
    TR-7 gave the Hadoop file system plugin to Terrier the ability to check that files exist. However, when using HOD and HadoopIndexing, indexing would fail in TrecTerrier stating that the index did not exit. This is because, in TrecTerrier, HOD has not been called, and hence the HDFS is not yet available.
  1. TR-8.v1.patch
    (9 kB)
    Craig Macdonald
    29/Jan/09 1:12 AM

Activity

Hide
Craig Macdonald added a comment - 29/Jan/09 1:12 AM

This patch remove index checking from TrecTerrier. Instead, HadoopIndexing and TRECIndexing should check the index path themselves. This patch provides such functionality.

Show
Craig Macdonald added a comment - 29/Jan/09 1:12 AM This patch remove index checking from TrecTerrier. Instead, HadoopIndexing and TRECIndexing should check the index path themselves. This patch provides such functionality.
Hide
Craig Macdonald added a comment - 29/Jan/09 3:40 PM

Committed.

Show
Craig Macdonald added a comment - 29/Jan/09 3:40 PM Committed.

People

Dates

  • Created:
    29/Jan/09 1:11 AM
    Updated:
    29/Jan/09 7:32 PM
    Resolved:
    29/Jan/09 3:40 PM