Terrier Core

Delay index path checking during indexing till HDFS is loaded

Details

  • Type: Improvement Improvement
  • Status: Closed Closed
  • Priority: Minor Minor
  • Resolution: Fixed
  • Affects Version/s: None
  • Fix Version/s: 2.2.1
  • Component/s: None
  • Description:
    Hide
    TR-7 gave the Hadoop file system plugin to Terrier the ability to check that files exist. However, when using HOD and HadoopIndexing, indexing would fail in TrecTerrier stating that the index did not exit. This is because, in TrecTerrier, HOD has not been called, and hence the HDFS is not yet available.
    Show
    TR-7 gave the Hadoop file system plugin to Terrier the ability to check that files exist. However, when using HOD and HadoopIndexing, indexing would fail in TrecTerrier stating that the index did not exit. This is because, in TrecTerrier, HOD has not been called, and hence the HDFS is not yet available.

Attachments

  1. Text File
    TR-8.v1.patch
    (9 kB)
    Craig Macdonald
    29/Jan/09 1:12 AM

Activity

Ascending order - Click to sort in descending order
Hide
Permalink
Craig Macdonald added a comment - 29/Jan/09 1:12 AM

This patch remove index checking from TrecTerrier. Instead, HadoopIndexing and TRECIndexing should check the index path themselves. This patch provides such functionality.

Show
Craig Macdonald added a comment - 29/Jan/09 1:12 AM This patch remove index checking from TrecTerrier. Instead, HadoopIndexing and TRECIndexing should check the index path themselves. This patch provides such functionality.
Hide
Permalink
Craig Macdonald added a comment - 29/Jan/09 3:40 PM

Committed.

Show
Craig Macdonald added a comment - 29/Jan/09 3:40 PM Committed.

People

Dates

  • Created:
    29/Jan/09 1:11 AM
    Updated:
    29/Jan/09 7:32 PM
    Resolved:
    29/Jan/09 3:40 PM
Atlassian JIRA (v4.1#519) | Report a problem | Request a feature | Contact administrators
Powered by a free Atlassian JIRA open source license for Univ of Glasgow. Try JIRA - bug tracking software for your team.