Terrier Core

Property values for single-pass indexing are not scaled

Details

  • Type: Bug Bug
  • Status: Resolved Resolved
  • Priority: Major Major
  • Resolution: Fixed
  • Affects Version/s: 2.2.1
  • Fix Version/s: None
  • Component/s: .indexing, .utility
  • Description:
    Hide
    The value of property "memory.reserved" is interpreted as bytes, and not in MB as stated in the sample property file.
    The value of "memory.heap.usage" is interpreted as real number, and not as percentage.

    When using the default settings (memory.reserved=50, memory.heap.usage=85), the condition in RuntimeMemoryChecker therefore never gets true and the intermediate runs are not written to disk. When indexing large datasets, this leads to out of memory exceptions.

    Maybe it would be a good idea to check the valid range of these properties and output a warning.
    Show
    The value of property "memory.reserved" is interpreted as bytes, and not in MB as stated in the sample property file. The value of "memory.heap.usage" is interpreted as real number, and not as percentage. When using the default settings (memory.reserved=50, memory.heap.usage=85), the condition in RuntimeMemoryChecker therefore never gets true and the intermediate runs are not written to disk. When indexing large datasets, this leads to out of memory exceptions. Maybe it would be a good idea to check the valid range of these properties and output a warning.

Issue Links

Related
 

Activity

Ascending order - Click to sort in descending order
Hide
Permalink
Craig Macdonald added a comment - 23/Feb/10 5:27 PM

Thanks for the heads up about the sample properties file. I have added suitable checks to the relevant Terrier class for checking that these properties are properly set in the future.

Show
Craig Macdonald added a comment - 23/Feb/10 5:27 PM Thanks for the heads up about the sample properties file. I have added suitable checks to the relevant Terrier class for checking that these properties are properly set in the future.
Hide
Permalink
Craig Macdonald added a comment - 24/Feb/10 5:19 PM

Committed to trunk. Thanks!

Show
Craig Macdonald added a comment - 24/Feb/10 5:19 PM Committed to trunk. Thanks!

People

Dates

  • Created:
    23/Feb/10 5:04 PM
    Updated:
    24/Feb/10 5:19 PM
    Resolved:
    24/Feb/10 5:19 PM
Atlassian JIRA (v4.1#519) | Report a problem | Request a feature | Contact administrators
Powered by a free Atlassian JIRA open source license for Univ of Glasgow. Try JIRA - bug tracking software for your team.