Terrier Core

It should be usefull to have some kind of policy for the pipeline (reset) that should be applied every Documents or every Query submitted to the system.

Example:
You want put in the pipeline a stage that is re-initialized every Query.

Solution:
Here i'm going to give my solution.

The solution refactoring the org.terrier.terms introducing the reset() method in
TermPipeline interface and TermPipelineAccessor.
This change in interface affected all the TermPipeline so new base class (BaseTermPipeline) was created and inerithed by all TermPipeline.
BaseTermPipeline give a default implementation of reset() method and also move the "next" atribute in it.

The patch also affected the Manager class and many Indexer classes:
Manager
Indexer
BasicIndexer
BasicSinglePassIndexer
BlockIndexer
Hadoop_BasicSinglePassIndexer

Thanks to all.

Tagging for 3.1

Hi Stilo,

Just working on this now. Two things that I changed:

• Reset is called AFTER a document/query
• Its not optional in the Manager, it is called after every query (i.e. no additional property)
• I didnt make the BaseTermPipeline class.

Can you think of a way of providing a Junit test for this?

Hi Craig.
Unfortunatly i can't provide a Test class (i'm not yet focused on this problem now).
But i think you can make a simple test by make a termpipeline that print something like "Hello wolrd" every document.
Anyway i did not agree on your approach; i my mind the reset option it's necessary to have a document/query oriented
"filtering" so in this sense a BEFORE approach may fit better then a AFTER reset approach.
I didn't understand why you remove BaseTermPipeline hierarcy is elegant for me but probablu u need less ineritanche?

Bye
GS

Hi Stilo,

Thanks for the quick response. My idea with a reset AFTER is that a TermPipeline instance could buffer some terms, and then only let them out once reset() is called. However, you still want them in the same document, so in this respect, reset() is like a flush().

About the base classes - I already made a Stemmer base class, which encapsulated most of the changes.

Other side of the coin.

GS

Committed to trunk for version 3.1 release.

Hi Giovanni, Your affiliation is still University of Rome Tor Vergata, right? Need to credit you in the changes documentation.

Craig,
thanks i'm:
University degli Studi dell'Aquila
and
Nestor Laboratory - University of Rome "Tor Vergata"

many many thanks
GS.