Package org.terrier.realtime.incremental

Class IncrementalIndex

java.lang.Object

org.terrier.structures.Index

org.terrier.realtime.multi.MultiIndex

org.terrier.realtime.incremental.IncrementalIndex

All Implemented Interfaces:

java.io.Closeable, java.lang.AutoCloseable, UpdatableIndex

public class IncrementalIndex
extends MultiIndex
implements UpdatableIndex

This is the main Index class for an incremental index. An incremental index is a MultiIndex where one index shard is stored in memory and the rest are stored on disk. Periodically, the memory index is then written do disk, defined as per a FlushPolicy. When the memory index has been flushed to disk, optionally the on-disk portion of the incremental index can then be merged together (based upon a MergePolicy) and/or deleted (based upon a DeletePolicy).

Properties

• incremental.flush: the flush policy to use. Four possible values are supported: noflush (default), flushdocs, flushmem, flushtime

• incremental.merge: the merge policy to use. Three possible values are supported: nomerge (default), single, geometric

• incremental.delete: the delete policy to use. Two possible values are supported: nodelete (default), deleteFixedSize

Since:

4.0

Author:

Richard McCreadie, Stuart Mackie

Nested Class Summary

Nested Classes

Modifier and Type	Class	Description
static class	IncrementalIndex.Loader

Nested classes/interfaces inherited from class org.terrier.structures.Index

Index.DirectIndexRef

Field Summary

Fields

Modifier and Type	Field	Description
protected static org.slf4j.Logger	logger
MemoryIndex	memory
java.lang.String	path
java.lang.String	prefix
int	prefixID

Fields inherited from class org.terrier.realtime.multi.MultiIndex

blocks, fields, indices, selectiveMatchingPolicy

Constructor Summary

Constructors

Modifier	Constructor	Description
protected	IncrementalIndex(Index[] indices)

Method Summary

Modifier and Type	Method	Description
boolean	addToDocument(int docid, Document doc)	Adds specified content contents to the named document id.
boolean	addToDocument(int docid, DocumentPostingList docContents)	Adds relevant terms to the named document id.
void	close()
void	flush()
static IncrementalIndex	get(java.lang.String path, java.lang.String prefix)	Construct a new incremental index.
java.lang.String	getPath()
java.lang.String	getPrefix()
int	getPrefixID()
java.lang.String	getTimeOfLastUpdate()	This method prints out the last time this index was updated as a String in GMT format
void	indexDocument(java.util.Map<java.lang.String,java.lang.String> docProperties, DocumentPostingList docContents)	Update the index with a new document.
void	indexDocument(Document doc)	Update the index with a new document.
boolean	removeDocument(int docid)	Removes a document from the index.
void	setPath(java.lang.String path)
void	setPrefix(java.lang.String prefix)
void	setPrefixID(int prefixID)

Methods inherited from class org.terrier.realtime.multi.MultiIndex

getCollectionStatistics, getDirectIndex, getDocumentIndex, getIndexRef, getIndexStructureInputStream, getInvertedIndex, getIthShard, getLexicon, getMetaIndex, getNumberOfShards, hasIndexStructure, toString

Methods inherited from class org.terrier.structures.Index

createIndex, createIndex, getEnd, getIndexStructure, getStart, hasIndexStructureInputStream, makeDirectIndexRef, setIndexLoadingProfileAsRetrieval

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, wait, wait, wait

Field Detail

logger

protected static final org.slf4j.Logger logger

memory

public MemoryIndex memory

path

public java.lang.String path

prefix

public java.lang.String prefix

prefixID

public int prefixID

Constructor Detail

IncrementalIndex

protected IncrementalIndex(Index[] indices)

Method Detail

get

public static IncrementalIndex get(java.lang.String path,
                                   java.lang.String prefix)

Construct a new incremental index.

indexDocument

public void indexDocument(Document doc)
                   throws java.lang.Exception

Update the index with a new document.

Specified by:

indexDocument in interface UpdatableIndex

Throws:

java.lang.Exception

indexDocument

public void indexDocument(java.util.Map<java.lang.String,java.lang.String> docProperties,
                          DocumentPostingList docContents)
                   throws java.lang.Exception

Update the index with a new document.

Specified by:

indexDocument in interface UpdatableIndex

Throws:

java.lang.Exception

flush

public void flush()
           throws java.io.IOException

Overrides:

flush in class MultiIndex

Throws:

java.io.IOException

close

public void close()
           throws java.io.IOException

Specified by:

close in interface java.lang.AutoCloseable

Specified by:

close in interface java.io.Closeable

Overrides:

close in class MultiIndex

Throws:

java.io.IOException

getTimeOfLastUpdate

public java.lang.String getTimeOfLastUpdate()

This method prints out the last time this index was updated as a String in GMT format

getPath

public java.lang.String getPath()

setPath

public void setPath(java.lang.String path)

getPrefix

public java.lang.String getPrefix()

setPrefix

public void setPrefix(java.lang.String prefix)

getPrefixID

public int getPrefixID()

setPrefixID

public void setPrefixID(int prefixID)

removeDocument

public boolean removeDocument(int docid)

Description copied from interface: UpdatableIndex

Removes a document from the index. Returns true if successful. No known operable implementations at this time.

Specified by:

removeDocument in interface UpdatableIndex

addToDocument

public boolean addToDocument(int docid,
                             Document doc)
                      throws java.lang.Exception

Description copied from interface: UpdatableIndex

Adds specified content contents to the named document id.

Specified by:

addToDocument in interface UpdatableIndex

Returns:

true if supported & successful.

Throws:

java.lang.Exception

addToDocument

public boolean addToDocument(int docid,
                             DocumentPostingList docContents)
                      throws java.lang.Exception

Description copied from interface: UpdatableIndex

Adds relevant terms to the named document id.

Specified by:

addToDocument in interface UpdatableIndex

Returns:

true if supported & successful.

Throws:

java.lang.Exception