IdentityTokeniser (Terrier Information Retrieval Platform 5.5 API)

java.lang.Object
- org.terrier.indexing.tokenisation.Tokeniser
- - org.terrier.indexing.tokenisation.IdentityTokeniser

All Implemented Interfaces:

java.io.Serializable
```
public class IdentityTokeniser
extends Tokeniser
```
A Tokeniser implementation that returns the input as is.

Author:

Craig Macdonald

See Also:

Serialized Form

- Field Summary
  - Fields inherited from class org.terrier.indexing.tokenisation.Tokeniser
    EMPTY_STREAM
- Constructor Summary
  
  Constructors
  Constructor Description
  
  IdentityTokeniser()
- Method Summary
  
  All Methods Instance Methods Concrete Methods
  Modifier and Type Method Description
  
  TokenStream tokenise(java.io.Reader reader)
  Tokenises the text obtained from the specified reader.
  - Methods inherited from class org.terrier.indexing.tokenisation.Tokeniser
    getTokeniser, getTokens, getTokens
  - Methods inherited from class java.lang.Object
    clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

- Constructor Detail
  - IdentityTokeniser
```
public IdentityTokeniser()
```
- Method Detail
  - tokenise
```
public TokenStream tokenise(java.io.Reader reader)
```
    Description copied from class: Tokeniser
    
    Tokenises the text obtained from the specified reader.
    
    Specified by:
    
    tokenise in class Tokeniser
    
    Parameters:
    
    reader - Stream of text to be tokenised
    
    Returns:
    
    a TokenStream of the tokens found in the text.