java.lang.Object
- org.terrier.structures.collections.ReadOnlyMap<K,V>
- - org.terrier.structures.collections.FSOrderedMapFile<K,V>

Type Parameters:

K - Type of the keys

V - Type of the values

All Implemented Interfaces:

java.io.Closeable, java.lang.AutoCloseable, java.util.Map<K,V>, java.util.SortedMap<K,V>, OrderedMap<K,V>
```
public class FSOrderedMapFile<K extends org.apache.hadoop.io.WritableComparable,V extends org.apache.hadoop.io.Writable>
extends ReadOnlyMap<K,V>
implements OrderedMap<K,V>, java.io.Closeable, java.util.SortedMap<K,V>
```
An implementation of java.util.Map that can be accessed from disk. Key and value types are assumed to have a fixed size. Their factories must be passed to the constructor. In the name, FSOrderedMapFile, FS stands for Fixed Size.

Since:

3.0

Author:

Craig Macdonald

Nested Class Summary

Nested Classes
Modifier and Type	Class	Description
`static class`	`FSOrderedMapFile.EntryIterator<IK extends org.apache.hadoop.io.Writable,IV extends org.apache.hadoop.io.Writable>`	an iterator for entries.
`static interface`	`FSOrderedMapFile.FSOMapFileBSearchShortcut<KEY>`	interface FSOMapFileBSearchShortcut
`static class`	`FSOrderedMapFile.MapFileInMemory<IK extends org.apache.hadoop.io.Writable,IV extends org.apache.hadoop.io.Writable>`	MapFileInMemory class
`static interface`	`FSOrderedMapFile.MapFileWriter`	Interface for writing a FSOMapFile
`static class`	`FSOrderedMapFile.MultiFSOMapWriter`	Writes out a FSOMapFile, but assumes that input data need not be sorted by key.

Nested classes/interfaces inherited from interface java.util.Map
java.util.Map.Entry<K extends java.lang.Object,V extends java.lang.Object>

Field Summary

Fields
Modifier and Type	Field	Description
`protected RandomDataInput`	`dataFile`	actual underlying data file
`protected java.lang.String`	`dataFilename`	filename of the underlying file
`protected int`	`entrySize`	total size of one key,value pair
`protected FixedSizeWriteableFactory<K>`	`keyFactory`
`protected static org.slf4j.Logger`	`logger`	The logger used for this class
`protected int`	`numberOfEntries`	The number of entries in the file.
`protected FSOrderedMapFile.FSOMapFileBSearchShortcut<K>`	`shortcut`
`static java.lang.String`	`USUAL_EXTENSION`	USUAL_EXTENSION
`protected FixedSizeWriteableFactory<V>`	`valueFactory`

Constructor Summary

Constructors
Constructor	Description
`FSOrderedMapFile(java.lang.String filename, boolean updateable, FixedSizeWriteableFactory<K> _keyFactory, FixedSizeWriteableFactory<V> _valueFactory)`	Construct a new object to access the underlying file data structure
`FSOrderedMapFile(IndexOnDisk index, java.lang.String structureName)`	constructor
`FSOrderedMapFile(RandomDataInput file, java.lang.String filename, FixedSizeWriteableFactory<K> _keyFactory, FixedSizeWriteableFactory<V> _valueFactory)`	constructor

Method Summary

All Methods Static Methods Instance Methods Concrete Methods
Modifier and Type	Method	Description
`protected void`	`_clear()`
`void`	`clear()`	Remove all entries from this map
`void`	`close()`
`java.util.Comparator<? super K>`	`comparator()`	Always returns null, as keys for FSOMapFile are always Comparable, and their Comparable implementation are used.
`boolean`	`containsKey(java.lang.Object o)`
`boolean`	`containsValue(java.lang.Object o)`
`java.util.Set<java.util.Map.Entry<K,V>>`	`entrySet()`
`K`	`firstKey()`
`java.util.Map.Entry<K,V>`	`get(int entryNumber)`	Return the entry at the specified index
`V`	`get(java.lang.Object _key)`
`protected org.terrier.structures.collections.FSOrderedMapFile.MapFileEntry<K,V>`	`getEntry(K key)`	this method is the one which does the actual disk lookup of entries.
`WriteableFactory<K>`	`getKeyFactory()`	Get the key factory
`WriteableFactory<V>`	`getValueFactory()`	Get the value factory
`java.util.SortedMap<K,V>`	`headMap(K to)`
`boolean`	`isEmpty()`
`java.util.Set<K>`	`keySet()`
`K`	`lastKey()`
`static FSOrderedMapFile.MapFileWriter`	`mapFileWrite(java.lang.String filename)`	returns a utility class which can be used to write a FSOrderedMapFile.
`static void`	`mapFileWrite(java.lang.String filename, java.lang.Iterable<java.util.Map.Entry<org.apache.hadoop.io.WritableComparable,org.apache.hadoop.io.Writable>> t)`	writes an entire map FSOrderedMapFile at once, to the specified filename, and using the data contained in the specified iterator
`static void`	`mapFileWrite(java.lang.String filename, java.util.Iterator<java.util.Map.Entry<org.apache.hadoop.io.WritableComparable,org.apache.hadoop.io.Writable>> ti)`	writes an entire map FSOrderedMapFile at once, to the specified filename, and using the data contained in the specified iterator
`static int`	`numberOfEntries(java.lang.String filename, FixedSizeWriteableFactory<?> _keyFactory, FixedSizeWriteableFactory<?> _valueFactory)`	Return number of entries
`void`	`putAll(java.util.Map<? extends K,? extends V> m)`
`void`	`setBSearchShortcut(FSOrderedMapFile.FSOMapFileBSearchShortcut<K> _shortcut)`	Set the FSOMapFileBSearchShortcut
`int`	`size()`	Returns the number of entries in this map
`java.util.SortedMap<K,V>`	`subMap(K from, K to)`
`java.util.SortedMap<K,V>`	`tailMap(K from)`
`java.util.Collection<V>`	`values()`
`protected RandomDataOutput`	`write()`

Methods inherited from class org.terrier.structures.collections.ReadOnlyMap
put, remove

Methods inherited from class java.lang.Object
clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Methods inherited from interface java.util.Map
compute, computeIfAbsent, computeIfPresent, equals, forEach, getOrDefault, hashCode, merge, put, putIfAbsent, remove, remove, replace, replace, replaceAll

- Field Detail
  - USUAL_EXTENSION
```
public static final java.lang.String USUAL_EXTENSION
```
    USUAL_EXTENSION
    
    See Also:
    
    Constant Field Values
  - logger
```
protected static final org.slf4j.Logger logger
```
    The logger used for this class
  - dataFile
```
protected RandomDataInput dataFile
```
    actual underlying data file
  - dataFilename
```
protected java.lang.String dataFilename
```
    filename of the underlying file
  - numberOfEntries
```
protected int numberOfEntries
```
    The number of entries in the file.
  - entrySize
```
protected int entrySize
```
    total size of one key,value pair
  - shortcut
```
protected FSOrderedMapFile.FSOMapFileBSearchShortcut<K extends org.apache.hadoop.io.WritableComparable> shortcut
```
  - keyFactory
```
protected FixedSizeWriteableFactory<K extends org.apache.hadoop.io.WritableComparable> keyFactory
```
  - valueFactory
```
protected FixedSizeWriteableFactory<V extends org.apache.hadoop.io.Writable> valueFactory
```
- Constructor Detail
  - FSOrderedMapFile
```
public FSOrderedMapFile(IndexOnDisk index,
                        java.lang.String structureName)
                 throws java.io.IOException
```
    constructor
    
    Parameters:
    
    index -
    
    structureName -
    
    Throws:
    
    java.io.IOException
  - FSOrderedMapFile
```
public FSOrderedMapFile(java.lang.String filename,
                        boolean updateable,
                        FixedSizeWriteableFactory<K> _keyFactory,
                        FixedSizeWriteableFactory<V> _valueFactory)
                 throws java.io.IOException
```
    Construct a new object to access the underlying file data structure
    
    Parameters:
    
    filename - Filename of the file containing the structure
    
    updateable - Whether the file can be updated in this JVM
    
    _keyFactory - factory object for keys
    
    _valueFactory - factory object for values
    
    Throws:
    
    java.io.IOException - thrown if an IO problem occurs
  - FSOrderedMapFile
```
public FSOrderedMapFile(RandomDataInput file,
                        java.lang.String filename,
                        FixedSizeWriteableFactory<K> _keyFactory,
                        FixedSizeWriteableFactory<V> _valueFactory)
                 throws java.io.IOException
```
    constructor
    
    Parameters:
    
    file -
    
    filename -
    
    _keyFactory -
    
    _valueFactory -
    
    Throws:
    
    java.io.IOException
- Method Detail
  - write
```
protected RandomDataOutput write()
```
  - numberOfEntries
```
public static int numberOfEntries(java.lang.String filename,
                                  FixedSizeWriteableFactory<?> _keyFactory,
                                  FixedSizeWriteableFactory<?> _valueFactory)
```
    Return number of entries
    
    Parameters:
    
    filename -
    
    _keyFactory -
    
    _valueFactory -
    
    Returns:
    
    number of entries
  - getKeyFactory
```
public WriteableFactory<K> getKeyFactory()
```
    Get the key factory
  - getValueFactory
```
public WriteableFactory<V> getValueFactory()
```
    Get the value factory
  - clear
```
public void clear()
```
    Remove all entries from this map
    
    Specified by:
    
    clear in interface java.util.Map<K extends org.apache.hadoop.io.WritableComparable,V extends org.apache.hadoop.io.Writable>
    
    Overrides:
    
    clear in class ReadOnlyMap<K extends org.apache.hadoop.io.WritableComparable,V extends org.apache.hadoop.io.Writable>
  - putAll
```
public void putAll(java.util.Map<? extends K,? extends V> m)
```
    Specified by:
    
    putAll in interface java.util.Map<K extends org.apache.hadoop.io.WritableComparable,V extends org.apache.hadoop.io.Writable>
  - _clear
```
protected void _clear()
```
  - entrySet
```
public java.util.Set<java.util.Map.Entry<K,V>> entrySet()
```
    Specified by:
    
    entrySet in interface java.util.Map<K extends org.apache.hadoop.io.WritableComparable,V extends org.apache.hadoop.io.Writable>
    
    Specified by:
    
    entrySet in interface java.util.SortedMap<K extends org.apache.hadoop.io.WritableComparable,V extends org.apache.hadoop.io.Writable>
  - keySet
```
public java.util.Set<K> keySet()
```
    Specified by:
    
    keySet in interface java.util.Map<K extends org.apache.hadoop.io.WritableComparable,V extends org.apache.hadoop.io.Writable>
    
    Specified by:
    
    keySet in interface java.util.SortedMap<K extends org.apache.hadoop.io.WritableComparable,V extends org.apache.hadoop.io.Writable>
  - values
```
public java.util.Collection<V> values()
```
    Specified by:
    
    values in interface java.util.Map<K extends org.apache.hadoop.io.WritableComparable,V extends org.apache.hadoop.io.Writable>
    
    Specified by:
    
    values in interface java.util.SortedMap<K extends org.apache.hadoop.io.WritableComparable,V extends org.apache.hadoop.io.Writable>
  - size
```
public int size()
```
    Returns the number of entries in this map
    
    Specified by:
    
    size in interface java.util.Map<K extends org.apache.hadoop.io.WritableComparable,V extends org.apache.hadoop.io.Writable>
  - containsValue
```
public boolean containsValue(java.lang.Object o)
```
    Specified by:
    
    containsValue in interface java.util.Map<K extends org.apache.hadoop.io.WritableComparable,V extends org.apache.hadoop.io.Writable>
  - containsKey
```
public boolean containsKey(java.lang.Object o)
```
    Specified by:
    
    containsKey in interface java.util.Map<K extends org.apache.hadoop.io.WritableComparable,V extends org.apache.hadoop.io.Writable>
  - isEmpty
```
public boolean isEmpty()
```
    Specified by:
    
    isEmpty in interface java.util.Map<K extends org.apache.hadoop.io.WritableComparable,V extends org.apache.hadoop.io.Writable>
  - setBSearchShortcut
```
public void setBSearchShortcut(FSOrderedMapFile.FSOMapFileBSearchShortcut<K> _shortcut)
```
    Set the FSOMapFileBSearchShortcut
  - getEntry
```
protected org.terrier.structures.collections.FSOrderedMapFile.MapFileEntry<K,V> getEntry(K key)
```
    this method is the one which does the actual disk lookup of entries. If an entry is not found, then a MapFileEntry is returned where the index field indicates the (-(insertion point) -1) of the specified key. See also Arrays.binarySearch()
  - firstKey
```
public K firstKey()
```
    Specified by:
    
    firstKey in interface java.util.SortedMap<K extends org.apache.hadoop.io.WritableComparable,V extends org.apache.hadoop.io.Writable>
  - lastKey
```
public K lastKey()
```
    Specified by:
    
    lastKey in interface java.util.SortedMap<K extends org.apache.hadoop.io.WritableComparable,V extends org.apache.hadoop.io.Writable>
  - headMap
```
public java.util.SortedMap<K,V> headMap(K to)
```
    Specified by:
    
    headMap in interface java.util.SortedMap<K extends org.apache.hadoop.io.WritableComparable,V extends org.apache.hadoop.io.Writable>
  - subMap
```
public java.util.SortedMap<K,V> subMap(K from,
                                             K to)
```
    Specified by:
    
    subMap in interface java.util.SortedMap<K extends org.apache.hadoop.io.WritableComparable,V extends org.apache.hadoop.io.Writable>
  - tailMap
```
public java.util.SortedMap<K,V> tailMap(K from)
```
    Specified by:
    
    tailMap in interface java.util.SortedMap<K extends org.apache.hadoop.io.WritableComparable,V extends org.apache.hadoop.io.Writable>
  - comparator
```
public final java.util.Comparator<? super K> comparator()
```
    Always returns null, as keys for FSOMapFile are always Comparable, and their Comparable implementation are used.
    
    Specified by:
    
    comparator in interface java.util.SortedMap<K extends org.apache.hadoop.io.WritableComparable,V extends org.apache.hadoop.io.Writable>
  - get
```
public V get(java.lang.Object _key)
```
    Specified by:
    
    get in interface java.util.Map<K extends org.apache.hadoop.io.WritableComparable,V extends org.apache.hadoop.io.Writable>
  - get
```
public java.util.Map.Entry<K,V> get(int entryNumber)
```
    Return the entry at the specified index
    
    Specified by:
    
    get in interface OrderedMap<K extends org.apache.hadoop.io.WritableComparable,V extends org.apache.hadoop.io.Writable>
  - close
```
public void close()
           throws java.io.IOException
```
    Specified by:
    
    close in interface java.lang.AutoCloseable
    
    Specified by:
    
    close in interface java.io.Closeable
    
    Throws:
    
    java.io.IOException
  - mapFileWrite
```
public static void mapFileWrite(java.lang.String filename,
                                java.lang.Iterable<java.util.Map.Entry<org.apache.hadoop.io.WritableComparable,org.apache.hadoop.io.Writable>> t)
                         throws java.io.IOException
```
    writes an entire map FSOrderedMapFile at once, to the specified filename, and using the data contained in the specified iterator
    
    Throws:
    
    java.io.IOException
  - mapFileWrite
```
public static void mapFileWrite(java.lang.String filename,
                                java.util.Iterator<java.util.Map.Entry<org.apache.hadoop.io.WritableComparable,org.apache.hadoop.io.Writable>> ti)
                         throws java.io.IOException
```
    writes an entire map FSOrderedMapFile at once, to the specified filename, and using the data contained in the specified iterator
    
    Throws:
    
    java.io.IOException
  - mapFileWrite
```
public static FSOrderedMapFile.MapFileWriter mapFileWrite(java.lang.String filename)
                                                   throws java.io.IOException
```
    returns a utility class which can be used to write a FSOrderedMapFile. Input data MUST be sorted by key.
    
    Throws:
    
    java.io.IOException

Class FSOrderedMapFile<K extends org.apache.hadoop.io.WritableComparable,​V extends org.apache.hadoop.io.Writable>

Nested Class Summary

Nested classes/interfaces inherited from interface java.util.Map

Field Summary

Constructor Summary

Method Summary

Methods inherited from class org.terrier.structures.collections.ReadOnlyMap

Methods inherited from class java.lang.Object

Methods inherited from interface java.util.Map

Field Detail

USUAL_EXTENSION

logger

dataFile

dataFilename

numberOfEntries

entrySize

shortcut

keyFactory

valueFactory

Constructor Detail

FSOrderedMapFile

FSOrderedMapFile

FSOrderedMapFile

Method Detail

write

numberOfEntries

getKeyFactory

getValueFactory

clear

putAll

_clear

entrySet

keySet

values

size

containsValue

containsKey

isEmpty

setBSearchShortcut

getEntry

firstKey

lastKey

headMap

subMap

tailMap

comparator

get

get

close

mapFileWrite

mapFileWrite

mapFileWrite

Class FSOrderedMapFile<K extends org.apache.hadoop.io.WritableComparable,V extends org.apache.hadoop.io.Writable>