uk.ac.starlink.parquet

Class CachedParquetStarTable

java.lang.Object

uk.ac.starlink.table.AbstractStarTable

uk.ac.starlink.parquet.ParquetStarTable

uk.ac.starlink.parquet.CachedParquetStarTable

All Implemented Interfaces:

Closeable, AutoCloseable, StarTable

public class CachedParquetStarTable
extends ParquetStarTable

ParquetStarTable implementation that does a parallel read of all the column data at construction time.

Since:

2 Mar 2021

Author:

Mark Taylor

Nested Class Summary

Nested classes/interfaces inherited from class uk.ac.starlink.parquet.ParquetStarTable

ParquetStarTable.Config

Field Summary

Fields inherited from class uk.ac.starlink.parquet.ParquetStarTable

CREATEDBY_INFO, NAME_KEY, REQUIRED_VOTMETAVERSION, UNSUPPORTED_INFO, VOTMETA_KEY, VOTMETA_NAMESPACE, VOTMETAVERSION_KEY

Constructor Summary

Constructors

Constructor and Description
CachedParquetStarTable(IOSupplier<org.apache.parquet.hadoop.ParquetFileReader> pfrSupplier, ParquetStarTable.Config config, int nthread) Constructor.

Method Summary

Modifier and Type	Method and Description
void	close() The AbstractStarTable implementation of this method does nothing.
Object	getCell(long irow, int icol) The AbstractStarTable implementation of this method throws an UnsupportedOperationException, since unless otherwise provided there is no random access.
Object[]	getRow(long irow) The AbstractStarTable implementation of this method constructs a row by repeated invocation of AbstractStarTable.getCell(long, int).
RowAccess	getRowAccess() Returns an object which can provide random access to this table's data, if random access is implemented.
RowSequence	getRowSequence() Returns an object which can iterate over all the rows in the table sequentially.
RowSplittable	getRowSplittable() Returns a default splittable which relies on table random access if available, or otherwise provides only sequential access (no splits).
boolean	isRandom() The AbstractStarTable implementation of this method returns false.

Methods inherited from class uk.ac.starlink.parquet.ParquetStarTable

getColumnCount, getColumnInfo, getColumnReadStore, getExtraMetadataMap, getInputColumn, getParquetFileReader, getRowCount, getSchema, getVOTableMetadataText, setVOTableMetadataText

Methods inherited from class uk.ac.starlink.table.AbstractStarTable

checkedLongToInt, getColumnAuxDataInfos, getName, getParameters, getURL, setName, setParameters, setURL

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Methods inherited from interface uk.ac.starlink.table.StarTable

getParameterByName, setParameter

Constructor Detail

CachedParquetStarTable

public CachedParquetStarTable(IOSupplier<org.apache.parquet.hadoop.ParquetFileReader> pfrSupplier,
                              ParquetStarTable.Config config,
                              int nthread)
                       throws IOException

Constructor.

Parameters:

pfrSupplier - access to parquet data file

config - table reading configuration

nthread - number of threads to use for concurrent column reads; if <=0, a value is chosen based on the number of available processors

Throws:

IOException

Method Detail

isRandom

public boolean isRandom()

Description copied from class: AbstractStarTable

The AbstractStarTable implementation of this method returns false.

Specified by:

isRandom in interface StarTable

Overrides:

isRandom in class AbstractStarTable

Returns:

true if table random access methods are available

getRowSequence

public RowSequence getRowSequence()
                           throws IOException

Description copied from interface: StarTable

Returns an object which can iterate over all the rows in the table sequentially. Each such returned object is safe for use within a single thread, but not in general from multiple threads concurrently.

Specified by:

getRowSequence in interface StarTable

Specified by:

getRowSequence in class AbstractStarTable

Returns:

new RowSequence

Throws:

IOException - if there is an error providing access

getRowAccess

public RowAccess getRowAccess()
                       throws IOException

Description copied from interface: StarTable

Returns an object which can provide random access to this table's data, if random access is implemented. Each such returned object is safe for use within a single thread, but not in general from multiple threads concurrently.

Specified by:

getRowAccess in interface StarTable

Overrides:

getRowAccess in class AbstractStarTable

Returns:

new RowAccess

Throws:

IOException - if there is an error setting up access

getRowSplittable

public RowSplittable getRowSplittable()
                               throws IOException

Description copied from class: AbstractStarTable

Returns a default splittable which relies on table random access if available, or otherwise provides only sequential access (no splits).

It is often possible to provide a better implementation than this.

Specified by:

getRowSplittable in interface StarTable

Overrides:

getRowSplittable in class AbstractStarTable

Returns:

Tables.getDefaultRowSplittable(this)

Throws:

IOException

See Also:

RowRunner

getCell

public Object getCell(long irow,
                      int icol)
               throws IOException

Description copied from class: AbstractStarTable

The AbstractStarTable implementation of this method throws an UnsupportedOperationException, since unless otherwise provided there is no random access.

Specified by:

getCell in interface StarTable

Overrides:

getCell in class AbstractStarTable

Parameters:

irow - the index of the cell's row

icol - the index of the cell's column

Returns:

the contents of this cell

Throws:

IOException - if there is an error reading the data

getRow

public Object[] getRow(long irow)
                throws IOException

Description copied from class: AbstractStarTable

The AbstractStarTable implementation of this method constructs a row by repeated invocation of AbstractStarTable.getCell(long, int).

Specified by:

getRow in interface StarTable

Overrides:

getRow in class AbstractStarTable

Parameters:

irow - the index of the row to retrieve

Returns:

an array of the objects in each cell in row irow

Throws:

IOException - if there is an error reading the data

close

public void close()
           throws IOException

Description copied from class: AbstractStarTable

The AbstractStarTable implementation of this method does nothing.

Specified by:

close in interface Closeable

Specified by:

close in interface AutoCloseable

Specified by:

close in interface StarTable

Overrides:

close in class AbstractStarTable

Throws:

IOException