T
- the type of the materialized recordpublic abstract class ReadSupport<T> extends Object
ParquetInputFormat
to materialize recordsModifier and Type | Class and Description |
---|---|
static class |
ReadSupport.ReadContext
information to read the file
|
Modifier and Type | Field and Description |
---|---|
static String |
PARQUET_READ_SCHEMA
configuration key for a parquet read projection schema
|
Constructor and Description |
---|
ReadSupport() |
Modifier and Type | Method and Description |
---|---|
static MessageType |
getSchemaForRead(MessageType fileMessageType,
MessageType projectedMessageType) |
static MessageType |
getSchemaForRead(MessageType fileMessageType,
String partialReadSchemaString)
attempts to validate and construct a
MessageType from a read projection schema |
ReadSupport.ReadContext |
init(org.apache.hadoop.conf.Configuration configuration,
Map<String,String> keyValueMetaData,
MessageType fileSchema)
Deprecated.
override
init(InitContext) instead |
ReadSupport.ReadContext |
init(InitContext context)
called in
InputFormat.getSplits(org.apache.hadoop.mapreduce.JobContext) in the front end |
abstract RecordMaterializer<T> |
prepareForRead(org.apache.hadoop.conf.Configuration configuration,
Map<String,String> keyValueMetaData,
MessageType fileSchema,
ReadSupport.ReadContext readContext)
called in
RecordReader.initialize(org.apache.hadoop.mapreduce.InputSplit, org.apache.hadoop.mapreduce.TaskAttemptContext) in the back end
the returned RecordConsumer will materialize the records and add them to the destination |
public static final String PARQUET_READ_SCHEMA
public static MessageType getSchemaForRead(MessageType fileMessageType, String partialReadSchemaString)
MessageType
from a read projection schemafileMessageType
- the typed schema of the sourcepartialReadSchemaString
- the requested projection schemapublic static MessageType getSchemaForRead(MessageType fileMessageType, MessageType projectedMessageType)
@Deprecated public ReadSupport.ReadContext init(org.apache.hadoop.conf.Configuration configuration, Map<String,String> keyValueMetaData, MessageType fileSchema)
init(InitContext)
insteadInputFormat.getSplits(org.apache.hadoop.mapreduce.JobContext)
in the front endconfiguration
- the job configurationkeyValueMetaData
- the app specific metadata from the filefileSchema
- the schema of the filepublic ReadSupport.ReadContext init(InitContext context)
InputFormat.getSplits(org.apache.hadoop.mapreduce.JobContext)
in the front endcontext
- the initialisation contextpublic abstract RecordMaterializer<T> prepareForRead(org.apache.hadoop.conf.Configuration configuration, Map<String,String> keyValueMetaData, MessageType fileSchema, ReadSupport.ReadContext readContext)
RecordReader.initialize(org.apache.hadoop.mapreduce.InputSplit, org.apache.hadoop.mapreduce.TaskAttemptContext)
in the back end
the returned RecordConsumer will materialize the records and add them to the destinationconfiguration
- the job configurationkeyValueMetaData
- the app specific metadata from the filefileSchema
- the schema of the filereadContext
- returned by the init methodCopyright © 2015. All rights reserved.