• 如果您觉得本站非常有看点,那么赶紧使用Ctrl+D 收藏吧

Java RCFileRecordReader类的典型用法和代码示例

java 1次浏览

本文整理汇总了Java中org.apache.hadoop.hive.ql.io.RCFileRecordReader的典型用法代码示例。如果您正苦于以下问题:Java RCFileRecordReader类的具体用法?Java RCFileRecordReader怎么用?Java RCFileRecordReader使用的例子?那么恭喜您, 这里精选的类代码示例或许可以为您提供帮助。

RCFileRecordReader类属于org.apache.hadoop.hive.ql.io包,在下文中一共展示了RCFileRecordReader类的2个代码示例,这些例子默认根据受欢迎程度排序。您可以为喜欢或者感觉有用的代码点赞,您的评价将有助于我们的系统推荐出更棒的Java代码示例。

示例1: initialize

点赞 3

import org.apache.hadoop.hive.ql.io.RCFileRecordReader; //导入依赖的package包/类
@SuppressWarnings("deprecation")
   @Override
   public void initialize(InputSplit split, TaskAttemptContext ctx)
    throws IOException, InterruptedException {

FileSplit fileSplit = (FileSplit) split;
Configuration conf = ctx.getConfiguration();
splitPath = fileSplit.getPath();

rcFileRecordReader = new RCFileRecordReader<LongWritable, BytesRefArrayWritable>(
	conf, new org.apache.hadoop.mapred.FileSplit(splitPath,
		fileSplit.getStart(), fileSplit.getLength(),
		new org.apache.hadoop.mapred.JobConf(conf)));

key = rcFileRecordReader.createKey();
value = rcFileRecordReader.createValue();

   }
 

开发者ID:sigmoidanalytics,
项目名称:spork-streaming,
代码行数:19,
代码来源:HiveRCRecordReader.java

示例2: readRow

点赞 3

import org.apache.hadoop.hive.ql.io.RCFileRecordReader; //导入依赖的package包/类
private ColumnarStruct readRow(File outputFile, Path outputPath, String schema) throws IOException,
        InterruptedException, SerDeException {

    FileSplit fileSplit = new FileSplit(outputPath, 0L, outputFile.length(), (String[])null);


    Path splitPath = fileSplit.getPath();

    RCFileRecordReader<LongWritable, BytesRefArrayWritable> rcFileRecordReader = new RCFileRecordReader<LongWritable, BytesRefArrayWritable>(
        new Configuration(false), new org.apache.hadoop.mapred.FileSplit(splitPath,
            fileSplit.getStart(), fileSplit.getLength(),
            new org.apache.hadoop.mapred.JobConf(conf)));

    LongWritable key = rcFileRecordReader.createKey();
    BytesRefArrayWritable value = rcFileRecordReader.createValue();
    rcFileRecordReader.next(key, value);
    rcFileRecordReader.close();

    ColumnarStruct struct = readColumnarStruct(value, schema);
    return struct;
}
 

开发者ID:sigmoidanalytics,
项目名称:spork-streaming,
代码行数:22,
代码来源:TestHiveColumnarStorage.java


版权声明:本文转自网络文章,转载此文章仅为分享知识,如有侵权,请联系管理员进行删除。
喜欢 (0)