Java FileTap类的典型用法和代码示例

本文整理汇总了Java中cascading.tap.local.FileTap类的典型用法代码示例。如果您正苦于以下问题：Java FileTap类的具体用法？Java FileTap怎么用？Java FileTap使用的例子？那么恭喜您, 这里精选的类代码示例或许可以为您提供帮助。

FileTap类属于cascading.tap.local包，在下文中一共展示了FileTap类的6个代码示例，这些例子默认根据受欢迎程度排序。您可以为喜欢或者感觉有用的代码点赞，您的评价将有助于我们的系统推荐出更棒的Java代码示例。

示例1: testCreateCommonCrawlFlowDef

import cascading.tap.local.FileTap; //导入依赖的package包/类
@Test
public void testCreateCommonCrawlFlowDef() throws Exception {
    Properties properties = new ConfigReader().renderProperties(CommonCrawlIndexTest.class);

    String sourcePath = properties.getProperty("inPath");
    String sinkPath = properties.getProperty("testCreateCommonCrawlFlowDefOutput");
    String sinkValidationPath = properties.getProperty("testCreateCommonCrawlFlowDefOutputValidation");

    // create the Cascading "source" (input) tap to read the commonCrawl WAT file(s)
    Tap source = new FileTap(new TextLine(new Fields("line")) ,sourcePath);

    // create the Cascading "sink" (output) tap to dump the results
    Tap sink = new FileTap(new TextLine(new Fields("line")) ,sinkPath);

    //Build the Cascading Flow Definition
    FlowDef flowDef = CommonCrawlIndex.createCommonCrawlFlowDef(source, sink);
    new LocalFlowConnector(properties).connect(flowDef).complete();

    Assert.sameContent(sinkPath, sinkValidationPath);
}

开发者ID:awslabs，
项目名称:aws-big-data-blog，
代码行数:21，
代码来源:CommonCrawlIndexTest.java

示例2: LocalTemplateTap

import cascading.tap.local.FileTap; //导入依赖的package包/类
@ConstructorProperties({ "parent", "pathTemplate", "pathFields",
    "openTapsThreshold" })
public LocalTemplateTap(FileTap parent, Granularity input, Granularity output
    , int openTapsThreshold) {
  super(parent, input,output, openTapsThreshold);
  this.input = input;
  this.output = output;
}

开发者ID:guokr，
项目名称:hebo，
代码行数:9，
代码来源:LocalTemplateTap.java

示例3: sourceTap

import cascading.tap.local.FileTap; //导入依赖的package包/类
private Tap sourceTap() {
    return new FileTap(new TextLine(new Fields("line")), TestUtils.sampleArtistsJson());
}

开发者ID:xushjie1987，
项目名称:es-hadoop-v2.2.0，
代码行数:4，
代码来源:AbstractCascadingLocalJsonSaveTest.java

示例4: sourceTap

import cascading.tap.local.FileTap; //导入依赖的package包/类
private Tap sourceTap() {
    return new FileTap(new TextDelimited(new Fields("id", "name", "url", "picture", "ts")), INPUT);
}

开发者ID:xushjie1987，
项目名称:es-hadoop-v2.2.0，
代码行数:4，
代码来源:AbstractCascadingLocalSaveTest.java

示例5: launchFlow

import cascading.tap.local.FileTap; //导入依赖的package包/类
List<Tuple> launchFlow() throws IOException {
    File tmpDir = Files.createTempDir();
    try {
        FlowDef flowDef = FlowDef.flowDef().setName("testFlow");
        for(int sourceId=0; sourceId<sources.length; sourceId++) {
            Source source = sources[sourceId];
            File input = new File(tmpDir, "input"+sourceId+".csv");
            FileWriter writer = new FileWriter(input);
            try {
                for(Tuple tuple : source.tuples) {
                    if(source.fields.size() != tuple.size()) {
                        throw new IllegalArgumentException("Number of input fields is not the same of value of input tuple");
                    }
                    writer.write((tuple.getString(0) == null)? "" : tuple.getString(0));
                    for(int i=1; i<tuple.size(); i++) {
                        writer.write("\t");
                        writer.write((tuple.getString(i) == null)? "" : tuple.getString(i));
                    }
                    writer.write("\n");
                }
            } finally {
                writer.flush();
                writer.close();
            }
            FileTap inputTap = new FileTap(new TextDelimited(source.fields), input.getAbsolutePath());
            flowDef.addSource(source.pipe, inputTap);
        }
        File output = new File(tmpDir, "output.csv");
        FileTap outputTap = new FileTap(new TextDelimited(true, "\t"), output.getAbsolutePath());
        flowDef.addTailSink(tail, outputTap);
        Flow<?> flow = new LocalFlowConnector(new Properties()).connect(flowDef);
        flow.complete();
        
        List<Tuple> result = new ArrayList<Tuple>();
        TupleEntryIterator iterator = flow.openSink();
        while(iterator.hasNext()) {
            result.add(iterator.next().getTupleCopy());
        }
        iterator.close();
        return result;
    } finally {
        FileUtils.deleteDirectory(tmpDir);
    }
}

开发者ID:vbehar，
项目名称:cascading-flapi，
代码行数:45，
代码来源:TestHelper.java

示例6: sourceTap

import cascading.tap.local.FileTap; //导入依赖的package包/类
private Tap sourceTap() {
    return new FileTap(new TextDelimited(new Fields("id", "name", "url", "picture", "ts", "tag")), INPUT);
}

开发者ID:elastic，
项目名称:elasticsearch-hadoop，
代码行数:4，
代码来源:AbstractCascadingLocalSaveTest.java