• 如果您觉得本站非常有看点,那么赶紧使用Ctrl+D 收藏吧

Java DepthFirstWalker类的典型用法和代码示例

java 1次浏览

本文整理汇总了Java中org.apache.pig.impl.plan.DepthFirstWalker的典型用法代码示例。如果您正苦于以下问题:Java DepthFirstWalker类的具体用法?Java DepthFirstWalker怎么用?Java DepthFirstWalker使用的例子?那么恭喜您, 这里精选的类代码示例或许可以为您提供帮助。

DepthFirstWalker类属于org.apache.pig.impl.plan包,在下文中一共展示了DepthFirstWalker类的21个代码示例,这些例子默认根据受欢迎程度排序。您可以为喜欢或者感觉有用的代码点赞,您的评价将有助于我们的系统推荐出更棒的Java代码示例。

示例1: MRCompiler

点赞 3

import org.apache.pig.impl.plan.DepthFirstWalker; //导入依赖的package包/类
public MRCompiler(PhysicalPlan plan,
        PigContext pigContext) throws MRCompilerException {
    super(plan, new DepthFirstWalker<PhysicalOperator, PhysicalPlan>(plan));
    this.plan = plan;
    this.pigContext = pigContext;
    splitsSeen = new HashMap<OperatorKey, MapReduceOper>();
    MRPlan = new MROperPlan();
    nig = NodeIdGenerator.getGenerator();
    udfFinder = new UDFFinder();
    List<PhysicalOperator> roots = plan.getRoots();
    if((roots == null) || (roots.size() <= 0)) {
        int errCode = 2053;
        String msg = "Internal error. Did not find roots in the physical plan.";
        throw new MRCompilerException(msg, errCode, PigException.BUG);
    }
    scope = roots.get(0).getOperatorKey().getScope();
    messageCollector = new CompilationMessageCollector() ;
    phyToMROpMap = new HashMap<PhysicalOperator, MapReduceOper>();

    fileConcatenationThreshold = Integer.parseInt(pigContext.getProperties()
            .getProperty(FILE_CONCATENATION_THRESHOLD, "100"));
    optimisticFileConcatenation = pigContext.getProperties().getProperty(
            OPTIMISTIC_FILE_CONCATENATION, "false").equals("true");
    LOG.info("File concatenation threshold: " + fileConcatenationThreshold
            + " optimistic? " + optimisticFileConcatenation);
}
 

开发者ID:sigmoidanalytics,
项目名称:spork,
代码行数:27,
代码来源:MRCompiler.java

示例2: PactCompiler

点赞 3

import org.apache.pig.impl.plan.DepthFirstWalker; //导入依赖的package包/类
public PactCompiler(PactPlan plan, PigContext pc, UDFContext udfCon) {
    super(plan, new DepthFirstWalker<PactOperator, PactPlan>(plan));
    this.plan = plan;
    this.pc = pc;
    UDFContext.setUdfContext(udfCon);
    compiledInputs = new Contract[plan.size()];	//each PactOperator will become one InputContract
    nig = NodeIdGenerator.getGenerator();
    r = new Random(1331);
    FileLocalizer.setR(r);
    List<PactOperator> roots = plan.getRoots();
    if((roots == null) || (roots.size() <= 0)) {
    	String msg = "Internal error. Did not find roots in the pact plan.";
    	log.info(msg);
    }
    scope = roots.get(0).getOperatorKey().getScope();
    pactOpToContract = new HashMap<PactOperator, Contract>();
    udfFinder = new UDFFinder();
}
 

开发者ID:PonIC,
项目名称:PonIC,
代码行数:19,
代码来源:PactCompiler.java

示例3: SPrinter

点赞 2

import org.apache.pig.impl.plan.DepthFirstWalker; //导入依赖的package包/类
public SPrinter(PrintStream ps, SOperPlan plan, PigContext pc) {
	super(plan, new DepthFirstWalker<StormOper, SOperPlan>(plan));
       this.pc = pc;
	mStream = ps;
       mStream.println("#--------------------------------------------------");
       mStream.println("# Storm Topology Plan                              ");
       mStream.println("#--------------------------------------------------");
}
 

开发者ID:JamesLampton,
项目名称:piggybank-squeal,
代码行数:9,
代码来源:SPrinter.java

示例4: AliasVisitor

点赞 2

import org.apache.pig.impl.plan.DepthFirstWalker; //导入依赖的package包/类
public AliasVisitor(PhysicalPlan plan, List<String> alias, List<String> aliasLocation) {
    super(plan, new DepthFirstWalker<PhysicalOperator, PhysicalPlan>(
            plan));
    this.alias = alias;
    this.aliasLocation = aliasLocation;
    aliasSet = new HashSet<String>();
    if (!alias.isEmpty()) {
        for (String s : alias) aliasSet.add(s);
    }
}
 

开发者ID:sigmoidanalytics,
项目名称:spork-streaming,
代码行数:11,
代码来源:ScriptState.java

示例5: MRPrinter

点赞 2

import org.apache.pig.impl.plan.DepthFirstWalker; //导入依赖的package包/类
/**
 * @param ps PrintStream to output plan information to
 * @param plan MR plan to print
 */
public MRPrinter(PrintStream ps, MROperPlan plan) {
    super(plan, new DepthFirstWalker<MapReduceOper, MROperPlan>(plan));
    mStream = ps;
    mStream.println("#--------------------------------------------------");
    mStream.println("# Map Reduce Plan                                  ");
    mStream.println("#--------------------------------------------------");
}
 

开发者ID:sigmoidanalytics,
项目名称:spork-streaming,
代码行数:12,
代码来源:MRPrinter.java

示例6: LimitAdjuster

点赞 2

import org.apache.pig.impl.plan.DepthFirstWalker; //导入依赖的package包/类
public LimitAdjuster(MROperPlan plan, PigContext pigContext) {
    super(plan, new DepthFirstWalker<MapReduceOper, MROperPlan>(plan));
    this.pigContext = pigContext;
    nig = NodeIdGenerator.getGenerator();
    List<MapReduceOper> roots = plan.getRoots();
    scope = roots.get(0).getOperatorKey().getScope();
}
 

开发者ID:sigmoidanalytics,
项目名称:spork-streaming,
代码行数:8,
代码来源:LimitAdjuster.java

示例7: JoinDistributedCacheVisitor

点赞 2

import org.apache.pig.impl.plan.DepthFirstWalker; //导入依赖的package包/类
public JoinDistributedCacheVisitor(PhysicalPlan plan,
        PigContext pigContext, Configuration conf) {
    super(plan, new DepthFirstWalker<PhysicalOperator, PhysicalPlan>(
            plan));
    this.pigContext = pigContext;
    this.conf = conf;
}
 

开发者ID:sigmoidanalytics,
项目名称:spork-streaming,
代码行数:8,
代码来源:JobControlCompiler.java

示例8: UdfDistributedCacheVisitor

点赞 2

import org.apache.pig.impl.plan.DepthFirstWalker; //导入依赖的package包/类
public UdfDistributedCacheVisitor(PhysicalPlan plan,
        PigContext pigContext,
        Configuration conf) {
    super(plan, new DepthFirstWalker<PhysicalOperator, PhysicalPlan>(
            plan));
    this.pigContext = pigContext;
    this.conf = conf;
}
 

开发者ID:sigmoidanalytics,
项目名称:spork-streaming,
代码行数:9,
代码来源:JobControlCompiler.java

示例9: MRCompiler

点赞 2

import org.apache.pig.impl.plan.DepthFirstWalker; //导入依赖的package包/类
public MRCompiler(PhysicalPlan plan,
        PigContext pigContext) throws MRCompilerException {
    super(plan, new DepthFirstWalker<PhysicalOperator, PhysicalPlan>(plan));
    this.plan = plan;
    this.pigContext = pigContext;
    splitsSeen = new HashMap<OperatorKey, MapReduceOper>();
    MRPlan = new MROperPlan();
    nig = NodeIdGenerator.getGenerator();
    r = new Random(1331);
    FileLocalizer.setR(r);
    udfFinder = new UDFFinder();
    List<PhysicalOperator> roots = plan.getRoots();
    if((roots == null) || (roots.size() <= 0)) {
    	int errCode = 2053;
    	String msg = "Internal error. Did not find roots in the physical plan.";
    	throw new MRCompilerException(msg, errCode, PigException.BUG);
    }
    scope = roots.get(0).getOperatorKey().getScope();
    messageCollector = new CompilationMessageCollector() ;
    phyToMROpMap = new HashMap<PhysicalOperator, MapReduceOper>();
    
    fileConcatenationThreshold = Integer.parseInt(pigContext.getProperties()
            .getProperty(FILE_CONCATENATION_THRESHOLD, "100"));
    optimisticFileConcatenation = pigContext.getProperties().getProperty(
            OPTIMISTIC_FILE_CONCATENATION, "false").equals("true");
    LOG.info("File concatenation threshold: " + fileConcatenationThreshold
            + " optimistic? " + optimisticFileConcatenation);
}
 

开发者ID:sigmoidanalytics,
项目名称:spork-streaming,
代码行数:29,
代码来源:MRCompiler.java

示例10: CombinerOptimizer

点赞 2

import org.apache.pig.impl.plan.DepthFirstWalker; //导入依赖的package包/类
public CombinerOptimizer(MROperPlan plan, boolean doMapAgg, 
        CompilationMessageCollector messageCollector) {

    super(plan, new DepthFirstWalker<MapReduceOper, MROperPlan>(plan));
    this.messageCollector = messageCollector;
    this.doMapAgg = doMapAgg;
}
 

开发者ID:sigmoidanalytics,
项目名称:spork-streaming,
代码行数:8,
代码来源:CombinerOptimizer.java

示例11: IllustratorAttacher

点赞 2

import org.apache.pig.impl.plan.DepthFirstWalker; //导入依赖的package包/类
public IllustratorAttacher(PhysicalPlan plan, LineageTracer lineage, int maxRecords,
    Map<POLoad, LogicalSchema> poLoadToSchemaMap, PigContext hadoopPigContext) throws VisitorException {
    super(plan, new DepthFirstWalker<PhysicalOperator, PhysicalPlan>(plan));
    pigContext = hadoopPigContext;
    this.lineage = lineage;
    poToEqclassesMap = new HashMap<PhysicalOperator, Collection<IdentityHashSet<Tuple>>>();
    poToDataMap = new HashMap<PhysicalOperator, DataBag>();
    this.maxRecords = maxRecords;
    this.poloadToSchemaMap = poLoadToSchemaMap;
}
 

开发者ID:sigmoidanalytics,
项目名称:spork-streaming,
代码行数:11,
代码来源:IllustratorAttacher.java

示例12: revisit

点赞 2

import org.apache.pig.impl.plan.DepthFirstWalker; //导入依赖的package包/类
/**
 * revisit an enhanced physical plan from MR compilation
 * @param plan a physical plan to be traversed
 */
public void revisit(PhysicalPlan plan) throws VisitorException {
    pushWalker(new DepthFirstWalker<PhysicalOperator, PhysicalPlan>(plan));
    revisit = true;
    PhysicalPlan oriPlan = mPlan;
    mPlan = plan;
    visit();
    mPlan = oriPlan;
    popWalker();
}
 

开发者ID:sigmoidanalytics,
项目名称:spork-streaming,
代码行数:14,
代码来源:IllustratorAttacher.java

示例13: AliasVisitor

点赞 2

import org.apache.pig.impl.plan.DepthFirstWalker; //导入依赖的package包/类
public AliasVisitor(PhysicalPlan plan, List<String> alias, List<String> aliasLocation) {
    super(plan, new DepthFirstWalker<PhysicalOperator, PhysicalPlan>(plan));
    this.alias = alias;
    this.aliasLocation = aliasLocation;
    aliasSet = new HashSet<String>();
    if (!alias.isEmpty()) {
        for (String s : alias) aliasSet.add(s);
    }
}
 

开发者ID:sigmoidanalytics,
项目名称:spork,
代码行数:10,
代码来源:ScriptState.java

示例14: TezCompiler

点赞 2

import org.apache.pig.impl.plan.DepthFirstWalker; //导入依赖的package包/类
public TezCompiler(PhysicalPlan plan, PigContext pigContext)
        throws TezCompilerException {
    super(plan, new DepthFirstWalker<PhysicalOperator, PhysicalPlan>(plan));
    this.plan = plan;
    this.pigContext = pigContext;

    pigProperties = pigContext.getProperties();
    splitsSeen = Maps.newHashMap();
    tezPlan = new TezOperPlan();
    nig = NodeIdGenerator.getGenerator();
    udfFinder = new UDFFinder();
    List<PhysicalOperator> roots = plan.getRoots();
    if((roots == null) || (roots.size() <= 0)) {
        int errCode = 2053;
        String msg = "Internal error. Did not find roots in the physical plan.";
        throw new TezCompilerException(msg, errCode, PigException.BUG);
    }
    scope = roots.get(0).getOperatorKey().getScope();
    localRearrangeFactory = new POLocalRearrangeTezFactory(scope, nig);
    phyToTezOpMap = Maps.newHashMap();

    fileConcatenationThreshold = Integer.parseInt(pigProperties
            .getProperty(FILE_CONCATENATION_THRESHOLD, "100"));
    optimisticFileConcatenation = pigProperties.getProperty(
            OPTIMISTIC_FILE_CONCATENATION, "false").equals("true");
    LOG.info("File concatenation threshold: " + fileConcatenationThreshold
            + " optimistic? " + optimisticFileConcatenation);
}
 

开发者ID:sigmoidanalytics,
项目名称:spork,
代码行数:29,
代码来源:TezCompiler.java

示例15: XMLMRPrinter

点赞 2

import org.apache.pig.impl.plan.DepthFirstWalker; //导入依赖的package包/类
/**
 * @param ps PrintStream to output plan information to
 * @param plan MR plan to print
 * @throws ParserConfigurationException 
 */
public XMLMRPrinter(PrintStream ps, MROperPlan plan) throws ParserConfigurationException {
    super(plan, new DepthFirstWalker<MapReduceOper, MROperPlan>(plan));
    mStream = ps;
    DocumentBuilderFactory factory = DocumentBuilderFactory.newInstance();
    DocumentBuilder builder = factory.newDocumentBuilder();
    this.doc = builder.newDocument();
    this.root = this.doc.createElement("mapReducePlan");
    this.doc.appendChild(this.root);
}
 

开发者ID:sigmoidanalytics,
项目名称:spork,
代码行数:15,
代码来源:XMLMRPrinter.java

示例16: AliasVisitor

点赞 2

import org.apache.pig.impl.plan.DepthFirstWalker; //导入依赖的package包/类
public AliasVisitor(PhysicalPlan plan, List<String> alias) {
    super(plan, new DepthFirstWalker<PhysicalOperator, PhysicalPlan>(
            plan));
    this.alias = alias;
    aliasSet = new HashSet<String>();
    if (!alias.isEmpty()) {
        for (String s : alias) aliasSet.add(s);
    }
}
 

开发者ID:PonIC,
项目名称:PonIC,
代码行数:10,
代码来源:ScriptState.java

示例17: JoinDistributedCacheVisitor

点赞 2

import org.apache.pig.impl.plan.DepthFirstWalker; //导入依赖的package包/类
public JoinDistributedCacheVisitor(PhysicalPlan plan, 
        PigContext pigContext, Configuration conf) {
    super(plan, new DepthFirstWalker<PhysicalOperator, PhysicalPlan>(
            plan));
    this.pigContext = pigContext;
    this.conf = conf;
}
 

开发者ID:PonIC,
项目名称:PonIC,
代码行数:8,
代码来源:JobControlCompiler.java

示例18: UdfDistributedCacheVisitor

点赞 2

import org.apache.pig.impl.plan.DepthFirstWalker; //导入依赖的package包/类
public UdfDistributedCacheVisitor(PhysicalPlan plan, 
                                  PigContext pigContext,
                                  Configuration conf) {
    super(plan, new DepthFirstWalker<PhysicalOperator, PhysicalPlan>(
            plan));
    this.pigContext = pigContext;
    this.conf = conf;
}
 

开发者ID:PonIC,
项目名称:PonIC,
代码行数:9,
代码来源:JobControlCompiler.java

示例19: UdfDistributedCacheVisitor

点赞 2

import org.apache.pig.impl.plan.DepthFirstWalker; //导入依赖的package包/类
public UdfDistributedCacheVisitor(PhysicalPlan plan, 
        PigContext pigContext,
        Configuration conf) {
    super(plan, new DepthFirstWalker<PhysicalOperator, PhysicalPlan>(
            plan));
    this.pigContext = pigContext;
    this.conf = conf;
}
 

开发者ID:kaituo,
项目名称:sedge,
代码行数:9,
代码来源:JobControlCompiler.java

示例20: FeatureVisitor

点赞 2

import org.apache.pig.impl.plan.DepthFirstWalker; //导入依赖的package包/类
public FeatureVisitor(PhysicalPlan plan, BitSet feature) {
    super(plan, new DepthFirstWalker<PhysicalOperator, PhysicalPlan>(
            plan));
    this.feature = feature;
}
 

开发者ID:sigmoidanalytics,
项目名称:spork-streaming,
代码行数:6,
代码来源:ScriptState.java

示例21: LogicalPlanFeatureVisitor

点赞 2

import org.apache.pig.impl.plan.DepthFirstWalker; //导入依赖的package包/类
protected LogicalPlanFeatureVisitor(LogicalPlan plan, BitSet feature) throws FrontendException {
    super(plan, new org.apache.pig.newplan.DepthFirstWalker(plan));
    this.feature = feature;
}
 

开发者ID:sigmoidanalytics,
项目名称:spork-streaming,
代码行数:5,
代码来源:ScriptState.java


版权声明:本文转自网络文章,转载此文章仅为分享知识,如有侵权,请联系管理员进行删除。
喜欢 (0)