• 如果您觉得本站非常有看点，那么赶紧使用Ctrl+D 收藏吧
> java > Java ElementDescriptor类的典型用法和代码示例

Java ElementDescriptor类的典型用法和代码示例

java 1次浏览

本文整理汇总了Java中org.apache.pig.backend.datastorage.ElementDescriptor类的典型用法代码示例。如果您正苦于以下问题：Java ElementDescriptor类的具体用法？Java ElementDescriptor怎么用？Java ElementDescriptor使用的例子？那么恭喜您, 这里精选的类代码示例或许可以为您提供帮助。

ElementDescriptor类属于org.apache.pig.backend.datastorage包，在下文中一共展示了ElementDescriptor类的25个代码示例，这些例子默认根据受欢迎程度排序。您可以为喜欢或者感觉有用的代码点赞，您的评价将有助于我们的系统推荐出更棒的Java代码示例。

示例1: processMove

​ 点赞 3 ​ 

import org.apache.pig.backend.datastorage.ElementDescriptor; //导入依赖的package包/类
@Override
protected void processMove(String src, String dst) throws IOException
{
    if(mExplain == null) { // process only if not in "explain" mode

        executeBatch();

        try {
            ElementDescriptor srcPath = mDfs.asElement(src);
            ElementDescriptor dstPath = mDfs.asElement(dst);

            if (!srcPath.exists()) {
                throw new IOException("File or directory " + src + " does not exist.");
            }

            srcPath.rename(dstPath);
        }
        catch (DataStorageException e) {
            throw new IOException("Failed to move " + src + " to " + dst, e);
        }
    } else {
        log.warn("'mv' statement is ignored while processing 'explain -script' or '-check'");
    }
}

开发者ID:sigmoidanalytics，
项目名称:spork-streaming，
代码行数:25，
代码来源:GruntParser.java

示例2: processCopy

​ 点赞 3 ​ 

import org.apache.pig.backend.datastorage.ElementDescriptor; //导入依赖的package包/类
@Override
protected void processCopy(String src, String dst) throws IOException
{
    if(mExplain == null) { // process only if not in "explain" mode

        executeBatch();

        try {
            ElementDescriptor srcPath = mDfs.asElement(src);
            ElementDescriptor dstPath = mDfs.asElement(dst);

            srcPath.copy(dstPath, mConf, false);
        }
        catch (DataStorageException e) {
            throw new IOException("Failed to copy " + src + " to " + dst, e);
        }
    } else {
        log.warn("'cp' statement is ignored while processing 'explain -script' or '-check'");
    }
}

开发者ID:sigmoidanalytics，
项目名称:spork-streaming，
代码行数:21，
代码来源:GruntParser.java

示例3: processCopyToLocal

​ 点赞 3 ​ 

import org.apache.pig.backend.datastorage.ElementDescriptor; //导入依赖的package包/类
@Override
protected void processCopyToLocal(String src, String dst) throws IOException
{
    if(mExplain == null) { // process only if not in "explain" mode

        executeBatch();

        try {
            ElementDescriptor srcPath = mDfs.asElement(src);
            ElementDescriptor dstPath = mLfs.asElement(dst);

            srcPath.copy(dstPath, false);
        }
        catch (DataStorageException e) {
            throw new IOException("Failed to copy " + src + "to (locally) " + dst, e);
        }
    } else {
        log.warn("'copyToLocal' statement is ignored while processing 'explain -script' or '-check'");
    }
}

开发者ID:sigmoidanalytics，
项目名称:spork-streaming，
代码行数:21，
代码来源:GruntParser.java

示例4: processCopyFromLocal

​ 点赞 3 ​ 

import org.apache.pig.backend.datastorage.ElementDescriptor; //导入依赖的package包/类
@Override
protected void processCopyFromLocal(String src, String dst) throws IOException
{
    if(mExplain == null) { // process only if not in "explain" mode

        executeBatch();

        try {
            ElementDescriptor srcPath = mLfs.asElement(src);
            ElementDescriptor dstPath = mDfs.asElement(dst);

            srcPath.copy(dstPath, false);
        }
        catch (DataStorageException e) {
            throw new IOException("Failed to copy (loally) " + src + "to " + dst, e);
        }
    } else {
        log.warn("'copyFromLocal' statement is ignored while processing 'explain -script' or '-check'");
    }
}

开发者ID:sigmoidanalytics，
项目名称:spork-streaming，
代码行数:21，
代码来源:GruntParser.java

示例5: processRemove

​ 点赞 3 ​ 

import org.apache.pig.backend.datastorage.ElementDescriptor; //导入依赖的package包/类
@Override
protected void processRemove(String path, String options ) throws IOException
{
    if(mExplain == null) { // process only if not in "explain" mode

        ElementDescriptor dfsPath = mDfs.asElement(path);
        executeBatch();

        if (!dfsPath.exists()) {
            if (options == null || !options.equalsIgnoreCase("force")) {
                throw new IOException("File or directory " + path + " does not exist.");
            }
        }
        else {

            dfsPath.delete();
        }
    } else {
        log.warn("'rm/rmf' statement is ignored while processing 'explain -script' or '-check'");
    }
}

开发者ID:sigmoidanalytics，
项目名称:spork-streaming，
代码行数:22，
代码来源:GruntParser.java

示例6: getSize

​ 点赞 3 ​ 

import org.apache.pig.backend.datastorage.ElementDescriptor; //导入依赖的package包/类
public static long getSize(String fileName, Properties properties) throws IOException {
	DataStorage dds = new HDataStorage(properties);
    ElementDescriptor elem = dds.asElement(fileName);
   
    // recursively get all the files under this path
    ElementDescriptor[] allElems = getFileElementDescriptors(elem);
    
    long size = 0;
    
    // add up the sizes of all files found
    for (int i=0; i<allElems.length; i++) {
    	Map<String, Object> stats = allElems[i].getStatistics();
    	size += (Long) (stats.get(ElementDescriptor.LENGTH_KEY));
    }
    
    return size;
}

开发者ID:sigmoidanalytics，
项目名称:spork-streaming，
代码行数:18，
代码来源:FileLocalizer.java

示例7: create

​ 点赞 3 ​ 

import org.apache.pig.backend.datastorage.ElementDescriptor; //导入依赖的package包/类
static public OutputStream create(String fileSpec, boolean append, PigContext pigContext) throws IOException {
    fileSpec = checkDefaultPrefix(pigContext.getExecType(), fileSpec);
    if (!fileSpec.startsWith(LOCAL_PREFIX)) {
        ElementDescriptor elem = pigContext.getDfs().asElement(fileSpec);
        return elem.create();
    }
    else {
        fileSpec = fileSpec.substring(LOCAL_PREFIX.length());

        // TODO probably this should be replaced with the local file system
        File f = (new File(fileSpec)).getParentFile();
        if (f!=null){
            boolean res = f.mkdirs();
            if (!res)
                log.warn("FileLocalizer.create: failed to create " + f);
        }
        
        return new FileOutputStream(fileSpec,append);
    }
}

开发者ID:sigmoidanalytics，
项目名称:spork-streaming，
代码行数:21，
代码来源:FileLocalizer.java

示例8: globMatchesFiles

​ 点赞 3 ​ 

import org.apache.pig.backend.datastorage.ElementDescriptor; //导入依赖的package包/类
private static boolean globMatchesFiles(ElementDescriptor elem,
                                        DataStorage fs)
        throws IOException
{
    try {
        // Currently, if you give a glob with non-special glob characters, hadoop
        // returns an array with your file name in it.  So check for that.
        ElementDescriptor[] elems = fs.asCollection(elem.toString());
        switch (elems.length) {
        case 0:
            return false;

        case 1:
            return !elems[0].equals(elem);

        default:
            return true;
        }
    }
    catch (DataStorageException e) {
        throw e;
    }
}

开发者ID:sigmoidanalytics，
项目名称:spork-streaming，
代码行数:24，
代码来源:FileLocalizer.java

示例9: testPigServer

​ 点赞 3 ​ 

import org.apache.pig.backend.datastorage.ElementDescriptor; //导入依赖的package包/类
@Test
public void testPigServer() throws Throwable {
    log.debug("creating pig server");
    PigContext pigContext = new PigContext(ExecType.MAPREDUCE, cluster.getProperties());
    PigServer pig = new PigServer(pigContext);
    System.out.println("testing capacity");
    long capacity = pig.capacity();
    assertTrue(capacity > 0);
    String sampleFileName = "/tmp/fileTest";
    if (!pig.existsFile(sampleFileName)) {
        ElementDescriptor path = pigContext.getDfs().asElement(sampleFileName);
        OutputStream os = path.create();
        os.write("Ben was here!".getBytes());
        os.close();
    }
    long length = pig.fileSize(sampleFileName);
    assertTrue(length > 0);
}

开发者ID:sigmoidanalytics，
项目名称:spork-streaming，
代码行数:19，
代码来源:TestMapReduce.java

示例10: processMove

​ 点赞 3 ​ 

import org.apache.pig.backend.datastorage.ElementDescriptor; //导入依赖的package包/类
@Override
protected void processMove(String src, String dst) throws IOException
{
    filter.validate(PigCommandFilter.Command.MV);
    if(mExplain == null) { // process only if not in "explain" mode

        executeBatch();

        try {
            ElementDescriptor srcPath = mDfs.asElement(src);
            ElementDescriptor dstPath = mDfs.asElement(dst);

            if (!srcPath.exists()) {
                throw new IOException("File or directory " + src + " does not exist.");
            }

            srcPath.rename(dstPath);
        }
        catch (DataStorageException e) {
            throw new IOException("Failed to move " + src + " to " + dst, e);
        }
    } else {
        log.warn("'mv' statement is ignored while processing 'explain -script' or '-check'");
    }
}

开发者ID:sigmoidanalytics，
项目名称:spork，
代码行数:26，
代码来源:GruntParser.java

示例11: processCopy

​ 点赞 3 ​ 

import org.apache.pig.backend.datastorage.ElementDescriptor; //导入依赖的package包/类
@Override
protected void processCopy(String src, String dst) throws IOException
{
    filter.validate(PigCommandFilter.Command.CP);
    if(mExplain == null) { // process only if not in "explain" mode

        executeBatch();

        try {
            ElementDescriptor srcPath = mDfs.asElement(src);
            ElementDescriptor dstPath = mDfs.asElement(dst);

            srcPath.copy(dstPath, mConf, false);
        }
        catch (DataStorageException e) {
            throw new IOException("Failed to copy " + src + " to " + dst, e);
        }
    } else {
        log.warn("'cp' statement is ignored while processing 'explain -script' or '-check'");
    }
}

开发者ID:sigmoidanalytics，
项目名称:spork，
代码行数:22，
代码来源:GruntParser.java

示例12: processCopyToLocal

​ 点赞 3 ​ 

import org.apache.pig.backend.datastorage.ElementDescriptor; //导入依赖的package包/类
@Override
protected void processCopyToLocal(String src, String dst) throws IOException
{
    filter.validate(PigCommandFilter.Command.COPYTOLOCAL);
    if(mExplain == null) { // process only if not in "explain" mode

        executeBatch();

        try {
            ElementDescriptor srcPath = mDfs.asElement(src);
            ElementDescriptor dstPath = mLfs.asElement(dst);

            srcPath.copy(dstPath, false);
        }
        catch (DataStorageException e) {
            throw new IOException("Failed to copy " + src + "to (locally) " + dst, e);
        }
    } else {
        log.warn("'copyToLocal' statement is ignored while processing 'explain -script' or '-check'");
    }
}

开发者ID:sigmoidanalytics，
项目名称:spork，
代码行数:22，
代码来源:GruntParser.java

示例13: processCopyFromLocal

​ 点赞 3 ​ 

import org.apache.pig.backend.datastorage.ElementDescriptor; //导入依赖的package包/类
@Override
protected void processCopyFromLocal(String src, String dst) throws IOException
{
    filter.validate(PigCommandFilter.Command.COPYFROMLOCAL);
    if(mExplain == null) { // process only if not in "explain" mode

        executeBatch();

        try {
            ElementDescriptor srcPath = mLfs.asElement(src);
            ElementDescriptor dstPath = mDfs.asElement(dst);

            srcPath.copy(dstPath, false);
        }
        catch (DataStorageException e) {
            throw new IOException("Failed to copy (loally) " + src + "to " + dst, e);
        }
    } else {
        log.warn("'copyFromLocal' statement is ignored while processing 'explain -script' or '-check'");
    }
}

开发者ID:sigmoidanalytics，
项目名称:spork，
代码行数:22，
代码来源:GruntParser.java

示例14: getSize

​ 点赞 3 ​ 

import org.apache.pig.backend.datastorage.ElementDescriptor; //导入依赖的package包/类
public static long getSize(String fileName, Properties properties) throws IOException {
	DataStorage dds = new HDataStorage(properties);
    ElementDescriptor elem = dds.asElement(fileName);

    // recursively get all the files under this path
    ElementDescriptor[] allElems = getFileElementDescriptors(elem);

    long size = 0;

    // add up the sizes of all files found
    for (int i=0; i<allElems.length; i++) {
    	Map<String, Object> stats = allElems[i].getStatistics();
    	size += (Long) (stats.get(ElementDescriptor.LENGTH_KEY));
    }

    return size;
}

开发者ID:sigmoidanalytics，
项目名称:spork，
代码行数:18，
代码来源:FileLocalizer.java

示例15: create

​ 点赞 3 ​ 

import org.apache.pig.backend.datastorage.ElementDescriptor; //导入依赖的package包/类
static public OutputStream create(String fileSpec, boolean append, PigContext pigContext) throws IOException {
    fileSpec = checkDefaultPrefix(pigContext.getExecType(), fileSpec);
    if (!fileSpec.startsWith(LOCAL_PREFIX)) {
        ElementDescriptor elem = pigContext.getDfs().asElement(fileSpec);
        return elem.create();
    }
    else {
        fileSpec = fileSpec.substring(LOCAL_PREFIX.length());

        // TODO probably this should be replaced with the local file system
        File f = (new File(fileSpec)).getParentFile();
        if (f!=null){
            boolean res = f.mkdirs();
            if (!res)
                log.warn("FileLocalizer.create: failed to create " + f);
        }

        return new FileOutputStream(fileSpec,append);
    }
}

开发者ID:sigmoidanalytics，
项目名称:spork，
代码行数:21，
代码来源:FileLocalizer.java

示例16: listPaths

​ 点赞 3 ​ 

import org.apache.pig.backend.datastorage.ElementDescriptor; //导入依赖的package包/类
/**
 * List the contents of a directory.
 * @param dir name of directory to list
 * @return array of strings, one for each file name
 * @throws IOException
 */
public String[] listPaths(String dir) throws IOException {
    // Check if this operation is permitted
    filter.validate(PigCommandFilter.Command.LS);

    Collection<String> allPaths = new ArrayList<String>();
    ContainerDescriptor container = pigContext.getDfs().asContainer(dir);
    Iterator<ElementDescriptor> iter = container.iterator();

    while (iter.hasNext()) {
        ElementDescriptor elem = iter.next();
        allPaths.add(elem.toString());
    }

    String[] type = new String[1];
    return allPaths.toArray(type);
}

开发者ID:sigmoidanalytics，
项目名称:spork，
代码行数:23，
代码来源:PigServer.java

示例17: testPigServer

​ 点赞 3 ​ 

import org.apache.pig.backend.datastorage.ElementDescriptor; //导入依赖的package包/类
@Test
public void testPigServer() throws Throwable {
    log.debug("creating pig server");
    PigContext pigContext = new PigContext(cluster.getExecType(), cluster.getProperties());
    PigServer pig = new PigServer(pigContext);
    System.out.println("testing capacity");
    long capacity = pig.capacity();
    assertTrue(capacity > 0);
    String sampleFileName = "/tmp/fileTest";
    if (!pig.existsFile(sampleFileName)) {
        ElementDescriptor path = pigContext.getDfs().asElement(sampleFileName);
        OutputStream os = path.create();
        os.write("Ben was here!".getBytes());
        os.close();
    }
    long length = pig.fileSize(sampleFileName);
    assertTrue(length > 0);
}

开发者ID:sigmoidanalytics，
项目名称:spork，
代码行数:19，
代码来源:TestMapReduce.java

示例18: processMove

​ 点赞 3 ​ 

import org.apache.pig.backend.datastorage.ElementDescriptor; //导入依赖的package包/类
@Override
protected void processMove(String src, String dst) throws IOException
{
    if(mExplain == null) { // process only if not in "explain" mode

        executeBatch();
    
        try {
            ElementDescriptor srcPath = mDfs.asElement(src);
            ElementDescriptor dstPath = mDfs.asElement(dst);
            
            if (!srcPath.exists()) {
                throw new IOException("File or directory " + src + " does not exist.");                
            }
            
            srcPath.rename(dstPath);
        }
        catch (DataStorageException e) {
            throw new IOException("Failed to move " + src + " to " + dst, e);
        }
    } else {
        log.warn("'mv' statement is ignored while processing 'explain -script' or '-check'");
    }
}

开发者ID:PonIC，
项目名称:PonIC，
代码行数:25，
代码来源:GruntParser.java

示例19: processCopy

​ 点赞 3 ​ 

import org.apache.pig.backend.datastorage.ElementDescriptor; //导入依赖的package包/类
@Override
protected void processCopy(String src, String dst) throws IOException
{
    if(mExplain == null) { // process only if not in "explain" mode

        executeBatch();
    
        try {
            ElementDescriptor srcPath = mDfs.asElement(src);
            ElementDescriptor dstPath = mDfs.asElement(dst);
            
            srcPath.copy(dstPath, mConf, false);
        }
        catch (DataStorageException e) {
            throw new IOException("Failed to copy " + src + " to " + dst, e);
        }
    } else {
        log.warn("'cp' statement is ignored while processing 'explain -script' or '-check'");
    }
}

开发者ID:PonIC，
项目名称:PonIC，
代码行数:21，
代码来源:GruntParser.java

示例20: processCopyToLocal

​ 点赞 3 ​ 

import org.apache.pig.backend.datastorage.ElementDescriptor; //导入依赖的package包/类
@Override
protected void processCopyToLocal(String src, String dst) throws IOException
{
    if(mExplain == null) { // process only if not in "explain" mode
        
        executeBatch();
    
        try {
            ElementDescriptor srcPath = mDfs.asElement(src);
            ElementDescriptor dstPath = mLfs.asElement(dst);
            
            srcPath.copy(dstPath, false);
        }
        catch (DataStorageException e) {
            throw new IOException("Failed to copy " + src + "to (locally) " + dst, e);
        }
    } else {
        log.warn("'copyToLocal' statement is ignored while processing 'explain -script' or '-check'");
    }
}

开发者ID:PonIC，
项目名称:PonIC，
代码行数:21，
代码来源:GruntParser.java

示例21: processCopyFromLocal

​ 点赞 3 ​ 

import org.apache.pig.backend.datastorage.ElementDescriptor; //导入依赖的package包/类
@Override
protected void processCopyFromLocal(String src, String dst) throws IOException
{
    if(mExplain == null) { // process only if not in "explain" mode
        
        executeBatch();
    
        try {
            ElementDescriptor srcPath = mLfs.asElement(src);
            ElementDescriptor dstPath = mDfs.asElement(dst);
            
            srcPath.copy(dstPath, false);
        }
        catch (DataStorageException e) {
            throw new IOException("Failed to copy (loally) " + src + "to " + dst, e);
        }
    } else {
        log.warn("'copyFromLocal' statement is ignored while processing 'explain -script' or '-check'");
    }
}

开发者ID:PonIC，
项目名称:PonIC，
代码行数:21，
代码来源:GruntParser.java

示例22: processRemove

​ 点赞 3 ​ 

import org.apache.pig.backend.datastorage.ElementDescriptor; //导入依赖的package包/类
@Override
protected void processRemove(String path, String options ) throws IOException
{
    if(mExplain == null) { // process only if not in "explain" mode

        ElementDescriptor dfsPath = mDfs.asElement(path);    
        executeBatch();
    
        if (!dfsPath.exists()) {
            if (options == null || !options.equalsIgnoreCase("force")) {
                throw new IOException("File or directory " + path + " does not exist."); 
            }
        }
        else {
            
            dfsPath.delete();
        }
    } else {
        log.warn("'rm/rmf' statement is ignored while processing 'explain -script' or '-check'");
    }
}

开发者ID:PonIC，
项目名称:PonIC，
代码行数:22，
代码来源:GruntParser.java

示例23: delete

​ 点赞 3 ​ 

import org.apache.pig.backend.datastorage.ElementDescriptor; //导入依赖的package包/类
static public boolean delete(String fileSpec, PigContext pigContext) throws IOException{
    fileSpec = checkDefaultPrefix(pigContext.getExecType(), fileSpec);
    if (!fileSpec.startsWith(LOCAL_PREFIX)) {
        ElementDescriptor elem = pigContext.getDfs().asElement(fileSpec);
        elem.delete();
        return true;
    }
    else {
        fileSpec = fileSpec.substring(LOCAL_PREFIX.length());
        boolean ret = true;
        // TODO probably this should be replaced with the local file system
        File f = (new File(fileSpec));
        // TODO this only deletes the file. Any dirs createdas a part of this
        // are not removed.
        if (f!=null){
            ret = f.delete();
        }
        
        return ret;
    }
}

开发者ID:PonIC，
项目名称:PonIC，
代码行数:22，
代码来源:FileLocalizer.java

示例24: getTemporaryPath

​ 点赞 3 ​ 

import org.apache.pig.backend.datastorage.ElementDescriptor; //导入依赖的package包/类
/**
 * @deprecated Use {@link #getTemporaryPath(PigContext)} instead
 */
@Deprecated
public static synchronized ElementDescriptor 
    getTemporaryPath(ElementDescriptor relative, 
                     PigContext pigContext) throws IOException {
    if (relative == null) {
        relative = relativeRoot(pigContext);
    }
    if (!relativeRoot(pigContext).exists()) {
        relativeRoot(pigContext).create();
    }
    ElementDescriptor elem= 
        pigContext.getDfs().asElement(relative.toString(), "tmp" + r.nextInt());
    toDelete().push(elem);
    return elem;
}

开发者ID:PonIC，
项目名称:PonIC，
代码行数:19，
代码来源:FileLocalizer.java

示例25: exists

​ 点赞 2 ​ 

import org.apache.pig.backend.datastorage.ElementDescriptor; //导入依赖的package包/类
private boolean exists(ElementDescriptor e) throws IOException {
    if (lookupCache.containsKey(e)) {
        return lookupCache.get(e);
    } else {
     boolean res = e.exists();
     lookupCache.put(e, res);
     return res;
    }
}

开发者ID:sigmoidanalytics，
项目名称:spork-streaming，
代码行数:10，
代码来源:JsonMetadata.java

版权声明：本文转自网络文章，转载此文章仅为分享知识，如有侵权，请联系管理员进行删除。
喜欢 (0)