• 如果您觉得本站非常有看点,那么赶紧使用Ctrl+D 收藏吧

Java InsertAllResponse类的典型用法和代码示例

未分类 8次浏览

本文整理汇总了Java中com.google.cloud.bigquery.InsertAllResponse的典型用法代码示例。如果您正苦于以下问题:Java InsertAllResponse类的具体用法?Java InsertAllResponse怎么用?Java InsertAllResponse使用的例子?那么恭喜您, 这里精选的类代码示例或许可以为您提供帮助。

InsertAllResponse类属于com.google.cloud.bigquery包,在下文中一共展示了InsertAllResponse类的18个代码示例,这些例子默认根据受欢迎程度排序。您可以为喜欢或者感觉有用的代码点赞,您的评价将有助于我们的系统推荐出更棒的Java代码示例。

示例1: testPutWhenPartitioningOnMessageTimeWhenNoTimestampType

点赞 3

import com.google.cloud.bigquery.InsertAllResponse; //导入依赖的package包/类
@Test(expected = ConnectException.class)
public void testPutWhenPartitioningOnMessageTimeWhenNoTimestampType() {
  final String topic = "test-topic";

  Map<String, String> properties = propertiesFactory.getProperties();
  properties.put(BigQuerySinkConfig.TOPICS_CONFIG, topic);
  properties.put(BigQuerySinkConfig.DATASETS_CONFIG, ".*=scratch");
  properties.put(BigQuerySinkTaskConfig.BIGQUERY_MESSAGE_TIME_PARTITIONING_CONFIG, "true");

  BigQuery bigQuery = mock(BigQuery.class);
  SinkTaskContext sinkTaskContext = mock(SinkTaskContext.class);
  InsertAllResponse insertAllResponse = mock(InsertAllResponse.class);

  when(bigQuery.insertAll(anyObject())).thenReturn(insertAllResponse);
  when(insertAllResponse.hasErrors()).thenReturn(false);

  BigQuerySinkTask testTask = new BigQuerySinkTask(bigQuery, null);
  testTask.initialize(sinkTaskContext);
  testTask.start(properties);

  testTask.put(Collections.singletonList(spoofSinkRecord(topic, "value", "message text", TimestampType.NO_TIMESTAMP_TYPE, null)));
}
 

开发者ID:wepay,
项目名称:kafka-connect-bigquery,
代码行数:23,
代码来源:BigQuerySinkTaskTest.java

示例2: publishEvents

点赞 3

import com.google.cloud.bigquery.InsertAllResponse; //导入依赖的package包/类
/**
 * publish all events that we have queued up to BigQuery. This is called both by
 * the timer and by writeEvent(). Need to be sure they don't step on each other.
 */
private void publishEvents() {
    InsertAllRequest request;
    InsertAllResponse response;
    
    synchronized (this) {
        if (timerTask != null) {
            timerTask.cancel();
            timerTask = null;
        }
        if (opCounter > 0) {
            request = InsertAllRequest.of(table, batch);
            response = bigquery.insertAll(request);
            logWriteErrors(response);
            opCounter = 0;
            batch.clear();
        }

        // ensure that we don't keep queued events around very long.
        timerTask = new FlushQueuedEvents();
        timer.schedule(timerTask, flushFreq);
    }
}
 

开发者ID:oracle,
项目名称:bdglue,
代码行数:27,
代码来源:BigQueryTable.java

示例3: performWriteRequest

点赞 3

import com.google.cloud.bigquery.InsertAllResponse; //导入依赖的package包/类
/**
 * Sends the request to BigQuery, and throws an exception if any errors occur as a result of doing
 * so.
 * @param tableId The PartitionedTableId.
 * @param rows The rows to write.
 * @param topic The Kafka topic that the row data came from (ignored).
 */
@Override
public void performWriteRequest(PartitionedTableId tableId,
                                List<InsertAllRequest.RowToInsert> rows,
                                String topic) {
  InsertAllRequest request = createInsertAllRequest(tableId, rows);
  InsertAllResponse writeResponse = bigQuery.insertAll(request);
  if (writeResponse.hasErrors()) {
    logger.warn(
        "You may want to enable auto schema updates by specifying "
        + "{}=true in the properties file",
        BigQuerySinkTaskConfig.SCHEMA_UPDATE_CONFIG
    );
    throw new BigQueryConnectException(writeResponse.getInsertErrors());
  } else {
    logger.debug("table insertion completed with no reported errors");
  }
}
 

开发者ID:wepay,
项目名称:kafka-connect-bigquery,
代码行数:25,
代码来源:SimpleBigQueryWriter.java

示例4: testSimplePut

点赞 3

import com.google.cloud.bigquery.InsertAllResponse; //导入依赖的package包/类
@Test
public void testSimplePut() {
  final String topic = "test-topic";

  Map<String, String> properties = propertiesFactory.getProperties();
  properties.put(BigQuerySinkConfig.TOPICS_CONFIG, topic);
  properties.put(BigQuerySinkConfig.DATASETS_CONFIG, ".*=scratch");

  BigQuery bigQuery = mock(BigQuery.class);
  SinkTaskContext sinkTaskContext = mock(SinkTaskContext.class);
  InsertAllResponse insertAllResponse = mock(InsertAllResponse.class);

  when(bigQuery.insertAll(anyObject())).thenReturn(insertAllResponse);
  when(insertAllResponse.hasErrors()).thenReturn(false);

  BigQuerySinkTask testTask = new BigQuerySinkTask(bigQuery, null);
  testTask.initialize(sinkTaskContext);
  testTask.start(properties);

  testTask.put(Collections.singletonList(spoofSinkRecord(topic)));
  testTask.flush(Collections.emptyMap());
  verify(bigQuery, times(1)).insertAll(any(InsertAllRequest.class));
}
 

开发者ID:wepay,
项目名称:kafka-connect-bigquery,
代码行数:24,
代码来源:BigQuerySinkTaskTest.java

示例5: testSimplePutWhenSchemaRetrieverIsNotNull

点赞 3

import com.google.cloud.bigquery.InsertAllResponse; //导入依赖的package包/类
@Test
public void testSimplePutWhenSchemaRetrieverIsNotNull() {
  final String topic = "test-topic";

  Map<String, String> properties = propertiesFactory.getProperties();
  properties.put(BigQuerySinkConfig.TOPICS_CONFIG, topic);
  properties.put(BigQuerySinkConfig.DATASETS_CONFIG, ".*=scratch");

  BigQuery bigQuery = mock(BigQuery.class);
  SinkTaskContext sinkTaskContext = mock(SinkTaskContext.class);
  InsertAllResponse insertAllResponse = mock(InsertAllResponse.class);

  when(bigQuery.insertAll(anyObject())).thenReturn(insertAllResponse);
  when(insertAllResponse.hasErrors()).thenReturn(false);

  SchemaRetriever schemaRetriever = mock(SchemaRetriever.class);

  BigQuerySinkTask testTask = new BigQuerySinkTask(bigQuery, schemaRetriever);
  testTask.initialize(sinkTaskContext);
  testTask.start(properties);

  testTask.put(Collections.singletonList(spoofSinkRecord(topic)));
  testTask.flush(Collections.emptyMap());
  verify(bigQuery, times(1)).insertAll(any(InsertAllRequest.class));
  verify(schemaRetriever, times(1)).setLastSeenSchema(any(TableId.class), any(String.class), any(Schema.class));
}
 

开发者ID:wepay,
项目名称:kafka-connect-bigquery,
代码行数:27,
代码来源:BigQuerySinkTaskTest.java

示例6: insertAll

点赞 2

import com.google.cloud.bigquery.InsertAllResponse; //导入依赖的package包/类
/**
* Example of inserting rows into a table without running a load job.
*/
// [TARGET insertAll(InsertAllRequest)]
// [VARIABLE "my_dataset_name"]
// [VARIABLE "my_table_name"]
public InsertAllResponse insertAll(String datasetName, String tableName) {
// [START insertAll]
TableId tableId = TableId.of(datasetName, tableName);
// Values of the row to insert
Map<String, Object> rowContent = new HashMap<String, Object>();
rowContent.put("booleanField", true);
// Bytes are passed in base64
rowContent.put("bytesField", "Cg0NDg0="); // 0xA, 0xD, 0xD, 0xE, 0xD in base64
// Records are passed as a map
Map<String, Object> recordsContent = new HashMap<String, Object>();
recordsContent.put("stringField", "Hello, World!");
rowContent.put("recordField", recordsContent);
InsertAllResponse response = bigquery.insertAll(InsertAllRequest.newBuilder(tableId)
.addRow("rowId", rowContent)
// More rows can be added in the same RPC by invoking .addRow() on the builder
.build());
if (response.hasErrors()) {
// If any of the insertions failed, this lets you inspect the errors
for (Entry<Long, List<BigQueryError>> entry : response.getInsertErrors().entrySet()) {
// inspect row error
}
}
// [END insertA


版权声明:本文转自网络文章,转载此文章仅为分享知识,如有侵权,请联系管理员进行删除。
喜欢 (0)