Druiddefault

Druid--Druid中数据管理

2020-03-09  本文已影响0人  李小李的路

schema变化

替换Segment

foo_2015-01-01/2015-01-02_v1_0
foo_2015-01-01/2015-01-02_v1_1
foo_2015-01-01/2015-01-02_v1_2
foo_2015-01-01/2015-01-02_v2_0
foo_2015-01-01/2015-01-02_v2_1
foo_2015-01-01/2015-01-02_v2_2
foo_2015-01-01/2015-01-02_v1_0
foo_2015-01-02/2015-01-03_v1_1
foo_2015-01-03/2015-01-04_v1_2
foo_2015-01-01/2015-01-02_v1_0
foo_2015-01-02/2015-01-03_v2_1
foo_2015-01-03/2015-01-04_v1_2

Segment中不同的schema

压缩和重建索引

{
    "type": "compact",
    "id": <task_id>,
    "dataSource": <task_datasource>,
    "ioConfig": <IO config>,
    "dimensionsSpec" <custom dimensionsSpec>,
    "metricsSpec" <custom metricsSpec>,
    "segmentGranularity": <segment granularity after compaction>,
    "tuningConfig" <parallel indexing task tuningConfig>,
    "context": <task context>
}
Field Description Required
type Task type. Should be compact Yes
id Task id No
dataSource DataSource name to be compacted Yes
ioConfig ioConfig for compaction task. See Compaction IOConfig for details. Yes
dimensionsSpec Custom dimensionsSpec. Compaction task will use this dimensionsSpec if exist instead of generating one. See below for more details. No
metricsSpec Custom metricsSpec. Compaction task will use this metricsSpec if specified rather than generating one. No
segmentGranularity If this is set, compactionTask will change the segment granularity for the given interval. See segmentGranularity of granularitySpec for more details. See the below table for the behavior. No
tuningConfig Parallel indexing task tuningConfig No
context Task context No
{
  "type" : "compact",
  "dataSource" : "wikipedia",
  "ioConfig" : {
    "type": "compact",
    "inputSpec": {
      "type": "interval",
      "interval": "2017-01-01/2018-01-01"
    }
  }
}

添加新数据

更新已存在的数据

采用Lookup

Reingesting数据(清洁数据)

With Hadoop-based ingestion

Reindexing with Native Batch Ingestion

deleting data

上一篇 下一篇

猜你喜欢

热点阅读