码迷,mamicode.com
首页 > 其他好文 > 详细

Kylin-基本知识

时间:2016-05-27 11:40:15      阅读:371      评论:0      收藏:0      [点我收藏+]

标签:

CUBE

  • Table - This is definition of hive tables as source of cubes, which must be synced before building cubes.
    技术分享

  • Data Model - This describes a STAR SCHEMA data model, which defines fact/lookup tables and filter condition.
    技术分享

  • Cube Descriptor - This describes definition and settings for a cube instance, defining which data model to use, what dimensions and measures to have, how to partition to segments and how to handle auto-merge etc.
    技术分享

  • Cube Instance - This is instance of cube, built from one cube descriptor, and consist of one or more cube segments according partition settings.
    技术分享

  • Partition - User can define a DATE/STRING column as partition column on cube descriptor, to separate one cube into several segments with different date periods.
    技术分享

  • Cube Segment - This is actual carrier of cube data, and maps to a HTable in HBase. One building job creates one new segment for the cube instance. Once data change on specified data period, we can refresh related segments to avoid rebuilding whole cube.
    技术分享

  • Aggregation Group - Each aggregation group is subset of dimensions, and build cuboid with combinations inside. It aims at pruning for optimization.
    技术分享

DIMENSION & MEASURE

  • Mandotary - This dimension type is used for cuboid pruning, if a dimension is specified as “mandatory”, then those combinations without such dimension are pruned.
  • Hierarchy - This dimension type is used for cuboid pruning, if dimension A,B,C forms a “hierarchy” relation, then only combinations with A, AB or ABC shall be remained.
  • Derived - On lookup tables, some dimensions could be generated from its PK, so there’s specific mapping between them and FK from fact table. So those dimensions are DERIVED and don’t participate in cuboid generation.
    技术分享

  • Count Distinct(HyperLogLog) - Immediate COUNT DISTINCT is hard to calculate, a approximate algorithm - HyperLogLog is introduced, and keep error rate in a lower level.
  • Count Distinct(Precise) - Precise COUNT DISTINCT will be pre-calculated basing on RoaringBitmap, currently only int or bigint are supported.
  • Top N - For example, with this measure type, user can easily get specified numbers of top sellers/buyers etc.
    技术分享

CUBE ACTIONS

  • BUILD - Given an interval of partition column, this action is to build a new cube segment.
  • REFRESH - This action will rebuilt cube segment in some partition period, which is used in case of source table increasing.
  • MERGE - This action will merge multiple continuous cube segments into single one. This can be automated with auto-merge settings in cube descriptor.
  • PURGE - Clear segments under a cube instance. This will only update metadata, and won’t delete cube data from HBase.
    技术分享

JOB STATUS

  • NEW - This denotes one job has been just created.
  • PENDING - This denotes one job is paused by job scheduler and waiting for resources.
  • RUNNING - This denotes one job is running in progress.
  • FINISHED - This denotes one job is successfully finished.
  • ERROR - This denotes one job is aborted with errors.
  • DISCARDED - This denotes one job is cancelled by end users.
    技术分享

JOB ACTION

  • RESUME - Once a job in ERROR status, this action will try to restore it from latest successful point.
  • DISCARD - No matter status of a job is, user can end it and release resources with DISCARD action.
    技术分享

Kylin-基本知识

标签:

原文地址:http://blog.csdn.net/jianghuxiaojin/article/details/51508662

(0)
(0)
   
举报
评论 一句话评论(0
登录后才能评论!
© 2014 mamicode.com 版权所有  联系我们:gaon5@hotmail.com
迷上了代码!