Timetombs

泛义的工具是文明的基础,而确指的工具却是愚人的器物

66h / 117a
,更新于 2024-03-10T17:38:47Z+08:00 by   63c7f91

[HBase] Region Split Policy

版权声明 - CC BY-NC-SA 4.0

1 概述

待完善。。。

2 配置方式

2.1 RegionServer

不推荐:原因是不够灵活。

hbase.regionserver.region.split.policy1。默认配置org.apache.hadoop.hbase.regionserver.SteppingSplitPolicy2

<property>
  <name>hbase.regionserver.region.split.policy</name>
  <value>org.apache.hadoop.hbase.regionserver.SteppingSplitPolicy</value>
</property>

2.2 Table

推荐:原因是表范围内的配置,每个表可以独立设置适合的策略。

方式1(不推荐)

HTableDescriptor tableDesc = new HTableDescriptor("test");
tableDesc.setValue(HTableDescriptor.SPLIT_POLICY, ConstantSizeRegionSplitPolicy.class.getName());
tableDesc.addFamily(new HColumnDescriptor(Bytes.toBytes("cf1")));
admin.createTable(tableDesc);

方式2(推荐)

# old
hbase> create 'blog', {METADATA => {'SPLIT_POLICY' => 'org.apache.hadoop.hbase.regionserver.ConstantSizeRegionSplitPolicy'}},{NAME => 'cf'}

# new
hbase> create 'blog',{CONFIGURATION => {'hbase.regionserver.region.split_restriction.type' => 'KeyPrefix','hbase.regionserver.region.split_restriction.prefix_length' => '1'}},{NAME=>'cf'}

3 RegionSplitPolicy

RegionSplitPolicy的继承关系如下:

  1. RegionSplitPolicy3
    1. DisabledRegionSplitPolicy4
    2. ConstantSizeRegionSplitPolicy5
    3. IncreasingToUpperBoundRegionSplitPolicy6
      1. SteppingSplitPolicy2
      2. KeyPrefixRegionSplitPolicy7
      3. DelimitedKeyPrefixRegionSplitPolicy8
      4. BusyRegionSplitPolicy9

4 RegionSplitRestriction

KeyPrefixRegionSplitPolicy7DelimitedKeyPrefixRegionSplitPolicy8在源码中已被标记为@Deprecated

/**
 * A custom RegionSplitPolicy implementing a SplitPolicy that groups
 * rows by a prefix of the row-key
 *
 * This ensures that a region is not split "inside" a prefix of a row key.
 * I.e. rows can be co-located in a region by their prefix.
 *
 * @deprecated since 2.4.3 and will be removed in 4.0.0. Use {@link RegionSplitRestriction},
 *   instead.
 */
@Deprecated
@InterfaceAudience.Private
public class KeyPrefixRegionSplitPolicy extends IncreasingToUpperBoundRegionSplitPolicy {
  // some code
}

推荐使用RegionSplitRestriction10代替。目前包含KeyPrefixRegionSplitRestriction11DelimitedKeyPrefixRegionSplitRestriction12两种策略。

/**
 * A split restriction that restricts the pattern of the split point.
 * <p>
 * The difference between {@link RegionSplitPolicy} and RegionSplitRestriction is that
 * RegionSplitRestriction defines how to split while {@link RegionSplitPolicy} defines when we need
 * to split.
 * <p>
 * We can specify a split restriction, "KeyPrefix" or "DelimitedKeyPrefix", to a table with the
 * "hbase.regionserver.region.split_restriction.type" property. The "KeyPrefix" split restriction
 * groups rows by a prefix of the row-key. And the "DelimitedKeyPrefix" split restriction groups
 * rows by a prefix of the row-key with a delimiter.
 *
 * For example:
 * <pre>
 * <code>
 * # Create a table with a "KeyPrefix" split restriction, where the prefix length is 2 bytes
 * hbase> create 'tbl1', 'fam',
 *   {CONFIGURATION => {'hbase.regionserver.region.split_restriction.type' => 'KeyPrefix',
 *                      'hbase.regionserver.region.split_restriction.prefix_length' => '2'}}
 *
 * # Create a table with a "DelimitedKeyPrefix" split restriction, where the delimiter is a comma
 * hbase> create 'tbl2', 'fam',
 *   {CONFIGURATION => {'hbase.regionserver.region.split_restriction.type' => 'DelimitedKeyPrefix',
 *                      'hbase.regionserver.region.split_restriction.delimiter' => ','}}
 * </code>
 * </pre>
 *
 * Instead of specifying a split restriction to a table directly, we can also set the properties
 * in hbase-site.xml. In this case, the specified split restriction is applied for all the tables.
 * <p>
 * Note that the split restriction is also applied to a user-specified split point so that we don't
 * allow users to break the restriction.
 *
 * @see NoRegionSplitRestriction
 * @see KeyPrefixRegionSplitRestriction
 * @see DelimitedKeyPrefixRegionSplitRestriction
 */
@InterfaceAudience.Private
public abstract class RegionSplitRestriction {
  // some code
}

2 参考

HBase原理(hbasefly.com)
HBase原理 – 所有Region切分的细节都在这里了(hbasefly.com)


  1. regionserver-config : https://hbase.apache.org/2.3/book.html#hbase.regionserver.region.split.policy ↩︎

  2. SteppingSplitPolicy Source Code: https://github.com/apache/hbase/blob/rel/2.4.9/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/SteppingSplitPolicy.java ↩︎ ↩︎

  3. RegionSplitPolicy Source Code: https://github.com/apache/hbase/blob/rel/2.4.9/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/RegionSplitPolicy.java ↩︎

  4. DisabledRegionSplitPolicy Source Code: https://github.com/apache/hbase/blob/rel/2.4.9/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/DisabledRegionSplitPolicy.java ↩︎

  5. ConstantSizeRegionSplitPolicy Source Code: https://github.com/apache/hbase/blob/rel/2.4.9/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/ConstantSizeRegionSplitPolicy.java ↩︎

  6. IncreasingToUpperBoundRegionSplitPolicy Source Code: https://github.com/apache/hbase/blob/rel/2.4.9/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/IncreasingToUpperBoundRegionSplitPolicy.java ↩︎

  7. KeyPrefixRegionSplitPolicy Source Code: https://github.com/apache/hbase/blob/rel/2.4.9/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/KeyPrefixRegionSplitPolicy.java ↩︎ ↩︎

  8. DelimitedKeyPrefixRegionSplitPolicy Source Code: https://github.com/apache/hbase/blob/rel/2.4.9/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/DelimitedKeyPrefixRegionSplitPolicy.java ↩︎ ↩︎

  9. BusyRegionSplitPolicy Source Code: https://github.com/apache/hbase/blob/rel/2.4.9/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/BusyRegionSplitPolicy.java ↩︎

  10. RegionSplitRestriction Source Code: https://github.com/apache/hbase/blob/rel/2.4.9/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/RegionSplitRestriction.java ↩︎

  11. KeyPrefixRegionSplitRestriction Source Code: https://github.com/apache/hbase/blob/rel/2.4.9/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/KeyPrefixRegionSplitRestriction.java ↩︎

  12. DelimitedKeyPrefixRegionSplitRestriction Source Code: https://github.com/apache/hbase/blob/rel/2.4.9/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/DelimitedKeyPrefixRegionSplitRestriction.java ↩︎

上一篇 : [HBase] KeyValue
下一篇 : [HBase] Best Practice