hbase定时memflush PeriodicMemstoreFlusher
PeriodicMemstoreFlusher类 定时判断是否触发memflush,
判断间隔hbase.server.thread.wakefrequency 默认10S
@Override protected void chore() { for (HRegion r : this.server.onlineRegions.values()) { if (r == null) continue; if (r.shouldFlush()) { FlushRequester requester = server.getFlushRequester(); if (requester != null) { long randomDelay = rand.nextInt(RANGE_OF_DELAY) + MIN_DELAY_TIME; LOG.info(getName() + " requesting flush for region " + r.getRegionNameAsString() + " after a delay of " + randomDelay); //Throttle the flushes by putting a delay. If we don't throttle, and there //is a balanced write-load on the regions in a table, we might end up //overwhelming the filesystem with too many flushes at once. requester.requestDelayedFlush(r, randomDelay); } } } }
触发条件should Flush方法
boolean shouldFlush() { if(this.completeSequenceId + this.flushPerChanges < this.sequenceId.get()) {//防止memstore变化太多 return true; } if (flushCheckInterval <= 0) { //disabled return false; } long now = EnvironmentEdgeManager.currentTimeMillis(); //if we flushed in the recent past, we don't need to do again now if ((now - getLastFlushTime() < flushCheckInterval)) {//每次flush间隔时间,hbase.regionserver.optionalcacheflushinterval默认为1小时 return false; } //since we didn't flush in the recent past, flush now if certain conditions //are met. Return true on first such memstore hit. for (Store s : this.getStores().values()) { if (s.timeOfOldestEdit() < now - flushCheckInterval) { // oldest edit of store , one hour ago , now 有store在一小时钱修改过 // we have an old enough edit in the memstore, flush return true; } } return false; }
1.依照sequeceid,判断memstore没有太多flush,进行flush
2.比较每次flush的间隔时间,没到时间,不进行flush(单从时间上看,memstore够老)
3.超过间隔时间,并且有个store的edit超过间隔时间(从用户修改上看,memstore够老)
相关推荐
晨曦之星 2020-08-14
lwb 2020-07-26
eternityzzy 2020-07-19
大而话之BigData 2020-06-16
ITwangnengjie 2020-06-14
gengwx00 2020-06-11
大而话之BigData 2020-06-10
鲸鱼写程序 2020-06-08
needyit 2020-06-04
strongyoung 2020-06-04
WeiHHH 2020-05-30
ITwangnengjie 2020-05-09
gengwx00 2020-05-08
gengwx00 2020-05-09
大而话之BigData 2020-05-06
Buerzhu 2020-05-01
gengwx00 2020-04-30