码迷,mamicode.com
首页 > 其他好文 > 详细

Tcmalloc SetMemoryReleaseRate(double) 和 ReleaseFreeMemory()浅析

时间:2021-06-23 16:41:42      阅读:0      评论:0      收藏:0      [点我收藏+]

标签:big   asi   tst   效率   ref   toc   extra   rtu   释放   

问题

Tcmalloc 由于使用了精心设计的 cache,进而大大提高了malloc 和 free 的效率,但由之而来的是 cache 大小的难以把控,容易出现 cache 占用过大,进而 OOM 的问题,tcmalloc 提供了两个方案解决这个问题

  • 展现 tcmalloc 维护的内存状态解决

    //virtual void GetStats(char* buffer, int buffer_length);
    MallocExtension::instance()->GetStats(buf, length);
    

解决

1.ReleaseFreeMemory

作用

  • 一次性,全部释放 tcmalloc 的 cache 内存

函数实现

// Same as ReleaseToSystem() but release as much memory as possible.
virtual void ReleaseFreeMemory();
// 实现
void MallocExtension::ReleaseFreeMemory() {
  ReleaseToSystem(static_cast<size_t>(-1));   // SIZE_T_MAX
}
//加锁释放,尽可能多的cache
virtual void ReleaseToSystem(size_t num_bytes) {
  SpinLockHolder h(Static::pageheap_lock());
  if (num_bytes <= extra_bytes_released_) {
    // We released too much on a prior call, so don‘t release any
    // more this time.
    extra_bytes_released_ = extra_bytes_released_ - num_bytes;
    return;
  }
  num_bytes = num_bytes - extra_bytes_released_;
  // num_bytes might be less than one page.  If we pass zero to
  // ReleaseAtLeastNPages, it won‘t do anything, so we release a whole
  // page now and let extra_bytes_released_ smooth it out over time.
  Length num_pages = max<Length>(num_bytes >> kPageShift, 1);
  size_t bytes_released = Static::pageheap()->ReleaseAtLeastNPages(
    num_pages) << kPageShift;
  if (bytes_released > num_bytes) {
    extra_bytes_released_ = bytes_released - num_bytes;
  } else {
    // The PageHeap wasn‘t able to release num_bytes.  Don‘t try to
    // compensate with a big release next time.  Specifically,
    // ReleaseFreeMemory() calls ReleaseToSystem(LONG_MAX).
    extra_bytes_released_ = 0;
  }
}

劣势

  • 对 page heap 加锁,cache 较大时,占用锁时间较长,可能阻塞较长时间
  • 对 tcmalloc 源码的阅读,会在之后的博客中分享

2.SetMemoryReleaseRate

作用

  • 控制释放 cache 给 OS的速度

函数实现

// Sets the rate at which we release unused memory to the system.
// Zero means we never release memory back to the system.  Increase
// this flag to return memory faster; decrease it to return memory
// slower.  Reasonable rates are in the range [0,10].  (Currently
// only implemented in tcmalloc).
virtual void SetMemoryReleaseRate(double rate);
// 实现
virtual void SetMemoryReleaseRate(double rate) {
  FLAGS_tcmalloc_release_rate = rate;
}

rate的含义

  • 官方建议 [0 , 10]
  • 0 代表不归还,数字越大归还越快,但最大是多少呢?以及是否是线性的呢?
// Incrementally release some memory to the system.
// IncrementalScavenge(n) is called whenever n pages are freed.
// 大意是,释放 n 个 pages 时, 通过 IncrementalScavenge 归还一定大小 cache 给 OS
void IncrementalScavenge(Length n);
// 实现
void PageHeap::IncrementalScavenge(Length n) {
  // Fast path; not yet time to release memory
  // 需要等待的 pages 数,每释放 scavenge_counter_ 个 pages 归还一次给OS
  scavenge_counter_ -= n;
  if (scavenge_counter_ >= 0) return;  // Not yet time to scavenge
  const double rate = FLAGS_tcmalloc_release_rate;
  if (rate <= 1e-6) {
    // 0 不释放
    scavenge_counter_ = kDefaultReleaseDelay;
    return;
  }
  ++stats_.scavenge_count;
  Length released_pages = ReleaseAtLeastNPages(1);
  if (released_pages == 0) {
    scavenge_counter_ = kDefaultReleaseDelay;
  } else {
    // Compute how long to wait until we return memory.
    // FLAGS_tcmalloc_release_rate==1 means wait for 1000 pages
    // after releasing one page.
    // 反比关系,1000 / rate 为要等待的 pages,所以 rate 可以理解为 [1, 1000]
    // 1 时等待1000个pages, 1000时等待1个page
    const double mult = 1000.0 / rate;
    double wait = mult * static_cast<double>(released_pages);
    if (wait > kMaxReleaseDelay) {
      // Avoid overflow and bound to reasonable range.
      wait = kMaxReleaseDelay;
    }
    scavenge_counter_ = static_cast<int64_t>(wait);
  }
}
  • 由此可见,建议在 [0, 1000] 配置
  • 1 时归还率为 1 / 1000, 1000时归还率为 50%
  • 数字越大越好,非线性反比关系

Tcmalloc SetMemoryReleaseRate(double) 和 ReleaseFreeMemory()浅析

标签:big   asi   tst   效率   ref   toc   extra   rtu   释放   

原文地址:https://www.cnblogs.com/jamgun/p/14920755.html

(0)
(0)
   
举报
评论 一句话评论(0
登录后才能评论!
© 2014 mamicode.com 版权所有  联系我们:gaon5@hotmail.com
迷上了代码!