【LevelDB源码阅读】Arena
是什么
内存分配管理器,主要为skiplist即Memtable服务而不是整个项目。申请内存时,将申请到的内存直接放入vector中,在Arena的生命周期结束后,统一释放掉所有申请的内存,内部结构如下图:
为什么要用
- 避免内存碎片,skiplist里面记录的都是用户传进来的key/value,这些字符串有长有短,放到内存中很容易导致内存碎片
学到什么
- 与(&)操作进行内存对齐
- atomic保证原子性
- 使用delete避免拷贝和赋值操作
源码分析
首先看一下内部成员
// Allocation state char *alloc_ptr_; // 内存偏移量指针,指向未使用内存的首地址 size_t alloc_bytes_remaining_; // 剩下内存数 // Array of new[] allocated memory blocks std::vector<char*> blocks_; // 存储每次分配的内存指针 // Total memory usage of the arena. // // TODO(costan): This member is accessed via atomics, but the others are // accessed without any locking. Is this OK? std::atomic<size_t> memory_usage_;
构造函数和析构函数
Arena::Arena() // vector调用默认构造函数初始化 : alloc_ptr_(nullptr), alloc_bytes_remaining_(0), memory_usage_(0) {} Arena::~Arena() { for (size_t i = 0; i < blocks_.size(); i++) { delete[] blocks_[i]; } }
拷贝构造和拷贝赋值
Arena(const Arena&) = delete; Arena& operator=(const Arena&) = delete;
主要对外接口
// Return a pointer to a newly allocated memory block of "bytes" bytes. char* Allocate(size_t bytes); // Allocate memory with the normal alignment guarantees provided by malloc. char* AllocateAligned(size_t bytes);
具体实现如下:
inline char* Arena::Allocate(size_t bytes) { // The semantics of what to return are a bit messy if we allow // 0-byte allocations, so we disallow them here (we don‘t need // them for our internal use). assert(bytes > 0); if (bytes <= alloc_bytes_remaining_) { // 需要内存小于剩余内存,直接分配 char *result = alloc_ptr_; // 保存指针,用于返回 alloc_ptr_ += bytes; alloc_bytes_remaining_ -= bytes; return result; } return AllocateFallback(bytes); // 需要内存大于剩余内存 }
需要内存大于剩余内存时,调用AllocateFallback()分配内存:
- 如果需要内存大于4096/4,则直接分配一块大小为bytes内存,避免每次剩余内存不可用,造成浪费。
- 否则,重新分配一个内存块(默认大小4096)用于存储数据,虽然浪费了当前内存块的剩余内存,但当下次再分配小内存时,可以直接使用,减少内存分配次数。
具体实现如下:
char* Arena::AllocateFallback(size_t bytes) { if (bytes > kBlockSize / 4) { // Object is more than a quarter of our block size. Allocate it separately // to avoid wasting too much space in leftover bytes. char *result = AllocateNewBlock(bytes); return result; } // We waste the remaining space in the current block. alloc_ptr_ = AllocateNewBlock(kBlockSize); alloc_bytes_remaining_ = kBlockSize; char *result = alloc_ptr_; alloc_ptr_ += bytes; alloc_bytes_remaining_ -= bytes; return result; }
其中分配新内存块实现如下:
memory_order_relaxed:针对只要求原子操作,除此之外不需要其它同步保证,计数器是一种典型应用场景。
char* Arena::AllocateNewBlock(size_t block_bytes) { char *result = new char[block_bytes]; blocks_.push_back(result); // 申请块放入vector,以便析构函数释放内存 memory_usage_.fetch_add(block_bytes + sizeof(char*), std::memory_order_relaxed); return result; }
对齐内存分配
char* Arena::AllocateAligned(size_t bytes) { const int align = (sizeof(void*) > 8) ? sizeof(void*) : 8; // 与系统相关,4或8 static_assert((align & (align - 1)) == 0, "Pointer size should be a power of 2"); // 确保是2的指数次方 size_t current_mod = reinterpret_cast<uintptr_t>(alloc_ptr_) & (align - 1); // 当前指针模对齐值 size_t slop = (current_mod == 0 ? 0 : align - current_mod); // 还差slop个字节对齐 size_t needed = bytes + slop; char *result; if (needed <= alloc_bytes_remaining_) { result = alloc_ptr_ + slop; // 对齐地址 alloc_ptr_ += needed; alloc_bytes_remaining_ -= needed; } else { // AllocateFallback always returned aligned memory result = AllocateFallback(bytes); } assert((reinterpret_cast<uintptr_t>(result) & (align - 1)) == 0); return result; }
读取内存使用
// Returns an estimate of the total memory usage of data allocated // by the arena. size_t MemoryUsage() const { return memory_usage_.load(std::memory_order_relaxed); }
相关推荐
guangyacyb 2020-06-14
江沐涵 2020-06-09
tmshasha 2019-12-15
Finnnnnnn 2019-11-05
longzuo 2019-07-09
longzuo 2012-04-03
tmshasha 2019-07-09
adwen00 2019-07-01
tmshasha 2015-03-24
江沐涵 2019-06-28
tmshasha 2019-06-28
guangyacyb 2019-06-26
longzuo 2019-05-27
江沐涵 2018-07-24
tmshasha 2019-04-29
YENCSDN 2017-12-21
guangyacyb 2011-08-09