mirror of
https://github.com/Zenithsiz/ftmemsim-valgrind.git
synced 2026-02-04 10:21:20 +00:00
53df23f0a6
include/pub_tool_deduppoolalloc.h coregrind/pub_core_deduppoolalloc.h coregrind/m_deduppoolalloc.c and uses it (currently only) for the strings in m_debuginfo/storage.c The idea is that such ddup pool allocator will also be used for other highly duplicated information (e.g. the DiCFSI information), where significant gains can also be achieved. The dedup pool for strings also decreases significantly the memory needed by the read inline information (patch still to be committed, see bug 278972). When testing with a big executable (tacot_process), this reduces the size of the dinfo arena from trunk: 158941184/109760512 max/curr mmap'd, 156775944/107882728 max/curr, to ddup: 157892608/106614784 max/curr mmap'd, 156362160/101414712 max/curr (so 3Mb less mmap-ed once debug info is read, 1Mb less mmap-ed in peak, 6Mb less allocated once debug info is read). This is all gained due to the string which changes from: trunk: 17,434,704 in 266: di.storage.addStr.1 to ddup: 10,966,608 in 750: di.storage.addStr.1 (6.5Mb less memory used by strings) The gain in mmap-ed memory is smaller due to fragmentation. Probably one could decrease the fragmentation by using bigger size for the dedup pool, but then we would lose memory on the last allocated pool (and for small libraries, we often do not use much of a big pool block). Solution might be to increase the pool size but have a "shrink_block" operation. To be looked at in the future. In terms of performance, startup of a big executable (on an old pentium) is not influenced significantly (something like 0.1 seconds on 15 seconds startup for a big executable, on a slow pentium). The dedup pool uses a hash table. The hash function used currently is the VG_(adler32) check sum. It is reported (and visible also here) that this checksum is not a very good hash function (many collisions). To have statistics about collisions, use --stats -v -v -v As an example of the collisions, on the strings in debug info of memcheck tool on x86, one obtain: --4789-- dedupPA:di.storage.addStr.1 9983 allocs (8174 uniq) 11 pools (4820 bytes free in last pool) --4789-- nr occurences of chains of len N, N-plicated keys, N-plicated elts --4789-- N: 0 : nr chain 6975, nr keys 0, nr elts 0 --4789-- N: 1 : nr chain 3670, nr keys 6410, nr elts 8174 --4789-- N: 2 : nr chain 1070, nr keys 226, nr elts 0 --4789-- N: 3 : nr chain 304, nr keys 100, nr elts 0 --4789-- N: 4 : nr chain 104, nr keys 84, nr elts 0 --4789-- N: 5 : nr chain 72, nr keys 42, nr elts 0 --4789-- N: 6 : nr chain 44, nr keys 34, nr elts 0 --4789-- N: 7 : nr chain 18, nr keys 13, nr elts 0 --4789-- N: 8 : nr chain 17, nr keys 8, nr elts 0 --4789-- N: 9 : nr chain 4, nr keys 6, nr elts 0 --4789-- N:10 : nr chain 9, nr keys 4, nr elts 0 --4789-- N:11 : nr chain 1, nr keys 0, nr elts 0 --4789-- N:13 : nr chain 1, nr keys 1, nr elts 0 --4789-- total nr of unique chains: 12289, keys 6928, elts 8174 which shows that on 8174 different strings, we have only 6410 strings which have a unique hash value. As other examples, N:13 line shows we have 13 strings mapping to the same key. N:14 line shows we have 4 groups of 10 strings mapping to the same key, etc. So, adler32 is definitely a bad hash function. Trials have been done with another hash function, giving a much lower collision rate. So, a better (but still fast) hash function would probably be beneficial. To be looked at ... git-svn-id: svn://svn.valgrind.org/valgrind/trunk@14029
96 lines
3.9 KiB
C
96 lines
3.9 KiB
C
|
|
/*--------------------------------------------------------------------*/
|
|
/*--- A simple pool (memory) allocator. pub_tool_poolalloc.h ---*/
|
|
/*--------------------------------------------------------------------*/
|
|
|
|
/*
|
|
This file is part of Valgrind, a dynamic binary instrumentation
|
|
framework.
|
|
|
|
Copyright (C) 2011-2013 OpenWorks LLP info@open-works.co.uk,
|
|
Philippe Waroquiers philippe.waroquiers@skynet.be
|
|
|
|
This program is free software; you can redistribute it and/or
|
|
modify it under the terms of the GNU General Public License as
|
|
published by the Free Software Foundation; either version 2 of the
|
|
License, or (at your option) any later version.
|
|
|
|
This program is distributed in the hope that it will be useful, but
|
|
WITHOUT ANY WARRANTY; without even the implied warranty of
|
|
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
|
|
General Public License for more details.
|
|
|
|
You should have received a copy of the GNU General Public License
|
|
along with this program; if not, write to the Free Software
|
|
Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA
|
|
02111-1307, USA.
|
|
|
|
The GNU General Public License is contained in the file COPYING.
|
|
*/
|
|
|
|
#ifndef __PUB_TOOL_POOLALLOC_H
|
|
#define __PUB_TOOL_POOLALLOC_H
|
|
|
|
#include "pub_tool_basics.h" // UWord
|
|
|
|
//--------------------------------------------------------------------
|
|
// PURPOSE: Provides efficient allocation and free of elements of
|
|
// the same size.
|
|
// This pool allocator manages elements alloc/free by allocating
|
|
// "pools" of many elements from a lower level allocator (typically
|
|
// pub_tool_mallocfree.h).
|
|
// Single elements can then be allocated and released from these pools.
|
|
// A pool allocator is faster and has less memory overhead than
|
|
// calling directly pub_tool_mallocfree.h
|
|
// Note: the pools of elements are not freed, even if all the
|
|
// single elements have been freed. The only way to free the underlying
|
|
// pools of elements is to delete the pool allocator.
|
|
//--------------------------------------------------------------------
|
|
|
|
|
|
typedef struct _PoolAlloc PoolAlloc;
|
|
|
|
/* Create new PoolAlloc, using given allocation and free function, and
|
|
for elements of the specified size. Alloc fn must not fail (that
|
|
is, if it returns it must have succeeded.) */
|
|
extern PoolAlloc* VG_(newPA) ( UWord elemSzB,
|
|
UWord nPerPool,
|
|
void* (*alloc)(const HChar*, SizeT),
|
|
const HChar* cc,
|
|
void (*free_fn)(void*) );
|
|
|
|
|
|
/* Free all memory associated with a PoolAlloc. */
|
|
extern void VG_(deletePA) ( PoolAlloc* pa);
|
|
|
|
/* Allocates an element from pa. */
|
|
extern void* VG_(allocEltPA) ( PoolAlloc* pa);
|
|
|
|
/* Free element of pa. */
|
|
extern void VG_(freeEltPA) ( PoolAlloc* pa, void* p);
|
|
|
|
/* A pool allocator can be shared between multiple data structures.
|
|
For example, multiple OSet* can allocate/free nodes from the same
|
|
pool allocator.
|
|
The Pool Allocator provides support to use a ref counter
|
|
to detect a pool allocator is not needed anymore.
|
|
It is the caller responsibility to call VG_(addRefPA) for
|
|
each new reference to a pool and VG_(releasePA) when such a reference
|
|
disappears.
|
|
VG_(releasePA) will automatically call VG_(deletePA)
|
|
to delete the PA when the ref counter drops to 0. */
|
|
|
|
// VG_(addRefPA) indicates there is a new reference to pa.
|
|
extern void VG_(addRefPA) ( PoolAlloc* pa);
|
|
|
|
// VG_(releasePA) decrements the pa reference count and deletes the pa if that
|
|
// reference count has dropped to zero. Returns the new value of the reference
|
|
// count.
|
|
extern UWord VG_(releasePA) ( PoolAlloc* pa);
|
|
|
|
#endif // __PUB_TOOL_POOLALLOC_
|
|
|
|
/*--------------------------------------------------------------------*/
|
|
/*--- end pub_tool_poolalloc.h ---*/
|
|
/*--------------------------------------------------------------------*/
|