conflict cache size.
The current policy is:
A 'more or less' LRU policy is implemented by giving
to each OldRef a generation nr in which it was last touched.
A new generation is created every 50000 new access.
GC is done when the nr of OldRef reaches --conflict-cache-size.
The GC consists in removing enough generations to free
half of the entries.
After GC of OldRef, the RCEC (Ref Counted Exe Contexts)
not referenced anymore are GC-ed.
The new policy is:
An exact LRU policy is implemented using a doubly linked list
of OldRef.
When reaching --conflict-cache-size, the LRU entry is re-used.
The not referenced RCEC are GC-ed when less than 75% of the RCEC
are referenced, and the nr of RCEC is 'big' (at least half the
size of the contextTab, and at least the max nr of RCEC reached
previously).
(note: tried to directly recover a unref'ed RCEC when recovering
the LRU oldref, but that gives a lot of re-creation of RCEC).
new policy has the following advantages/disadvantages:
1. It is faster (at least for big applications)
On a firefox startup/exit, we gain about 1m30 second on 11m.
Similar 5..10% speed up encountered on other big applications
or on the new perf/memrw test.
The speed increase depends on the amount of memory
touched by the application. For applications with a
working set fitting in conflict-cache-size, the new policy
might be marginally slower than previous policy on platforms
having a small cache : the current policy only sets a generation
nr when an address is re-accessed, while the new policy
has to unchain and rechain the OldRef access in the LRU
doubly linked list.
2. It uses less memory (at least for big applications)
Firefox startup/exit "core" arena max use decreases from
1175MB mmap-ed/1060MB alloc-ed
to
994MB mmap-ed/913MB alloc-ed
The decrease in memory is the result of having a lot less RCEC:
The current policy let the nr of RCEC grow till the conflict
cache size is GC-ed.
The new policy limits the nr of RCEC to 133% of the RCEC
really referenced. So, we end up with a max nr of RCEC
a lot smaller with the new policy : max RCEC 191000
versus 1317000, for a total nr of discard RCEC operations
almost the same: 33M versus 32M.
Also, the current policy allocates a big temporary array
to do the GC of OldRef.
With the new policy, size of an OldRef increases because
we need 2 pointers for the LRU doubly linked list, and
we need the accessed address.
In total, the OldRef increase is limited to one Word,
as we do not need anymore the gen, and the 'magic'
for sanity check was removed (the check somewhat
becomes less needed, because an OldRef is never freed
anymore. Also, we do a new cross-check between
the ga in the OldRef and the sparseWA key).
For applications using small memory and having
a small nr of different stack traces accessing memory,
the new policy causes an increase in memory (one Word
per OldRef).
3. Functionally, the new policy gives better past information:
once the steady state is reached (i.e. the conflict cache
is full), the new policy has always --conflict-cache-size
entries of past information.
The current policy has a nr of past information varying
between --conflict-cache-size/2 and --conflict-cache-size
(so in average, 75% of conflict-cache-size).
4. The new code is a little bit smaller/simpler:
The generation based GC is replaced by a simpler LRU policy.
So, in summary, this patch should allow big applications
to use less cpu/memory, while having very little
or no impact on memory/cpu of small applications.
Note that the OldRef data structure LRU policy
is not really explicitely tested by a regtest.
Not easy at first sight to make such a test portable
between platforms/OS/compilers/....
git-svn-id: svn://svn.valgrind.org/valgrind/trunk@15119
* modified perf/heap.c so that it optionally creates a partially defined bytes every N bytes
* created perf/heap_pdb4.vgperf calling heap 4 (so one byte on 4 is PDB in what heap allocates).
before/after pool alloc, here are the performances on a ppc64. So, it looks like
pool alloc also significantly improves the speed of PDB handling.
perl perf/vg_perf --vg=../pool_alloc --vg=../before_pool_trunk_untouched --reps=5 perf/heap_pdb4.vgperf
-- heap_pdb4 --
heap_pdb4 pool_alloc:0.41s no: 3.0s ( 7.3x, -----) me:11.9s (29.0x, -----)
heap_pdb4 before_pool_trunk_untouched:0.41s no: 3.1s ( 7.6x, -3.7%) me:16.9s (41.1x,-41.7%)
git-svn-id: svn://svn.valgrind.org/valgrind/trunk@12342
* move memcheck/perf/many-loss-records test to perf directory
massif/perf/many-xpts test to perf directory
* modified many-loss-records.vgperf and many-xpts.vgperf,
so as to have tool specific options prefixed with their tool
* remove directory memcheck/perf and massif/perf (containing no test anymore)
git-svn-id: svn://svn.valgrind.org/valgrind/trunk@12315
following improvements:
- Arch/OS/platform-specific files are now included/excluded via the
preprocessor, rather than via the build system. This is more consistent
(we use the pre-processor for small arch/OS/platform-specific chunks
within files) and makes the build system much simpler, as the sources for
all programs are the same on all platforms.
- Vast amounts of cut+paste Makefile.am code has been factored out. If a
new platform is implemented, you need to add 11 extra Makefile.am lines.
Previously it was over 100 lines.
- Vex has been autotoolised. Dependency checking now works in Vex (no more
incomplete builds). Parallel builds now also work. --with-vex no longer
works; it's little use and a pain to support. VEX/Makefile is still in
the Vex repository and gets overwritten at configure-time; it should
probably be renamed Makefile-gcc to avoid possible problems, such as
accidentally committing a generated Makefile. There's a bunch of hacky
copying to deal with the fact that autotools don't handle same-named files
in different directories. Julian plans to rename the files to avoid this
problem.
- Various small Makefile.am things have been made more standard automake
style, eg. the use of pkginclude/pkglib prefixes instead of rolling our
own.
- The existing five top-level Makefile.am include files have been
consolidated into three.
- Most Makefile.am files now are structured more clearly, with comment
headers separating sections, declarations relating to the same things next
to each other, better spacing and layout, etc.
- Removed the unused exp-ptrcheck/tests/x86 directory.
- Renamed some XML files.
- Factored out some duplicated dSYM handling code.
- Split auxprogs/ into auxprogs/ and mpi/, which allowed the resulting
Makefile.am files to be much more standard.
- Cleaned up m_coredump by merging a bunch of files that had been
overzealously separated.
The net result is 630 fewer lines of Makefile.am code, or 897 if you exclude
the added Makefile.vex.am, or 997 once the hacky file copying for Vex is
removed. And the build system is much simpler.
git-svn-id: svn://svn.valgrind.org/valgrind/trunk@10364
- Created Makefile.tool-tests.am, put standard AM_CFLAGS et al for tests in
it.
- A number of tests are shared between Helgrind and DRD. They used to be
built in both directories. Now they are only built in helgrind/tests/,
and the DRD .vgtest files just point to the executable in helgrind/tests/.
Most of these (about 30) had the source files in helgrind/tests/; I moved
the three that were in drd/tests/ into helgrind/tests/ for consistency.
- Fixed rwlock_test, which was failing to run due to a wrong name in the
.vgtest file.
- Removed remnants of unused 'hello' test for Memcheck.
- Avoided redundant flag specification in various places, esp.
memcheck/tests/Makefile.am.
- Removed unnecessary _AIX guards in some Linux-only tests.
git-svn-id: svn://svn.valgrind.org/valgrind/trunk@9202
distinction between primary and secondary build targets, and (2) make
it independent of the default behaviour of gcc (iow, what gcc does
when you specify neither -m32 nor -m64).
As a result, an out-of-the-box build on ppc64-linux now builds a
system which is basically for 64-bit PowerPC, but also has the ability
to run 32-bit ppc-linux binaries (exactly the same arrangement as you
get when building on amd64-linux).
There are various twists and turns. multiple-architectures.txt is
updated all the gory details.
This will break amd64 builds until such time as
<tool>/tests/{amd64,x86}/Makefile.am are fixed up (shortly).
git-svn-id: svn://svn.valgrind.org/valgrind/trunk@5493
program more uniformly measures the speed of FP +/-/* load/store
across different architectures.
git-svn-id: svn://svn.valgrind.org/valgrind/trunk@5455
x86 isms in tinycc.c, (2) giving it a fixed, preprocessed input file
to chew on, and (3) just compiling to a .o with no attempt to link.
git-svn-id: svn://svn.valgrind.org/valgrind/trunk@5439
(use "make perf" to run) that executes test programs and times their
slowdowns under various tools. It works a lot like the vg_regtest script.
It's a bit rough around the edges -- eg. you can't currently directly
compare two different versions of Valgrind, which would be useful -- but it
is a good start.
There are currently two test programs in perf/. More will be added as time
goes on. This stuff will be built on so that performance changes can be
tracked over time.
git-svn-id: svn://svn.valgrind.org/valgrind/trunk@5323
