Necessary changes to support nanoMIPS on Linux.
Part 4/4 - Other changes (mainly include/*)
Patch by Aleksandar Rikalo, Dimitrije Nikolic, Tamara Vlahovic,
Nikola Milutinovic and Aleksandra Karadzic.
Related KDE issue: #400872.
This commit is the bulk of the new code.
There is however no functional impact yet : the new modules are not used by anybody.
git-svn-id: svn://svn.valgrind.org/valgrind/trunk@16123
only allowed to be called in certain contexts which is
enforced at runtime.
Change callgrind accordingly.
New header file pub_tool_transtab.h added.
git-svn-id: svn://svn.valgrind.org/valgrind/trunk@14867
to make it available to tools. This allows to remove quite a bit of
ifdeffery from memcheck's mc_machine.c
git-svn-id: svn://svn.valgrind.org/valgrind/trunk@14601
include/pub_tool_deduppoolalloc.h
coregrind/pub_core_deduppoolalloc.h
coregrind/m_deduppoolalloc.c
and uses it (currently only) for the strings in m_debuginfo/storage.c
The idea is that such ddup pool allocator will also be used for other
highly duplicated information (e.g. the DiCFSI information), where
significant gains can also be achieved.
The dedup pool for strings also decreases significantly the memory
needed by the read inline information (patch still to be committed,
see bug 278972).
When testing with a big executable (tacot_process),
this reduces the size of the dinfo arena from
trunk: 158941184/109760512 max/curr mmap'd, 156775944/107882728 max/curr,
to
ddup: 157892608/106614784 max/curr mmap'd, 156362160/101414712 max/curr
(so 3Mb less mmap-ed once debug info is read, 1Mb less mmap-ed in peak,
6Mb less allocated once debug info is read).
This is all gained due to the string which changes from:
trunk: 17,434,704 in 266: di.storage.addStr.1
to
ddup: 10,966,608 in 750: di.storage.addStr.1
(6.5Mb less memory used by strings)
The gain in mmap-ed memory is smaller due to fragmentation.
Probably one could decrease the fragmentation by using bigger
size for the dedup pool, but then we would lose memory on the last
allocated pool (and for small libraries, we often do not use much
of a big pool block).
Solution might be to increase the pool size but have a "shrink_block"
operation. To be looked at in the future.
In terms of performance, startup of a big executable (on an old pentium)
is not influenced significantly (something like 0.1 seconds on 15 seconds
startup for a big executable, on a slow pentium).
The dedup pool uses a hash table. The hash function used currently
is the VG_(adler32) check sum. It is reported (and visible also here)
that this checksum is not a very good hash function (many collisions).
To have statistics about collisions, use --stats -v -v -v
As an example of the collisions, on the strings in debug info of memcheck tool on x86,
one obtain:
--4789-- dedupPA:di.storage.addStr.1 9983 allocs (8174 uniq) 11 pools (4820 bytes free in last pool)
--4789-- nr occurences of chains of len N, N-plicated keys, N-plicated elts
--4789-- N: 0 : nr chain 6975, nr keys 0, nr elts 0
--4789-- N: 1 : nr chain 3670, nr keys 6410, nr elts 8174
--4789-- N: 2 : nr chain 1070, nr keys 226, nr elts 0
--4789-- N: 3 : nr chain 304, nr keys 100, nr elts 0
--4789-- N: 4 : nr chain 104, nr keys 84, nr elts 0
--4789-- N: 5 : nr chain 72, nr keys 42, nr elts 0
--4789-- N: 6 : nr chain 44, nr keys 34, nr elts 0
--4789-- N: 7 : nr chain 18, nr keys 13, nr elts 0
--4789-- N: 8 : nr chain 17, nr keys 8, nr elts 0
--4789-- N: 9 : nr chain 4, nr keys 6, nr elts 0
--4789-- N:10 : nr chain 9, nr keys 4, nr elts 0
--4789-- N:11 : nr chain 1, nr keys 0, nr elts 0
--4789-- N:13 : nr chain 1, nr keys 1, nr elts 0
--4789-- total nr of unique chains: 12289, keys 6928, elts 8174
which shows that on 8174 different strings, we have only 6410 strings which have
a unique hash value. As other examples, N:13 line shows we have 13 strings
mapping to the same key. N:14 line shows we have 4 groups of 10 strings mapping to the
same key, etc.
So, adler32 is definitely a bad hash function.
Trials have been done with another hash function, giving a much lower
collision rate. So, a better (but still fast) hash function would probably
be beneficial. To be looked at ...
git-svn-id: svn://svn.valgrind.org/valgrind/trunk@14029
of memcheck and helgrind in a common module:
pub_tool_addrinfo.h pub_core_addrinfo.h m_addrinfo.c
At the same time, the factorised code is made usable by other
tools also (and is used by the gdbserver command 'v.info location'
which replaces the helgrind 'describe addr' introduced 1 week ago
and which is now callable by all tools).
The new address description code can describe more addresses
(e.g. for memcheck, if the block is not on the free list anymore,
but is in an arena free list, this will also be described).
Similarly, helgrind address description can now describe more addresses
when --read-var-info=no is given (e.g. global symbols are
described, or addresses on the stack are described as
being on the stack, freed blocks in the arena free list are
described, ...).
See e.g. the change in helgrind/tests/annotate_rwlock.stderr.exp
or locked_vs_unlocked2.stderr.exp
The patch touches many files, but is basically a lot of improvements
in helgrind output files.
The code changes are mostly refactorisation of existing code.
git-svn-id: svn://svn.valgrind.org/valgrind/trunk@13965
VALGRIND_DISABLE_ADDR_ERROR_REPORTING_IN_RANGE and
VALGRIND_ENABLE_ADDR_ERROR_REPORTING_IN_RANGE
and supporting machinery for managing whole-address-space sparse
mappings. n-i-bz. In support of
https://bugzilla.mozilla.org/show_bug.cgi?id=970643
git-svn-id: svn://svn.valgrind.org/valgrind/trunk@13884
Necessary changes to Valgrind to support MIPS64LE on Linux.
Minor cleanup/style changes embedded in the patch as well.
The change corresponds to r2687 in VEX.
Patch written by Dejan Jevtic and Petar Jovanovic.
More information about this issue:
https://bugs.kde.org/show_bug.cgi?id=313267
git-svn-id: svn://svn.valgrind.org/valgrind/trunk@13292
It's reorg only. No new cache autodetection stuff has been added.
coregrind
pub_tool_cpuid.h is removed as it is no longer exposed to tools.
Its contents has moved to pub_core_cpuid.h.
New file: coregrind/m_cache.c to contain the autodetect code for
cache configurations and define other cache characteristics that
cannot be autodetected (i.e. icaches_maintain_coherence). Most of
cg-arch/x86-amd64.c was moved here. The cache detection code for
x86-64 needs to be fixed to properly initialise VexCacheInfo. It
currently has cachegrind bias.
m_cache.c exports a single function (to coregrind):
VG_(machine_get_cache_info)(VexArchInfo *vai)
This function is called from VG_(machine_get_hwcaps) after hwcaps have
been detected.
cachegrind
Remove cachegrind/cg-{ppc32,ppc43,arm,mips32,s390x,x86-amd64}.c
With the exception of x86/mamd64 those were only establishing a
default cache configuration and that is so small a code snippet that
a separate file is no longer warranted. So, the code was moved to
cg-arch.c. Code was added to extract the relevant info from
x86-amd64.
New function maybe_tweak_LLc which captures the code to massage the
LLc cache configuration into something the simulator can handle. This
was originally in cg-x86-amd64.c but should be used to all architectures.
Changed warning message about missing cache auto-detect feature
to be more useful. Adapted filter-stderr scripts accordingly.
git-svn-id: svn://svn.valgrind.org/valgrind/trunk@13028
From: Ian Campbell <Ian.Campbell@citrix.com>
Under Xen the toolstack is responsible for managing the domains in
the system, e.g. creating, destroying, and otherwise manipulating
them.
To do this it uses a number of ioctls on the /proc/xen/privcmd
device. Most of these (the MMAPBATCH ones) simply set things up such
that a subsequenct mmap call will map the desired guest memory. Since
valgrind has no way of knowing what the memory contains we assume
that it is all initialised (to do otherwise would require valgrind to
be observing the complete state of the system and not just the given
process).
The most interesting ioctl is XEN_IOCTL_PRIVCMD_HYPERCALL which
allows the toolstack to make arbitrary hypercalls. Although the
mechanism here is specific to the OS of the guest running the
toolstack the hypercalls themselves are defined solely by the
hypervisor. Therefore I have split support for this ioctl into a part
in syswrap-linux.c which handles the ioctl itself and passes things
onto a new syswrap-xen.c which handles the specifics of the
hypercalls themselves. Porting this to another OS should just be a
matter of wiring up syswrap-$OS.c to decode the ioctl and call into
syswrap-xen.c. In the future we may want to split this into
syswrap-$ARCH-xen.c but for now this is x86 only.
The hypercall coverage here is pretty small but is enough to get
reasonable(-ish) results out of the xl toolstack when listing,
creating and destroying domains.
One issue is that the hypercalls which are exlusively used by the
toolstacks (as opposed to those used by guest operating systems) are
not considered a stable ABI, since the hypervisor and the lowlevel
tools are considered a matched pair. This covers the sysctl and
domctl hypercalls which are a fairly large chunk of the support
here. I'm not sure how to solve this without invoking a massive
amount of duplication. Right now this targets the Xen unstable
interface (which will shortly be released as Xen 4.2), perhaps I can
get away with deferring this problem until the first change .
On the plus side the vast majority of hypercalls are not of interest
to the toolstack (they are used by guests) so we can get away without
implementing them.
Note: a hypercall only reads as many words from the ioctl arg
struct as there are actual arguments to that hypercall and the
toolstack only initialises the arguments which are used. However
there is no space in the DEFN_PRE_TEMPLATE prototype to allow this to
be communicated from syswrap-xen.c back to syswrap-linux.c. Since a
hypercall can have at most 5 arguments I have hackily stolen ARG8 for
this purpose.
git-svn-id: svn://svn.valgrind.org/valgrind/trunk@12963
* manual-core.xml : fix a typo
* include/pub_tool_inner.h : new file, defining macros for inner annotation
include/Makefile.am : reference this new file.
* syswrap-linux.c : when ENABLE_INNER, register the stacks for the outer.
(otherwise, nothing works properly).
* m_redir.c : avoid inner interpreting the outer vgpreload instructions.
* sema.c : annotate the semaphore with RWLOCK annotations for helgrind
* ticket-lock-linux.c : similar.
git-svn-id: svn://svn.valgrind.org/valgrind/trunk@12414
* new files include/pub_tool_groupalloc.h and coregrind/m_groupalloc.c
implementing a group allocator (based on helgrind group alloc).
* include/Makefile.am coregrind/Makefile.am : added pub_tool_groupalloc.h
and m_groupalloc.c
* helgrind/libhb_core.c : use pub_tool_groupalloc.h/m_groupalloc.c
instead of the local implementation.
* include/pub_tool_oset.h coregrind/m_oset.c : new function
allowing to create an oset that will use a pool allocator.
new function allowing to clone an oset (so as to share the pool alloc)
* memcheck/tests/unit_oset.c drd/tests/unit_bitmap.c : modified
so that it compiles with the new m_oset.c
* memcheck/mc_main.c : use group alloc for MC_Chunk
memcheck/mc_include.h : declare the MC_Chunk group alloc
* memcheck/mc_main.c : use group alloc for the nodes of the secVBitTable OSet
* include/pub_tool_hashtable.h coregrind/m_hashtable.c : pass the free node
function in the VG_(HT_destruct).
(needed as the hashtable user can allocate a node with its own alloc,
the hash table destroy must be able to free the nodes with the user
own free).
* coregrind/m_gdbserver/m_gdbserver.c : pass free function to VG_(HT_destruct)
* memcheck/mc_replace_strmem.c memcheck/mc_machine.c
memcheck/mc_malloc_wrappers.c memcheck/mc_leakcheck.c
memcheck/mc_errors.c memcheck/mc_translate.c : new include needed
due to group alloc.
git-svn-id: svn://svn.valgrind.org/valgrind/trunk@12341
__builtin_setjmp and __builtin_longjmp so that they can be selectively
replaced, on a platform by platform basis. Does not change any
functionality. Related to #259977.
git-svn-id: svn://svn.valgrind.org/valgrind/trunk@11687
side components. (Florian Krohm <britzel@acm.org> and Christian
Borntraeger <borntraeger@de.ibm.com>). Fixes#243404.
git-svn-id: svn://svn.valgrind.org/valgrind/trunk@11604
following improvements:
- Arch/OS/platform-specific files are now included/excluded via the
preprocessor, rather than via the build system. This is more consistent
(we use the pre-processor for small arch/OS/platform-specific chunks
within files) and makes the build system much simpler, as the sources for
all programs are the same on all platforms.
- Vast amounts of cut+paste Makefile.am code has been factored out. If a
new platform is implemented, you need to add 11 extra Makefile.am lines.
Previously it was over 100 lines.
- Vex has been autotoolised. Dependency checking now works in Vex (no more
incomplete builds). Parallel builds now also work. --with-vex no longer
works; it's little use and a pain to support. VEX/Makefile is still in
the Vex repository and gets overwritten at configure-time; it should
probably be renamed Makefile-gcc to avoid possible problems, such as
accidentally committing a generated Makefile. There's a bunch of hacky
copying to deal with the fact that autotools don't handle same-named files
in different directories. Julian plans to rename the files to avoid this
problem.
- Various small Makefile.am things have been made more standard automake
style, eg. the use of pkginclude/pkglib prefixes instead of rolling our
own.
- The existing five top-level Makefile.am include files have been
consolidated into three.
- Most Makefile.am files now are structured more clearly, with comment
headers separating sections, declarations relating to the same things next
to each other, better spacing and layout, etc.
- Removed the unused exp-ptrcheck/tests/x86 directory.
- Renamed some XML files.
- Factored out some duplicated dSYM handling code.
- Split auxprogs/ into auxprogs/ and mpi/, which allowed the resulting
Makefile.am files to be much more standard.
- Cleaned up m_coredump by merging a bunch of files that had been
overzealously separated.
The net result is 630 fewer lines of Makefile.am code, or 897 if you exclude
the added Makefile.vex.am, or 997 once the hacky file copying for Vex is
removed. And the build system is much simpler.
git-svn-id: svn://svn.valgrind.org/valgrind/trunk@10364
- Introduced VG_SYSNUM_STRING and VG_SYSNUM_STRING_EXTRA which factor out
differences in the way syscall numbers are printed on different platforms.
This gets rid of seven "DDD" fixme-style comments.
- This also meant that Darwin syscall numbers are now printed in a
non-ambiguous way -- previously Unix, machine-dependent and diagnostic
syscalls were all printed the same way, even though their numbers overlap.
Now each number is prefixed with "unix", "mdep", etc. And Mach trap
numbers aren't printed as negative numbers now that they have a "mach"
prefix.
- Split each of pub_core_vkiscnums.h and pub_tool_vkiscnums.h into two
parts, one suitable for inclusion in asm files, one suitable for inclusion
in C files; in both cases the latter includes the former. This makes
this module more like other modules that have asm-only components (eg.
m_transtab); it also allows the hacky VG_IN_ASSEMBLY_SOURCE macros and
tests to be removed.
- Removed some of the VG_DARWIN_SYSNO_* macros that were no longer needed,
and renamed some of the existing ones to make their meanings clearer.
- Added comments on the encoding of Darwin syscall numbers so it's
possible for mortals to understand without reading the kernel code..
git-svn-id: svn://svn.valgrind.org/valgrind/trunk@10218
matching, in the function VG_(generic_match). Patterns to be matched
against may contain only '*'-style wildcards (matches any number of
elements, we don't care what they are), '?' wildcards (matches exactly
one element, we don't care what it is) and literal elements.
It is totally abstractified, in the sense that the pattern and input
arrays may be arrays of anything. The caller provides enough
information so that VG_(generic_match) can step along both arrays, and
can ask the questions "is this pattern element a '*' ?", "is this
pattern element a '?' ?", and "does this pattern element match an
input element ?".
The existing function VG_(string_match) is reimplemented using
VG_(generic_match), although the ability to escape metacharacters in
the pattern string is removed -- I don't think it was ever used.
In m_errormgr, matching of suppression stacks (including wildcard
"..." lines) against error stacks is re-implemented using
VG_(generic_match).
Further detailed comments are in m_seqmatch.h and pub_tool_seqmatch.h.
A negative side effect is that VG_(string_match) will be much slower
than before, due to the abstractification. It may be necessary to
reimplement a specialised version later.
git-svn-id: svn://svn.valgrind.org/valgrind/trunk@8816
which is also indexed by UWord. This can be used as a replacement for
WordFM with unboxed keys, when the key ranges are dense. It is
implemented as a 256-way radix tree (4-deep on 32-bit platforms,
8-deep on 64-bit platforms).
git-svn-id: svn://svn.valgrind.org/valgrind/trunk@8806
helgrind/ into the core. It's just too darn useful to not be in the
core. There is some overlap in functionality between OSet and WordFM,
but OSet is more space efficient in some circumstances, whereas WordFM
is easier to use and a bit more flexible in some cases.
Also in this new module (m_wordfm) is a type WordBag, build on top of
WordFM. This provides Multiset of UWords functionality.
git-svn-id: svn://svn.valgrind.org/valgrind/trunk@8524
Makefile.am changes for AIX5. Almost all boilerplate stuff fitting in
with the existing factorisation scheme. The only change of interest
is that configure.in now generates automake symbols of name
VGP_platform and VGO_os, whereas previously it just made VG_platform
which was a bit inconsistent with the VGP/VGO/VGA scheme used in C
code.
git-svn-id: svn://svn.valgrind.org/valgrind/trunk@6242
interface, except for the syscall numbers, into that. Mostly this
means moving include/vki-*.h to include/vki/vki-*.h.
include/pub_tool_basics.h previously dragged in the entire kernel
interface. I've done away with that, so that modules which need to
see the kernel interface now have to include pub_{core,tool}_vki.h
explicitly. This is why there are many modified .c files -- they have
all acquired an extra #include line.
This certainly breaks all platforms except x86. Will fix shortly.
git-svn-id: svn://svn.valgrind.org/valgrind/trunk@6225
changes from r4341 through r4787 inclusive). That branch is now dead.
Please do not commit anything else to it.
For the most part the merge was not troublesome. The main areas of
uncertainty are:
- build system: I had to import by hand Makefile.core-AM_CPPFLAGS.am
and include it in a couple of places. Building etc seems to still
work, but I haven't tried building the documentation.
- syscall wrappers: Following analysis by Greg & Nick, a whole lot of
stuff was moved from -generic to -linux after the branch was created.
I think that is satisfactorily glued back together now.
- Regtests: although this appears to work, no .out files appear, which
is strange, and makes it hard to diagnose regtest failures. In
particular memcheck/tests/x86/scalar.stderr.exp remains in a
conflicted state.
- amd64 is broken (slightly), and ppc32 will be unbuildable. I'll
attend to the former shortly.
git-svn-id: svn://svn.valgrind.org/valgrind/trunk@4789
which is a sorted set with no duplicates. This is derived from
m_skiplist, which it will hopefully replace. The interface has the
following improvements:
- Avoided all mention of how the data structure is implemented in the
interface, so it could be replaced with another data structure without
changing external code.
- Two kinds of comparison: fast -- use the first word of each element
for comparison; slow -- use a custom function. The custom function
compares a key with an element, so non-overlapping interval lists can
be supported easily. m_skiplist only supports the slow variant, and it
makes things almost 2x faster.
- Users pass in malloc() and free() functions, so m_oset.c it doesn't
rely on any particular allocator.
- It has a Destroy() function which will deallocate all the nodes.
- It allows variable-sized nodes.
- No static constructor; I needed the flexibility of being able to
execute arbitrary code in the constructor. This also means no type
internals are exposed.
No part of Valgrind actually uses OSet yet, although I've privately
converted several data structures, and so I'm confident that the
interface is basically sound. Some functions may be added later.
The implementation uses AVL trees, and has the following
characteristics:
- Lookup is much faster than for skiplists -- around 3x. This is
because the inner lookup loop is much tighter.
- Insertion and removal is similar speed to skiplists, maybe a little
slower, but there's still some fat to be trimmed.
- The code is a bit longer and more complex than the skiplist code.
This was intended to replace the need for the VgHashTable type. But my
experiments have shown that VgHashTable is really fast, faster than both
AVL trees and skiplists in all but extreme cases (eg. if the hashtable
becomes way too full): insertion takes constant time, because you always
prepend to chains; lookup depends on chain length, but the inner loop
is so tight that you need about 20 elements per chain before it gets
worse than the AVL tree; removal is similar to lookup. And because
insertion uses prepending, any locality in accesses will help things. If
VgHashTable had its interface cleaned up to look like OSet's, and was made
to auto-resize when it got too full, it might be a better OSet (although
it's not sorted).
So, it's currently unclear exactly how the AVL tree OSet will be used.
The skiplist could be converted to the new interface (I have a 90%
complete version which I used in the comparison experiments). If
VgHashTable was converted to the same interface (or as close as
possible) it would make direct comparison of important places (eg.
Memcheck's malloc_lists) simple.
git-svn-id: svn://svn.valgrind.org/valgrind/trunk@4410
bit-rotted badly and was clogging up the code.
I put the useful remnants in docs/porting-to-ARM in case anyone ever
wants to try porting to ARM again.
git-svn-id: svn://svn.valgrind.org/valgrind/trunk@4092
Plenty still to do, but simple programs like ls seem to run ok
Thanks, Paul, for having your ppc port of valgrind 2.4 to work from!
git-svn-id: svn://svn.valgrind.org/valgrind/trunk@3969
- Broke part of m_scheduler off into a new module m_threadstate. It
contains ThreadState, VG_(threads)[] and some basic operations on the
thread table. All simple stuff, the complex stuff stays in m_scheduler.
This avoids lots of circular dependencies between m_scheduler and other
modules.
- Managed to finally remove core.h and tool.h, double hurrah!
- Introduced pub_tool_basics.h and pub_core_basics.h, one of which is
include by every single C file.
- Lots of little cleanups and changes related to the above.
- I even did a small amount of documentation updating.
git-svn-id: svn://svn.valgrind.org/valgrind/trunk@3944
