This implements the interception of all globally public allocation
functions by default. It works by adding a flag to the spec to say the
interception only applies to global functions. Which is set for the
somalloc spec. The librarypath to match is set to "*" unless the user
overrides it. Then each DiSym keeps track of whether the symbol is local
or global. For a spec which has isGlobal set only isGlobal symbols will
match.
Note that because of padding to keep the addresses in DiSym aligned the
addition of the extra bool isGlobal doesn't actually grow the struct.
The comments explain how the struct could be made more compact on 32bit
systems, but this isn't as easy on 64bit systems. So I didn't try to do
that in this patch.
For ELF symbols keeping track of which are global is trivial. For pdb I
had to guess and made only the "Public" symbols global. I don't know
how/if macho keeps track of global symbols or not. For now I just mark
all of them local (which just means things work as previously on platforms
that use machos, no non-system symbols are matches by default for somalloc
unless the user explicitly tells which library name to match).
Included are two testcases for shared libraries (wrapmalloc) and staticly
linked (wrapmallocstatic) malloc/free overrides that depend on the new
default. One existing testcase (new_override) was adjusted to explicitly
not use the new somalloc default because it depends on a user defined
new implementation that has side-effects and should explicitly not be
intercepted.
git-svn-id: svn://svn.valgrind.org/valgrind/trunk@15726
The ELF debug info reader on Solaris now performs a quick pre-scan of section
headers for .rodata sections. If there are multiple .rodata sections
present then symbols from .symtab are scanned which section they point to.
The "true" .rodata section is thus determined.
Fixes BZ#353802.
git-svn-id: svn://svn.valgrind.org/valgrind/trunk@15719
Valgrind aspects, to match vex r3124.
See bug 339778 - Linux/TileGx platform support to Valgrind
git-svn-id: svn://svn.valgrind.org/valgrind/trunk@15080
Left justification of strings in myvprintf_str was mixed up.
Now fixed and %s formats changed accordingly.
In function myvprintf_int64: the local buffer was not large
enough to hold ULONG_MAX in binary notation. Numbers were
truncated at 39 digits.
Testcases added.
git-svn-id: svn://svn.valgrind.org/valgrind/trunk@14808
(/system/bin/ls, /system/bin/date) run. Still to do:
* enable more malloc/free intercepts
* enable wrappers for ashmem and binder syscalls
* check to see if any special ioctl support is required for ARM Mali GPUs
git-svn-id: svn://svn.valgrind.org/valgrind/trunk@14690
that once an element has been allocated and added to the pool it must
not be modified afterwards. See the documentation in pub_tool_deduppoolalloc.h
The rest of the patch is ripple.
git-svn-id: svn://svn.valgrind.org/valgrind/trunk@14654
truncate overlaps in the DebugInfoMappings that have been collected by
the DebugInfo's FSM. Not doing so can confuse ML_(read_elf_debug_info)'s
computation of bias values. Observed to be a problem when reading EDIDX
sections for objects mangled by Mike Hommey's elfhack program.
See http://bugzilla.mozilla.org/show_bug.cgi?id=788974
git-svn-id: svn://svn.valgrind.org/valgrind/trunk@14632
Since valgrind 3.9.0 the STABS support was already disabled completely.
But the code was still there being compiled and we were still searching
for stabs sections in binaries. Completely remove all sources, tests and
references. Add a note to coregrind/m_debuginfo/README.txt to mention
the old code can be found in the subversion repository.
git-svn-id: svn://svn.valgrind.org/valgrind/trunk@14550
- Document that the allocation function must ot return NULL.
- As a conequence of the previous requirement the various Create and AllocNode
functions cannot return NULL. Remove pointless asserts at call sites.
- Remove documentation of undefined function CreateWithCmp.
- Names of library functions (such as 'free') are reserved as a are names
beginning with underscores. Don't use those.
git-svn-id: svn://svn.valgrind.org/valgrind/trunk@14531
Revision r14464 made it so that debug alt files could be found by their
build-id or their (relative) file path. Debug alt files are matched using
the given build-id, but by crc. Calculating the full CRC is costly, but
currently still needed to avoid misidentifying the main file as debug
file. Slightly more efficient would be to use fstat to check we aren't
actually opening the main file under any other name (but that only works
for local DiImages). Or we could check that the file being opened actually
has at least one .debug* section. But this change was the minimal patch
to make things work as before.
git-svn-id: svn://svn.valgrind.org/valgrind/trunk@14474
readdwarf3 would only look for alt dwz files using the build-id.
But alt files can be installed relative to the debug (or main) file.
Fix find_debug_file to allow searching of relative files even if
we don't want an ET_REL (rel_ok) file, and pass the build-id to
open_debug_file so it can be checked. Add the debug file path to
_DebugInfoFSM and set it in find_debug_file once opened. Pass the
dbgname or filename as relative file to resolve an altfile in
read_elf_debug_info when we ahava an debugaltlink_escn.
git-svn-id: svn://svn.valgrind.org/valgrind/trunk@14464
(used for ppc64 platforms) #ifdef-ed and accessed by macros
that becomes NOP on non ppc64 platforms.
This decreases the debuginfo memory by about 2.5 Mb on a big 32 bit application.
Note : doing that, some questions were encountered in the way
tocptr and local_ep have (or do not have) to be copied/maintained
in storage.c canonicaliseSymtab
git-svn-id: svn://svn.valgrind.org/valgrind/trunk@14273
to add PPC64 LE support. The other two patches can be found in Bugzillas
334384 and 334836.
POWER PC, add the functional Little Endian support, patch 2
The IBM POWER processor now supports both Big Endian and Little Endian.
The ABI for Little Endian also changes. Specifically, the function
descriptor is not used, the stack size changed, accessing the TOC
changed. Functions now have a local and a global entry point. Register
r2 contains the TOC for local calls and register r12 contains the TOC
for global calls. This patch makes the functional changes to the
Valgrind tool. The patch makes the changes needed for the
none/tests/ppc32 and none/tests/ppc64 Makefile.am. A number of the
ppc specific tests have Endian dependencies that are not fixed in
this patch. They are fixed in the next patch.
Per Julian's comments renamed coregrind/m_dispatch/dispatch-ppc64-linux.S
to coregrind/m_dispatch/dispatch-ppc64be-linux.S Created new file for LE
coregrind/m_dispatch/dispatch-ppc64le-linux.S. The same was done for
coregrind/m_syswrap/syscall-ppc-linux.S.
Signed-off-by: Carl Love <carll@us.ibm.com>
git-svn-id: svn://svn.valgrind.org/valgrind/trunk@14239
to add PPC64 LE support. The other two patches can be found in Bugzillas
334834 and 334836. The commit does not have a VEX commit associated with it.
POWER PC, add initial Little Endian support
The IBM POWER processor now supports both Big Endian and Little Endian.
This patch renames the #defines with the name ppc64 to ppc64be for the BE
specific code. This patch adds the Little Endian #define ppc64le to the
Additionally, a few functions are renamed to remove BE from the name if the
function is used by BE and LE. Functions that are BE specific have BE put
in the name.
The goals of this patch is to make sure #defines, function names and
variables consistently use PPC64/ppc64 if it refers to BE and LE,
PPC64BE/ppc64be if it is specific to BE, PPC64LE/ppc64le if it is LE
specific. The patch does not break the code for PPC64 Big Endian.
The test files memcheck/tests/atomic_incs.c, tests/power_insn_available.c
and tests/power_insn_available.c are also updated to the new #define
definition for PPC64 BE.
Signed-off-by: Carl Love <carll@us.ibm.com>
git-svn-id: svn://svn.valgrind.org/valgrind/trunk@14238
On a big executable, the trunk needs:
dinfo: 134873088/71438336 max/curr mmap'd, 134607808/66717872 max/curr
With the patch, we have:
dinfo: 99065856/56836096 max/curr mmap'd, 97883776/51663656 max/curr
So, peak dinfo memory decreases by about 36Mb, and final by 15Mb.
(for info, valgrind 3.9.0 uses
dinfo: 158941184/109666304 max/curr mmap'd, 156775944/107590656 max/curr
So, compared to 3.9.0, dinfo peak decreases by about 40%, and the final
memory is divided by more than 2).
The memory decrease is obtained by:
* using a dedup pool to store filename/dirname pair for the loctab source/line
information.
As typically, there is not a lot of such pairs, typically a UShort is
good enough to identify a fn/dn pair in a dedup pool.
To avoid losing memory due to alignment, the fndn indexes are stored
in a "parallel" array to the DiLoc loctab array, with entries having
1, or 2 or 4 bytes according to the nr of fn/dn pairs in the dedup pool.
See priv_storage.h comments for details.
(there was a extensible WordArray local implementation in readdwarf.c.
As with this change, we use an xarray, the local implementation was
removed).
* the memory needed for --read-inline-info is slightly decreased (-2Mb)
by removing the (unused) dirname from the DiInlLoc struct.
Handling dirname for inlined function caller implies to rework
the dwarf3 parser read_filename_table common to the var and inlinfo parser.
Waiting for this to be done, the dirname component is removed from DiInlLoc.
* the stabs reader (readstabs.c) is broken since 3.9.0.
For this change, the code has been updated to make it compile with the new
DiLoc/FnDn dedup pool. As the code is completely broken, a vg_assert(0)
has been put at the begin of the stabs reader.
* the pdb reader (readpdb.c) has been trivially updated and should still work.
It has not been tested (how do we test this ?).
A follow-up patch will be done to avoid doing too many calls to
ML_(addFnDn) : instead of having one call per ML_(addLineInfo), one
should have a single call done when reading the filename table.
This has also be tested in an outer/inner setup, to verify no
memory leak/bugs.
git-svn-id: svn://svn.valgrind.org/valgrind/trunk@14158
On a big executable, the trunk needs:
dinfo: 155844608/106737664 max/curr mmap'd 155572624/102276760 max/curr
With the patch, we have:
dinfo: 134873088/70389760 max/curr mmap'd 134607808/66717512 max/curr
So, peak dinfo memory decreases by 21Mb, and final by 36Mb.
The memory decrease is obtained by:
* using a dedup pool to store the machine dependent part (cfsi_m)
of the cfsi information as this information is highly duplicated.
For x86 and arm64, the duplication factor of cfsi machine dependent
part is very high (up to a factor 60).
For arm64, it is more like a factor 3.
A 'variable size' (1, 2 or 4 bytes) is automatically used to identify
the cfsi_m, if there is less than or more than 255/64K different cfsi_m.
* not storing explicitely the length of a range for which a cfsi_m
is to be used: in a large majority of the cases, ranges are
consecutive, and so the end of a range is just one byte before
the start of the next range.
So, we do not store the length of the ranges.
If there is a hole between 2 ranges, the hole is stored explicitely
as a range in which we have no cfsi_m information.
On x86 and amd64, we have quite some holes (something like one hole
every 7 cfsi). On arm64, we have very few holes (less than one hole
every 50 cfsi).
Even with the nr of holes on x86/amd64, it is more memory efficient
to store the holes rather than to store the length of each cfsi.
* Merging consecutive ranges that have the same cfsi_m info:
Many cfsi are "mergeable": there is no hole between 2 cfsi, and their
machine dependent part is identical
(I guess the unwind info needed by valgrind is subset of the full
unwind info, and so, the cfsi entries are not merged by the compiler,
but can be merged for simple unwind). Depending on the platform
(x86, amd64, arm64) and of the library/object file, we can have a
significant nr of mergeable entries.
The patch is not very small, but a lot is mechanical changes.
The patch has been compiled and tested on x86/amd64/ppc32/ppc64
(but ppc does not use cfsi so that just verifies it compiles).
It has been compiled on arm64, and "tested" by launching valgrind on
one executable.
It has not been compiled on s390 and mips.
With some luck, maybe it will compile on these platforms.
And if that uses the whole provision of luck for 2014, it might even work
on these platforms :).
If it does not compile, the fix should be straightforward.
Runtime problems might be more tricky (but arm64 "worked out of the box"
once x86/amd64 were ok).
This has also be tested in an outer/inner setup, to verify no memory leak/bugs.
git-svn-id: svn://svn.valgrind.org/valgrind/trunk@14129
showing inlined function calls.
See 278972 valgrind stacktraces and suppression do not handle inlined function call debuginfo
Reading the inlined dwarf call info is activated using the new clo
--read-inline-info=yes
Default is currently no but an objective is to optimise the performance
and memory in order to possibly set it on by default.
(see below discussion about performances).
Basically, the patch provides the following pieces:
1. Implement a new dwarf3 reader that reads the inlined call info
2. Some performance improvements done for this new parser, and
on some common code between the new parser and the var info parser.
3. Use the parsed inlined info to produce stacktrace showing inlined calls
4. Use the parsed inlined info in the suppression matching and suppression generation
5. and of course, some reg tests
1. new dwarf3 reader:
---------------------
Two options were possible: add the reading of the inlined info
in the current var info dwarf reader, or add a 2nd reader.
The 2nd approach was preferred, for the following reasons:
The var info reader is slow, memory hungry and quite complex.
Having a separate parsing phase for the inlined information
is simpler/faster when just reading the inlined info.
Possibly, a single parser would be faster when using both
--read-var-info=yes and --read-inline-info=yes.
However, var-info being extremely memory/cpu hungry, it is unlikely
to be used often, and having a separate parsing for inlined info
does in any case make not much difference.
(--read-var-info=yes is also now less interesting thanks to commit
r13991, which provides a fast and low memory "reasonable" location
for an address).
The inlined info parser reads the dwarf info to make calls
to priv_storage.h ML_(addInlInfo).
2. performance optimisations
----------------------------
* the abbrev cache has been improved in revision r14035.
* The new parser skips the non interesting DIEs
(the var-info parser has no logic to skip uninteresting DIEs).
* Some other minor perf optimisation here and there.
In total now, on a big executable, 15 seconds CPU are needed to
create the inlined info (on my slow x86 pentium).
With regards to memory, the dinfo arena:
with inlined info: 172281856/121085952 max/curr mmap'd
without : 157892608/106721280 max/curr mmap'd,
So, basically, inlined information costs about 15Mb of memory for
my big executable (compared to first version of the patch, this is
already using less memory, thanks to the strpool deduppoolalloc.
The needed memory can probably be decreased somewhat more.
3. produce better stack traces
------------------------------
VG_(describe_IP) has a new argument InlIPCursor *iipc which allows
to describe inlined function calls by doing repetitive calls
to describe_IP. See pub_tool_debuginfo.h for a description.
4. suppression generation and matching
--------------------------------------
* suppression generation now also uses an InlIPCursor *iipc
to generate a line for each inlined fn call.
* suppression matching: to allow suppression matching to
match one IP to several function calls in a suppression entry,
the 'inputCompleter' object (that allows to lazily generate
function or object names for a stacktrace when matching
an error with a suppression) has been generalised a little bit
more to also lazily generate the input sequence.
VG_(generic_match) has been updated so as to be more generic
with respect to the input completer : when providing an
input completer, VG_(generic_match) does not need anymore
to produce/compute any input itself : this is all delegated
to the input completer.
5. various regtests
-------------------
to test stack traces with inlined calls, and suppressions
of (some of) these errors using inlined fn calls matching.
Work still to do:
-----------------
* improve parsing performance
* improve the memory overhead.
* handling the directory name for files of the inlined function calls is not yet done.
(probably implies to refactor some code)
* see if m_errormgr.c *offsets arrays cannot be managed via xarray
git-svn-id: svn://svn.valgrind.org/valgrind/trunk@14036
include/pub_tool_deduppoolalloc.h
coregrind/pub_core_deduppoolalloc.h
coregrind/m_deduppoolalloc.c
and uses it (currently only) for the strings in m_debuginfo/storage.c
The idea is that such ddup pool allocator will also be used for other
highly duplicated information (e.g. the DiCFSI information), where
significant gains can also be achieved.
The dedup pool for strings also decreases significantly the memory
needed by the read inline information (patch still to be committed,
see bug 278972).
When testing with a big executable (tacot_process),
this reduces the size of the dinfo arena from
trunk: 158941184/109760512 max/curr mmap'd, 156775944/107882728 max/curr,
to
ddup: 157892608/106614784 max/curr mmap'd, 156362160/101414712 max/curr
(so 3Mb less mmap-ed once debug info is read, 1Mb less mmap-ed in peak,
6Mb less allocated once debug info is read).
This is all gained due to the string which changes from:
trunk: 17,434,704 in 266: di.storage.addStr.1
to
ddup: 10,966,608 in 750: di.storage.addStr.1
(6.5Mb less memory used by strings)
The gain in mmap-ed memory is smaller due to fragmentation.
Probably one could decrease the fragmentation by using bigger
size for the dedup pool, but then we would lose memory on the last
allocated pool (and for small libraries, we often do not use much
of a big pool block).
Solution might be to increase the pool size but have a "shrink_block"
operation. To be looked at in the future.
In terms of performance, startup of a big executable (on an old pentium)
is not influenced significantly (something like 0.1 seconds on 15 seconds
startup for a big executable, on a slow pentium).
The dedup pool uses a hash table. The hash function used currently
is the VG_(adler32) check sum. It is reported (and visible also here)
that this checksum is not a very good hash function (many collisions).
To have statistics about collisions, use --stats -v -v -v
As an example of the collisions, on the strings in debug info of memcheck tool on x86,
one obtain:
--4789-- dedupPA:di.storage.addStr.1 9983 allocs (8174 uniq) 11 pools (4820 bytes free in last pool)
--4789-- nr occurences of chains of len N, N-plicated keys, N-plicated elts
--4789-- N: 0 : nr chain 6975, nr keys 0, nr elts 0
--4789-- N: 1 : nr chain 3670, nr keys 6410, nr elts 8174
--4789-- N: 2 : nr chain 1070, nr keys 226, nr elts 0
--4789-- N: 3 : nr chain 304, nr keys 100, nr elts 0
--4789-- N: 4 : nr chain 104, nr keys 84, nr elts 0
--4789-- N: 5 : nr chain 72, nr keys 42, nr elts 0
--4789-- N: 6 : nr chain 44, nr keys 34, nr elts 0
--4789-- N: 7 : nr chain 18, nr keys 13, nr elts 0
--4789-- N: 8 : nr chain 17, nr keys 8, nr elts 0
--4789-- N: 9 : nr chain 4, nr keys 6, nr elts 0
--4789-- N:10 : nr chain 9, nr keys 4, nr elts 0
--4789-- N:11 : nr chain 1, nr keys 0, nr elts 0
--4789-- N:13 : nr chain 1, nr keys 1, nr elts 0
--4789-- total nr of unique chains: 12289, keys 6928, elts 8174
which shows that on 8174 different strings, we have only 6410 strings which have
a unique hash value. As other examples, N:13 line shows we have 13 strings
mapping to the same key. N:14 line shows we have 4 groups of 10 strings mapping to the
same key, etc.
So, adler32 is definitely a bad hash function.
Trials have been done with another hash function, giving a much lower
collision rate. So, a better (but still fast) hash function would probably
be beneficial. To be looked at ...
git-svn-id: svn://svn.valgrind.org/valgrind/trunk@14029
Bug #327837. The buildid from the .gnu_debugaltlink section was parsed
incorrectly (from the wrong offset). Causing the debug alt file not to
be found.
git-svn-id: svn://svn.valgrind.org/valgrind/trunk@13715
exceeds the allowable range. With this change, it should be
essentially impossible to crash V by feeding it invalid ELF or Dwarf.
git-svn-id: svn://svn.valgrind.org/valgrind/branches/DISRV@13432
* add a new flag --allow-mismatched-debuginfo to override the
CRC32/build-id checks, if needed
* tidy up logic for finding files on the --extra-debuginfo-path
and at the --debuginfo-server
* don't assert if connection to the debuginfo server is lost;
instead print a reasonable message and quit.
git-svn-id: svn://svn.valgrind.org/valgrind/branches/DISRV@13431
Necessary changes to Valgrind to support MIPS64LE on Linux.
Minor cleanup/style changes embedded in the patch as well.
The change corresponds to r2687 in VEX.
Patch written by Dejan Jevtic and Petar Jovanovic.
More information about this issue:
https://bugs.kde.org/show_bug.cgi?id=313267
git-svn-id: svn://svn.valgrind.org/valgrind/trunk@13292
instead checking the shdrs:
The separate .debug file has wrong phdrs. This isn't normally fatal
since .debug files are never directly loaded. But since valgrind
uses the phdrs to locate the build-id it will fail. The attached
patch makes it so that the code falls back to using the shdrs to
locate the NOTE sections so that the buildid can be matched anyway.
Fixes#305431. (Mark Wielaard, mjw@redhat.com)
git-svn-id: svn://svn.valgrind.org/valgrind/trunk@13160
specification of an extra directory in which to look for debuginfo
objects. Fixes#310792. (Alex Chiang, achiang@canonical.com)
git-svn-id: svn://svn.valgrind.org/valgrind/trunk@13154
If a segment is mapped with permission rwx, then map->rx
and map->rw will be true.
But due to the if (map->rx) {
...
} else if (map->rw) {
...
the (map->rw) part will not be executed.
If this mapping is the one which "gives" the nonempty rw map,
then this mapping will not be seen, and the following
vg_assert(has_nonempty_rw);
will fail.
This assert can be reproduced by doing
setarch i686 -X
./vg-in-place --tool=none none/tests/map_unmap
Note: the setarch i686 -X above has as effect to make all read
mapping also executable. So, a rw mapping becomes rwx and then
triggers the above asserts.
The setarch i686 -X also introduces a discrepancy between
the kernel mappings (rwx) and the valgrind aspacemgr view
(which believes it is a rw mapping).
This discrepancy causes a crash if giving --sanity-level=3.
A possible fix is to have valgrind calling the personality system call
and detecting if the READ_IMPLIES_EXEC bit (the -X arg to setarch)
was set, and then modify aspacemgr so that all read mapped segments
are automatically mapped x also.
This commit is the minimal fix allowing to run executables
launched with this READ_IMPLIES_EXEC.
git-svn-id: svn://svn.valgrind.org/valgrind/trunk@12810
di->soname was not freed, so was leaked when debug info is removed.
free(soname) added in free_Debuginfo, after having verified
and then ensured that all soname are allocated in dinfo.
regtested on deb6/amd64
git-svn-id: svn://svn.valgrind.org/valgrind/trunk@12442
