NDK's appear to #define __unused to __attribute__((__unused__)),
causing the build to fail in bizarre ways.
git-svn-id: svn://svn.valgrind.org/valgrind/trunk@14412
r14384 introduced use of getdents64 syscall and we missed a system call
value for MIPS64, so it broke the build for it. Add missing values now.
git-svn-id: svn://svn.valgrind.org/valgrind/trunk@14409
There are a couple of issues with helgrind on arm-linux with glibc:
- Thread creation stack traces cannot unwind through clone
(cfi ends right after syscall)
- ld.so has a special "hard float" name that isn't recognized as special
(ld-linux-armhf.so.3)
- Races are found when manipulating GOT sections.
git-svn-id: svn://svn.valgrind.org/valgrind/trunk@14407
An address below the sp will be described as being on a stack, but below sp.
The stack for such an address is found in the registered stacks.
Also, if there is a guard page at the end of the stack (lowest address)
an address in this page will be described as being in thread guard page.
A guard page is recognised as being a page not readable/writable/executable.
git-svn-id: svn://svn.valgrind.org/valgrind/trunk@14399
it is different (ld-linux-aarch64.so.1) from all other targets.
(Why?) This makes Helgrind at least somewhat usable on arm64-linux.
git-svn-id: svn://svn.valgrind.org/valgrind/trunk@14396
At various places, there were either some assumption that the 'end'
boundary (highest address) was either not included, included,
or was the highest addressable word, or the highest addressable byte.
This e.g. was very visible when doing:
./vg-in-place -d -d ./helgrind/tests/tc01_simple_race|&grep regi
giving
--24040:2:stacks register 0xBEDB4000-0xBEDB4FFF as stack 0
--24040:2:stacks register 0x402C000-0x4A2C000 as stack 1
showing that the main stack end was (on x86) not the highest word
but the highest byte, while for the thread 1, the registered end
was a byte not part of the stack.
The attached patch ensures that stack bounds semantic are documented and
consistent. Also, some of the stack handling code is factorised.
The convention that the patch ensures and documents is:
start is the lowest addressable byte, end is the highest addressable byte.
(the words 'min' and 'max' have been kept when already used, as this wording is
consistent with the new semantic of start/end).
In various debug log, used brackets [ and ] to make clear that
both bounds are included.
The code to guess and register the client stack was duplicated
in all the platform specific syswrap-<plat>-<os>.c files.
Code has been factorised in syswrap-generic.c
The patch has been regression tested on
x86, amd64, ppc32/64, s390x.
It has been compiled and one test run on arm64.
Not compiled/not tested on darwin, android, mips32/64, arm
More in details, the patch does the following:
coregrind/pub_core_aspacemgr.h
include/valgrind.h
include/pub_tool_machine.h
coregrind/pub_core_scheduler.h
coregrind/pub_core_stacks.h
- document start/end semantic in various functions
also in pub_tool_machine.h:
- replaces unclear 'bottommost address' by 'lowest address'
(unclear as stack bottom is or at least can be interpreted as
the 'functional' bottom of the stack, which is the highest
address for 'stack growing downwards').
coregrind/pub_core_initimg.h
replace unclear clstack_top by clstack_end
coregrind/m_main.c
updated to clstack_end
coregrind/pub_core_threadstate.h
renamed client_stack_highest_word to client_stack_highest_byte
coregrind/m_scheduler/scheduler.c
computes client_stack_highest_byte as the highest addressable byte
Update comments in call to VG_(show_sched_status)
coregrind/m_machine.c
coregrind/m_stacktrace.c
updated to client_stack_highest_byte, and switched
stack_lowest/highest_word to stack_lowest/highest_byte accordingly
coregrind/m_stacks.c
clarify semantic of start/end,
added a comment to indicate why we invert start/end in register call
(note that the code find_stack_by_addr was already assuming that
end was included as the checks were doing e.g.
sp >= i->start && sp <= i->end
coregrind/pub_core_clientstate.h
coregrind/m_clientstate.c
renames Addr VG_(clstk_base) to Addr VG_(clstk_start_base)
(start to indicate it is the lowest address, base suffix kept
to indicate it is the initial lowest address).
coregrind/m_initimg/initimg-darwin.c
updated to VG_(clstk_start_base)
replace unclear iicii.clstack_top by iicii.clstack_end
updated clstack_max_size computation according to both bounds included.
coregrind/m_initimg/initimg-linux.c
updated to VG_(clstk_start_base)
updated VG_(clstk_end) computation according to both bounds included.
replace unclear iicii.clstack_top by iicii.clstack_end
coregrind/pub_core_aspacemgr.h
extern Addr VG_(am_startup) : clarify semantic of the returned value
coregrind/m_aspacemgr/aspacemgr-linux.c
removed a copy of a comment that was already in pub_core_aspacemgr.h
(avoid double maintenance)
renamed unclear suggested_clstack_top to suggested_clstack_end
(note that here, it looks like suggested_clstack_top was already
the last addressable byte)
* factorisation of the stack guessing and registration causes
mechanical changes in the following files:
coregrind/m_syswrap/syswrap-ppc64-linux.c
coregrind/m_syswrap/syswrap-x86-darwin.c
coregrind/m_syswrap/syswrap-amd64-linux.c
coregrind/m_syswrap/syswrap-arm-linux.c
coregrind/m_syswrap/syswrap-generic.c
coregrind/m_syswrap/syswrap-mips64-linux.c
coregrind/m_syswrap/syswrap-ppc32-linux.c
coregrind/m_syswrap/syswrap-amd64-darwin.c
coregrind/m_syswrap/syswrap-mips32-linux.c
coregrind/m_syswrap/priv_syswrap-generic.h
coregrind/m_syswrap/syswrap-x86-linux.c
coregrind/m_syswrap/syswrap-s390x-linux.c
coregrind/m_syswrap/syswrap-darwin.c
coregrind/m_syswrap/syswrap-arm64-linux.c
Some files to look at more in details:
syswrap-darwin.c : the handling of sysctl(kern.usrstack) looked
buggy to me, and has probably be made correct by the fact that
VG_(clstk_end) is now the last addressable byte. However,unsure
about this, as I could not find any documentation about
sysctl(kern.usrstack). I only find several occurences on the web,
showing that the result of this is page aligned, which I guess
means it must be 1+ the last addressable byte.
syswrap-x86-darwin.c and syswrap-amd64-darwin.c
I suspect the code that was computing client_stack_highest_word
was wrong, and the patch makes it correct.
syswrap-mips64-linux.c
not sure what to do for this code. This is the only code
that was guessing the stack differently from others.
Kept (almost) untouched. To be discussed with mips maintainers.
coregrind/pub_core_libcassert.h
coregrind/m_libcassert.c
* void VG_(show_sched_status):
renamed Bool valgrind_stack_usage to Bool stack_usage
if stack_usage, shows both the valgrind stack usage and
the client stack boundaries
coregrind/m_scheduler/scheduler.c
coregrind/m_gdbserver/server.c
coregrind/m_gdbserver/remote-utils.c
Updated comments in callers to VG_(show_sched_status)
git-svn-id: svn://svn.valgrind.org/valgrind/trunk@14392
getdents has been deprecated since linux 2.4 and newer arches (arm64)
might no longer provide the getdents syscall. Use getdents64 for reading
the /proc/self/fd/ dir so --track-fds=yes works reliable on all arches.
Without this the none/tests/fdleak*vgtest might fail.
git-svn-id: svn://svn.valgrind.org/valgrind/trunk@14384
This caused memcheck/tests/linux/proc-auxv.vgtest to fail because
our internal VG(lseek) would return ENOSYS.
git-svn-id: svn://svn.valgrind.org/valgrind/trunk@14383
of clo which are (or should be) 'enum set'.
* pub_tool_options.h : add new macrox VG_USET_CLO and VG_USETX_CLO to
parse an 'enum set' command line option (with or without "all" keyword).
* use VG_USET_CLO for existing enum set clo options:
memcheck --errors-for-leak-kinds, --show-leak-kinds, --leak-check-heuristics
coregrind --vgdb-stop-at
* change --sim-hints and --kernel-variants to enum set
(this allows to detect user typos: currently, a typo in a sim-hint
or kernel variant is silently ignored. Now, an error will be given
to the user)
* The 2 new sets (--sim-hints and --kernel-variants) should not make
use of the 'all' keyword => VG_(parse_enum_set) has a new argument
to enable/disable the use of the "all" keyword.
* The macros defining an 'all enum' set definition was duplicating
all enum values (so addition of a new enum value could easily
give a bug). Removing these macros as they are unused
(to the exception of the leak-kind set).
For this set, the 'all macro' has been replaced by an 'all function',
coded using parse_enum_set parsing the "all" keyword.
git-svn-id: svn://svn.valgrind.org/valgrind/trunk@14301
(used for ppc64 platforms) #ifdef-ed and accessed by macros
that becomes NOP on non ppc64 platforms.
This decreases the debuginfo memory by about 2.5 Mb on a big 32 bit application.
Note : doing that, some questions were encountered in the way
tocptr and local_ep have (or do not have) to be copied/maintained
in storage.c canonicaliseSymtab
git-svn-id: svn://svn.valgrind.org/valgrind/trunk@14273
around VG_(sysnum_string). Also remove associated macro
VG_SYSNUM_STRING_EXTRA.
The VG_SYSNUM_STRING macro returned a pointer to a variable which
is out of scope. Using that value may cause undefined behaviour.
Change VG_(sysnum_string) to return pointer to static buffer instead.
Fix call sites.
git-svn-id: svn://svn.valgrind.org/valgrind/trunk@14264
to add PPC64 LE support. The other two patches can be found in Bugzillas
334384 and 334836.
POWER PC, add the functional Little Endian support, patch 2
The IBM POWER processor now supports both Big Endian and Little Endian.
The ABI for Little Endian also changes. Specifically, the function
descriptor is not used, the stack size changed, accessing the TOC
changed. Functions now have a local and a global entry point. Register
r2 contains the TOC for local calls and register r12 contains the TOC
for global calls. This patch makes the functional changes to the
Valgrind tool. The patch makes the changes needed for the
none/tests/ppc32 and none/tests/ppc64 Makefile.am. A number of the
ppc specific tests have Endian dependencies that are not fixed in
this patch. They are fixed in the next patch.
Per Julian's comments renamed coregrind/m_dispatch/dispatch-ppc64-linux.S
to coregrind/m_dispatch/dispatch-ppc64be-linux.S Created new file for LE
coregrind/m_dispatch/dispatch-ppc64le-linux.S. The same was done for
coregrind/m_syswrap/syscall-ppc-linux.S.
Signed-off-by: Carl Love <carll@us.ibm.com>
git-svn-id: svn://svn.valgrind.org/valgrind/trunk@14239
to add PPC64 LE support. The other two patches can be found in Bugzillas
334834 and 334836. The commit does not have a VEX commit associated with it.
POWER PC, add initial Little Endian support
The IBM POWER processor now supports both Big Endian and Little Endian.
This patch renames the #defines with the name ppc64 to ppc64be for the BE
specific code. This patch adds the Little Endian #define ppc64le to the
Additionally, a few functions are renamed to remove BE from the name if the
function is used by BE and LE. Functions that are BE specific have BE put
in the name.
The goals of this patch is to make sure #defines, function names and
variables consistently use PPC64/ppc64 if it refers to BE and LE,
PPC64BE/ppc64be if it is specific to BE, PPC64LE/ppc64le if it is LE
specific. The patch does not break the code for PPC64 Big Endian.
The test files memcheck/tests/atomic_incs.c, tests/power_insn_available.c
and tests/power_insn_available.c are also updated to the new #define
definition for PPC64 BE.
Signed-off-by: Carl Love <carll@us.ibm.com>
git-svn-id: svn://svn.valgrind.org/valgrind/trunk@14238
rather than throwing to the default case. This stops Memcheck
reporting false positives for the NETLINK case.
git-svn-id: svn://svn.valgrind.org/valgrind/trunk@14237
Some syscall verification code is allocating memory to generate
the string used to build an error, e.g. syswrap-generic.c verifying fields of
e.g socket addresses (pre_mem_read_sockaddr) or sendmsg/recvmsg args
(msghdr_foreachfield)
The allocated pointer was copied in the error created by VG_(maybe_record_error).
This was wrong for 2 reasons:
1. If the error is a new error, it is stored in a list of errors,
but the string memory was freed by pre_mem_read_sockaddr, msghdr_foreachfield, ...
This causes a dangling reference. Was at least visible when giving -v, which
re-prints all errors at the end of execution.
Probably this could have some consequences during run while generating new errors,
and comparing for equality with a recorded error having a dangling reference.
2. the same allocated string is re-used for each piece/field of the verified struct.
The code in mc_errors.c that checks that 2 errors are identical was then wrongly
considereing that 2 successive errors for 2 different fields for the same syscall
arg are identical, just because the error string happened to be produced at
the same address.
(it is believed that initially, the error string was assumed to be a static
string, which is not the case anymore, causing the above 2 problems).
Changes:
* The fix consists in duplicating in m_errormgr.c the given error string when
the error is recorded. In other words, the error string is now duplicated similarly
to the (optional) extra component of the error.
* memcheck/tests/linux/rfcomm.c test modified as now an error is reported
for each uninit field.
* socketaddr unknown family is also better reported (using sa_data field name,
rather than an empty field name.
* minor reformatting in m_errormgr.c, to be below 80 characters.
Some notes:
1. the string is only duplicated if the error is recorded
(ie. printed or the first time an error matches a suppression).
The string is not duplicated for duplicated errors or following errors
matching the first (suppressed) error.
The string is also not duplicated for 'unique errors' (that are printed
and then not recorded).
2. duplicating the string for each recorded error is not deemed to
use a lot of memory:
* error strings are usually NULL or short (often 10 bytes or so).
* we expect no program has a huge number of errors
If ever this string duplicate would be significant, having a DedupPoolAlloc
in m_errormgr.c for these strings would reduce this memory (as we expect to
have very few different strings, even with millions of errors).
git-svn-id: svn://svn.valgrind.org/valgrind/trunk@14214
and stack address description.
* A race condition on an allocated block shows the stacktrace, but
does not show the thread # that allocated the block.
This patch adds the output of the thread # that allocated the block.
* The patch also fixes the confusion that might appear between
the core threadid and the helgrind thread nr in Stack address description:
A printed stack addrinfo was containing a thread id, while all other helgrind
messages are using (supposed to use) an 'helgrind thread #' which
is used in the thread announcement.
Basically, the idea is to let a tool set a "tool specific thread nr'
in an addrinfo.
The pretty printing of the addrinfo is then by preference showing this
thread nr (if it was set, i.e. different of 0).
Currently, only helgrind uses this addrinfo tnr.
Note: in xml mode, the output is matching the protocol description.
I.e., GUI should not be impacted by this change, if they properly implement
the xml protocol.
* Also, make the output produced by m_addrinfo consistent:
The message 'block was alloc'd at' is changed to be like all other
output : one character indent, and starting with an uppercase
git-svn-id: svn://svn.valgrind.org/valgrind/trunk@14175
The dedup pool can now be used to allocate elements and identify
them with a number rather than an address.
This new feature is not used (yet) but is intended to be used to
decrease the memory needed to store the CFSI information.
git-svn-id: svn://svn.valgrind.org/valgrind/trunk@14123
only decrease the size of a block, does not change the address,
does not need to alloc another block and copy the memory,
and (if big enough) makes the excess memory available for other
allocations.
VG_(arena_realloc_shrink) is then used for debuginfo storage.c
(replacing an allocation + copy).
Also use it in the dedup pool, to recuperate the unused
memory of the last pool.
This also allows to re-increase the string pool size to the original
3.9.0 value of 64Kb. All this slightly decrease the peak and in use
memory of dinfo.
VG_(arena_realloc_shrink) will also be used to implement (in another patch)
a dedup pool which "numbers" the allocated elements.
git-svn-id: svn://svn.valgrind.org/valgrind/trunk@14122
showing inlined function calls.
See 278972 valgrind stacktraces and suppression do not handle inlined function call debuginfo
Reading the inlined dwarf call info is activated using the new clo
--read-inline-info=yes
Default is currently no but an objective is to optimise the performance
and memory in order to possibly set it on by default.
(see below discussion about performances).
Basically, the patch provides the following pieces:
1. Implement a new dwarf3 reader that reads the inlined call info
2. Some performance improvements done for this new parser, and
on some common code between the new parser and the var info parser.
3. Use the parsed inlined info to produce stacktrace showing inlined calls
4. Use the parsed inlined info in the suppression matching and suppression generation
5. and of course, some reg tests
1. new dwarf3 reader:
---------------------
Two options were possible: add the reading of the inlined info
in the current var info dwarf reader, or add a 2nd reader.
The 2nd approach was preferred, for the following reasons:
The var info reader is slow, memory hungry and quite complex.
Having a separate parsing phase for the inlined information
is simpler/faster when just reading the inlined info.
Possibly, a single parser would be faster when using both
--read-var-info=yes and --read-inline-info=yes.
However, var-info being extremely memory/cpu hungry, it is unlikely
to be used often, and having a separate parsing for inlined info
does in any case make not much difference.
(--read-var-info=yes is also now less interesting thanks to commit
r13991, which provides a fast and low memory "reasonable" location
for an address).
The inlined info parser reads the dwarf info to make calls
to priv_storage.h ML_(addInlInfo).
2. performance optimisations
----------------------------
* the abbrev cache has been improved in revision r14035.
* The new parser skips the non interesting DIEs
(the var-info parser has no logic to skip uninteresting DIEs).
* Some other minor perf optimisation here and there.
In total now, on a big executable, 15 seconds CPU are needed to
create the inlined info (on my slow x86 pentium).
With regards to memory, the dinfo arena:
with inlined info: 172281856/121085952 max/curr mmap'd
without : 157892608/106721280 max/curr mmap'd,
So, basically, inlined information costs about 15Mb of memory for
my big executable (compared to first version of the patch, this is
already using less memory, thanks to the strpool deduppoolalloc.
The needed memory can probably be decreased somewhat more.
3. produce better stack traces
------------------------------
VG_(describe_IP) has a new argument InlIPCursor *iipc which allows
to describe inlined function calls by doing repetitive calls
to describe_IP. See pub_tool_debuginfo.h for a description.
4. suppression generation and matching
--------------------------------------
* suppression generation now also uses an InlIPCursor *iipc
to generate a line for each inlined fn call.
* suppression matching: to allow suppression matching to
match one IP to several function calls in a suppression entry,
the 'inputCompleter' object (that allows to lazily generate
function or object names for a stacktrace when matching
an error with a suppression) has been generalised a little bit
more to also lazily generate the input sequence.
VG_(generic_match) has been updated so as to be more generic
with respect to the input completer : when providing an
input completer, VG_(generic_match) does not need anymore
to produce/compute any input itself : this is all delegated
to the input completer.
5. various regtests
-------------------
to test stack traces with inlined calls, and suppressions
of (some of) these errors using inlined fn calls matching.
Work still to do:
-----------------
* improve parsing performance
* improve the memory overhead.
* handling the directory name for files of the inlined function calls is not yet done.
(probably implies to refactor some code)
* see if m_errormgr.c *offsets arrays cannot be managed via xarray
git-svn-id: svn://svn.valgrind.org/valgrind/trunk@14036
include/pub_tool_deduppoolalloc.h
coregrind/pub_core_deduppoolalloc.h
coregrind/m_deduppoolalloc.c
and uses it (currently only) for the strings in m_debuginfo/storage.c
The idea is that such ddup pool allocator will also be used for other
highly duplicated information (e.g. the DiCFSI information), where
significant gains can also be achieved.
The dedup pool for strings also decreases significantly the memory
needed by the read inline information (patch still to be committed,
see bug 278972).
When testing with a big executable (tacot_process),
this reduces the size of the dinfo arena from
trunk: 158941184/109760512 max/curr mmap'd, 156775944/107882728 max/curr,
to
ddup: 157892608/106614784 max/curr mmap'd, 156362160/101414712 max/curr
(so 3Mb less mmap-ed once debug info is read, 1Mb less mmap-ed in peak,
6Mb less allocated once debug info is read).
This is all gained due to the string which changes from:
trunk: 17,434,704 in 266: di.storage.addStr.1
to
ddup: 10,966,608 in 750: di.storage.addStr.1
(6.5Mb less memory used by strings)
The gain in mmap-ed memory is smaller due to fragmentation.
Probably one could decrease the fragmentation by using bigger
size for the dedup pool, but then we would lose memory on the last
allocated pool (and for small libraries, we often do not use much
of a big pool block).
Solution might be to increase the pool size but have a "shrink_block"
operation. To be looked at in the future.
In terms of performance, startup of a big executable (on an old pentium)
is not influenced significantly (something like 0.1 seconds on 15 seconds
startup for a big executable, on a slow pentium).
The dedup pool uses a hash table. The hash function used currently
is the VG_(adler32) check sum. It is reported (and visible also here)
that this checksum is not a very good hash function (many collisions).
To have statistics about collisions, use --stats -v -v -v
As an example of the collisions, on the strings in debug info of memcheck tool on x86,
one obtain:
--4789-- dedupPA:di.storage.addStr.1 9983 allocs (8174 uniq) 11 pools (4820 bytes free in last pool)
--4789-- nr occurences of chains of len N, N-plicated keys, N-plicated elts
--4789-- N: 0 : nr chain 6975, nr keys 0, nr elts 0
--4789-- N: 1 : nr chain 3670, nr keys 6410, nr elts 8174
--4789-- N: 2 : nr chain 1070, nr keys 226, nr elts 0
--4789-- N: 3 : nr chain 304, nr keys 100, nr elts 0
--4789-- N: 4 : nr chain 104, nr keys 84, nr elts 0
--4789-- N: 5 : nr chain 72, nr keys 42, nr elts 0
--4789-- N: 6 : nr chain 44, nr keys 34, nr elts 0
--4789-- N: 7 : nr chain 18, nr keys 13, nr elts 0
--4789-- N: 8 : nr chain 17, nr keys 8, nr elts 0
--4789-- N: 9 : nr chain 4, nr keys 6, nr elts 0
--4789-- N:10 : nr chain 9, nr keys 4, nr elts 0
--4789-- N:11 : nr chain 1, nr keys 0, nr elts 0
--4789-- N:13 : nr chain 1, nr keys 1, nr elts 0
--4789-- total nr of unique chains: 12289, keys 6928, elts 8174
which shows that on 8174 different strings, we have only 6410 strings which have
a unique hash value. As other examples, N:13 line shows we have 13 strings
mapping to the same key. N:14 line shows we have 4 groups of 10 strings mapping to the
same key, etc.
So, adler32 is definitely a bad hash function.
Trials have been done with another hash function, giving a much lower
collision rate. So, a better (but still fast) hash function would probably
be beneficial. To be looked at ...
git-svn-id: svn://svn.valgrind.org/valgrind/trunk@14029
--read-var-info=yes is very memory and cpu intensive.
This patch ensures that even witout --read-var-info=yes that
the frame where the address point is reported in the address
description.
git-svn-id: svn://svn.valgrind.org/valgrind/trunk@13991
This reverts valgrind svn r13975. This was a work in progress, still being
discussed in bug #334802. It should not yet been pushed.
git-svn-id: svn://svn.valgrind.org/valgrind/trunk@13978
Add an explanation of why an option was bad to fmsg_bad_option calls that
were just using "" as argument. Fixes bug #334802.
git-svn-id: svn://svn.valgrind.org/valgrind/trunk@13975
that mingw64 also defines __MINGW32__, which led to the 32-bit definitions
being used in the 64-bit case. n-i-bz. (Bernhard.Loos@ruecker.de)
git-svn-id: svn://svn.valgrind.org/valgrind/trunk@13971
of memcheck and helgrind in a common module:
pub_tool_addrinfo.h pub_core_addrinfo.h m_addrinfo.c
At the same time, the factorised code is made usable by other
tools also (and is used by the gdbserver command 'v.info location'
which replaces the helgrind 'describe addr' introduced 1 week ago
and which is now callable by all tools).
The new address description code can describe more addresses
(e.g. for memcheck, if the block is not on the free list anymore,
but is in an arena free list, this will also be described).
Similarly, helgrind address description can now describe more addresses
when --read-var-info=no is given (e.g. global symbols are
described, or addresses on the stack are described as
being on the stack, freed blocks in the arena free list are
described, ...).
See e.g. the change in helgrind/tests/annotate_rwlock.stderr.exp
or locked_vs_unlocked2.stderr.exp
The patch touches many files, but is basically a lot of improvements
in helgrind output files.
The code changes are mostly refactorisation of existing code.
git-svn-id: svn://svn.valgrind.org/valgrind/trunk@13965
