slightly increases the performance. It also moderately improves
the nr of cases where helgrind can provide the stack trace of the old
access (when using the same amount of memory for the OldRef entries).
The patch also provides a new helgrind monitor command to show
the recorded accesses for an address+len, and adds an optional argument
lock_address to the monitor command 'info locks', to show the info
about just this lock.
Currently, oldref are maintained in a sparse WA, that points to N
entries, as specified by --conflict-cache-size=N.
For each entry (associated to an address), we have the last 5 accesses.
Old entries are recycled in an exact LRU order.
But inside an entry, we could have a recent access, and 4 very
old accesses that are kept 'alive' by a single thread accessing
repetitively the address shared with the 4 other old entries.
The attached patch replaces the sparse WA that maintains the OldREf
by an hash table.
Each OldRef now also only maintains one single access for an address.
As an OldRef now maintains only one access, all the entries are now
strictly in LRU mode.
Memory used for OldRef
-----------------------
For the trunk, an OldRef has a size of 72 bytes (on 32 bits archs)
maintaining up to 5 accesses to the same address.
On 64 bits arch, an OldRef is 104 bytes.
With the patch, an OldRef has a size of 32 bytes (on 32 bits archs)
or 56 bytes (on 64 bits archs).
So, for one single access, the new code needs (on 32 bits)
32 bytes, while the trunk needs only 14.4 bytes.
However, that is the worst case, assuming that the 5 entries in the
accs array are all used.
Looking on 2 big apps (one of them being firefox), we see that
we have very few OldRef entries that have the 5 entries occupied.
On a firefox startup, of the 5x1,000,000 accesses, we only have
1,406,939 accesses that are used.
So, in average, the trunk uses in reality around 52 bytes per access.
The default value for --conflict-cache-size has been doubled to 2000000.
This ensures that the memory used for the OldRef is more or less the
same as the trunk (104Mb for OldRef entries).
Memory used for sparseWA versus hashtable
-----------------------------------------
Looking on 2 big apps (one of them being firefox), we see that
there are big variations on the size of the WA : it can go in a few
seconds from 10MB to 250MB, or can decrease back to 10 MB.
This all depends where the last N accesses were done: if well localised,
the WA will be small.
If the last N accesses were distributed over a big address space,
then the WA will be big: the last level of WA (the biggest memory consumer)
uses slightly more than 1KB (2KB on 64 bits) for each '256 bytes' memory
zone where there is an oldref. So, in the worst case, on 32 bits, we
need > 1_000_000_000 sparseWA memory to keep 1_000_000 OldRef.
The hash table has between 1 to 2 Word overhead per OldRef
(as the chain array is +- doubled each time the hash table is full).
So, unless the OldRef are extremely localised, the overhead of the
hash table will be significantly less.
With the patch, the core arena total alloc is:
5299535/1201448632 totalloc-blocks/bytes
The trunk is
6693111/3959050280 totalloc-blocks/bytes
(so, around 1.20Gb versus 3.95Gb).
This big difference is due to the fact that the sparseWA repetitively
allocates then frees Level0 or LevelN when OldRef in the region covered
by the Level0/N have all been recycled.
In terms of CPU
---------------
With the patch, on amd64, a firefox startup seems slightly faster (around 1%).
The peak memory mmaped/used decreases by 200Mb.
For a libreoffice test, the memory decreases by 230Mb. CPU also decreases
slightly (1%).
In terms of correctness:
-----------------------
The trunk could potentially show not the most recent access
to the memory of a race : the first OldRef entry matching the raced upon
address was used, while we could have a more recent access in a following
OldRef entry. In other words, the trunk only guaranteed to find the
most recent access in an OldRef, but not between the several OldRef that
could cover the raced upon address.
So, assuming it is important to show the most recent access, this patch
ensures we really show the most recent access, even in presence of overlapping
accesses.
git-svn-id: svn://svn.valgrind.org/valgrind/trunk@15289
of the dynamically allocated Lock addresses.
This restores helgrind/tests/locked_vs_unlocked2.stderr.exp
from r14931.
While regtesting the patch I've observed intermittent failures
of helgrind/tests/hg05_race2 like so:
--- ../../helgrind/tests/hg05_race2.stderr.exp (revision 15001)
+++ ../../helgrind/tests/hg05_race2.stderr.exp (working copy)
@@ -26,8 +26,7 @@
at 0x........: th (hg05_race2.c:17)
by 0x........: mythread_wrapper (hg_intercepts.c:...)
...
- Location 0x........ is 0 bytes inside foo.poot[5].plop[11],
- declared at hg05_race2.c:24, in frame #x of thread x
+ Address 0x........ is on thread #x's stack
----------------------------------------------------------------
@@ -42,8 +41,7 @@
at 0x........: th (hg05_race2.c:17)
by 0x........: mythread_wrapper (hg_intercepts.c:...)
...
- Location 0x........ is 0 bytes inside foo.poot[5].plop[11],
- declared at hg05_race2.c:24, in frame #x of thread x
+ Address 0x........ is on thread #x's stack
Surely, that's something else.
git-svn-id: svn://svn.valgrind.org/valgrind/trunk@15009
anonymous or file mmap-ed segments and shared memory segments.
* pub_tool_addrinfo.h:
new AddrTag Addr_SegmentKind // Client segment (mapped memory)
new struct SegmentKind in AddrInfo
* m_addrinfo.c:
If address is still undescribed, try to describe by findinf a client segment.
* update various tests
* mc_errors.c:
add a call to VG_(clear_addrinfo) in MC_(pp_describe_addr)
as the memory allocated in the local AddrInfo has to be cleared once
info is printed.
git-svn-id: svn://svn.valgrind.org/valgrind/trunk@14779
Two things:
- remove the buffer argument from VG_(DebugInfo_sect_kind)
- allocate AddrInfo::SectKind::objname dynamically
git-svn-id: svn://svn.valgrind.org/valgrind/trunk@14719
The change eliminates the fixed size buffers in gen_suppression and
show_used_suppressions. This is achieved by changing the return type from
VG_TDICT_CALL(tool_get_extra_suppression_info and
VG_TDICT_CALL(tool_print_extra_suppression_use from Bool to SizeT.
A return value of 0 indicates that nothing (except the terminating '\0'
which is always inserted) was written to the buffer. This corresponds to the
previous False return value. A return value which is equal to the buffer
size (that was passed in as function argument) indicates that the buffer was
not large enough. The caller then resizes the buffer and retries.
Otherwise, the buffer was large enough.
Regtested with a resize value of 1.
git-svn-id: svn://svn.valgrind.org/valgrind/trunk@14606
First, as the allocator function does not fail, there is no need
to assert its return value.
Second, remove commented out (since r8765) function VG_(isEmptyFM).
Third, remove VG_(getNodeSizeFM) from the API. The details of the
implementation do not need to be exposed.
Fourth, for consistency require that the copy functions for keys and
values in VG_(dopyFM) (which are essentially like allocators) return
non-NULL values for non-NULL arguments if they return.
Fifth, document NULL-ness of return values for VG_(newFM), VG_(dopyFM),
and VG_(newBag). Remove pointless asserts at call sites.
Six, change avl_dopy to assert that the node the function is
supposed to copy is not NULL. It is called that way anyhow. With
that change the function never returns NULL which allows us to
simplify the call sites. Checking the return value is no longer needed.
git-svn-id: svn://svn.valgrind.org/valgrind/trunk@14535
There are a couple of issues with helgrind on arm-linux with glibc:
- Thread creation stack traces cannot unwind through clone
(cfi ends right after syscall)
- ld.so has a special "hard float" name that isn't recognized as special
(ld-linux-armhf.so.3)
- Races are found when manipulating GOT sections.
git-svn-id: svn://svn.valgrind.org/valgrind/trunk@14407
* Add lock announcements in various helgrind errors that were not
announcing the locks
* ensure locks are also announced in xml (note that this is compatible
with xml protocol version 4, so no impact on GUI which properly
implement the protocol)
Changes done:
* Like other HG record_error functions, HG_(record_error_LockOrder) is
now passing Lock* rather than lock guest addresses.
* update exp files for tests that were showing locks without announcing them
* change tc14_laog_dinphils.c and tc15_laog_lockdel.c so as to
have same sizes on 32 and 64 bits systems for allocated or symbol sizes.
* factorise all code that was announcing first lock observation
* enable xml lock announcement
git-svn-id: svn://svn.valgrind.org/valgrind/trunk@14204
(note that some error messages are not announcing the lock,
which is not that nice).
At least the lock order violation message do not announce locks.
That should be improved/fixed
git-svn-id: svn://svn.valgrind.org/valgrind/trunk@14188
and stack address description.
* A race condition on an allocated block shows the stacktrace, but
does not show the thread # that allocated the block.
This patch adds the output of the thread # that allocated the block.
* The patch also fixes the confusion that might appear between
the core threadid and the helgrind thread nr in Stack address description:
A printed stack addrinfo was containing a thread id, while all other helgrind
messages are using (supposed to use) an 'helgrind thread #' which
is used in the thread announcement.
Basically, the idea is to let a tool set a "tool specific thread nr'
in an addrinfo.
The pretty printing of the addrinfo is then by preference showing this
thread nr (if it was set, i.e. different of 0).
Currently, only helgrind uses this addrinfo tnr.
Note: in xml mode, the output is matching the protocol description.
I.e., GUI should not be impacted by this change, if they properly implement
the xml protocol.
* Also, make the output produced by m_addrinfo consistent:
The message 'block was alloc'd at' is changed to be like all other
output : one character indent, and starting with an uppercase
git-svn-id: svn://svn.valgrind.org/valgrind/trunk@14175
of memcheck and helgrind in a common module:
pub_tool_addrinfo.h pub_core_addrinfo.h m_addrinfo.c
At the same time, the factorised code is made usable by other
tools also (and is used by the gdbserver command 'v.info location'
which replaces the helgrind 'describe addr' introduced 1 week ago
and which is now callable by all tools).
The new address description code can describe more addresses
(e.g. for memcheck, if the block is not on the free list anymore,
but is in an arena free list, this will also be described).
Similarly, helgrind address description can now describe more addresses
when --read-var-info=no is given (e.g. global symbols are
described, or addresses on the stack are described as
being on the stack, freed blocks in the arena free list are
described, ...).
See e.g. the change in helgrind/tests/annotate_rwlock.stderr.exp
or locked_vs_unlocked2.stderr.exp
The patch touches many files, but is basically a lot of improvements
in helgrind output files.
The code changes are mostly refactorisation of existing code.
git-svn-id: svn://svn.valgrind.org/valgrind/trunk@13965
- Helgrind GDB server monitor command 'describe <address>'
allowing to describe an address (e.g. where it was allocated).
- Helgrind GDB server monitor command 'info locks' giving
the list of locks, their location, and their status.
In a further patch, it is intended to
1. factorise the describe address code between memcheck and helgrind
2. generalise the describe address gdbsrv command so as to make
it usable for all tools.
git-svn-id: svn://svn.valgrind.org/valgrind/trunk@13930
Option -v outputs a list of used suppressions. This only gives
the nr of times a suppression was used.
For a leak search, this only gives the nr of loss records that
have been suppressed, but it does not give additional needed details
to understand more precisely what has been suppressed
(i.e. nr of blocks and nr of bytes).
=> Add in the tool interface update_extra_suppression_use and
print_extra_suppression_info functions to allow the tool to record
additioonal use statistics for a suppression. These statistics
can be done depending on the error (and its data) which is suppressed.
Use this in memcheck for the leak suppressions, to maintain and output
the nr of blocks and bytes suppressed by a suppression during
the last leak search.
git-svn-id: svn://svn.valgrind.org/valgrind/trunk@13651
If a suppression file contains an error, the lineno reported could be wrong.
Also, give filename and lineno of the used suppressions in -v debugging output.
The fix consists in ensuring that tool specific read_extra function gets
the Int* lineno pointer, together with other VG_(get_line) parameters.
git-svn-id: svn://svn.valgrind.org/valgrind/trunk@13469
Also fix all usages of the wordFM data structure. Once upon a time
wordFM used Words but now it uses UWords.
Likewise for WordBag.
git-svn-id: svn://svn.valgrind.org/valgrind/trunk@13070
* performance and scalability improvements
* show locks held by both threads in a race
* show all 4 locks involved in a lock order violation
* better delimited error messages
git-svn-id: svn://svn.valgrind.org/valgrind/trunk@11824
to previous behaviour, in which it was constructed but any resulting
errors were not shown, hence wasting CPU and memory.) Partial fix
for #255353. (Philippe Waroquiers, philippe.waroquiers@skynet.be)
git-svn-id: svn://svn.valgrind.org/valgrind/trunk@11574
lock, also report the stack where the lock was previously locked.
This makes it easier to diagnose deadlocks.
git-svn-id: svn://svn.valgrind.org/valgrind/trunk@11234
* Add new client request VALGRIND_HG_CLEAN_MEMORY_HEAPBLOCK. This is
like VALGRIND_HG_CLEAN_MEMORY but doesn't take an address range.
Instead it takes a single argument which is supposed to be a pointer
to the start of, or anywhere within, a heap allocated block.
Helgrind then finds the block and paints it as belonging to the
calling thread. This is needed for correctly describing the
behaviour of threadsafe reference counting when applied to classes
involving inheritance of release methods or involving multiple
inheritance.
* Add statistics counters for all basic VTS operations (tick, join,
cmpLEQ, cmp_structural).
* Rewrite VTS__cmp_structural to be much faster.
git-svn-id: svn://svn.valgrind.org/valgrind/trunk@11123
is inside a heap block, and if so, print the allocation point of the
heap block. It's stupid not to do this considering that the
implementation already keeps track of all mallocs and frees.
git-svn-id: svn://svn.valgrind.org/valgrind/trunk@11089
both wrapped up in XML tags (as before) but also in plain text in a
sequence of CDATA blocks. Normally only one, but in the worst case
the raw data will have ]]> in it, in which case it needs to be split
across two CDATA blocks.
This apparently simple change involved a lot of refactoring of the
suppression printing machinery:
* in the core-tool iface, change "print_extra_suppression_info" (which
prints any auxiliary info) to "get_extra_suppression_info", which
parks the text in a caller-supplied buffer. Adjust tools to match.
* VG_(apply_StackTrace): accept a void* argument, which is passed to
each invokation of the functional parameter (a poor man's closure
implementation).
* move PRINTF_CHECK into put_tool_basics.h, where it should have been
all along
* move private printf-into-an-XArray-of-character functions from
m_debuginfo into m_xarray, and make them public
* gen_suppression itself: use all the above changes. Basically we
always generate the plaintext version into an XArray. In text mode
that's just printed. In XML mode, we print the XMLery as before,
but the plaintext version is dumped into a CDATA block too.
* update the Protocol 4 specification to match all this.
This still isn't 100% right in the sense that the CDATA block data
needs to be split across multiple blocks if it should ever contain the
CDATA end mark "]]>". The Protocol 4 spec has this right even though
the implementation currently doesn't.
Fixes#191189.
git-svn-id: svn://svn.valgrind.org/valgrind/trunk@10822
* VG_(find_seginfo): incrementally rearrange the DebugInfo list, like
most of the other list-searching functions do.
* rename all VG_(*seginfo*) functions exported from m_debuginfo to
VG_(*DebugInfo*). "seginfo" was a historical name which was mostly
but not completely, done away with some time back.
git-svn-id: svn://svn.valgrind.org/valgrind/trunk@10678
* rename many functions to do with shadow memory handling, to
more clearly differentiate reads and writes directly of the
shadow state from client reads and writes, each of which
generate both a read and a write of the client state. It was
getting confusing (== hard to verify) in there.
* use idempotency of memory state machine transition rules to
speed up long sequential sections, speedups in range 0% to 28%
* remove 4-way Pord (EQ, LT, GT, UN) and associated machinery,
and replace it with something that merely computes LEQ in the
partial ordering, since that's all that is necessary, and
this simplifies some fast-case paths.
* add optional approx history mechanism a la DRD (start/end stack
of conflicting segment), much faster if you don't need exact
conflicting-access details
* libhb_so_recv: tick the VTS in the receiving thread; don't just
join with the VC in the SO. It's probably correct without this
modification, but that correctness is fragile and depends on
complex properties of how SOs are used/created. Much better to
be completely safe. (Needs cache-isation).
* get rid of unnecessary shadow memory state "SVal_NOACCESS"
and simplify associated fast-case paths in msmc{read,write}
git-svn-id: svn://svn.valgrind.org/valgrind/trunk@10589
were longer than 200 chars. Now dynamic memory is used and so they can be
arbitrarily long in theory, although in practice it bombs out at 100,000 for
sanity purposes. This required adjusting the core/tool interface for
read_extra_suppression_info().
git-svn-id: svn://svn.valgrind.org/valgrind/trunk@10581
find conflicting accesses that overlap the current access in any way,
rather than just match at the addresses. This allows reporting of
conflicts between accesses which overlap but are not the same
size/alignment.
Doesn't seem to always work reliably, for reasons I don't understand,
but I so far failed to make a small test case.
git-svn-id: svn://svn.valgrind.org/valgrind/trunk@8811
read-or-writeness of each access, so that these can be displayed in
error messages.
* Use recorded read-or-writeness info to avoid producing error
messages that claim claim two reads race against each other -- this
is clearly silly. For each pair of racing accesses now reported, at
least one of them will (should!) always now be a write, and (as
previously ensured) they will be from different threads.
* Lookups in the conflicting-access map is expensive, so don't do that
as soon as a race is detected. Instead wait until the update_extra
method is called.
git-svn-id: svn://svn.valgrind.org/valgrind/trunk@8809
minor changes to make stack unwinding on amd64-linux approximately
twice as fast as it was before.
git-svn-id: svn://svn.valgrind.org/valgrind/trunk@8707
