Commit Graph

57 Commits

Author SHA1 Message Date
Philippe Waroquiers
2bd926a725 outer/inner setup: new perf/vg_perf options to run perf tests + support translation chaining in inner.
* perf/vg_perf:
Similarly to tests/vg_regtest, perf/vg_perf now accepts the 3 
optional arguments:
    --outer-valgrind
    --outer-tool
    --outer-args

This allows easy analysis or comparison of performance between
different Valgrind versions (e.g. using callgrind, or cachegrind/cg_diff).

* See README_DEVELOPERS for more details.

* vg_regtest modified so as to use the 'in-place' build of inner, rather
  than the installed version.

* added option --smc-check=all-non-file to vg_perf and vg_regtest 
  outer default arguments (needed when evaluating a Valgrind which does
  translation chaining).




git-svn-id: svn://svn.valgrind.org/valgrind/trunk@12496
2012-04-08 19:52:38 +00:00
Philippe Waroquiers
aa50a7e4df Allows to run regression tests in an outer/inner setup.
A '3 lines how to':
   perl tests/vg_regtest --outer-valgrind=../trunk_untouched/install/bin/valgrind --all
           (the outer results for a test xxx is in xxx.outer.log)
   To run with another tool (e.g. drd), add the argument --outer-tool=drd


Still to do/things to improve:

* Most (inner) tests are successful when running under an outer
  memcheck. Need to analyse the reasons of remaining failures.

* The memcheck annotations in m_mallocfree.c can be improved:
  - A superblock is marked 'undefined', it should rather be marked
    'no access'.
  - When a free block is splitted, the remaining free block is
    not made 'no access'. Instead, it is made 'undefined'.
      => this decreases the chance to find bugs.
      => this is not very efficient (e.g. the rest of a superblock
         is often marked undefined repetitively).
    Similarly, the free block created by VG_(arena_memalign)
    is marked 'undefined'. 'No access' would be preferrable.
  - mkInuseBlock marks the new block as undefined. This is probably
    not needed, as VALGRIND_MALLOCLIKE_BLOCK will do it already.
  - VG_(arena_malloc) should give the requested size to
    VALGRIND_MALLOCLIKE_BLOCK, not the malloc usable size,
    as this decreases the chance to find buffer overrun bugs.
    But giving the requested size is tricky (see comments in
    the code).

* need to do memcheck annotations in m_poolalloc.c
   so as to allow leak checking for pool allocated elements.

* vg_regtest.in
  - should analyse the results of the outer and should
    produce a separate result for the tests for which
    the outer detects an error or a memory leak or ...


Changes done:
   README_DEVELOPERS: document the new outer/inner features.
   manual-core.xml: document the new sim-hint no-inner-prefix
   tests/outer_inner.supp: new file, containing the suppressions for inner.
   vg_regtest.in: implement new args --outer-valgrind, --outer-tool, --outer-args.
   m_mallocfree.c: annotations for memcheck.
   m_libcprint.c: handle the new sim-hint no-inner-prefix
   m_main.c: do an (early) parse of --sim-hints




git-svn-id: svn://svn.valgrind.org/valgrind/trunk@12441
2012-03-11 22:24:03 +00:00
Florian Krohm
a84d8c766f Also parse stdoutB_filter_args and stderrB_filter_args.
Forgot to do that in r12045.


git-svn-id: svn://svn.valgrind.org/valgrind/trunk@12085
2011-10-03 00:19:05 +00:00
Florian Krohm
387089d67d Change the backtrace filtering machinery for the helgrind regression
bucket. Instead of removing what we don't want to see in a backtrace
(e.g. path segments through libc and libpthread), we simply keep what
we do want to see. That way .exp files can be generic.
We need to make sure that GCC inlining does not get in the way. So all
the ..._WRK function in hg_intercepts.c are attributed as noinline.
The backtrace filtering is done in the new filter_helgrind script.
filter_stderr is simplified quite a bit.
Fixes bug #281468. See also the comments #5 and #6 there.


git-svn-id: svn://svn.valgrind.org/valgrind/trunk@12045
2011-09-26 00:29:44 +00:00
Julian Seward
b37b05d214 Improvements for testing and compilation breakage for the GDB server
on various platforms:

* In all gdbserver_tests using gdb:
  Made a more general way to remove the initial start message.

* tests using threads burning cpu modified to have only 1 thread.
  This makes them independent of the scheduler fairness.

* filter_gdb and filter_vgdb enhanced to anonymise
    some debian 6.0/ppc specific things
    some s390x/gdb 7.0, gdb 7.1  specific things

* vgdb.c: added an #include <linux/ptrace.h> to fix compilation
  on s390x fedora and suse. (Christian Boerntrager)

* fixed a bug in valgrind-low.c debug log :
  when a register size is 0, its image cannot be output (and register
  should not be transferred).

* added a parameter --keep-unfiltered to vg_regtest.in
  This will make it easier to update filter_gdb:
  in case gdbserver_tests are failing due to "artificial"
  differences to be filtered, re-run the tests using:
     perl tests/vg_regtest --keep-unfiltered gdbserver_tests
  Then a tar file with all the *.out in gdbserver_tests
  will allow me to better/faster update the filter_gdb.

* made a better detection of a working PTRACE_GETREGS at compile time
  and/or at run-time.

This is the patch on bug 214909 comment 69.
(Philippe Waroquiers, philippe.waroquiers@skynet.be)



git-svn-id: svn://svn.valgrind.org/valgrind/trunk@11740
2011-05-10 11:01:07 +00:00
Julian Seward
2ee9e90486 Implement a GDB server in Valgrind. See #214909.
(Philippe Waroquiers, philippe.waroquiers@skynet.be)



git-svn-id: svn://svn.valgrind.org/valgrind/trunk@11727
2011-05-06 21:02:55 +00:00
Bart Van Assche
5c9590f939 Expand ${PWD} in the vgopts: line of .vgtest files.
git-svn-id: svn://svn.valgrind.org/valgrind/trunk@11311
2010-08-31 15:15:35 +00:00
Nicholas Nethercote
52cf1f5f05 Merge r9103 and r9105 (add --ignore-fn to Massif) from the Darwin branch.
git-svn-id: svn://svn.valgrind.org/valgrind/trunk@9567
2009-04-17 04:26:41 +00:00
Nicholas Nethercote
4c23a0b3e9 tests/arch_test is currently being used for two purposes:
- by vg_regtest for determining if a directory name matches an architecture;
- by various .vgtest files for detecting x86/AMD64 features.

This commit splits it in two for the two different purposes, which makes
things clearer.

Specific changes

- Moved the x86/AMD64 feature detection stuff out of arch_test.c, and
  into the new x86_amd64_feature.c.  Updated the relevant .vgtest files for
  the change.

- In vg_regtest, now a prereq command must return 0 (prereq satisfied) or 1
  (prereq not satisfied).  Anything else makes vg_regtest abort.  This
  makes obvious any problems with prereq tests rather than just making the
  tests skip innocuously.  (We previously had exactly such a problem on the
  DARWIN branch;  the x86 feature detection tests caused segfaults so the
  tests were incorrectly skipped.  This change will catch any similar future
  problem.)

- Changed os_test from a script to a C program, matching cpu_test.

- Removed some unintentional darwin stuff from platform_test.


git-svn-id: svn://svn.valgrind.org/valgrind/trunk@9316
2009-03-04 04:15:16 +00:00
Nicholas Nethercote
95f49af2f2 Use 'diff -u' for regtests if it's supported, its output it much easier to
read than plain 'diff'.



git-svn-id: svn://svn.valgrind.org/valgrind/trunk@9263
2009-02-25 04:57:56 +00:00
Nicholas Nethercote
108768ee98 Merge r9175 (don't run inappropriate OS- and platform-specific tests) from
the DARWIN branch.



git-svn-id: svn://svn.valgrind.org/valgrind/trunk@9176
2009-02-16 00:42:10 +00:00
Nicholas Nethercote
2e4c67743e Ignore Emacs temporary files. Also, produce the correct .diff filename when
the .exp filename has a non-numeric suffix.



git-svn-id: svn://svn.valgrind.org/valgrind/trunk@7538
2008-03-02 22:48:14 +00:00
Nicholas Nethercote
745f47f01c Make comments reflect the code.
git-svn-id: svn://svn.valgrind.org/valgrind/trunk@7494
2008-02-27 18:10:19 +00:00
Julian Seward
e70c4211ff Marginally enhance help message.
git-svn-id: svn://svn.valgrind.org/valgrind/trunk@7258
2007-11-30 21:24:05 +00:00
Julian Seward
e63e873227 Merge (from branches/THRCHECK) r7043:
Allow a test to have any number of .stderr.exp* files, so long as at
least one is found.  In fact the comments in the code that claimed
that .stderr.exp[0-9] are the only ones allowed were misleading;
.stderr.exp* has always been allowed.  The only functional change here
is to mandate that at least one such file exists; prior to the change
the script mandated that at least a .stderr.exp (with no other suffix)
file existed.

Purpose is to allow collections of .stderr.exp files with suffixes
arranged in some meaningful way, eg, foo.stderr.exp-glibc25-amd64.
This might help in making testsuites more maintainable when there
have to be multiple .stderr.exp files.  Naming them merely as
.stderr.exp1, .stderr.exp2, etc, makes it impossible to remember
what the differences between the files actually is.



git-svn-id: svn://svn.valgrind.org/valgrind/trunk@7127
2007-11-09 23:29:46 +00:00
Nicholas Nethercote
62eed24c67 Merged the MASSIF2 branch to the trunk. Main changes:
- ms_main.c: completely overhauled.

- massif/tests/*:  lots of them now.

- massif/perf/:  added.

- massif/hp2ps:  removed.  No longer used.

- vg_regtest: renamed the previously unused "posttest" notion to "post".
  Using it for checking ms_print's output.

Although the code has changed dramatically, as has the form of the tool's
output, the information presented in the output is basically the same,
although it's now (hopefully) much more useful.  So the tool name is
unchanged.



git-svn-id: svn://svn.valgrind.org/valgrind/trunk@7069
2007-11-01 04:40:37 +00:00
Nicholas Nethercote
8b2049f475 Update a comment.
git-svn-id: svn://svn.valgrind.org/valgrind/trunk@6373
2006-11-26 22:49:58 +00:00
Nicholas Nethercote
aa2ce63f67 Add --valgrind-lib, to complement --valgrind, to the regtests script
(from Dave Nomura).



git-svn-id: svn://svn.valgrind.org/valgrind/trunk@6338
2006-10-23 18:38:35 +00:00
Julian Seward
2e44b6c1d4 Don't use -C0 for diff; apparently this is a GNU-ism.
git-svn-id: svn://svn.valgrind.org/valgrind/trunk@6300
2006-10-17 02:24:51 +00:00
Julian Seward
d419c4762b Read extra args for all tests from $EXTRA_REGTEST_OPTS, which is
useful (eg) for running tests with --sanity-level= set.  Print a
big warning before and after the tests if it is set.



git-svn-id: svn://svn.valgrind.org/valgrind/trunk@5931
2006-05-26 00:13:21 +00:00
Nicholas Nethercote
0ec806741e Add an optional post-test check. Based on a patch from Josef W.
git-svn-id: svn://svn.valgrind.org/valgrind/trunk@5817
2006-04-03 14:25:23 +00:00
Nicholas Nethercote
991367c922 Merge in the COMPVBITS branch to the trunk. This is a big change to
Memcheck, replacing the 9-bits-per-byte shadow memory representation to a
2-bits-per-byte representation (with possibly a little more on the side) by
taking advantage of the fact that extremely few memory bytes are partially
defined.

For the SPEC2k benchmarks with "test" inputs, this speeds up Memcheck by a
(geometric mean) factor of 1.20, and reduces the size of shadow memory by a
(geometric mean) factor of 4.26.

At the same time, Addrcheck is removed.  It hadn't worked for quite some
time, and with these improvements in Memcheck its raisons-d'etre have
shrivelled so much that it's not worth the effort to keep around.  Hooray!

Nb: this code hasn't been tested on PPC.  If things go wrong, look first in
the fast stack-handling functions (eg. mc_new_mem_stack_160,
MC_(helperc_MAKE_STACK_UNINIT)).


git-svn-id: svn://svn.valgrind.org/valgrind/trunk@5791
2006-03-27 11:37:07 +00:00
Nicholas Nethercote
6fd30a473c Remove dead line.
git-svn-id: svn://svn.valgrind.org/valgrind/trunk@5322
2005-12-09 21:01:46 +00:00
Nicholas Nethercote
bf86440453 Fail gracefully if 'make check' hasn't been run.
git-svn-id: svn://svn.valgrind.org/valgrind/trunk@5317
2005-12-08 22:39:04 +00:00
Nicholas Nethercote
e72b808c98 Make regtests work with --enable-inner.
git-svn-id: svn://svn.valgrind.org/valgrind/trunk@5098
2005-11-12 16:08:09 +00:00
Julian Seward
cc8ccbbfb4 This commit merges in changes from branches/ASPACEM (specifically,
changes from r4341 through r4787 inclusive).  That branch is now dead.
Please do not commit anything else to it.

For the most part the merge was not troublesome.  The main areas of
uncertainty are:

- build system: I had to import by hand Makefile.core-AM_CPPFLAGS.am
  and include it in a couple of places.  Building etc seems to still
  work, but I haven't tried building the documentation.

- syscall wrappers: Following analysis by Greg & Nick, a whole lot of
  stuff was moved from -generic to -linux after the branch was created.
  I think that is satisfactorily glued back together now.

- Regtests: although this appears to work, no .out files appear, which
  is strange, and makes it hard to diagnose regtest failures.  In
  particular memcheck/tests/x86/scalar.stderr.exp remains in a 
  conflicted state.

- amd64 is broken (slightly), and ppc32 will be unbuildable.  I'll
  attend to the former shortly.



git-svn-id: svn://svn.valgrind.org/valgrind/trunk@4789
2005-09-27 19:20:21 +00:00
Nicholas Nethercote
e074647282 Fix vg_regtests so it can properly handle alternative output filenames with
more than one numeric char at the end (eg. xml1.stderr.exp64).


git-svn-id: svn://svn.valgrind.org/valgrind/trunk@4050
2005-06-29 04:02:24 +00:00
Nicholas Nethercote
fe432113e6 A crucial, crucial change: update my email address.
git-svn-id: svn://svn.valgrind.org/valgrind/trunk@3632
2005-05-08 02:10:27 +00:00
Nicholas Nethercote
5ef4243598 Default to using --leak-check=summary.
MERGED FROM CVS HEAD



git-svn-id: svn://svn.valgrind.org/valgrind/trunk@3312
2005-03-12 20:38:13 +00:00
Julian Seward
0356d27ca6 Merge in changes from the 2.4.0 line. This basically brings in the
overhaul of the thread support.  Many things are now probably broken,
but at least with --tool=none, simple and not-so-simple threaded and
non-thread programs work.



git-svn-id: svn://svn.valgrind.org/valgrind/trunk@3265
2005-03-10 23:59:00 +00:00
Nicholas Nethercote
ce2585d447 Changed message at the top of files, and the startup message, and the
string in valgrind.pc.in, so that they describe Valgrind as a "dynamic
binary instrumentation framework", and don't mention platforms at all.  

I had to tweak the regtest filters a bit for this.



git-svn-id: svn://svn.valgrind.org/valgrind/trunk@3178
2004-12-01 14:14:42 +00:00
Nicholas Nethercote
10c3e3c927 Arch-abstraction:
- Added a hacky mechanism which prevents the regtest script from entering
directories for other architectures.  (Eg. when running on x86/, it won't enter
a ppc/ subdir.)


git-svn-id: svn://svn.valgrind.org/valgrind/trunk@3045
2004-11-18 18:21:56 +00:00
Nicholas Nethercote
3b4dc0bdba Generalised the reg test script again: replaced the "cpu_test" line,
which caused the test to be skipped if the CPU type wasn't appropriate,
with a "prereq" line, which specifies a command that must succeed before
the test is run.


git-svn-id: svn://svn.valgrind.org/valgrind/trunk@3041
2004-11-18 12:48:17 +00:00
Nicholas Nethercote
a2d1e021a4 Generalised reg test script: instead of having a "delete:" line, for removing
files once a test has completed, we have a "cleanup:" line, which specifies a
cleanup command.


git-svn-id: svn://svn.valgrind.org/valgrind/trunk@3039
2004-11-18 11:57:00 +00:00
Nicholas Nethercote
f1f1e2ffe3 Arch-abstraction:
- Change regtest script to print a message if a test is skipped due to the cpu
  being of the wrong type.


git-svn-id: svn://svn.valgrind.org/valgrind/trunk@2803
2004-10-19 16:56:41 +00:00
Nicholas Nethercote
9ce21c9e69 Fixed up various command line option scenarios:
- If no tool is specified, V now gives a short message and a list of
    available tools.  This was meant to happen previously, but a bug prevented
    it from working properly;  it gave the usage message instead.

  - If a bad option is given, V now gives a short message rather than the full
    --help.  This make V consistent with all other programs I looked at.

  - Now returning 0 when you do 'valgrind --help' and 'valgrind --version'
    as other programs do.

  - Removed VG_(startup_logging)() and VG_(shutdown_logging)() as they were
    empty and have been for a long time (always?).

  - Added various tests for these scenarios.  Had to change the regtest
    script slightly to allow for malformed command lines.

This addresses bug (wishlist) #82999.


git-svn-id: svn://svn.valgrind.org/valgrind/trunk@2418
2004-06-15 10:54:40 +00:00
Nicholas Nethercote
2fc96dcc06 Add a "delete:" line to the regression test .vgtest files, to clean up any file
created by the test.  Added appropriate lines to the Cachegrind and Massif
tests.  Should prevent large numbers of files clogging up directories.


git-svn-id: svn://svn.valgrind.org/valgrind/trunk@2372
2004-04-17 17:25:08 +00:00
Nicholas Nethercote
fd5dafaac2 Make vg_regtest return 1 if any tests fail. Useful for scripts that call it.
git-svn-id: svn://svn.valgrind.org/valgrind/trunk@2283
2004-03-01 08:27:37 +00:00
Nicholas Nethercote
01e3fb111c Allow multiple .exp files, eg. foo.stderr.exp, foo.stderr.exp2; test will pass
as long as one matches.


git-svn-id: svn://svn.valgrind.org/valgrind/trunk@2275
2004-02-29 01:31:09 +00:00
Nicholas Nethercote
5630025b84 Patch from Tom Hughes, for bug 72643:
Patch to improve SSE/SS2 support

  This patch should implement most of the missing SSE/SSE2 opcodes. About
  the only ones it doesn't do are the MASKMOVxxx ones as they are quite
  horrible and involved an implicit reference to EDI so I need to think
  about them a bit more.

  The patch also includes a set of tests for the MMX/SSE/SSE2 opcodes to
  validate that they have the same effect under valgrind as they do when
  run normally. In one or two cases this wasn't actually the case even
  for some of the implemented opcodes, so I fixed those as well ;-)


git-svn-id: svn://svn.valgrind.org/valgrind/trunk@2202
2004-01-19 19:14:18 +00:00
Jeremy Fitzhardinge
918c3a7b7e This jumbo-checkin is the Full Virtualization checkin. This eliminates
Valgrind's dependency on the dynamic linker for getting started, and
instead takes things into its own hands.

This checkin doesn't add much in the way of new functionality, but it
is the basis for all future work on Valgrind.  It allows us much more
flexibility in implementation, and well as increasing the reliability
of Valgrind by protecting it more from its clients.

This patch requires some changes to tools to update them to the changes
in the tool API, but they are straightforward.  See the posting "Heads
up: Full Virtualization" on valgrind-developers for a more complete
description of this change and its effects on you.


git-svn-id: svn://svn.valgrind.org/valgrind/trunk@2118
2003-12-16 02:05:15 +00:00
Dirk Mueller
2446c65296 only say something if there is something to say
git-svn-id: svn://svn.valgrind.org/valgrind/trunk@2037
2003-11-19 00:55:32 +00:00
Nicholas Nethercote
fb2d543ee8 Started to rescind the name "skin", replacing it with "tool". Did this in all
the places that normal users will see:

  - command line: --tool=foo (although --skin=foo still works)
  - docs: removed all traces (included renaming coregrind_skins.html to
    coregrind_tools.html)
  - in the usage messages
  - in error messages

Also did in in some places that I judged were unlikely to cause clashes with
existing workspaces:

  - in the header comments of many files (eg. "This file is part of Memcheck, a
    Valgrind tool for...")
  - in the regtests script
  - in the .supp files
  - in AUTHORS
  - in README_MISSING_SYSCALL_OR_IOCTL

Also update the AUTHORS file to mention Jeremy.


git-svn-id: svn://svn.valgrind.org/valgrind/trunk@2027
2003-11-14 17:47:54 +00:00
Jeremy Fitzhardinge
1e6361234c A complete reworking of Valgrind's handling of system calls and signals,
with the aim of making it more robust, more correct and perhaps faster.

This patch removes the need to poll blocking syscalls, by adding a proxy
LWP for each application thread.  This LWP is a kernel thread whose job
is to run all (potentially) blocking syscalls, and also to handle signals.

This allows the kernel to do more of the work of dealing with signals,
so on kernels which do this properly (2.6), Valgrind's behavious is a
lot more posix compliant.  On base 2.4 kernels, we emulate some of the
missing 2.6 functionality.


git-svn-id: svn://svn.valgrind.org/valgrind/trunk@1918
2003-10-13 22:26:55 +00:00
Nicholas Nethercote
ba72812e02 This commit allows Valgrind to be run in-place from the source tree, without
requiring "make install".  This is good for two reasons:

1. shorter compile times ("make" instead of "make install")
2. don't need to install before running the regression tests

Changes:
 - root Makefile now builds a libpthread.so.0 symlink to libpthread.so, which
   was needed.

 - startup script accepts a --in-place=<dir> option for running a non-installed
   version, where <dir> is the root of the src tree

 - VG_(mash_LD_PRELOAD_and_LD_LIBRARY_PATH) had to be fiddled with a bit

 - tests/vg_regtest now uses the --in-place option to test the non-installed
   version


git-svn-id: svn://svn.valgrind.org/valgrind/trunk@1614
2003-05-05 12:47:25 +00:00
Nicholas Nethercote
dc0fe27537 Changed things so that the regression test script (vg_regtest) is no longer
installed, as it was silly since the reg tests themselves aren't installed.
Updated the "make regtest" target to use the uninstalled version in tests/.

Also, vg_regtest now no longer uses the installed version of Valgrind to do the
testing, but instead coregrind/valgrind.  This means that you don't have to do
make install with your buggy changes before running the regtests on them :)

I even updated the RPM package .spec file!

Thanks to Dirk M�ller for the suggestion.


git-svn-id: svn://svn.valgrind.org/valgrind/trunk@1551
2003-04-23 21:48:20 +00:00
Nicholas Nethercote
dd6eacf7ff Now counting tests and stdout failures and stderr failures separately and
correctly.


git-svn-id: svn://svn.valgrind.org/valgrind/trunk@1547
2003-04-23 17:47:13 +00:00
Dirk Mueller
ba862b918e - fix regression tests counting
- unset $VALGRIND_OPTS (fixes 6 testsuite failures)


git-svn-id: svn://svn.valgrind.org/valgrind/trunk@1545
2003-04-23 16:52:06 +00:00
Nicholas Nethercote
ac7027c441 Updated copyright notices for 2003. Only 4 months late.
git-svn-id: svn://svn.valgrind.org/valgrind/trunk@1526
2003-04-15 14:58:06 +00:00
Nicholas Nethercote
0756a50bc3 Removed support for the 1.0.X series from the regression test suite -- this
was present from before the core/skin split, which is now dead.  Means the
script is slightly simpler, and we can dispense with lots of expected
foo.stderr.hd files.

Also undid accidental change to required Automake version in main Makefile.am
from my last commit, whoops.


git-svn-id: svn://svn.valgrind.org/valgrind/trunk@1518
2003-04-08 00:47:05 +00:00