Commit Graph

790 Commits

Author SHA1 Message Date
Zach Mullen 6ea9213244 Coverage glob should search in Source dir 2010-04-23 11:22:54 -04:00
Zach Mullen 7049dff85a Implement extra coverage globbing 2010-04-23 09:20:53 -04:00
David Cole 003ffe5908 Fix intermingling of test environments when tests run in parallel.
The SaveRestoreEnvironment helper object makes sure that the
original environment is restored immediately after the
StartProcess call returns rather than waiting for the end
of the test. When tests are executed in parallel, there is
no guarantee about the ordering of EndTest calls relative
to StartTest calls. In fact, it would be odd for them to
be nested nicely. Therefore, to avoid the corruption of
the calling ctest's environment, the original environment
must be restored before ForkProcess returns.
2010-03-30 14:08:31 -04:00
Zach Mullen 4b32ee01f2 Better detection of stop_time being passed. 2010-03-19 09:09:02 -04:00
Zach Mullen 9eea95c90f Fix StopTime to also account for localtime being a day ahead of gmtime 2010-03-18 14:48:42 -04:00
Zach Mullen 9676c52c3e Fix for StopTime for cases when gmtime is a day ahead of localtime 2010-03-18 13:53:40 -04:00
Zach Mullen a2fe175647 More debugging of StopTime test 2010-03-18 10:29:02 -04:00
Zach Mullen 0d48f81d32 Test using std::min 2010-03-17 12:12:04 -04:00
Zach Mullen 4c59570bfb Replace min() call with its literal definition 2010-03-17 12:07:10 -04:00
Zach Mullen 0ba9d04117 Add the --stop-time argument
Unit test and script hook for STOP_TIME
2010-03-17 11:04:13 -04:00
Zach Mullen 2fd9b8072b Add unit test for RESOURCE_LOCK test property 2010-03-03 10:33:09 -05:00
Zach Mullen 767ffba8ff Added RESOURCE_LOCK test property. 2010-03-02 15:38:02 -05:00
Zach Mullen 5cfbdb1f80 Close ifstream. 2010-03-01 12:05:58 -05:00
Zach Mullen 348f6c4d8c If tests failed on the last run, have them run first next time 2010-03-01 11:59:00 -05:00
Zach Mullen 09e748c69a Configurable path to CTest cost data file
Allow the user to set the CMake variable CTEST_COST_DATA_FILE, which will be used to store the cost data from test runs.  If not set, defaults to the original location in the build tree Testing/Temporary dir.
2010-03-01 11:06:28 -05:00
Zach Mullen b4d27dc041 Use historical average of test times to schedule tests. 2010-02-26 11:02:05 -05:00
Zach Mullen 57efb4a497 BUG: We shouldn't be setting the HideWindow option on the test processes we create. 2010-02-19 13:28:04 -05:00
David Cole 90f6a7270a Fix for fix for issue #2336 - do not specify CMAKE_BUILD_TYPE when the ctest -C configuration type string is empty. 2010-02-10 14:47:21 -05:00
Brad King 8bd231866a cmCTestGIT: Refresh index for local modifications
We use 'git diff-index' to detect local modifications after pull.  On
some filesystems the work tree timestamps of a few files may be dated
after the index, making them appear as locally modified.  We address the
problem by using 'git update-index --refresh' to refresh the index and
avoid false local modifications.
2010-02-09 13:31:32 -05:00
Brad King 2ec78b4de7 cmCTestBZR: Strip trailing slashes from paths
Our internal path processing methods assume no trailing slashes, but bzr
adds trailing slashes to updated directories.  This can lead to empty
entries in Update.xml files.  We address the problem by stripping the
slashes as soon as they are parsed.
2010-02-09 13:31:12 -05:00
Zach Mullen a0cc49a915 Support for relative paths to executables in MemCheck tests. Also fixed a bug causing memcheck args to be overwritten repetitively. 2010-02-04 11:24:57 -05:00
Zach Mullen c87282ce52 Re-enabled Scheduler test and fixed the underlying problem. 2010-02-03 16:32:26 -05:00
David Cole b0d5fa0bc7 Fix for issue #5041 - improve error messages that may occur during ctest_build and ctest_configure.
The improved text gives very clear information when either the CMakeLists.txt or CTestConfig.cmake file is missing. Hopefully, it makes it easier for those poor future souls who encounter these messages to solve their problems quickly.
2009-12-29 18:03:45 -05:00
David Cole aad15c3abf Fix issue #10060 - add APPEND arg to ctest_start command.
If APPEND is given to ctest_start, it will read the tag from the current existing Testing/TAG file rather than creating a new one based on the current time stamp. This allows a developer to run several dashboard scripts in a row, all of which will share the same tag/stamp/buildid when they finally get submitted to CDash. Now you can split the running of build phases and test phases for the same dashboard row into multiple scripts.
2009-12-29 14:38:31 -05:00
David Cole 4d3a387ac6 Preserve environment variables across calls to cmCTestScriptHandler::RunConfigurationScript. This will help prevent problems like we are currently experiencing (regarding CMake continuous dashboards only running once each day because the HOME environment variable is modified indirectly by a command in a called script, but affects the environment in the calling script). 2009-12-23 14:34:46 -05:00
Brad King cb27cfb1cc Use human-readable Git commit times in Update.xml
Previously we produced commit times formatted like

  1261403774 -0500

which is what the Git plumbing prints.  Now we use a human-readable
format like

  2009-12-21 15:28:06 -0500

which is still easy to machine-parse.
2009-12-21 10:29:00 -05:00
Zach Mullen 3312e3d710 Fix size type conversion warning in RunTest. 2009-12-21 09:52:56 -05:00
Zach Mullen 66f44f64d5 Fix segfault on Borland compiler resulting from divide by 0. 2009-12-18 11:42:58 -05:00
Brad King a4cdd73636 Use literal quotes in Update.xml UpdateCommand
Previously we escaped quotes in <UpdateCommand>...</UpdateCommand>
values using '&quot;'.  This is not necessary because the value is in
xml CDATA and not an xml attribute.
2009-12-18 10:54:24 -05:00
Brad King 3e724b2725 Submit Subversion directory path in Update.xml
A Subversion revision is unique across the entire repository, but work
trees typically correspond only to a subdirectory below the root path.
In order to specify the version of the source code that was tested,
CTest now submits a <SVNPath> element in Update.xml that specifies the
directory of the repository that corresponds to the work tree.  In
combination with the revision number this uniquely specifies the tested
source.  See issue #7541.
2009-12-18 08:19:41 -05:00
Brad King a73833d037 Submit global tree revision in Update.xml
We teach CTest to report in a <Revision> element the revision of the
source tree that was tested.  This makes sense for all modern VCS tools
because they version the whole tree.  We simply omit this element for
CVS because it only versions files.  See issue #7541.
2009-12-18 08:13:51 -05:00
Zach Mullen 1842d2eae0 Fix output compression error message 2009-12-17 11:17:57 -05:00
Zach Mullen ff916b48bd CTest output submitted to the dashboard is now compressed by default. 2009-12-17 11:14:49 -05:00
Zach Mullen 9add940eb6 Added an option to conditionally attach files to a test submission only if the test does not pass. Also some preliminary changes for test output compression. 2009-12-16 14:50:16 -05:00
Zach Mullen 5bfe1a1962 Changed XML formatting of files attached to test so that each file is archived and marked up individually, in order to make things easier on the CDash side. Also switched to using the NamedMeasurement tag instead of a new tag. 2009-12-15 14:24:24 -05:00
Zach Mullen dc66139f7c CTest-side changes to allow users to attach arbitrary files to test results that will be submitted to cdash using the ATTACHED_FILES test property. 2009-12-15 12:07:15 -05:00
Zach Mullen b2e7da885d Added a "-http1.0" option to ctest to make it submit using curl's http 1.0 option. Also added parsing of html reponse output to determine whether errors or warnings were sent in response from the server. If errors or warnings occurred, the response is output to stdout, and the "submission successful" message has been changed to accurately reflect whether or not warnings or errors were returned with the response. 2009-12-11 14:10:37 -05:00
Zach Mullen 5e427677c3 Restore the "***Not Run" message to ctest command line output for tests which did not start due to missing/bad executables or missing required files. 2009-12-10 15:51:56 -05:00
Zach Mullen 4de7cc3621 Unfortunately, I noticed the comment on bug 8668 too late. This changes my last implementation of the exe wrapper to something which makes much more sense: a REQUIRED_FILES property on tests. 2009-12-10 15:37:04 -05:00
Zach Mullen 48b6133928 [0008668: CTest Dev: Missing executables shown as failed tests when using MPI.] Added a wrapping option to add_test so that exes built by the project can be safely wrapped in other exes and be listed as "not run" rather than "failed" if they are not built. 2009-12-10 14:38:32 -05:00
Zach Mullen 08ac33fe6a Fix memory leak that occurred when a test executable could not be started. (See BadExe test) 2009-12-10 09:39:19 -05:00
Zach Mullen c32088cfee Local test timeouts will now always take precedence over the global --timeout option. 2009-12-08 10:26:43 -05:00
Zach Mullen b30f627190 CMake global timeout (--timeout option) should prevail over individual test timeouts if it is lower than the individual timeout. 2009-12-07 13:25:54 -05:00
David Cole 0b38bb4c53 Fix issue #2336 - honor the -C arg to ctest. Honor it for all stages of running -D dashboards from the command line and running ctest_configure, ctest_build and ctest_test commands in -S scripts. Also, allow a script to change it by setting the CTEST_CONFIGURATION_TYPE variable: allows for multiple configuration build/test cycles within one script. Add a new signature for the cmake command build_command that accepts CONFIGURATION as one argument. The original build_command signature is still there, but now marked as deprecated in the documentation. Of course... also add CTestConfig tests to verify that -C is honored for -D dashboards and -S scripts. 2009-12-04 12:09:01 -05:00
Bill Hoffman 22c37b2b19 Hanle the case where a test can not be run because it is a bad executable. 2009-12-02 16:37:43 -05:00
Zach Mullen c17ce46c73 Align test output for timeout condition. 2009-11-30 16:09:00 -05:00
Zach Mullen a52c0118d4 Added the --timeout option to ctest command line. This sets a global timeout on all tests if no more specific timeout is set on them. 2009-11-30 16:08:11 -05:00
Brad King e1548142fb CTest: Move initial checkout to ctest_start()
In CTest command-driven script mode we support starting without a source
tree.  Previously the ctest_start() command would do some initialization
but could not do anything that required CTestConfig.cmake from the input
source tree.  Later, ctest_update() would run CTEST_CHECKOUT_COMMAND to
create the source tree, and then re-initialize everything.  This
delayed-initialization approach led to many complicated cases of which
only some worked.  For example, the second initialization only worked
correctly in Nightly mode and simply failed for Experimental and
Continuous builds.

A simpler solution is to run CTEST_CHECKOUT_COMMAND during ctest_start()
and then have a single initialization path.  In principle this change in
behavior could break scripts that set the checkout command after
ctest_start() but before ctest_update().  However, the convention we've
always followed has been to set all variables before ctest_start().

See issue #9450.
2009-11-24 08:58:59 -05:00
Zach Mullen 2921fa96ed Handle multiple carriage return issue on windows VS9. Also make sure that running ctest in showonly mode does not kill our cost store, since many ctest tests do this and corrupt our cost data. 2009-11-16 10:58:43 -05:00
Bill Hoffman 1b48d361d8 fix number of callers to work with newer versions of valgrind 2009-11-11 08:43:15 -05:00
Zach Mullen c9c0ee4056 Change logic of ctest subdirs command to allow for absolute paths. Also added test coverage for passing absolute paths to subdirs. 2009-11-10 10:40:24 -05:00
Zach Mullen f7899c7445 Fixed a bug where it was possible for a test to be started twice if a lower-indexed test depended on it. 2009-11-09 16:07:47 -05:00
Zach Mullen 17afd5257b Bug 9090: CTest does not handle absolute paths in CTestTestfile SUBDIR( ) entries.
The ctest subdirs command now checks the relative path first, and if that does not exist, also checks if the given path was absolute.  Thanks vodall for the patch.
2009-11-09 14:07:36 -05:00
Zach Mullen 9f105a3f6d Fix style warning in cmsystemtools, cosmetic change to ctest code 2009-11-05 11:37:47 -05:00
Zach Mullen 9aa4e134b5 Fixed infinite loop condition when a test's dependent test executable could not be found. 2009-11-05 11:03:07 -05:00
Zach Mullen 85912a3777 Added TARGET, CONFIGURATION, PROJECT_NAME, and FLAGS arguments to ctest_build command 2009-11-03 16:46:30 -05:00
Zach Mullen d47ada823a Added logic to check for cycles in the test dependency graph before any tests are run. Previously a cycle resulted in a segfault from stack overflow. 2009-11-02 12:52:50 -05:00
Zach Mullen 8612aa10b6 Hook for scheduling tests in a random order
This may help statistically detect implicit dependencies among unit
tests while running in parallel.
2009-10-29 15:30:12 -04:00
Bill Hoffman d7a3277fb3 Fix bug#9767 catch missing leak. 2009-10-29 08:38:04 -04:00
Zach Mullen e2b63e473d Fix working dir issue for ctest show only mode (-N) 2009-10-27 10:16:01 -04:00
Zach Mullen 99697308f3 Add a ctest test to cover cmCTestBatchTestHandler. Since its behavior isn't fully defined yet, we just run it as is and don't care much about the result. 2009-10-26 09:44:08 -04:00
Zach Mullen f2010871ba Ctest was broken for subdirs. Restored working directory state for tests so that their executables could be found. 2009-10-20 10:54:02 -04:00
Zach Mullen 727fa7bcb6 Fix ctest output alignment for cases where total tests run is not the same width as max test index. 2009-10-15 12:02:22 -04:00
David Cole 85feea2d0d Fix floating point comparison warnings. Thanks to Alex Neundorf for the patch. 2009-10-13 16:39:48 -04:00
David Cole 4439f17b20 Fix issue #5668 - use CollapseFullPath when determining if covered file is within source or binary tree. Allows gcc/gcov coverage analysis using MinGW on Windows. 2009-10-12 14:51:54 -04:00
Zach Mullen 7fd3d7cf80 BUG: #0009648 Change "The following tests FAILED" message to print on stdout rather than stderr 2009-10-05 12:47:09 -04:00
Zach Mullen b304186bc5 Match width of ctest "Start xx: " line to line up with the end test line 2009-10-05 10:20:52 -04:00
Alexander Neundorf b3040beb41 fix two more warnings from icpc
asfGlob and abfGlob were there since rev 1.3 of this file (Oct 17th 2004) and unused since
then, so remove them

Alex
2009-10-04 11:40:02 -04:00
David Cole ccb0cf1306 Fix warnings in CMake source code. 2009-10-02 15:30:01 -04:00
David Cole 44bcba7461 Fix warnings in CMake source code. Suppress rampant warnings emanating from Qt files. 2009-10-01 16:47:08 -04:00
Zach Mullen a5be445ca9 BUG: CTest should honor test timeouts. 2009-09-29 14:31:58 -04:00
Zach Mullen 202bb45695 BUG: 0009612: --output-on-failure option doesn't work with the new parallel CTest handler 2009-09-29 09:45:43 -04:00
Brad King 96afb12087 Convert CMake to OSI-approved BSD License
This converts the CMake license to a pure 3-clause OSI-approved BSD
License.  We drop the previous license clause requiring modified
versions to be plainly marked.  We also update the CMake copyright to
cover the full development time range.
2009-09-28 11:43:28 -04:00
Zach Mullen 5e06f0a557 Fixed uninitialized memory issue in ctest show-only mode 2009-09-25 11:45:19 -04:00
Zach Mullen 5a2f24856e Output a message in ctest when each test is started 2009-09-24 13:49:20 -04:00
Zach Mullen 65c418e56f Reformat ctest -N output. Removed the "Start processing tests" message as well. 2009-09-24 09:49:46 -04:00
Zach Mullen 84f3f6cad6 Tests which are not run should be added to the failed test list. 2009-09-23 15:37:50 -04:00
Zach Mullen 4342e37d63 More SLURM experimentation (ctest batch mode) 2009-09-21 15:29:34 -04:00
Zach Mullen a4c6f178fe Fixed a slurm batch argument identifier. 2009-09-21 14:58:30 -04:00
Zach Mullen 87b4c66ca8 Need to quote args when generating batch scripts from ctest 2009-09-21 14:21:41 -04:00
Zach Mullen e4293b4b38 Re-enabled failing tests; fixed ctest_build output to be consistent in the error condition across different make programs so that these tests would pass. 2009-09-21 13:40:40 -04:00
Bill Hoffman 932dac93b7 Fix uninitialized errors. 2009-09-16 11:49:09 -04:00
Zach Mullen 243bbae954 Removed fork-and-continue option from ctest generated batch script entries 2009-09-14 11:23:20 -04:00
Brad King bcbb4626a3 Avoid shadowing std::vector member
The cmProcess::Buffer class derives from std::vector.  We were using
local variable 'data' in the GetLine method but this name shadowed a
member of vector with GNU.  This renames it to 'text'.
2009-09-12 10:20:00 -04:00
Bill Hoffman 63a7d85249 Fix for bug#9442, ctest crash if CTEST_SOURCE_DIRECTORY was not set. 2009-09-11 17:15:02 -04:00
Brad King 8c24c1dcc3 Fix new CTest output handling for no newline
When we clear the buffer for an output pipe after returning the last
partial line (without a newline) we need to set the partial line range
to empty.  Otherwise the buffer object is left in an inconsistent state.
2009-09-11 16:20:24 -04:00
Bill Hoffman 8a690289c2 Add label summary times to ctest default output. Also, remove parallel time output. Add flag to disable label summary. 2009-09-11 13:34:35 -04:00
Brad King 6a7eae7184 Rewrite CTest child output handling
This commit fixes cmCTestRunTest and cmProcess to more efficiently
handle child output.  We now use the buffer for each child output pipe
to hold at most a partial line plus one new block of data at a time.
All complete lines are scanned in-place, and then only the partial line
at the end of the buffer is moved back to the beginning before appending
new data.

We also simplify the cmProcess interface by making GetNextOutputLine the
only method that needs to be called while the process is running.  This
simplifies cmCTestRunTest so that CheckOutput can be called until it
returns false when the process is done.
2009-09-11 12:26:41 -04:00
Brad King b6c26cded2 Initialize cmCTestRunTest instances robustly
All instances of this class need a cmCTestTestHandler, so we now require
one to construct it.  The instance also provides the cmCTest instance
too.
2009-09-11 10:09:48 -04:00
Brad King b41a548d86 Add parentheses around '&&' between '||' for gcc
The GNU compiler warns about possible operator precedence mistakes and
asks for explicit parentheses (-Wparentheses).  We add the parentheses
to silence the warning.  This also fixes one real logic error in the
find_package() implementation by correcting expression evaluation order.
2009-09-11 08:18:15 -04:00
Zach Mullen c57fb18920 Added some ctest batch capabilities 2009-09-10 11:18:05 -04:00
Zach Mullen 4b4e801eba BUG: Fixed segfault and bad reporting if a ctest executable could not be found. Also added some batch testing code that is not yet complete. 2009-09-10 11:16:08 -04:00
Zach Mullen a516040579 ENH: ctest now writes time cost data to a file after a test set is run, and uses these time costs to schedule the processes the next time ctest is run in that build tree. 2009-09-08 17:10:35 -04:00
Zach Mullen 6727834a8c BUG: Fixed extraneous newlines from ctest process output 2009-09-08 14:48:23 -04:00
Zach Mullen 39e5f9d963 ENH: Replaced the EXPENSIVE test property with a COST test property taking a floating point value. Tests are now started in descending order of their cost, which defaults to 0 if none is specified. 2009-09-08 13:39:13 -04:00
Zach Mullen a02ef56401 BUG: Fixed issue where ctest would hang if a process terminated with output in its buffers but no newline 2009-09-08 10:16:16 -04:00
Zach Mullen 384f4d1f3f Fixed warning 2009-09-08 09:12:44 -04:00
Zach Mullen 5fb958fde9 ENH: Added ctest test options PROCESSORS and RUN_SERIAL. These allow specification of resource allocation for given tests running with the ctest -j N option. RUN_SERIAL ensures that a given test does not run in parallel with any other test. Also forced appending of "..." to the longest test name in ctest. 2009-09-07 10:26:17 -04:00
Bill Hoffman cdb966532b Fix memory and process leak in ctest_run_script. 2009-09-04 17:01:47 -04:00
David Cole 68b8332088 Increase curl submit timeout. A submit will timeout if there are 120 seconds of very little activity. 30 seconds was too short. 2009-09-04 14:00:26 -04:00