Commit Graph

840 Commits

Author SHA1 Message Date
Bill Hoffman 3f94c7cc39 When processing DartMeasurements use the tests working directory.
Restore CMake 2.6 behavoir where the dart processing apparently was run from within the tests directory instead of the root of the build tree. This addresses issue reported on the VTK Dashboard where the Testing/Temporary directory is defined as ../../../Testing/Temporary but the DartProcessing is run with respect to the root of the build tree. This causes the regression/difference images to not be located by ctest and thus not uploaded to the dashboard.
2010-10-19 16:15:32 -04:00
Zach Mullen 32242affea Added CTest command --print-labels
This command allows a user to quickly see the list of all available
test labels.  The labels are also printed in verbose show only mode,
alongside their corresponding tests.
2010-08-31 10:41:23 -04:00
Brad King 5383657357 CTest: Avoid use of old EscapeSpaces method
Refactor how cmCTestMemCheckHandler computes the memory tester command
line options to avoid encoding them in a single string just to parse
them again.  The EscapeSpaces uses backslahes to escape spaces on UNIX
platforms, so replace other calls to it in CTest that are used to create
human-readable strings with simple double-quoting.
2010-08-18 10:14:09 -04:00
Brad King e6067ca969 Merge topic 'ctest-git-flexibility'
a7319cf ctest_update: Run 'git submodule' at top level
7bf8dc1 ctest_update: Support ".git file" work trees
65cb72f ctest_update: Abort if Git FETCH_HEAD has no candidates
2010-08-03 16:07:31 -04:00
Brad King a7319cf1c1 ctest_update: Run 'git submodule' at top level
The git submodule porcelain must be executed from the top level of the
work tree.  Use 'git rev-parse --show-cdup' to find the top level
relative to the source tree.  This is better than searching up the tree
for .git ourselves because it will always work the same way Git does and
thus honors settings like GIT_DISCOVERY_ACROSS_FILESYSTEM.
2010-07-27 11:31:59 -04:00
Brad King 7bf8dc1ac9 ctest_update: Support ".git file" work trees
Commit c3781efb (Support Git upstream branch rewrites, 2010-06-08)
assumed that ".git/FETCH_HEAD" exists inside the source tree.  Fix the
implementation to handle a work tree using a ".git file" to link to its
repository.  Use "git rev-parse --git-dir" to locate the real .git dir.
2010-07-26 11:40:20 -04:00
Brad King 65cb72f758 ctest_update: Abort if Git FETCH_HEAD has no candidates
If .git/FETCH_HEAD provides no merge candidate do not attempt to update.
Also log FETCH_HEAD lines as we parse them since they are essentially
output from the git fetch command.
2010-07-20 16:57:50 -04:00
Brad King e1a4c02bac Merge topic 'resolve/doc-spelling/CPackRPM'
7739d78 Merge CPackRPM changes into doc-spelling
9203e91 Fix spelling errors reported by Lintian.
2010-07-20 16:00:29 -04:00
Brad King f7a0386fc5 Merge topic 'resolve/ctest-file-checksum/remove-CTestTest3'
38c762c Merge 'remove-CTestTest3' into ctest-file-checksum
46df0b4 Activate retry code on any curl submit failure.
8705497 Checksum test should use CMAKE_TESTS_CDASH_SERVER
d0d1cdd Mock checksum failure output for old CDash versions
af5ef0c Testing for CTest checksum
86e81b5 CTest should resubmit in the checksum failed case
d6b7107 Fix subscript out of range crash
082c87e Cross-platform fixes for checksum/retry code
e525649 Checksums on CTest submit files, and retry timed out submissions.
2010-07-13 17:05:33 -04:00
Brad King 38c762c728 Merge 'remove-CTestTest3' into ctest-file-checksum
Conflicts:
	Tests/CMakeLists.txt
2010-07-13 17:01:39 -04:00
Brad King a570ba6df7 Merge topic 'external-project-launcher'
b2634b6 Detect CMake warnings and errors in build output.
012c986 ExternalProject: Add LOG_* options to hide step output
2010-07-13 14:58:11 -04:00
Brad King 7739d786a4 Merge CPackRPM changes into doc-spelling
Conflicts:
	Modules/CPackRPM.cmake
2010-07-13 09:46:52 -04:00
Kai Wasserbäch 9203e9187e Fix spelling errors reported by Lintian.
During a Lintian run on the binary packages of CMake in Debian I was
notified of many spelling mistakes.
2010-07-13 09:41:37 -04:00
David Cole 46df0b44ac Activate retry code on any curl submit failure.
Previously, we were only going into the retry block
for time out conditions. But a "could not connect"
response, or really any sort of curl failure, is
also a condition where we should retry the submit
if the user has requested a retry.
2010-07-12 16:48:38 -04:00
David Cole b2634b6aa3 Detect CMake warnings and errors in build output.
Fix niggly in ExternalProject that would inadvertently
create a log file of output when a command was explicitly
set to the empty string.

Also, increase the default value for showing context
around build errors to 10 lines, up from 6. Also add
the ability to customize this setting so that users
can put it back to 6 or even lower if they want, or
show more if they want.
2010-07-10 15:04:31 -04:00
Brad King 28a16d1cc4 Merge branch 'improve-test-cost-sorting' 2010-07-06 10:24:31 -04:00
Zach Mullen 142edf8ad4 More robust cost-based scheduling impl 2010-07-01 14:10:49 -04:00
Zach Mullen 3e52000a07 Fix cycle detection for test dependencies 2010-06-30 10:39:17 -04:00
David Cole 415900ba77 Eliminate -Wconversion warnings.
Change types of local variables, or casting, or re-arrange
expressions to get rid of "conversion may alter value" warnings
as seen on recent dashboard submissions from londinium.kitware.
2010-06-27 11:22:05 -04:00
Brad King 566bb2d078 Merge branch 'ctest-git-empty-commits' 2010-06-24 10:41:20 -04:00
Brad King 5890f562e3 Merge branch 'fix-ctest-parallel' 2010-06-24 10:40:19 -04:00
Brad King 73144a8f1c Merge branch 'fix_ctest_failure_code' 2010-06-24 10:35:10 -04:00
Zach Mullen 1a4c0c55c7 Parallel CTest hangs if serial test has depends 2010-06-23 17:18:59 -04:00
Brad King bbfe241c6a CTest: Parse empty Git commits correctly
Git's diff-tree format has no '\n'-terminated blank line at the end of
its commit message body block if there are no diff lines.  Instead the
message body is terminated by '\0' and there is no diff section.  Teach
CTest to parse the format in this case.
2010-06-23 09:14:43 -04:00
Brad King 0336d822cc Merge branch 'doc_ctest_build_target' 2010-06-22 09:56:53 -04:00
Brad King 3efb7f34a4 Merge branch 'fix-stoptime-mem-leak' 2010-06-22 09:56:39 -04:00
Bill Hoffman 8f8c1edca2 Fix for bug #10859, ctest exit exception incorrectly reported.
CTest was using the return value from the program instead of the
exit exception value for the process.
2010-06-22 09:55:09 -04:00
Brad King b4aa834900 Merge branch 'ctest-git-flexibility' 2010-06-15 14:10:41 -04:00
Zach Mullen 6e7f182318 Document ctest_build() TARGET option 2010-06-15 10:43:30 -04:00
Zach Mullen 960dc2b10f Do not exit if stoptime is passed. 2010-06-15 10:29:35 -04:00
Zach Mullen 9d6567a269 Extra coverage glob should subtract the explicitly defined excluded files 2010-06-11 14:17:00 -04:00
Zach Mullen d0d1cdd71b Mock checksum failure output for old CDash versions 2010-06-10 15:02:24 -04:00
Zach Mullen af5ef0c969 Testing for CTest checksum 2010-06-10 12:25:49 -04:00
Brad King c3781efb28 ctest_update: Support Git upstream branch rewrites
Use 'git fetch' followed by 'git reset' to update the source tree.  This
is better than 'git pull' because it can handle a rewritten upstream
branch and does not leave local modifications.  After fetch, parse
FETCH_HEAD to find the merge head that 'git pull' would choose to track
the upstream branch.  Then reset to the selected head.

In the normal fast-forward case the behavior remains unchanged.
However, now local modifications and commits will be erased, and
upstream rewrites are handled smoothly.  This ensures that the upstream
branch is tested as expected.
2010-06-08 16:50:17 -04:00
Brad King 2eae651acc ctest_update: Support custom Git update command
Define CTest configuration variable CTEST_GIT_UPDATE_CUSTOM to set a
custom command line for updating Git-managed source trees.
2010-06-08 16:12:28 -04:00
Brad King 608406bf55 Merge branch 'ctest-update-git-submodule' 2010-06-07 14:34:28 -04:00
Brad King cbd0fa3013 Merge branch 'php-coverage' 2010-06-07 14:19:56 -04:00
Zach Mullen 86e81b53c1 CTest should resubmit in the checksum failed case 2010-06-05 10:36:23 -04:00
Brad King d0a1b9efd0 Fix ctest_update log prefix for git submodule update
Commit 67277bac (Teach ctest_update about Git submodules, 2010-05-04)
accidentally logged "git submodule update" with the prefixes "pull-out"
and "pull-err".  Fix it to use "submodule-out" and "submodule-err"
instead.
2010-06-04 17:01:23 -04:00
Zach Mullen d6b71078da Fix subscript out of range crash 2010-06-03 13:52:48 -04:00
Zach Mullen 082c87e528 Cross-platform fixes for checksum/retry code 2010-06-03 13:27:26 -04:00
Zach Mullen e525649a4e Checksums on CTest submit files, and retry timed out submissions. 2010-06-03 10:34:34 -04:00
Bill Hoffman 1d4e121d9c Add php coverage to ctest. 2010-05-25 09:23:25 -04:00
Zach Mullen fa3c1817b1 Cost-based test scheduling should only be done in parallel mode. 2010-05-19 11:20:30 -04:00
Zach Mullen 38cabcd9ff Fix shadowed local warning in coverage handler 2010-05-12 09:37:43 -04:00
Zach Mullen 231cbbef80 Change coverage globbing mechanism to resolve paths earlier 2010-05-07 11:00:21 -04:00
Brad King 678073bec2 CTest: Submit author email in Update.xml
Add the <Email>...</Email> element in Update.xml for each commit
reported.  This field was defined by Dart but never really used.
Distributed version control systems use author name and email
instead of a user id, so now it makes sense to use this field.
2010-05-07 09:17:26 -04:00
Brad King 67277bacca Teach ctest_update about Git submodules
Git does not automatically checkout the matching version of a submodule
when it checks out a new version of the parent project in the work tree.
If the submodule reference changed in the parent project then we were
reporting the submodule path as a local modification.  Work around the
problem in ctest_update using "git submodule update" after "git pull".
For projects with no submodules this is a no-op.  See issue #10662.
Also add a submodule to the test project for CTest.UpdateGIT to test the
work-around.
2010-05-04 09:40:04 -04:00
Bill Hoffman fd5b75e3f4 Fix for for bug #10550, fix some errors for an old Sun compiler. 2010-04-28 17:52:46 -04:00
Brad King 1da611fd46 ctest_update: Report failure in Update.xml
Report in <UpdateReturnStatus> when the update command line returns
non-zero.  Otherwise the failure may be silently ignored since the error
was previously recorded only in the local log file.
2010-04-27 14:10:47 -04:00
Zach Mullen 6ea9213244 Coverage glob should search in Source dir 2010-04-23 11:22:54 -04:00
Zach Mullen 7049dff85a Implement extra coverage globbing 2010-04-23 09:20:53 -04:00
David Cole 003ffe5908 Fix intermingling of test environments when tests run in parallel.
The SaveRestoreEnvironment helper object makes sure that the
original environment is restored immediately after the
StartProcess call returns rather than waiting for the end
of the test. When tests are executed in parallel, there is
no guarantee about the ordering of EndTest calls relative
to StartTest calls. In fact, it would be odd for them to
be nested nicely. Therefore, to avoid the corruption of
the calling ctest's environment, the original environment
must be restored before ForkProcess returns.
2010-03-30 14:08:31 -04:00
Zach Mullen 4b32ee01f2 Better detection of stop_time being passed. 2010-03-19 09:09:02 -04:00
Zach Mullen 9eea95c90f Fix StopTime to also account for localtime being a day ahead of gmtime 2010-03-18 14:48:42 -04:00
Zach Mullen 9676c52c3e Fix for StopTime for cases when gmtime is a day ahead of localtime 2010-03-18 13:53:40 -04:00
Zach Mullen a2fe175647 More debugging of StopTime test 2010-03-18 10:29:02 -04:00
Zach Mullen 0d48f81d32 Test using std::min 2010-03-17 12:12:04 -04:00
Zach Mullen 4c59570bfb Replace min() call with its literal definition 2010-03-17 12:07:10 -04:00
Zach Mullen 0ba9d04117 Add the --stop-time argument
Unit test and script hook for STOP_TIME
2010-03-17 11:04:13 -04:00
Zach Mullen 2fd9b8072b Add unit test for RESOURCE_LOCK test property 2010-03-03 10:33:09 -05:00
Zach Mullen 767ffba8ff Added RESOURCE_LOCK test property. 2010-03-02 15:38:02 -05:00
Zach Mullen 5cfbdb1f80 Close ifstream. 2010-03-01 12:05:58 -05:00
Zach Mullen 348f6c4d8c If tests failed on the last run, have them run first next time 2010-03-01 11:59:00 -05:00
Zach Mullen 09e748c69a Configurable path to CTest cost data file
Allow the user to set the CMake variable CTEST_COST_DATA_FILE, which will be used to store the cost data from test runs.  If not set, defaults to the original location in the build tree Testing/Temporary dir.
2010-03-01 11:06:28 -05:00
Zach Mullen b4d27dc041 Use historical average of test times to schedule tests. 2010-02-26 11:02:05 -05:00
Zach Mullen 57efb4a497 BUG: We shouldn't be setting the HideWindow option on the test processes we create. 2010-02-19 13:28:04 -05:00
David Cole 90f6a7270a Fix for fix for issue #2336 - do not specify CMAKE_BUILD_TYPE when the ctest -C configuration type string is empty. 2010-02-10 14:47:21 -05:00
Brad King 8bd231866a cmCTestGIT: Refresh index for local modifications
We use 'git diff-index' to detect local modifications after pull.  On
some filesystems the work tree timestamps of a few files may be dated
after the index, making them appear as locally modified.  We address the
problem by using 'git update-index --refresh' to refresh the index and
avoid false local modifications.
2010-02-09 13:31:32 -05:00
Brad King 2ec78b4de7 cmCTestBZR: Strip trailing slashes from paths
Our internal path processing methods assume no trailing slashes, but bzr
adds trailing slashes to updated directories.  This can lead to empty
entries in Update.xml files.  We address the problem by stripping the
slashes as soon as they are parsed.
2010-02-09 13:31:12 -05:00
Zach Mullen a0cc49a915 Support for relative paths to executables in MemCheck tests. Also fixed a bug causing memcheck args to be overwritten repetitively. 2010-02-04 11:24:57 -05:00
Zach Mullen c87282ce52 Re-enabled Scheduler test and fixed the underlying problem. 2010-02-03 16:32:26 -05:00
David Cole b0d5fa0bc7 Fix for issue #5041 - improve error messages that may occur during ctest_build and ctest_configure.
The improved text gives very clear information when either the CMakeLists.txt or CTestConfig.cmake file is missing. Hopefully, it makes it easier for those poor future souls who encounter these messages to solve their problems quickly.
2009-12-29 18:03:45 -05:00
David Cole aad15c3abf Fix issue #10060 - add APPEND arg to ctest_start command.
If APPEND is given to ctest_start, it will read the tag from the current existing Testing/TAG file rather than creating a new one based on the current time stamp. This allows a developer to run several dashboard scripts in a row, all of which will share the same tag/stamp/buildid when they finally get submitted to CDash. Now you can split the running of build phases and test phases for the same dashboard row into multiple scripts.
2009-12-29 14:38:31 -05:00
David Cole 4d3a387ac6 Preserve environment variables across calls to cmCTestScriptHandler::RunConfigurationScript. This will help prevent problems like we are currently experiencing (regarding CMake continuous dashboards only running once each day because the HOME environment variable is modified indirectly by a command in a called script, but affects the environment in the calling script). 2009-12-23 14:34:46 -05:00
Brad King cb27cfb1cc Use human-readable Git commit times in Update.xml
Previously we produced commit times formatted like

  1261403774 -0500

which is what the Git plumbing prints.  Now we use a human-readable
format like

  2009-12-21 15:28:06 -0500

which is still easy to machine-parse.
2009-12-21 10:29:00 -05:00
Zach Mullen 3312e3d710 Fix size type conversion warning in RunTest. 2009-12-21 09:52:56 -05:00
Zach Mullen 66f44f64d5 Fix segfault on Borland compiler resulting from divide by 0. 2009-12-18 11:42:58 -05:00
Brad King a4cdd73636 Use literal quotes in Update.xml UpdateCommand
Previously we escaped quotes in <UpdateCommand>...</UpdateCommand>
values using '&quot;'.  This is not necessary because the value is in
xml CDATA and not an xml attribute.
2009-12-18 10:54:24 -05:00
Brad King 3e724b2725 Submit Subversion directory path in Update.xml
A Subversion revision is unique across the entire repository, but work
trees typically correspond only to a subdirectory below the root path.
In order to specify the version of the source code that was tested,
CTest now submits a <SVNPath> element in Update.xml that specifies the
directory of the repository that corresponds to the work tree.  In
combination with the revision number this uniquely specifies the tested
source.  See issue #7541.
2009-12-18 08:19:41 -05:00
Brad King a73833d037 Submit global tree revision in Update.xml
We teach CTest to report in a <Revision> element the revision of the
source tree that was tested.  This makes sense for all modern VCS tools
because they version the whole tree.  We simply omit this element for
CVS because it only versions files.  See issue #7541.
2009-12-18 08:13:51 -05:00
Zach Mullen 1842d2eae0 Fix output compression error message 2009-12-17 11:17:57 -05:00
Zach Mullen ff916b48bd CTest output submitted to the dashboard is now compressed by default. 2009-12-17 11:14:49 -05:00
Zach Mullen 9add940eb6 Added an option to conditionally attach files to a test submission only if the test does not pass. Also some preliminary changes for test output compression. 2009-12-16 14:50:16 -05:00
Zach Mullen 5bfe1a1962 Changed XML formatting of files attached to test so that each file is archived and marked up individually, in order to make things easier on the CDash side. Also switched to using the NamedMeasurement tag instead of a new tag. 2009-12-15 14:24:24 -05:00
Zach Mullen dc66139f7c CTest-side changes to allow users to attach arbitrary files to test results that will be submitted to cdash using the ATTACHED_FILES test property. 2009-12-15 12:07:15 -05:00
Zach Mullen b2e7da885d Added a "-http1.0" option to ctest to make it submit using curl's http 1.0 option. Also added parsing of html reponse output to determine whether errors or warnings were sent in response from the server. If errors or warnings occurred, the response is output to stdout, and the "submission successful" message has been changed to accurately reflect whether or not warnings or errors were returned with the response. 2009-12-11 14:10:37 -05:00
Zach Mullen 5e427677c3 Restore the "***Not Run" message to ctest command line output for tests which did not start due to missing/bad executables or missing required files. 2009-12-10 15:51:56 -05:00
Zach Mullen 4de7cc3621 Unfortunately, I noticed the comment on bug 8668 too late. This changes my last implementation of the exe wrapper to something which makes much more sense: a REQUIRED_FILES property on tests. 2009-12-10 15:37:04 -05:00
Zach Mullen 48b6133928 [0008668: CTest Dev: Missing executables shown as failed tests when using MPI.] Added a wrapping option to add_test so that exes built by the project can be safely wrapped in other exes and be listed as "not run" rather than "failed" if they are not built. 2009-12-10 14:38:32 -05:00
Zach Mullen 08ac33fe6a Fix memory leak that occurred when a test executable could not be started. (See BadExe test) 2009-12-10 09:39:19 -05:00
Zach Mullen c32088cfee Local test timeouts will now always take precedence over the global --timeout option. 2009-12-08 10:26:43 -05:00
Zach Mullen b30f627190 CMake global timeout (--timeout option) should prevail over individual test timeouts if it is lower than the individual timeout. 2009-12-07 13:25:54 -05:00
David Cole 0b38bb4c53 Fix issue #2336 - honor the -C arg to ctest. Honor it for all stages of running -D dashboards from the command line and running ctest_configure, ctest_build and ctest_test commands in -S scripts. Also, allow a script to change it by setting the CTEST_CONFIGURATION_TYPE variable: allows for multiple configuration build/test cycles within one script. Add a new signature for the cmake command build_command that accepts CONFIGURATION as one argument. The original build_command signature is still there, but now marked as deprecated in the documentation. Of course... also add CTestConfig tests to verify that -C is honored for -D dashboards and -S scripts. 2009-12-04 12:09:01 -05:00
Bill Hoffman 22c37b2b19 Hanle the case where a test can not be run because it is a bad executable. 2009-12-02 16:37:43 -05:00
Zach Mullen c17ce46c73 Align test output for timeout condition. 2009-11-30 16:09:00 -05:00
Zach Mullen a52c0118d4 Added the --timeout option to ctest command line. This sets a global timeout on all tests if no more specific timeout is set on them. 2009-11-30 16:08:11 -05:00
Brad King e1548142fb CTest: Move initial checkout to ctest_start()
In CTest command-driven script mode we support starting without a source
tree.  Previously the ctest_start() command would do some initialization
but could not do anything that required CTestConfig.cmake from the input
source tree.  Later, ctest_update() would run CTEST_CHECKOUT_COMMAND to
create the source tree, and then re-initialize everything.  This
delayed-initialization approach led to many complicated cases of which
only some worked.  For example, the second initialization only worked
correctly in Nightly mode and simply failed for Experimental and
Continuous builds.

A simpler solution is to run CTEST_CHECKOUT_COMMAND during ctest_start()
and then have a single initialization path.  In principle this change in
behavior could break scripts that set the checkout command after
ctest_start() but before ctest_update().  However, the convention we've
always followed has been to set all variables before ctest_start().

See issue #9450.
2009-11-24 08:58:59 -05:00
Zach Mullen 2921fa96ed Handle multiple carriage return issue on windows VS9. Also make sure that running ctest in showonly mode does not kill our cost store, since many ctest tests do this and corrupt our cost data. 2009-11-16 10:58:43 -05:00
Bill Hoffman 1b48d361d8 fix number of callers to work with newer versions of valgrind 2009-11-11 08:43:15 -05:00