This deterministic ordering makes it easier for a developer
comparing the ctest output of two test runs to see what changed
from run to run.
Thanks to Fraser Hutchison for the patch.
When add_test(NAME) is called without the CONFIGURATIONS argument then
the test is intended to run in any configuration. In multi-config
generators like the VS IDE and Xcode tests created by add_test(NAME) can
only be run when testing a known configuration (otherwise there is no
way to generate the test command line). If no test command line is
known for a particular configuration, or if no configuration is given to
ctest, report the test as not run instead of silently skipping it.
Also fix CMake's own TestsWorkingDirectory test invocation to correct a
previously silent failure exposed by this change.
3e32db7 cmCTestUploadCommand::CheckArgumentKeyword should return false if not FILES
6b6f309 Add the FILES keyword to ctest_upload command
28cdd0a Don't tar/gz ctest_upload() files
fbe4356 Change 'Files' tag to 'Upload' in Upload.xml
350546d Implement ctest_upload command
Some samples of things that got unnoticed by our nightly builds:
$ JAVA_HOME= mvn
Warning: JAVA_HOME environment variable is not set.
...
$ mvn
[INFO] Scanning for projects...
[INFO] ------------------------------------------------------------------------
[ERROR] BUILD FAILURE
...
Several major Linux distributions still do not provide Git >= 1.6.5.0 in
their stable package lists. Prior to commit 1173cc4a (Update Git
submodules with --recursive, 2011-02-22) CTest was able to use older Git
versions but simply silently failed to update submodules recursively.
Instead of failing with older Git versions preserve the status quo and
add a warning in the update log. Users testing projects with recursive
submodules may simply update to a Git new enough to support them.
Using sort results in a possibly-modified sorting
when all elements are "tied" - use stable_sort instead
to preserve the original ordering of tied elements.
To avoid name clashes.
Before this commit, name clashes can occur when invoking gcov on a
project that uses the same file name in different directories. The
--preserve-paths flag ensures all file names have a complete path,
avoiding name clashes.
Since commit e4beefeb (CTest: Do not munge UTF-8 output in XML files,
2009-12-08) we validate UTF-8 encoding of build and test output as it is
written to XML files. However, in cmCTestTestHandler::CleanTestOutput
we still processed test output one byte at a time and did not recognize
multi-byte UTF-8 characters. Presence of such characters caused early
truncation.
Teach CleanTestOutput to truncate test output at the limit but without
cutting it in the middle of a multi-byte encoding. Also, stop avoiding
truncation in the middle of an XML tag like "<MyElement>" because the
'<' and '>' will be properly escaped in the generated XML anyway.
667a90a Fix sentence break in add_test documentation
96309fc Make TestsWorkingDirectory test a C file
a4a5e37 Use iostream to make Borland happy
cfe53cd Fully specify the path to old-signature add_test
017d4e9 Group adding tests with its properties
561cc33 Only test the default cwd with Makefiles
d87bae7 Simplify the _default_cwd derivation
992c74f Use --><-- markers to denote the path
5249551 Flip slashes around on Windows
0a014da Add ctype.h include for toupper()
af12f83 Fix header includes for C++ and Visual Studio
5597aa2 Rename the project to match the test
9bf4165 Add tests for WORKING_DIRECTORY arg to add_test
42de5d0 Add WORKING_DIRECTORY argument to add_test
7679f9f Rename WorkingDirectory test
d95f817 Add the WORKING_DIRECTORY property to tests
A cycle exists when the DFS returns to the root node, not just when
multiple paths lead to the same node.
Inspired-By: Alexander Esilevich <aesilevich@pathscale.com>
Restore CMake 2.6 behavoir where the dart processing apparently was run from within the tests directory instead of the root of the build tree. This addresses issue reported on the VTK Dashboard where the Testing/Temporary directory is defined as ../../../Testing/Temporary but the DartProcessing is run with respect to the root of the build tree. This causes the regression/difference images to not be located by ctest and thus not uploaded to the dashboard.
This command allows a user to quickly see the list of all available
test labels. The labels are also printed in verbose show only mode,
alongside their corresponding tests.
Refactor how cmCTestMemCheckHandler computes the memory tester command
line options to avoid encoding them in a single string just to parse
them again. The EscapeSpaces uses backslahes to escape spaces on UNIX
platforms, so replace other calls to it in CTest that are used to create
human-readable strings with simple double-quoting.
a7319cf ctest_update: Run 'git submodule' at top level
7bf8dc1 ctest_update: Support ".git file" work trees
65cb72f ctest_update: Abort if Git FETCH_HEAD has no candidates
The git submodule porcelain must be executed from the top level of the
work tree. Use 'git rev-parse --show-cdup' to find the top level
relative to the source tree. This is better than searching up the tree
for .git ourselves because it will always work the same way Git does and
thus honors settings like GIT_DISCOVERY_ACROSS_FILESYSTEM.
Commit c3781efb (Support Git upstream branch rewrites, 2010-06-08)
assumed that ".git/FETCH_HEAD" exists inside the source tree. Fix the
implementation to handle a work tree using a ".git file" to link to its
repository. Use "git rev-parse --git-dir" to locate the real .git dir.
If .git/FETCH_HEAD provides no merge candidate do not attempt to update.
Also log FETCH_HEAD lines as we parse them since they are essentially
output from the git fetch command.
38c762c Merge 'remove-CTestTest3' into ctest-file-checksum
46df0b4 Activate retry code on any curl submit failure.
8705497 Checksum test should use CMAKE_TESTS_CDASH_SERVER
d0d1cdd Mock checksum failure output for old CDash versions
af5ef0c Testing for CTest checksum
86e81b5 CTest should resubmit in the checksum failed case
d6b7107 Fix subscript out of range crash
082c87e Cross-platform fixes for checksum/retry code
e525649 Checksums on CTest submit files, and retry timed out submissions.
Previously, we were only going into the retry block
for time out conditions. But a "could not connect"
response, or really any sort of curl failure, is
also a condition where we should retry the submit
if the user has requested a retry.
Fix niggly in ExternalProject that would inadvertently
create a log file of output when a command was explicitly
set to the empty string.
Also, increase the default value for showing context
around build errors to 10 lines, up from 6. Also add
the ability to customize this setting so that users
can put it back to 6 or even lower if they want, or
show more if they want.
Change types of local variables, or casting, or re-arrange
expressions to get rid of "conversion may alter value" warnings
as seen on recent dashboard submissions from londinium.kitware.
Git's diff-tree format has no '\n'-terminated blank line at the end of
its commit message body block if there are no diff lines. Instead the
message body is terminated by '\0' and there is no diff section. Teach
CTest to parse the format in this case.
Use 'git fetch' followed by 'git reset' to update the source tree. This
is better than 'git pull' because it can handle a rewritten upstream
branch and does not leave local modifications. After fetch, parse
FETCH_HEAD to find the merge head that 'git pull' would choose to track
the upstream branch. Then reset to the selected head.
In the normal fast-forward case the behavior remains unchanged.
However, now local modifications and commits will be erased, and
upstream rewrites are handled smoothly. This ensures that the upstream
branch is tested as expected.
Commit 67277bac (Teach ctest_update about Git submodules, 2010-05-04)
accidentally logged "git submodule update" with the prefixes "pull-out"
and "pull-err". Fix it to use "submodule-out" and "submodule-err"
instead.
Add the <Email>...</Email> element in Update.xml for each commit
reported. This field was defined by Dart but never really used.
Distributed version control systems use author name and email
instead of a user id, so now it makes sense to use this field.
Git does not automatically checkout the matching version of a submodule
when it checks out a new version of the parent project in the work tree.
If the submodule reference changed in the parent project then we were
reporting the submodule path as a local modification. Work around the
problem in ctest_update using "git submodule update" after "git pull".
For projects with no submodules this is a no-op. See issue #10662.
Also add a submodule to the test project for CTest.UpdateGIT to test the
work-around.
Report in <UpdateReturnStatus> when the update command line returns
non-zero. Otherwise the failure may be silently ignored since the error
was previously recorded only in the local log file.
The SaveRestoreEnvironment helper object makes sure that the
original environment is restored immediately after the
StartProcess call returns rather than waiting for the end
of the test. When tests are executed in parallel, there is
no guarantee about the ordering of EndTest calls relative
to StartTest calls. In fact, it would be odd for them to
be nested nicely. Therefore, to avoid the corruption of
the calling ctest's environment, the original environment
must be restored before ForkProcess returns.
Allow the user to set the CMake variable CTEST_COST_DATA_FILE, which will be used to store the cost data from test runs. If not set, defaults to the original location in the build tree Testing/Temporary dir.
We use 'git diff-index' to detect local modifications after pull. On
some filesystems the work tree timestamps of a few files may be dated
after the index, making them appear as locally modified. We address the
problem by using 'git update-index --refresh' to refresh the index and
avoid false local modifications.
Our internal path processing methods assume no trailing slashes, but bzr
adds trailing slashes to updated directories. This can lead to empty
entries in Update.xml files. We address the problem by stripping the
slashes as soon as they are parsed.
The improved text gives very clear information when either the CMakeLists.txt or CTestConfig.cmake file is missing. Hopefully, it makes it easier for those poor future souls who encounter these messages to solve their problems quickly.
If APPEND is given to ctest_start, it will read the tag from the current existing Testing/TAG file rather than creating a new one based on the current time stamp. This allows a developer to run several dashboard scripts in a row, all of which will share the same tag/stamp/buildid when they finally get submitted to CDash. Now you can split the running of build phases and test phases for the same dashboard row into multiple scripts.
Previously we produced commit times formatted like
1261403774 -0500
which is what the Git plumbing prints. Now we use a human-readable
format like
2009-12-21 15:28:06 -0500
which is still easy to machine-parse.
Previously we escaped quotes in <UpdateCommand>...</UpdateCommand>
values using '"'. This is not necessary because the value is in
xml CDATA and not an xml attribute.
A Subversion revision is unique across the entire repository, but work
trees typically correspond only to a subdirectory below the root path.
In order to specify the version of the source code that was tested,
CTest now submits a <SVNPath> element in Update.xml that specifies the
directory of the repository that corresponds to the work tree. In
combination with the revision number this uniquely specifies the tested
source. See issue #7541.
We teach CTest to report in a <Revision> element the revision of the
source tree that was tested. This makes sense for all modern VCS tools
because they version the whole tree. We simply omit this element for
CVS because it only versions files. See issue #7541.
In CTest command-driven script mode we support starting without a source
tree. Previously the ctest_start() command would do some initialization
but could not do anything that required CTestConfig.cmake from the input
source tree. Later, ctest_update() would run CTEST_CHECKOUT_COMMAND to
create the source tree, and then re-initialize everything. This
delayed-initialization approach led to many complicated cases of which
only some worked. For example, the second initialization only worked
correctly in Nightly mode and simply failed for Experimental and
Continuous builds.
A simpler solution is to run CTEST_CHECKOUT_COMMAND during ctest_start()
and then have a single initialization path. In principle this change in
behavior could break scripts that set the checkout command after
ctest_start() but before ctest_update(). However, the convention we've
always followed has been to set all variables before ctest_start().
See issue #9450.
The ctest subdirs command now checks the relative path first, and if that does not exist, also checks if the given path was absolute. Thanks vodall for the patch.
This converts the CMake license to a pure 3-clause OSI-approved BSD
License. We drop the previous license clause requiring modified
versions to be plainly marked. We also update the CMake copyright to
cover the full development time range.
The cmProcess::Buffer class derives from std::vector. We were using
local variable 'data' in the GetLine method but this name shadowed a
member of vector with GNU. This renames it to 'text'.
When we clear the buffer for an output pipe after returning the last
partial line (without a newline) we need to set the partial line range
to empty. Otherwise the buffer object is left in an inconsistent state.
This commit fixes cmCTestRunTest and cmProcess to more efficiently
handle child output. We now use the buffer for each child output pipe
to hold at most a partial line plus one new block of data at a time.
All complete lines are scanned in-place, and then only the partial line
at the end of the buffer is moved back to the beginning before appending
new data.
We also simplify the cmProcess interface by making GetNextOutputLine the
only method that needs to be called while the process is running. This
simplifies cmCTestRunTest so that CheckOutput can be called until it
returns false when the process is done.
The GNU compiler warns about possible operator precedence mistakes and
asks for explicit parentheses (-Wparentheses). We add the parentheses
to silence the warning. This also fixes one real logic error in the
find_package() implementation by correcting expression evaluation order.
CTest runs 'svn status' to identify modified and conflicting files in
the working directory. This commit fixes the interpretation of the 'X'
status, which corresponds to svn eXternals. This status should be
ignored rather than treated as a local modification.
The commit "Fix get_filename_component ABSOLUTE mode" broke the code
get_filename_component(cwd . ABSOLUTE)
because CTest scripts did not make cmMakefile::GetCurrentDirectory()
available. This commit fixes the problem by setting the proper
information on CTest script instances of cmMakefile.
This also makes CMAKE_CURRENT_SOURCE_DIR and CMAKE_CURRENT_BINARY_DIR
available to CTest scripts. They are set to the working directory at
script startup.
All global generator CreateLocalGenerator methods automatically
initialize the local generator instances with SetGlobalGenerator. In
several places we were calling SetGlobalGenerator again after receiving
the return value from CreateLocalGenerator. The double-initializations
leaked the resources allocated by the first call to SetGlobalGenerator.
This fix removes the unnecessary calls.
cmCTestScriptHandler, but have it load the new script CTestScriptMode.cmake
-> that makes it more flexible, also add a simple test that the system name
has been determined correctly
Alex
This creates cmCTestHG to drive CTest Update handling on hg-based work
trees. Currently we always update to the head of the remote tracking
branch (hg pull), so the nightly start time is ignored for Nightly
builds. A later change will address this.
See issue #7879. Patch from Emmanuel Christophe. I modified the patch
slightly for code style, to finish up some parsing details, and to fix
the test.
Previously tests marked with WILL_FAIL have been reported by CTest as
...............***Failed - supposed to fail
when they correctly failed. Now we just report ".....Passed" because
there is no reason to draw attention to something that works as
expected.
The TortoiseCVS version of cvs.exe includes the '.exe' in cvs update
messages for files removed from the repository. This change accounts
for it in the regular expressions that match such lines. Now removed
files are properly reported by ctest_update() when using TortoiseCVS.
We've chosen to drop our default dependence on xmlrpc. Thus we disable
the corresponding CTest submission method and remove the sources for
building xmlrpc locally. Users can re-enable the method by setting the
CTEST_USE_XMLRPC option to use a system-installed xmlrpc library.
The BZR xml output plugin can use some encodings that are not recognized
by expat, which leads to "Error parsing bzr log xml: unknown encoding".
This works around the problem by giving expat a mapping, and adds a
test. Patch from Tom Vercauteren. See issue #6857.
Previously CTest would drop dashboard submissions at public.kitware.com
on the PublicDashboard project if there was no configuration. The
server no longer supports forwarding to cdash.org, so there is no point
in this default. Furthermore, there should be no default at all because
it could leak information about proprietary projects that are not
configured correctly.
This creates cmCTestBZR to drive CTest Update handling on bzr-based work
trees. Currently we always update to the head of the remote tracking
branch (bzr pull), so the nightly start time is ignored for Nightly
builds. A later change will address this. Patch from Tom Vercauteren.
See issue #6857.
When CTest runs 'cvs log' to get revision information for updated files,
we were passing '-d<now'. The option seems useless since revisions
cannot be created in the future, and can lose revisions if the client
machine clock is behind the server.
This creates cmCTestGIT to drive CTest Update handling on git-based work
trees. Currently we always update to the head of the remote tracking
branch (git pull), so the nightly start time is ignored for Nightly
builds. A later change will address this. See issue #6994.
This factors parts of the svn update implementation that are useful for
any globally-versioning vcs tool into cmCTestGlobalVC. It will allow
the code to be shared among the support classes for most vcs tools.
Thomas Jarosch
David Cole
Johan Björk
Brad King
Bill Hoffman
Zach Mullen
Rolf Eike Beer
Marco Craveiro
Chuck Atkins
Clinton Stimpson
Kai Wasserbäch
Alexander Neundorf