Optionally hide the output of each external project build step by
wrapping its command in a launcher script. Make output available in log
files instead. This simplifies the output of external project builds.
At this point, CTestTest3 causes more problems than it's worth.
It uses CVS to grab a remote (over the network) copy of kwsys
code for testing. This causes some sort of problem nearly every
night on the nightly CMake dashboards. Worse: it causes problems
on different machines on different nights, then the next day, it's
fine again. So: remove this test and monitor the coverage.
If we lose a significant portion of code coverage, I will revert
this commit and re-activate the test. However, if we do not lose
a significant portion of code coverage, I will remove the code
for the test as well as removing it from the CMakeLists.txt file.
Brad King and I discussed this over the last few weeks, and we both
think we have sufficient coverage of all the checkout and update code
in other locally (non-network) based tests.
On the other hand, even if we do take a mild hit on coverage temporarily,
it should be relatively easy to increase our coverage again by adding
bits to those other locally based tests.
The bootstrap script works under MSYS, so test it. Use a launcher batch
file since 'ctest --build-and-test' is a Windows program and will not
honor the shebang line in the script.
Even though this test is checking that the ctest running it can handle
test output without newlines we should run the just-built CMake binary.
This allows the MemCheck test mode to check the correct CMake.
Use 'git fetch' followed by 'git reset' to update the source tree. This
is better than 'git pull' because it can handle a rewritten upstream
branch and does not leave local modifications. After fetch, parse
FETCH_HEAD to find the merge head that 'git pull' would choose to track
the upstream branch. Then reset to the selected head.
In the normal fast-forward case the behavior remains unchanged.
However, now local modifications and commits will be erased, and
upstream rewrites are handled smoothly. This ensures that the upstream
branch is tested as expected.
Teach (create|run)_dashboard_script macros to treat the argument as the
name of a build tree. Append '.cmake' to generate the dashboard script
name. This allows future re-use of the macros for multiple test
scripts.
Use it from ExternalProject and the ExternalProject test's
CMakeLists file rather than having duplicate find_program calls.
Add logic so that we do not try to use *.cmd variants of git
programs when using the MSYS Makefiles generator. Should fix
the last remaining dashboard issue with the new ExternalProject
git support additions.
Also, correct minor problem regarding placement of the local git
repo during test execution. On clean builds, it was being placed
incorrectly because of the ../.. relative reference. Use an absolute
path to place the local git repo in the proper directory, and only
use the relative reference when referring to it.
The Intel C compiler for Linux does not seem to reject any bad flags or
object files on its link lines. Work around the problem by using a
preprocessor #error directive to ensure that BADFLAG appears in the
build output. This does not really achieve the purpose of the tests but
it allows them to pass.
Double quote executable names that may have spaces in them.
Do not run the new git portions of the test on machines that
have git < version 1.6.5 on them.
The Borland librarian actually creates a BADFLAG.obj when the object is
missing the first time! This causes later tests to not reject it.
Instead use a Borland-specific variation on the flag.
Add archives of these file types and add to the test
cases covered in the ExternalProject test.
Also add an "Example" directory in the Tests/ExternalProject
directory containing the canonical simplest example of
ExternalProject usage.
Add a LinkFlags test series to check that these properties work. Since
no link flag is accepted everywhere we test for presence of flags by
adding a bad flag and looking for the complaint in the test output.
Echo results of calling git status before exiting with
an error. Add one special case so that the test may pass
on the dashmacmini2 continuous dashboard, despite a 'git
status' non-zero return code. More logic like this may
be required. I will re-evaluate based on tomorrow's
nightly dashboard runs.
Additionally, output some more information in both cvs
and git cases. When it is a cvs checkout, echo the contents
of CVS/Root and CVS/Repository to the test output. When it
is a git checkout, echo the output of 'git branch -a'.
This will allow us to see more details about any given CMake
source tree right in the CDash results for this test.
Improve FILE(DOWNLOAD ...):
- Add percent complete progress output to the FILE DOWNLOAD
command. This progress output is off by default to
preserve existing behavior. To turn it on, pass
SHOW_PROGRESS as an argument.
- Add EXPECTED_MD5 argument. Verify that the downloaded
file has the expected md5 sum after download is complete.
- Add documentation for SHOW_PROGRESS and EXPECTED_MD5.
When the destination file exists already and has the
expected md5 sum, then do not bother re-downloading
the file. ("Short circuit" return.)
Also, add a test that checks for the status output
indicating that the short circuit behavior is actually
occurring. Use a binary file for the test so that the
md5 sum is guaranteed to be the same on all platforms
regardless of "shifting text file line ending" issues.
Improve ExternalProject:
- Add argument URL_MD5.
- Add verify step that compares md5 sum of .tar.gz file
before extracting it.
- Add md5 check to download step, too, to prevent
unnecessary downloads.
- Emit a warning message when a file is not verified.
Indicate that the file may be corrupt or that no
checksum was specified.
Add the <Email>...</Email> element in Update.xml for each commit
reported. This field was defined by Dart but never really used.
Distributed version control systems use author name and email
instead of a user id, so now it makes sense to use this field.
Some Mac linkers produce the message
"file was built for unsupported file format which is not the
architecture being linked"
for this test. Update the test output regex to match it.
Git does not automatically checkout the matching version of a submodule
when it checks out a new version of the parent project in the work tree.
If the submodule reference changed in the parent project then we were
reporting the submodule path as a local modification. Work around the
problem in ctest_update using "git submodule update" after "git pull".
For projects with no submodules this is a no-op. See issue #10662.
Also add a submodule to the test project for CTest.UpdateGIT to test the
work-around.
For builds from Git repositories, add "-g<commit>" to the end of the
version number. If the source tree is modified, append "-dirty".
For builds from CVS checkouts, add "-cvs-<branch>".
If defined and non-empty, the value of CMAKE_TESTS_CDASH_SERVER should point
to a CDash server willing to accept submissions for a project named
PublicDashboard. On machines that also run a CDash dashboard, set this
variable to "http://localhost/CDash-trunk-Testing" so that the CMake tests
that submit dashboards do not have to send those submissions over the wire.
The CTestSubmitLargeOutput test runs a dashboard that has a test that produces
very large amount of output on stdout/stderr. Since we do not even want to
attempt to send such large output over the wire, this test is off by default
unless the CMAKE_TESTS_CDASH_SERVER server is localhost. This test is expected
to cause a submission failure when sent to CDash. It passes if the submit
results contain error output. It fails if the submit succeeds.
CMAKE_TESTS_CDASH_SERVER: CDash server used by CMake/Tests.
If not defined or "", this variable defaults to the server at
http://www.cdash.org/CDash.
If set explicitly to "NOTFOUND", curl tests and ctest tests that use the
network are skipped.
If set to something starting with "http://localhost/", the CDash is expected
to be an instance of CDash used for CDash testing, pointing to a
cdash4simpletest database. In these cases, the CDash dashboards should be
run first.
Commit "Teach CTest.Update tests to strongly check entries" (2010-02-09)
started checking Update.xml entries strongly. This revealed that some
cvs clients report "U CTestConfig.cmake" during update even though the
file did not change and it selects the same revision. As a result the
test fails with
Update.xml has extra unexpected entries:
Updated{CTestConfig.cmake}
We fix the test to tolerate this particular extra entry without failing.
We wrap the git executable in a shell script that touches one source
file after 'git pull'. This makes the file newer than the index even
though it has not actually changed. If CTest does not refresh the index
properly then the test will fail with a bogus modified file.
Previously these tests just checked for matching file names in the
Update.xml files. Now we check the update types (Updated, Modified, or
Conflicting) and reject unexpected extra entries.
At least one Fortran compiler does not provide a preprocessor symbol to
identify itself. Instead we try running unknown compilers with version
query flags known for each vendor and look for known output. Future
commits will add vendor-specific flags/output table entries.
The PathScale compiler silently accepts unknown options that start in
more than one '-':
$ touch foo.c
$ pathcc -c foo.c --junk
$ echo $?
0
$ pathcc -c foo.c ---junk
$ echo $?
0
$ pathcc -c foo.c -junk
pathcc ERROR parsing -junk: unknown flag
$ echo $?
2
We teach the TryCompile to pass a bogus flag with only one '-' instead
of three '-'s for this compiler.
We disable this test because PathScale Fortran mangles module symbols as
"MYSUB.in.MYMODULE" so we cannot interface with it from C. We already
did this for SunPro and MIPSpro.
Makefile dependencies must be escaped using cmLocalGenerator::Convert
with the cmLocalGenerator::MAKEFILE option. This fixes Fortran module
dependencies with spaces in the path. We test the fix by adding a space
to one of the module paths in the Fortran test.
If APPEND is given to ctest_start, it will read the tag from the current existing Testing/TAG file rather than creating a new one based on the current time stamp. This allows a developer to run several dashboard scripts in a row, all of which will share the same tag/stamp/buildid when they finally get submitted to CDash. Now you can split the running of build phases and test phases for the same dashboard row into multiple scripts.
The commit "Make CTestTestTimeout time configurable" added a CMake cache
variable CTestTestTimeout_TIME to configure the length of the timeout
used by the inner CTestTestTimeout test. The reason was to allow users
on slow machines to give this test some extra time without extending the
timeout for everyone. However, Cygwin CMake seems to load slowly enough
that it is worth a longer default timeout for that platform.
The commit "Submit Subversion directory path in Update.xml" added the
element <SVNPath>...</SVNPath> to Update.xml for Subversion work trees.
This commit teaches the CTest.UpdateSVN test to verify the presence of
the element.
We teach CTest to report in a <Revision> element the revision of the
source tree that was tested. This makes sense for all modern VCS tools
because they version the whole tree. We simply omit this element for
CVS because it only versions files. See issue #7541.
We teach ADD_TEST_MACRO to transform names of the form "Namespace.Name"
to the directory "Namespace/Name" and the project name "Name". This
will allow new tests to be better organized.
The test overrides the CMAKE_C_FLAGS and CMAKE_Fortran_FLAGS to test
passing a specific flag to the compiler wrapper scripts. We fix it to
honor any outside flags needed for the real compiler.
The commit "FortranCInterface: Honor language flags in checks" taught
the FortranCInterface module to pass C and Fortran flags into its
detection and verification checks. We improve on the change to allow
the '=' character in the language flags. This requires passing the
cache entry type with the -D options.
We create test FortranC.Flags to try passing per-language flags from a
project into its FortranCInterface detect/verify checks. We wrap the
compilers with scripts that enforce presence of expected flags.
CMake does not enable Fortran for its own build, but it needs to find a
Fortran compiler to know if it is possible to enable Fortran tests.
Previously we searched for a hard-coded list of Fortran compilers which
was duplicated from the CMakeDetermineFortranCompiler.cmake module. We
now run CMake on a small test project that enables the Fortran language
and reports the compiler it found. This represents a more realistic
check of whether the Fortran tests will be able to find a compiler.
Previously our EnforceConfig script that loads at test-time would only
enforce a non-empty CTEST_CONFIGURATION_TYPE for CMake 2.6.2 and lower.
Now we simply always enforce use of a configuration, and select one of
the configurations that was built if none is given.
This is necessary to run tests like CMake.Install that need to know the
configuration with which CMake was built.
We create option CMake_TEST_INSTALL to enable a new CMake.Install test.
It tests running the "make install" target to install CMake itself into
a test directory. We enable the option by default for dashboard builds.
We configure an EnforceConfig.cmake script to load at CTest time.
Previously we loaded it from Tests/CTestTestfile.cmake, but now we load
it from the top level so it applies to all tests.
Xcode 2.x forgets to create the target output directory before linking
the individual architecture pieces of a universal binary for the target
CMakeLibTests. Then it passes the directory to -L and -F options when
linking the and warns that the directory does not exist. We work around
the problem by using a pre-build rule on the target to create the output
directory.
We re-arrange EXECUTABLE_OUTPUT_PATH settings to avoid putting utility
and test executables in the 'bin' directory of the build tree. This
makes the directory look like that in the installation tree, except that
on multi-configuration generators we still use a per-config
subdirectory.
The DumpDocumentation executable and some supporting code and tests were
completely unused by CMake. Generation of documentation is done by the
individual executables with --help* options. In this commit we simply
remove the unused code, executable, and test.
In this test we start up a cmake script that runs a process that sleeps,
and the timeout for the script is shorter than the sleep time. However,
in order to properly detect that the sleeping grandchild is killed when
the script times out we need to give sufficient time for the script to
start the grandchild. Otherwise the log file for the grandchild is not
available.
On some (cygwin) builds our previous 1 second timeout for the script was
not long enough to let the interpreter load and start the grandchild.
We make the timeout time configurable by setting CTestTestTimeout_TIME
in the cache for CMake itself. It tells the test how long to let the
script run. The grandchild always sleeps for 4 seconds longer to ensure
a comfortable window during which the process tree can be killed.
This test requires that the dashboard script it drives be invoked with
"ctest -C <config> -S ...". We create a "CTestTest_CONFIG" variable to
hold a configuration selected at test time. We use the configuration
given to the outer CTest, if any, and then default to either Debug or
the CMAKE_BUILD_TYPE.
We extend the CTestTestTimeout test to check that when a test times out
its children (grandchildren of ctest) are killed. Instead of running
the timeout executable directly, we run it through a cmake script that
redirects the timeout executable output to a file. A second test later
runs and verifies that the timeout executable was unable to complete and
write data to the log file. Only if the first inner test times out and
the second inner test passes (log is empty) does the CTestTestTimeout
test pass.
Brad King
David Cole
Zach Mullen
Bill Hoffman