Commit Graph

31 Commits

Author SHA1 Message Date
Brad King 438fabf242 Tests: Teach RunCMake infrastructure to use custom check.cmake file 2015-09-17 10:21:08 -04:00
Brad King f38625be7a Tests: Teach RunCMake to tolerate 'Bullseye Testing' lines in test output
When testing under Bullseye coverage, some tests get lines on stderr of
the form:

 ... Bullseye Testing Technology ...

Remove such lines from output before matching because they are not
representative of the actual test output.
2015-09-17 10:20:59 -04:00
Brad King 1a75a96650 Tests: Teach RunCMake to tolerate 'Time Machine' lines in test output
On some OS X machines some tests get lines on stderr of the form:

 ... attempting to exclude an item from Time Machine by path ...

produced by the system.  Remove such lines from output before matching
because they are not representative of the actual test output.
2015-09-17 10:20:58 -04:00
Brad King dffc307c81 Tests: Teach RunCMake infrastructure to optionally timeout
Add a RunCMake_TEST_TIMEOUT option that tests can set to cause RunCMake
to limit the time it waits for the child process to finish.
2015-06-30 10:21:34 -04:00
Brad King 703e762529 Tests: Protect RunCMake tests from MAKEFLAGS in environment (#15542) 2015-04-29 16:14:52 -04:00
Brad King 7b8e7c4ac3 Deprecate Visual Studio 7 generator (.NET 2002)
Update documentation to mark the generator deprecated.  Add a warning at
the end of generation plus an option to turn off the warning.
2015-04-15 09:21:24 -04:00
Brad King 85c2626bab Deprecate Visual Studio 6 generator
Update documentation to mark the generator deprecated.  Add a warning at
the end of generation plus an option to turn off the warning.
2015-04-15 09:08:57 -04:00
Brad King 1535dcd894 Tests: Teach RunCMake to optionally merge command output to stdout
Some tests may need to read "cmake --build" output passed through from
native build tools and do not know if it will be on stdout or stderr.
Optionally use the same variable for the execute_process output so
that it merges them and we can always match using expected stdout.
2015-04-07 09:08:39 -04:00
Brad King 7ce9f6e29e Tests: Teach RunCMake to tolerate 'Error kstat returned' lines in test output
When RunCMake tests run on Solaris, the output may contain unexpected
lines of the form "Error kstat returned...".  These lines are printed by
SystemInformationImplementation::RunProcess when called from
SystemInformationImplementation::ParseValueFromKStat (see issue #12066).
Until someone investigates why kstat returns values outside the range it
documents, simply remove such lines from the actual output before
matching it against the expected output.
2015-02-23 10:01:58 -05:00
Brad King 69de0f7ea4 Tests: Teach RunCMake to tolerate Guard Malloc lines in test output
When RunCMake tests run under Xcode Guard Malloc, Guard Malloc may add
lines of the form "<tool>(<pid>) malloc:..." to the output.  Remove such
lines from the actual output before matching it against the expected
output.
2015-02-23 10:01:57 -05:00
Brad King 57f2aa7c37 Tests: Teach RunCMake to tolerate BullseyeCoverage lines in test output
When RunCMake tests run under dynamic analysis, Bullseye may add lines
of the form "BullseyeCoverage..." to the output.  Remove such lines from the
actual output before matching it against the expected output.
2015-02-23 10:01:57 -05:00
Stephen Kelly c0d8e71591 RunCMake: Allow specifying the stderr file for a test. 2015-02-09 19:15:04 +01:00
Brad King bf05938b8f Tests/RunCMake: Normalize newlines before checking actual output
We read the expected output with file(STRINGS) which converts CRLF
newlines to LF.  Do the same with the actual output before comparing.
2015-01-27 11:18:36 -05:00
Brad King 23bf98fc04 Tests/RunCMake: Honor CMAKE_MAKE_PROGRAM
If tests are using an explicit make program, pass it through to RunCMake
tests so that any that actually do build use the expected tool.
2015-01-24 07:35:44 -05:00
Brad King 6c121924ad Tests: Teach RunCMake to report command-line on failure
When a test sets RunCMake_TEST_COMMAND, report the command line
on failure to make it easier to run by hand for debugging.
2015-01-15 14:46:58 -05:00
Stephen Kelly 73ae888506 RunCMake: Expect empty output by default.
Expect tests to specify stderr content if it is present.

Fix the CMP0019 test, which has only been testing the WARN status
until now.  Specify in the CommandLine and FPHSA tests that content
is at least one character.

Set policies in the Language and CheckModules tests, which have empty
test output, modulo unrelated policies on some platforms.
2014-12-31 16:34:34 +01:00
Brad King 29533380ea Tests: Allow RunCMake expected result code to be a regex
This will allow the '-result.txt' files to specify a regex matching
different results on different platforms.
2014-12-16 13:34:34 -05:00
Brad King eb7d815649 cmake: Add -A option to specify a generator platform
Define the 'cmake -A' option to set CMAKE_GENERATOR_PLATFORM
without having to spell out the whole variable name.  We choose
the name '-A' for "platform" because '-P' is already taken, and
in the common use case the "platform" is actually an architecture
(e.g. x64).

Teach the RunCMake test infrastructure to use -A to pass the generator
platform.  Extend the RunCMake.GeneratorPlatform test with a case to
verify that the -A option cannot be repeated.
2014-09-15 10:26:59 -04:00
Brad King 09ab207c66 Tests: Add generator platform support
Propagate CMAKE_GENERATOR_PLATFORM through the test hierarchy so that all
tests can build with the selected generator platform, if any.
2014-09-10 11:21:50 -04:00
Brad King 6eee5d7449 Tests: Drop broken and now unused RunCMake_TEST_FILE option
We cannot use -DRunCMake_TEST=${RunCMake_TEST_FILE} because test cases
use project(${RunCMake_TEST}) and the project name cannot be a path to a
file.  The parent and grandparent commits removed the only uses of the
option.  Drop it now.
2014-06-30 14:35:52 -04:00
Stephen Kelly c869984ea0 RunCMake: Allow specifying the source dir and file to test.
This will allow decoupling the name of the test from the name and
location of the source file under test, which means one source
file can be used for multiple tests.

Rename the PARENT_SCOPE test in RunCMake.set to not use a keyword
of the if() command as a file name.  As the filename is now used
with an if condition, this causes a conflict.
2014-04-06 23:58:35 +02:00
Brad King 7cd239ed38 Tests/RunCMake: Tell cmake not to report unused command-line options
On OS X we pass -DCMAKE_POLICY_DEFAULT_CMP0025=NEW to all tests.
Once tests start using cmake_minimum_required(VERSION 3.0) then
CMake will warn that CMAKE_POLICY_DEFAULT_CMP0025 is unused.
These warnings are not part of the expected test output and can
cause such tests to fail.  Pass --no-warn-unused-cli to each test
cmake invocation to tell it not to produce these warnings.
2014-02-27 13:25:23 -05:00
Brad King d4ca30ae15 Tests/RunCMake: Add function to run a specified command-line
Add a 'run_cmake_command' function that can be used by tests to run a
given command-line and check the results rather than always running a
CMake configuration process.  This can be used in the future to test
'cmake -E' for example.
2014-01-20 13:46:51 -05:00
Brad King 12a7e2b10c Merge topic 'apple-clang-id'
1763c31 Set policy CMP0025 to NEW while building CMake itself
aa53ee5 Add policy CMP0025 for Apple Clang compiler id compatibility
ab65862 Clang: Add separate "AppleClang" compiler id
2013-10-09 10:21:28 -04:00
Brad King 1763c31c3b Set policy CMP0025 to NEW while building CMake itself
CMake is aware of the policy's NEW behavior and the AppleClang compiler
id.  Set the policy to NEW explicitly to avoid the warning and get the
NEW behavior.

Also teach the RunCMake test infrastructure to build tests with
-DCMAKE_POLICY_DEFAULT_CMP0025=NEW to avoid the policy warning
in test output that must match specific regular expressions.
2013-10-07 20:12:46 -04:00
Brad King 49533303a5 Tests/RunCMake: Tolerate valgrind lines in CMake output
When RunCMake tests run during dynamic analysis, valgrind may add lines
of the form "==[0-9]+==..." to the output.  Remove such lines from the
actual output before matching it against the expected output.
2013-09-16 09:59:52 -04:00
Brad King c307e1c911 Tests/RunCMake: Allow tests to control build tree behavior
Teach the run_cmake to allow tests to set a custom test build directory.
Also add an option to skip removing the build directory.
2013-02-18 10:47:27 -05:00
Brad King 56ca8d4e63 Tests: Add generator toolset support
Propagate CMAKE_GENERATOR_TOOLSET through the test hierarchy so that all
tests can build with the selected generator toolset, if any.
2013-02-07 11:09:56 -05:00
Brad King 4fd5342956 CMake: Add -T option to choose a generator toolset
Reject the option by default.  It will be implemented on a per-generator
basis.  Pass the setting into try_compile project generation.  Add cache
entry CMAKE_GENERATOR_TOOLSET and associated variable documentation to
hold the value persistently.

Add a RunCMake.GeneratorToolset test to cover basic "-T" option cases.
Verify that CMAKE_GENERATOR_TOOLSET is empty without -T, that -T is
rejected when the generator doesn't support it, and that two -T options
are always rejected.
2013-02-07 10:53:19 -05:00
Brad King 38c3943b6f Teach RunCMake tests to allow custom checks
Look for a <SubTest>-check.cmake script and load it to check side
effects of the sub test.  Provide it with the test source and build tree
paths in variables.  Check for a failure message in a result variable.
2012-04-19 08:32:57 -04:00
Brad King 42a81e7119 Add stronger infrastructure for CMake-only tests
The CMakeOnly directory added by commit 9a20abf0 (Add infrastructure for
CMake-only tests, 2012-01-11) was sufficient only for tests that always
run CMake to successfully configure a project.  Later commit eeaaffcb
(find_package: Test error and warning messages in failure cases,
2012-02-28) added a sample test that covers failure cases.

Generalize the above to create new "RunCMake" test infrastructure that
can run CMake multiple times for a single project with different
variations and check for expected result/stdout/stderr.  Allow for both
successful and failing CMake project configuration cases.  This will be
useful to test error messages and failure behavior.
2012-03-12 09:33:21 -04:00