kolan
/
CMake
1
0
Fork 0
Code Issues Pull Requests Releases Wiki Activity

Merge topic 'reorganize-cmCTest' 

61fcd08a Help: Fix documentation of ctest --build-and-test
7c67d401 Help: Consistently use quotes and fix punctuation in ctest(1) manual
0076fb10 cmCTest: Update doxygen to be consistent
30c5f94c cmCTest: Rename "ProcessTests()" to "ProcessSteps()" and improve doxygen
7c87ab75 cmCTest: Facilitate code reading adding consistent comments in Run()
6d8b9aa6 cmCTest: Improve readability adding "HandleTest(Action|Model)Argument()"
280d0a69 cmCTest: Improve readability adding "RunCMakeAndTest()" and "ExecuteTests()"
898cb987 cmCTest: Fix typo
This commit is contained in:
Brad King 2016-07-06 09:35:38 -04:00 committed by CMake Topic Stage
parent 8df2e68614 61fcd08a49
commit 50adbfcacb
3 changed files with 300 additions and 222 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

104
Help/manual/ctest.1.rst
View File

@ -56,7 +56,7 @@ Options
Enable failover. Enable failover.
This option allows ctest to resume a test set execution that was This option allows ctest to resume a test set execution that was
previously interrupted. If no interruption occurred, the -F option previously interrupted. If no interruption occurred, the ``-F`` option
will have no effect. will have no effect.
``-j <jobs>, --parallel <jobs>`` ``-j <jobs>, --parallel <jobs>``
@ -77,12 +77,12 @@ Options
Make ctest quiet. Make ctest quiet.
This option will suppress all the output. The output log file will This option will suppress all the output. The output log file will
still be generated if the --output-log is specified. Options such still be generated if the ``--output-log`` is specified. Options such
as --verbose, --extra-verbose, and --debug are ignored if --quiet is as ``--verbose``, ``--extra-verbose``, and ``--debug`` are ignored
specified. if ``--quiet`` is specified.
``-O <file>, --output-log <file>`` ``-O <file>, --output-log <file>``
Output to log file Output to log file.
This option tells ctest to write all its output to a log file. This option tells ctest to write all its output to a log file.
@ -90,7 +90,7 @@ Options
Disable actual execution of tests. Disable actual execution of tests.
This option tells ctest to list the tests that would be run but not This option tells ctest to list the tests that would be run but not
actually run them. Useful in conjunction with the -R and -E actually run them. Useful in conjunction with the ``-R`` and ``-E``
options. options.
``-L <regex>, --label-regex <regex>`` ``-L <regex>, --label-regex <regex>``
@ -118,7 +118,7 @@ Options
given regular expression. given regular expression.
``-D <dashboard>, --dashboard <dashboard>`` ``-D <dashboard>, --dashboard <dashboard>``
Execute dashboard test Execute dashboard test.
This option tells ctest to act as a CDash client and perform a This option tells ctest to act as a CDash client and perform a
dashboard test. All tests are <Mode><Test>, where Mode can be dashboard test. All tests are <Mode><Test>, where Mode can be
@ -126,26 +126,27 @@ Options
Update, Configure, Build, Test, Coverage, and Submit. Update, Configure, Build, Test, Coverage, and Submit.
``-D <var>:<type>=<value>`` ``-D <var>:<type>=<value>``
Define a variable for script mode Define a variable for script mode.
Pass in variable values on the command line. Use in conjunction Pass in variable values on the command line. Use in conjunction
with -S to pass variable values to a dashboard script. Parsing -D with ``-S`` to pass variable values to a dashboard script. Parsing ``-D``
arguments as variable values is only attempted if the value arguments as variable values is only attempted if the value
following -D does not match any of the known dashboard types. following ``-D`` does not match any of the known dashboard types.
``-M <model>, --test-model <model>`` ``-M <model>, --test-model <model>``
Sets the model for a dashboard Sets the model for a dashboard.
This option tells ctest to act as a CDash client where the TestModel This option tells ctest to act as a CDash client where the ``<model>``
can be Experimental, Nightly, and Continuous. Combining -M and -T can be ``Experimental``, ``Nightly``, and ``Continuous``.
is similar to -D Combining ``-M`` and ``-T`` is similar to ``-D``.
``-T <action>, --test-action <action>`` ``-T <action>, --test-action <action>``
Sets the dashboard action to perform Sets the dashboard action to perform.
This option tells ctest to act as a CDash client and perform some This option tells ctest to act as a CDash client and perform some
action such as start, build, test etc. Combining -M and -T is action such as ``start``, ``build``, ``test`` etc. See
similar to -D `Dashboard Client Steps`_ for the full list of actions.
Combining ``-M`` and ``-T`` is similar to ``-D``.
``--track <track>`` ``--track <track>``
Specify the track to submit dashboard to Specify the track to submit dashboard to
@ -156,24 +157,24 @@ Options
arbitrary. arbitrary.
``-S <script>, --script <script>`` ``-S <script>, --script <script>``
Execute a dashboard for a configuration Execute a dashboard for a configuration.
This option tells ctest to load in a configuration script which sets This option tells ctest to load in a configuration script which sets
a number of parameters such as the binary and source directories. a number of parameters such as the binary and source directories.
Then ctest will do what is required to create and run a dashboard. Then ctest will do what is required to create and run a dashboard.
This option basically sets up a dashboard and then runs ctest -D This option basically sets up a dashboard and then runs ``ctest -D``
with the appropriate options. with the appropriate options.
``-SP <script>, --script-new-process <script>`` ``-SP <script>, --script-new-process <script>``
Execute a dashboard for a configuration Execute a dashboard for a configuration.
This option does the same operations as -S but it will do them in a This option does the same operations as ``-S`` but it will do them in a
separate process. This is primarily useful in cases where the separate process. This is primarily useful in cases where the
script may modify the environment and you do not want the modified script may modify the environment and you do not want the modified
environment to impact other -S scripts. environment to impact other ``-S`` scripts.
``-A <file>, --add-notes <file>`` ``-A <file>, --add-notes <file>``
Add a notes file with submission Add a notes file with submission.
This option tells ctest to include a notes file when submitting This option tells ctest to include a notes file when submitting
dashboard. dashboard.
@ -188,19 +189,19 @@ Options
contains the same syntax as the command line. contains the same syntax as the command line.
``-U, --union`` ``-U, --union``
Take the Union of -I and -R Take the Union of ``-I`` and ``-R``.
When both -R and -I are specified by default the intersection of When both ``-R`` and ``-I`` are specified by default the intersection of
tests are run. By specifying -U the union of tests is run instead. tests are run. By specifying ``-U`` the union of tests is run instead.
``--rerun-failed`` ``--rerun-failed``
Run only the tests that failed previously Run only the tests that failed previously.
This option tells ctest to perform only the tests that failed during This option tells ctest to perform only the tests that failed during
its previous run. When this option is specified, ctest ignores all its previous run. When this option is specified, ctest ignores all
other options intended to modify the list of tests to run (-L, -R, other options intended to modify the list of tests to run (``-L``, ``-R``,
-E, -LE, -I, etc). In the event that CTest runs and no tests fail, ``-E``, ``-LE``, ``-I``, etc). In the event that CTest runs and no tests
subsequent calls to ctest with the --rerun-failed option will run fail, subsequent calls to ctest with the ``--rerun-failed`` option will run
the set of tests that most recently failed (if any). the set of tests that most recently failed (if any).
``--repeat-until-fail <n>`` ``--repeat-until-fail <n>``
@ -209,7 +210,7 @@ Options
This is useful in finding sporadic failures in test cases. This is useful in finding sporadic failures in test cases.
``--max-width <width>`` ``--max-width <width>``
Set the max width for a test name to output Set the max width for a test name to output.
Set the maximum width for each test name to show in the output. Set the maximum width for each test name to show in the output.
This allows the user to widen the output to avoid clipping the test This allows the user to widen the output to avoid clipping the test
@ -232,26 +233,25 @@ Options
label associated with the tests run. If there are no labels on the label associated with the tests run. If there are no labels on the
tests, nothing extra is printed. tests, nothing extra is printed.
``--build-and-test`` ``--build-and-test <path-to-source> <path-to-build>``
Configure, build and run a test. Configure, build and run a test.
This option tells ctest to configure (i.e. run cmake on), build, This option tells ctest to configure (i.e. run cmake on), build,
and or execute a test. The configure and test steps are optional. and or execute a test. The configure and test steps are optional.
The arguments to this command line are the source and binary The arguments to this command line are the source and binary
directories. By default this will run CMake on the Source/Bin directories.
directories specified unless --build-nocmake is specified. The ``--build-generator`` option *must* be provided to use
The --build-generator option *must* be provided to use ``--build-and-test``. If ``--test-command`` is specified then that will be
--build-and-test. If --test-command is specified then that will be
run after the build is complete. Other options that affect this run after the build is complete. Other options that affect this
mode are --build-target --build-nocmake, --build-run-dir, mode are ``--build-target``, ``--build-nocmake``, ``--build-run-dir``,
--build-two-config, --build-exe-dir, ``--build-two-config``, ``--build-exe-dir``,
--build-project,--build-noclean, --build-options ``--build-project``, ``--build-noclean`` and ``--build-options``.
``--build-target`` ``--build-target``
Specify a specific target to build. Specify a specific target to build.
This option goes with the --build-and-test option, if left out the This option goes with the ``--build-and-test`` option, if left out the
all target is built. ``all`` target is built.
``--build-nocmake`` ``--build-nocmake``
Run the build without running cmake first. Run the build without running cmake first.
@ -264,13 +264,13 @@ Options
Directory where programs will be after it has been compiled. Directory where programs will be after it has been compiled.
``--build-two-config`` ``--build-two-config``
Run CMake twice Run CMake twice.
``--build-exe-dir`` ``--build-exe-dir``
Specify the directory for the executable. Specify the directory for the executable.
``--build-generator`` ``--build-generator``
Specify the generator to use. Specify the generator to use. See the :manual:`cmake-generators(7)` manual.
``--build-generator-platform`` ``--build-generator-platform``
Specify the generator-specific platform. Specify the generator-specific platform.
@ -288,25 +288,23 @@ Options
Skip the make clean step. Skip the make clean step.
``--build-config-sample`` ``--build-config-sample``
A sample executable to use to determine the configuration
A sample executable to use to determine the configuration that A sample executable to use to determine the configuration that
should be used. e.g. Debug/Release/etc should be used. e.g. Debug/Release/etc.
``--build-options`` ``--build-options``
Add extra options to the build step. Add extra options to the build step.
This option must be the last option with the exception of This option must be the last option with the exception of
--test-command ``--test-command``
``--test-command`` ``--test-command``
The test to run with the --build-and-test option. The test to run with the ``--build-and-test`` option.
``--test-output-size-passed <size>`` ``--test-output-size-passed <size>``
Limit the output for passed tests to <size> bytes. Limit the output for passed tests to ``<size>`` bytes.
``--test-output-size-failed <size>`` ``--test-output-size-failed <size>``
Limit the output for failed tests to <size> bytes. Limit the output for failed tests to ``<size>`` bytes.
``--test-timeout`` ``--test-timeout``
The time limit in seconds, internal use only. The time limit in seconds, internal use only.
@ -335,14 +333,14 @@ Options
This option will submit extra files to the dashboard. This option will submit extra files to the dashboard.
``--force-new-ctest-process`` ``--force-new-ctest-process``
Run child CTest instances as new processes Run child CTest instances as new processes.
By default CTest will run child CTest instances within the same By default CTest will run child CTest instances within the same
process. If this behavior is not desired, this argument will process. If this behavior is not desired, this argument will
enforce new processes for child CTest processes. enforce new processes for child CTest processes.
``--schedule-random`` ``--schedule-random``
Use a random order for scheduling tests Use a random order for scheduling tests.
This option will run the tests in a random order. It is commonly This option will run the tests in a random order. It is commonly
used to detect implicit dependencies in a test suite. used to detect implicit dependencies in a test suite.
@ -361,7 +359,7 @@ Options
Set a time at which all tests should stop running. Set a time at which all tests should stop running.
Set a real time of day at which all tests should timeout. Example: Set a real time of day at which all tests should timeout. Example:
7:00:00 -0400. Any time format understood by the curl date parser ``7:00:00 -0400``. Any time format understood by the curl date parser
is accepted. Local time is assumed if no timezone is specified. is accepted. Local time is assumed if no timezone is specified.
``--http1.0`` ``--http1.0``

245
Source/cmCTest.cxx
View File

@ -886,7 +886,7 @@ int cmCTest::ExecuteHandler(const char* shandler)
return handler->ProcessHandler(); return handler->ProcessHandler();
} }
int cmCTest::ProcessTests() int cmCTest::ProcessSteps()
{ {
int res = 0; int res = 0;
bool notest = true; bool notest = true;
@ -2018,7 +2018,7 @@ int cmCTest::Run(std::vector<std::string>& args, std::string* output)
// handle the script arguments -S -SR -SP // handle the script arguments -S -SR -SP
this->HandleScriptArguments(i, args, SRArgumentSpecified); this->HandleScriptArguments(i, args, SRArgumentSpecified);
// handle a request for a dashboard // --dashboard: handle a request for a dashboard
std::string arg = args[i]; std::string arg = args[i];
if (this->CheckArgument(arg, "-D", "--dashboard") && i < args.size() - 1) { if (this->CheckArgument(arg, "-D", "--dashboard") && i < args.size() - 1) {
this->ProduceXML = true; this->ProduceXML = true;
@ -2044,54 +2044,18 @@ int cmCTest::Run(std::vector<std::string>& args, std::string* output)
this->AddVariableDefinition(input); this->AddVariableDefinition(input);
} }
if (this->CheckArgument(arg, "-T", "--test-action") && // --test-action: calls SetTest(<stage>, /*report=*/ false) to enable
(i < args.size() - 1)) { // the corresponding stage
this->ProduceXML = true; if (!this->HandleTestActionArgument(ctestExec, i, args)) {
i++; executeTests = false;
if (!this->SetTest(args[i].c_str(), false)) {
executeTests = false;
cmCTestLog(this, ERROR_MESSAGE,
"CTest -T called with incorrect option: " << args[i]
<< std::endl);
cmCTestLog(this, ERROR_MESSAGE, "Available options are:"
<< std::endl
<< " " << ctestExec << " -T all" << std::endl
<< " " << ctestExec << " -T start" << std::endl
<< " " << ctestExec << " -T update" << std::endl
<< " " << ctestExec << " -T configure" << std::endl
<< " " << ctestExec << " -T build" << std::endl
<< " " << ctestExec << " -T test" << std::endl
<< " " << ctestExec << " -T coverage" << std::endl
<< " " << ctestExec << " -T memcheck" << std::endl
<< " " << ctestExec << " -T notes" << std::endl
<< " " << ctestExec << " -T submit" << std::endl);
}
} }
// what type of test model // --test-model: what type of test model
if (this->CheckArgument(arg, "-M", "--test-model") && if (!this->HandleTestModelArgument(ctestExec, i, args)) {
(i < args.size() - 1)) { executeTests = false;
i++;
std::string const& str = args[i];
if (cmSystemTools::LowerCase(str) == "nightly") {
this->SetTestModel(cmCTest::NIGHTLY);
} else if (cmSystemTools::LowerCase(str) == "continuous") {
this->SetTestModel(cmCTest::CONTINUOUS);
} else if (cmSystemTools::LowerCase(str) == "experimental") {
this->SetTestModel(cmCTest::EXPERIMENTAL);
} else {
executeTests = false;
cmCTestLog(this, ERROR_MESSAGE,
"CTest -M called with incorrect option: " << str
<< std::endl);
cmCTestLog(this, ERROR_MESSAGE, "Available options are:"
<< std::endl
<< " " << ctestExec << " -M Continuous" << std::endl
<< " " << ctestExec << " -M Experimental" << std::endl
<< " " << ctestExec << " -M Nightly" << std::endl);
}
} }
// --extra-submit
if (this->CheckArgument(arg, "--extra-submit") && i < args.size() - 1) { if (this->CheckArgument(arg, "--extra-submit") && i < args.size() - 1) {
this->ProduceXML = true; this->ProduceXML = true;
this->SetTest("Submit"); this->SetTest("Submit");
@ -2106,6 +2070,7 @@ int cmCTest::Run(std::vector<std::string>& args, std::string* output)
cmakeAndTest = true; cmakeAndTest = true;
} }
// --schedule-random
if (this->CheckArgument(arg, "--schedule-random")) { if (this->CheckArgument(arg, "--schedule-random")) {
this->ScheduleType = "Random"; this->ScheduleType = "Random";
} }
@ -2124,6 +2089,7 @@ int cmCTest::Run(std::vector<std::string>& args, std::string* output)
} }
} // the close of the for argument loop } // the close of the for argument loop
// handle CTEST_PARALLEL_LEVEL environment variable
if (!this->ParallelLevelSetInCli) { if (!this->ParallelLevelSetInCli) {
if (const char* parallel = cmSystemTools::GetEnv("CTEST_PARALLEL_LEVEL")) { if (const char* parallel = cmSystemTools::GetEnv("CTEST_PARALLEL_LEVEL")) {
int plevel = atoi(parallel); int plevel = atoi(parallel);
@ -2131,75 +2097,144 @@ int cmCTest::Run(std::vector<std::string>& args, std::string* output)
} }
} }
// now what sould cmake do? if --build-and-test was specified then // now what should cmake do? if --build-and-test was specified then
// we run the build and test handler and return // we run the build and test handler and return
if (cmakeAndTest) { if (cmakeAndTest) {
this->Verbose = true; return this->RunCMakeAndTest(output);
cmCTestBuildAndTestHandler* handler =
static_cast<cmCTestBuildAndTestHandler*>(this->GetHandler("buildtest"));
int retv = handler->ProcessHandler();
*output = handler->GetOutput();
#ifdef CMAKE_BUILD_WITH_CMAKE
cmDynamicLoader::FlushCache();
#endif
if (retv != 0) {
cmCTestLog(this, DEBUG,
"build and test failing returning: " << retv << std::endl);
}
return retv;
} }
if (executeTests) { if (executeTests) {
int res; return this->ExecuteTests();
// call process directory
if (this->RunConfigurationScript) {
if (this->ExtraVerbose) {
cmCTestLog(this, OUTPUT, "* Extra verbosity turned on" << std::endl);
}
cmCTest::t_TestingHandlers::iterator it;
for (it = this->TestingHandlers.begin();
it != this->TestingHandlers.end(); ++it) {
it->second->SetVerbose(this->ExtraVerbose);
it->second->SetSubmitIndex(this->SubmitIndex);
}
this->GetHandler("script")->SetVerbose(this->Verbose);
res = this->GetHandler("script")->ProcessHandler();
if (res != 0) {
cmCTestLog(this, DEBUG,
"running script failing returning: " << res << std::endl);
}
} else {
// What is this? -V seems to be the same as -VV,
// and Verbose is always on in this case
this->ExtraVerbose = this->Verbose;
this->Verbose = true;
cmCTest::t_TestingHandlers::iterator it;
for (it = this->TestingHandlers.begin();
it != this->TestingHandlers.end(); ++it) {
it->second->SetVerbose(this->Verbose);
it->second->SetSubmitIndex(this->SubmitIndex);
}
std::string cwd = cmSystemTools::GetCurrentWorkingDirectory();
if (!this->Initialize(cwd.c_str(), CM_NULLPTR)) {
res = 12;
cmCTestLog(this, ERROR_MESSAGE, "Problem initializing the dashboard."
<< std::endl);
} else {
res = this->ProcessTests();
}
this->Finalize();
}
if (res != 0) {
cmCTestLog(this, DEBUG,
"Running a test(s) failed returning : " << res << std::endl);
}
return res;
} }
return 1; return 1;
} }
bool cmCTest::HandleTestActionArgument(const char* ctestExec, size_t& i,
const std::vector<std::string>& args)
{
bool success = true;
std::string arg = args[i];
if (this->CheckArgument(arg, "-T", "--test-action") &&
(i < args.size() - 1)) {
this->ProduceXML = true;
i++;
if (!this->SetTest(args[i].c_str(), false)) {
success = false;
cmCTestLog(this, ERROR_MESSAGE, "CTest -T called with incorrect option: "
<< args[i] << std::endl);
cmCTestLog(this, ERROR_MESSAGE, "Available options are:"
<< std::endl
<< " " << ctestExec << " -T all" << std::endl
<< " " << ctestExec << " -T start" << std::endl
<< " " << ctestExec << " -T update" << std::endl
<< " " << ctestExec << " -T configure" << std::endl
<< " " << ctestExec << " -T build" << std::endl
<< " " << ctestExec << " -T test" << std::endl
<< " " << ctestExec << " -T coverage" << std::endl
<< " " << ctestExec << " -T memcheck" << std::endl
<< " " << ctestExec << " -T notes" << std::endl
<< " " << ctestExec << " -T submit" << std::endl);
}
}
return success;
}
bool cmCTest::HandleTestModelArgument(const char* ctestExec, size_t& i,
const std::vector<std::string>& args)
{
bool success = true;
std::string arg = args[i];
if (this->CheckArgument(arg, "-M", "--test-model") &&
(i < args.size() - 1)) {
i++;
std::string const& str = args[i];
if (cmSystemTools::LowerCase(str) == "nightly") {
this->SetTestModel(cmCTest::NIGHTLY);
} else if (cmSystemTools::LowerCase(str) == "continuous") {
this->SetTestModel(cmCTest::CONTINUOUS);
} else if (cmSystemTools::LowerCase(str) == "experimental") {
this->SetTestModel(cmCTest::EXPERIMENTAL);
} else {
success = false;
cmCTestLog(this, ERROR_MESSAGE, "CTest -M called with incorrect option: "
<< str << std::endl);
cmCTestLog(this, ERROR_MESSAGE, "Available options are:"
<< std::endl
<< " " << ctestExec << " -M Continuous" << std::endl
<< " " << ctestExec << " -M Experimental" << std::endl
<< " " << ctestExec << " -M Nightly" << std::endl);
}
}
return success;
}
int cmCTest::ExecuteTests()
{
int res;
// call process directory
if (this->RunConfigurationScript) {
if (this->ExtraVerbose) {
cmCTestLog(this, OUTPUT, "* Extra verbosity turned on" << std::endl);
}
cmCTest::t_TestingHandlers::iterator it;
for (it = this->TestingHandlers.begin(); it != this->TestingHandlers.end();
++it) {
it->second->SetVerbose(this->ExtraVerbose);
it->second->SetSubmitIndex(this->SubmitIndex);
}
this->GetHandler("script")->SetVerbose(this->Verbose);
res = this->GetHandler("script")->ProcessHandler();
if (res != 0) {
cmCTestLog(this, DEBUG,
"running script failing returning: " << res << std::endl);
}
} else {
// What is this? -V seems to be the same as -VV,
// and Verbose is always on in this case
this->ExtraVerbose = this->Verbose;
this->Verbose = true;
cmCTest::t_TestingHandlers::iterator it;
for (it = this->TestingHandlers.begin(); it != this->TestingHandlers.end();
++it) {
it->second->SetVerbose(this->Verbose);
it->second->SetSubmitIndex(this->SubmitIndex);
}
std::string cwd = cmSystemTools::GetCurrentWorkingDirectory();
if (!this->Initialize(cwd.c_str(), CM_NULLPTR)) {
res = 12;
cmCTestLog(this, ERROR_MESSAGE, "Problem initializing the dashboard."
<< std::endl);
} else {
res = this->ProcessSteps();
}
this->Finalize();
}
if (res != 0) {
cmCTestLog(this, DEBUG,
"Running a test(s) failed returning : " << res << std::endl);
}
return res;
}
int cmCTest::RunCMakeAndTest(std::string* output)
{
this->Verbose = true;
cmCTestBuildAndTestHandler* handler =
static_cast<cmCTestBuildAndTestHandler*>(this->GetHandler("buildtest"));
int retv = handler->ProcessHandler();
*output = handler->GetOutput();
#ifdef CMAKE_BUILD_WITH_CMAKE
cmDynamicLoader::FlushCache();
#endif
if (retv != 0) {
cmCTestLog(this, DEBUG, "build and test failing returning: " << retv
<< std::endl);
}
return retv;
}
void cmCTest::SetNotesFiles(const char* notes) void cmCTest::SetNotesFiles(const char* notes)
{ {
if (!notes) { if (!notes) {

173
Source/cmCTest.h
View File

@ -53,6 +53,13 @@ class cmXMLWriter;
#endif #endif
#define cout no_cout_use_cmCTestLog #define cout no_cout_use_cmCTestLog
/** \class cmCTest
* \brief Represents a ctest invocation.
*
* This class represents a ctest invocation. It is the top level class when
* running ctest.
*
*/
class cmCTest class cmCTest
{ {
friend class cmCTestRunTest; friend class cmCTestRunTest;
@ -119,7 +126,7 @@ public:
typedef std::vector<cmsys::String> VectorOfStrings; typedef std::vector<cmsys::String> VectorOfStrings;
typedef std::set<std::string> SetOfStrings; typedef std::set<std::string> SetOfStrings;
///! Process Command line arguments /** Process Command line arguments */
int Run(std::vector<std::string>&, std::string* output = CM_NULLPTR); int Run(std::vector<std::string>&, std::string* output = CM_NULLPTR);
/** /**
@ -129,23 +136,30 @@ public:
void Finalize(); void Finalize();
/** /**
* Process the tests. This is the main routine. The execution of the * Process the dashboard client steps.
* tests should look like this:
* *
* Steps are enabled using SetTest()
*
* The execution of the steps (or #Part) should look like this:
*
* /code
* ctest foo; * ctest foo;
* foo.Initialize(); * foo.Initialize();
* // Set some things on foo * // Set some things on foo
* foo.ProcessTests(); * foo.ProcessSteps();
* foo.Finalize(); * foo.Finalize();
* /endcode
*
* \sa Initialize(), Finalize(), Part, PartInfo, SetTest()
*/ */
int ProcessTests(); int ProcessSteps();
/* /**
* A utility function that returns the nightly time * A utility function that returns the nightly time
*/ */
struct tm* GetNightlyTime(std::string const& str, bool tomorrowtag); struct tm* GetNightlyTime(std::string const& str, bool tomorrowtag);
/* /**
* Is the tomorrow tag set? * Is the tomorrow tag set?
*/ */
bool GetTomorrowTag() { return this->TomorrowTag; } bool GetTomorrowTag() { return this->TomorrowTag; }
@ -155,14 +169,14 @@ public:
*/ */
int TestDirectory(bool memcheck); int TestDirectory(bool memcheck);
///! what is the configuraiton type, e.g. Debug, Release etc. /** what is the configuraiton type, e.g. Debug, Release etc. */
std::string const& GetConfigType(); std::string const& GetConfigType();
double GetTimeOut() { return this->TimeOut; } double GetTimeOut() { return this->TimeOut; }
void SetTimeOut(double t) { this->TimeOut = t; } void SetTimeOut(double t) { this->TimeOut = t; }
double GetGlobalTimeout() { return this->GlobalTimeout; } double GetGlobalTimeout() { return this->GlobalTimeout; }
// how many test to run at the same time /** how many test to run at the same time */
int GetParallelLevel() { return this->ParallelLevel; } int GetParallelLevel() { return this->ParallelLevel; }
void SetParallelLevel(int); void SetParallelLevel(int);
@ -200,19 +214,19 @@ public:
cmCTest(); cmCTest();
~cmCTest(); ~cmCTest();
//! Set the notes files to be created. /** Set the notes files to be created. */
void SetNotesFiles(const char* notes); void SetNotesFiles(const char* notes);
void PopulateCustomVector(cmMakefile* mf, const std::string& definition, void PopulateCustomVector(cmMakefile* mf, const std::string& definition,
std::vector<std::string>& vec); std::vector<std::string>& vec);
void PopulateCustomInteger(cmMakefile* mf, const std::string& def, int& val); void PopulateCustomInteger(cmMakefile* mf, const std::string& def, int& val);
///! Get the current time as string /** Get the current time as string */
std::string CurrentTime(); std::string CurrentTime();
//! tar/gzip and then base 64 encode a file /** tar/gzip and then base 64 encode a file */
std::string Base64GzipEncodeFile(std::string const& file); std::string Base64GzipEncodeFile(std::string const& file);
//! base64 encode a file /** base64 encode a file */
std::string Base64EncodeFile(std::string const& file); std::string Base64EncodeFile(std::string const& file);
/** /**
@ -222,11 +236,13 @@ public:
*/ */
double GetRemainingTimeAllowed(); double GetRemainingTimeAllowed();
///! Open file in the output directory and set the stream /**
* Open file in the output directory and set the stream
*/
bool OpenOutputFile(const std::string& path, const std::string& name, bool OpenOutputFile(const std::string& path, const std::string& name,
cmGeneratedFileStream& stream, bool compress = false); cmGeneratedFileStream& stream, bool compress = false);
///! Should we only show what we would do? /** Should we only show what we would do? */
bool GetShowOnly(); bool GetShowOnly();
bool ShouldUseHTTP10() { return this->UseHTTP10; } bool ShouldUseHTTP10() { return this->UseHTTP10; }
@ -242,11 +258,11 @@ public:
std::string GetStopTime() { return this->StopTime; } std::string GetStopTime() { return this->StopTime; }
void SetStopTime(std::string const& time); void SetStopTime(std::string const& time);
// Used for parallel ctest job scheduling /** Used for parallel ctest job scheduling */
std::string GetScheduleType() { return this->ScheduleType; } std::string GetScheduleType() { return this->ScheduleType; }
void SetScheduleType(std::string type) { this->ScheduleType = type; } void SetScheduleType(std::string type) { this->ScheduleType = type; }
///! The max output width /** The max output width */
int GetMaxTestNameWidth() const; int GetMaxTestNameWidth() const;
void SetMaxTestNameWidth(int w) { this->MaxTestNameWidth = w; } void SetMaxTestNameWidth(int w) { this->MaxTestNameWidth = w; }
@ -274,31 +290,37 @@ public:
std::string* stdErr, int* retVal = CM_NULLPTR, std::string* stdErr, int* retVal = CM_NULLPTR,
const char* dir = CM_NULLPTR, double timeout = 0.0); const char* dir = CM_NULLPTR, double timeout = 0.0);
//! Clean/make safe for xml the given value such that it may be used as /**
// one of the key fields by CDash when computing the buildid. * Clean/make safe for xml the given value such that it may be used as
* one of the key fields by CDash when computing the buildid.
*/
static std::string SafeBuildIdField(const std::string& value); static std::string SafeBuildIdField(const std::string& value);
//! Start CTest XML output file /** Start CTest XML output file */
void StartXML(cmXMLWriter& xml, bool append); void StartXML(cmXMLWriter& xml, bool append);
//! End CTest XML output file /** End CTest XML output file */
void EndXML(cmXMLWriter& xml); void EndXML(cmXMLWriter& xml);
//! Run command specialized for make and configure. Returns process status /**
// and retVal is return value or exception. * Run command specialized for make and configure. Returns process status
* and retVal is return value or exception.
*/
int RunMakeCommand(const char* command, std::string& output, int* retVal, int RunMakeCommand(const char* command, std::string& output, int* retVal,
const char* dir, int timeout, std::ostream& ofs); const char* dir, int timeout, std::ostream& ofs);
/* /** Return the current tag */
* return the current tag
*/
std::string GetCurrentTag(); std::string GetCurrentTag();
//! Get the path to the build tree /** Get the path to the build tree */
std::string GetBinaryDir(); std::string GetBinaryDir();
//! Get the short path to the file. This means if the file is in binary or /**
// source directory, it will become /.../relative/path/to/file * Get the short path to the file.
*
* This means if the file is in binary or
* source directory, it will become /.../relative/path/to/file
*/
std::string GetShortPathToFile(const char* fname); std::string GetShortPathToFile(const char* fname);
enum enum
@ -308,7 +330,7 @@ public:
CONTINUOUS CONTINUOUS
}; };
// provide some more detailed info on the return code for ctest /** provide some more detailed info on the return code for ctest */
enum enum
{ {
UPDATE_ERRORS = 0x01, UPDATE_ERRORS = 0x01,
@ -320,14 +342,16 @@ public:
SUBMIT_ERRORS = 0x40 SUBMIT_ERRORS = 0x40
}; };
///! Are we producing XML /** Are we producing XML */
bool GetProduceXML(); bool GetProduceXML();
void SetProduceXML(bool v); void SetProduceXML(bool v);
//! Run command specialized for tests. Returns process status and retVal is /**
// return value or exception. If environment is non-null, it is used to set * Run command specialized for tests. Returns process status and retVal is
// environment variables prior to running the test. After running the test, * return value or exception. If environment is non-null, it is used to set
// environment variables are restored to their previous values. * environment variables prior to running the test. After running the test,
* environment variables are restored to their previous values.
*/
int RunTest(std::vector<const char*> args, std::string* output, int* retVal, int RunTest(std::vector<const char*> args, std::string* output, int* retVal,
std::ostream* logfile, double testTimeOut, std::ostream* logfile, double testTimeOut,
std::vector<std::string>* environment); std::vector<std::string>* environment);
@ -338,13 +362,13 @@ public:
*/ */
int ExecuteHandler(const char* handler); int ExecuteHandler(const char* handler);
/* /**
* Get the handler object * Get the handler object
*/ */
cmCTestGenericHandler* GetHandler(const char* handler); cmCTestGenericHandler* GetHandler(const char* handler);
cmCTestGenericHandler* GetInitializedHandler(const char* handler); cmCTestGenericHandler* GetInitializedHandler(const char* handler);
/* /**
* Set the CTest variable from CMake variable * Set the CTest variable from CMake variable
*/ */
bool SetCTestConfigurationFromCMakeVariable(cmMakefile* mf, bool SetCTestConfigurationFromCMakeVariable(cmMakefile* mf,
@ -352,37 +376,42 @@ public:
const std::string& cmake_var, const std::string& cmake_var,
bool suppress = false); bool suppress = false);
//! Make string safe to be send as an URL /** Make string safe to be send as an URL */
static std::string MakeURLSafe(const std::string&); static std::string MakeURLSafe(const std::string&);
/** Decode a URL to the original string. */ /** Decode a URL to the original string. */
static std::string DecodeURL(const std::string&); static std::string DecodeURL(const std::string&);
//! Should ctect configuration be updated. When using new style ctest /**
// script, this should be true. * Should ctect configuration be updated. When using new style ctest
* script, this should be true.
*/
void SetSuppressUpdatingCTestConfiguration(bool val) void SetSuppressUpdatingCTestConfiguration(bool val)
{ {
this->SuppressUpdatingCTestConfiguration = val; this->SuppressUpdatingCTestConfiguration = val;
} }
//! Add overwrite to ctest configuration. /**
// The format is key=value * Add overwrite to ctest configuration.
*
* The format is key=value
*/
void AddCTestConfigurationOverwrite(const std::string& encstr); void AddCTestConfigurationOverwrite(const std::string& encstr);
//! Create XML file that contains all the notes specified /** Create XML file that contains all the notes specified */
int GenerateNotesFile(const VectorOfStrings& files); int GenerateNotesFile(const VectorOfStrings& files);
//! Submit extra files to the server /** Submit extra files to the server */
bool SubmitExtraFiles(const char* files); bool SubmitExtraFiles(const char* files);
bool SubmitExtraFiles(const VectorOfStrings& files); bool SubmitExtraFiles(const VectorOfStrings& files);
//! Set the output log file name /** Set the output log file name */
void SetOutputLogFileName(const char* name); void SetOutputLogFileName(const char* name);
//! Set the visual studio or Xcode config type /** Set the visual studio or Xcode config type */
void SetConfigType(const char* ct); void SetConfigType(const char* ct);
//! Various log types /** Various log types */
enum enum
{ {
DEBUG = 0, DEBUG = 0,
@ -395,15 +424,15 @@ public:
OTHER OTHER
}; };
//! Add log to the output /** Add log to the output */
void Log(int logType, const char* file, int line, const char* msg, void Log(int logType, const char* file, int line, const char* msg,
bool suppress = false); bool suppress = false);
//! Get the version of dart server /** Get the version of dart server */
int GetDartVersion() { return this->DartVersion; } int GetDartVersion() { return this->DartVersion; }
int GetDropSiteCDash() { return this->DropSiteCDash; } int GetDropSiteCDash() { return this->DropSiteCDash; }
//! Add file to be submitted /** Add file to be submitted */
void AddSubmitFile(Part part, const char* name); void AddSubmitFile(Part part, const char* name);
std::vector<std::string> const& GetSubmitFiles(Part part) std::vector<std::string> const& GetSubmitFiles(Part part)
{ {
@ -411,7 +440,9 @@ public:
} }
void ClearSubmitFiles(Part part) { this->Parts[part].SubmitFiles.clear(); } void ClearSubmitFiles(Part part) { this->Parts[part].SubmitFiles.clear(); }
//! Read the custom configuration files and apply them to the current ctest /**
* Read the custom configuration files and apply them to the current ctest
*/
int ReadCustomConfigurationFileTree(const char* dir, cmMakefile* mf); int ReadCustomConfigurationFileTree(const char* dir, cmMakefile* mf);
std::vector<std::string>& GetInitialCommandLineArguments() std::vector<std::string>& GetInitialCommandLineArguments()
@ -419,7 +450,7 @@ public:
return this->InitialCommandLineArguments; return this->InitialCommandLineArguments;
} }
//! Set the track to submit to /** Set the track to submit to */
void SetSpecificTrack(const char* track); void SetSpecificTrack(const char* track);
const char* GetSpecificTrack(); const char* GetSpecificTrack();
@ -447,10 +478,13 @@ public:
{ {
return this->Definitions; return this->Definitions;
} }
// return the number of times a test should be run
/** Return the number of times a test should be run */
int GetTestRepeat() { return this->RepeatTests; } int GetTestRepeat() { return this->RepeatTests; }
// return true if test should run until fail
/** Return true if test should run until fail */
bool GetRepeatUntilFail() { return this->RepeatUntilFail; } bool GetRepeatUntilFail() { return this->RepeatUntilFail; }
private: private:
int RepeatTests; int RepeatTests;
bool RepeatUntilFail; bool RepeatUntilFail;
@ -485,7 +519,7 @@ private:
bool ShowOnly; bool ShowOnly;
//! Map of configuration properties /** Map of configuration properties */
typedef std::map<std::string, std::string> CTestConfigurationMap; typedef std::map<std::string, std::string> CTestConfigurationMap;
std::string CTestConfigFile; std::string CTestConfigFile;
@ -549,36 +583,47 @@ private:
*/ */
int Initialize(const char* binary_dir, cmCTestStartCommand* command); int Initialize(const char* binary_dir, cmCTestStartCommand* command);
//! parse the option after -D and convert it into the appropriate steps /** parse the option after -D and convert it into the appropriate steps */
bool AddTestsForDashboardType(std::string& targ); bool AddTestsForDashboardType(std::string& targ);
//! read as "emit an error message for an unknown -D value" /** read as "emit an error message for an unknown -D value" */
void ErrorMessageUnknownDashDValue(std::string& val); void ErrorMessageUnknownDashDValue(std::string& val);
//! add a variable definition from a command line -D value /** add a variable definition from a command line -D value */
bool AddVariableDefinition(const std::string& arg); bool AddVariableDefinition(const std::string& arg);
//! parse and process most common command line arguments /** parse and process most common command line arguments */
bool HandleCommandLineArguments(size_t& i, std::vector<std::string>& args, bool HandleCommandLineArguments(size_t& i, std::vector<std::string>& args,
std::string& errormsg); std::string& errormsg);
//! hande the -S -SP and -SR arguments /** hande the -S -SP and -SR arguments */
void HandleScriptArguments(size_t& i, std::vector<std::string>& args, void HandleScriptArguments(size_t& i, std::vector<std::string>& args,
bool& SRArgumentSpecified); bool& SRArgumentSpecified);
//! Reread the configuration file /** Reread the configuration file */
bool UpdateCTestConfiguration(); bool UpdateCTestConfiguration();
//! Create note from files. /** Create note from files. */
int GenerateCTestNotesOutput(cmXMLWriter& xml, const VectorOfStrings& files); int GenerateCTestNotesOutput(cmXMLWriter& xml, const VectorOfStrings& files);
//! Check if the argument is the one specified /** Check if the argument is the one specified */
bool CheckArgument(const std::string& arg, const char* varg1, bool CheckArgument(const std::string& arg, const char* varg1,
const char* varg2 = CM_NULLPTR); const char* varg2 = CM_NULLPTR);
//! Output errors from a test /** Output errors from a test */
void OutputTestErrors(std::vector<char> const& process_output); void OutputTestErrors(std::vector<char> const& process_output);
/** Handle the --test-action command line argument */
bool HandleTestActionArgument(const char* ctestExec, size_t& i,
const std::vector<std::string>& args);
/** Handle the --test-model command line argument */
bool HandleTestModelArgument(const char* ctestExec, size_t& i,
const std::vector<std::string>& args);
int RunCMakeAndTest(std::string* output);
int ExecuteTests();
bool SuppressUpdatingCTestConfiguration; bool SuppressUpdatingCTestConfiguration;
bool Debug; bool Debug;