When downloading an URL with ExternalProject, reattempt the download three
times if the file hash verification fails.
The re-attempt is limited to non-local URL's. The same download CMake script
is re-used after removing any file that resulted from the previous download
attempt. Up to three re-attempts are performed.
Manually revise the .rst format of the documentation. Use inline
literal quotes appropriately in paragraph text. Move the :: literal
block openers to the end of the preceding paragraphs. Convert the
command signature documentation and examples to cmake code-block
directives.
The VS IDE sets the environment variable VS_UNICODE_OUTPUT when
executing build rules in order to tell MS tools to report output through
a back door instead of through stdout/stderr. Unset this variable so
that CMake can capture or properly redirect all output from processes it
runs even when running inside a VS IDE build environment.
This generalizes the special cases fixed by commit 80d045b0 (When
GetPrerequisites.cmake runs dumpbin while running inside the VS IDE...,
2008-05-01) and commit 44aff73d (ExternalProject: Avoid bleed-through
output when logging, 2011-01-06), so drop special handling of
VS_UNICODE_OUTPUT in those instances.
We support multiple commands per external project step by using
the COMMAND keyword. Document this behavior and show an example.
While at it, document that shell operators and current working
directory behavior is not defined.
With
SVN_USERNAME ""
SVN_PASSWORD ""
in an ExternalProject_Add() call, the blank username and password
will be passed to the svn checkout/update step commands.
When the CMAKE_GENERATOR option is given to ExternalProject_Add, look
also for option CMAKE_GENERATOR_TOOLSET to select the value of the cmake
"-T" command-line flag. When no CMAKE_GENERATOR option is given
explicitly then use the current project's CMAKE_GENERATOR_TOOLSET (since
we already use its CMAKE_GENERATOR).
The ExternalProject_Add DEPENDS option adds two types of dependencies.
It adds a target-level build order dependency between the external
project target and the named targets. It also adds a file-level
dependency on the "done" stamp file of the named external project
targets. Targets not created by ExternalProject_Add have no such stamp
file and no _EP_STAMP_DIR property. Prior to commit d14c0243 (Refactor
repeated code into function, 2012-04-26) we unconditionally accepted an
empty stamp dir and generated a dependency on a non-existent file.
After that commit we generate an error that no stamp dir is set.
Skip the file-level dependency when the named dependency is not an
external project target in order to allow this use case. Teach the
ExternalProject test to cover the case.
In the current default update step for Git under the ExternalProject_Add
command, a 'git fetch' is always performed, followed by a 'git checkout' and
'git submodule update --recursive'. However, a 'git fetch' can be time
consuming and requires a network connection.
To save time, we look at the current checked out hash, and only perform the
fetch if required. This is performed in a CMake script so we can handle
the conditional logic in a cross platform manner.
Make the EXPECTED_HASH option take only a single value instead of two to
avoid handling sub-keyword arguments. This is also consistent with
URL_HASH in ExternalProject.
Some download URLs do not have the filename embedded in the url.
Add an interface to specify the local filename explicitly.
Suggested-by: James Goppert <james.goppert@gmail.com>
TLS has superseded SSL so rename the recently added file(DOWNLOAD) and
ExternalProject options using the newer terminology. Drop "CURLOPT"
from names because curl is an implementation detail.
This commit adds the ability to turn on and off ssl certificate
authority checking. It also adds the ability to specify a
certificate authority information file. This can be done
by setting global cmake variables CMAKE_CURLOPT_CAINFO_FILE
and or CMAKE_CURLOPT_SSL_VERIFYPEER in the project calling
ExternalProject_Add, or by passing those options to individual
ExternalProject_Add calls.
Add options HG_REPOSITORY and HG_TAG to specify an external project
hosted in a Mercurial repository. Teach ExternalProject to clone the
repository and update from it. Extend the ExternalProject test to try a
Mercurial repository when hg is available.
Ancient versions of CMake required else(), endif(), and similar block
termination commands to have arguments matching the command starting the
block. This is no longer the preferred style.
Run the following shell code:
for c in else endif endforeach endfunction endmacro endwhile; do
echo 's/\b'"$c"'\(\s*\)(.\+)/'"$c"'\1()/'
done >convert.sed &&
git ls-files -z -- bootstrap '*.cmake' '*.cmake.in' '*CMakeLists.txt' |
egrep -z -v '^(Utilities/cm|Source/kwsys/)' |
egrep -z -v 'Tests/CMakeTests/While-Endwhile-' |
xargs -0 sed -i -f convert.sed &&
rm convert.sed
Fix the build command when Ninja is the CMake Generator, but the
external project is created for a project that does not use CMake but
does use "make".
By tracking a stamp file within the git clone script itself.
Avoids a 2nd git clone operation after switching from Debug
to Release builds in Visual Studio, or vice-versa.
Add "private/internal-use-only" function _ep_get_step_stampfile
to get the name of the stamp file for a given step.
The functionality provided by this commit should be identical
to its parent commit.
In the case of git, only track the repository in the
repository info dependency tracking file. Not the tag.
The download step should only re-run if the repository changes.
The download step should NOT re-run if the tag changes.
The update step is an 'always' re-running step, and so should
already re-run, unless it's been eliminated by use of
UPDATE_COMMAND ""
Commit f67139ae added running a verify script in between running
the download and extract scripts. Since then, it has always been
missing the COMMAND keyword added in this commit.
It worked anyway (semi-accidentally) by running a command line like:
cmake -P script1.cmake cmake -P script2.cmake
CMake, when running -P scripts on the command line, runs them in order,
and apparently ignores spurious arguments in between (the middle "cmake"
in the above example) and so, all appeared to work as intended.
This commit adds the missing keyword and the commands that run are
now two separate sequential cmake invocations like:
cmake -P script1.cmake
cmake -P script2.cmake
...which was the original intent of commit f67139ae
When GIT_EXECUTABLE points at ".../Git/cmd/git.cmd" in an msysGit
installation we previously failed to detect the version number in a
subtle case. The "git.cmd" assumes 'chcp' is in PATH. It is typically
available at "C:\Windows\System32\chcp.com". On 64-bit Windows the File
System Redirector maps this location to "C:\Windows\SysWOW64\chcp.com"
for 32-bit processes. However, some Windows installations fail to
provide chcp.com at this path. Whenever git.cmd runs in a 32-bit
command shell, as it does under a 32-bit CMake binary, it reports
'chcp' is not recognized as an internal or external command,
operable program or batch file.
on stderr. Capture stderr separately so it does not affect parsing
of the version number.
See also msysGit issue 358:
http://code.google.com/p/msysgit/issues/detail?id=358
Note that FindGit prefers "git.cmd" over "git.exe" because it sets up
the proper HOME environment variable necessary for Git ssh connections
to work the same as they do from the Git bash prompt.
The previous commit added --non-interactive as a "side effect"
of turning on SVN_TRUST_CERT. While reviewing that commit, we
decided all ExternalProject svn usage should be non-interactive.
That way, if there's any sort of problem, svn will return an
error right away rather than hang forever waiting for input...
Which adds --non-interactive and --trust-server-cert to the svn
checkout and update command lines. This allows ExternalProject
clients to pull from an https:// based svn server even though
the server may not have a valid or trusted certificate.
Caveat emptor: I would NOT recommend using this except as a
short-term work-around. Rather, the server should have a valid,
trusted certificate, or the client should be using "http" instead
of "https".
Unset VS_UNICODE_OUTPUT when executing a command whose output
is being logged to a file. Previously, running Microsoft tools
in sub-processes of Visual Studio would send their output to
the Visual Studio output pipe named by this environment variable.
Unsetting it forces the output back to the normal stdout and stderr
channels where cmake can intercept it and direct it to the
appropriate log files.
Escaped the @var@ in the file writes - this was being expanded at file
write and so not causing a reconfigure at the right time. I also took
care of build up lists of lists in the variables, especially important
for things like MPI_EXTRA_LIBRARY. Added some error checking, and use
the tmp_dir for initial cache file.
On Windows the limit for command line arguments is 8192 characters, and
this was limiting longer paths with some of our more nested projects
such as Library. Placing the -D arguments into CMAKE_CACHE_ARGS will
write out an initial cache file, that will be passed to CMake with a -C
argument as the initial cache.
By forcing the cache variables we preserve the existing behavior with
-D, to change the values of cache variables in our inner projects.
Fixed parallel build for projects using generators that have the
CMAKE_EXTRA_GENERATOR as well as CMAKE_GENERATOR. Thanks to Bill Hoffman
for helping me to track this one down, I missed parallel builds.
This commit introduces the ability to add custom targets
that correspond to individual ExternalProject steps.
The main motivation behind this new feature is to drive
sub-project based dashboard steps as external projects
with separate targets for update, configure, build and
test output. This makes it easy to construct a ctest -S
script to drive such a dashboard.
With no STEP_TARGETS defined, there are no additional
custom targets introduced, to minimize the chatter in
the target name space. Clients may define STEP_TARGETS
on a per-ExternalProject_Add basis, or at the directory
level by setting the EP_STEP_TARGETS directory property.
The intent was to warn about misspelled keyword arguments,
but the effect was to warn about valid argument values far
too often. Let's stop annoying and confusing people.
ExternalProject_Add calls that did not explicitly request a
CMAKE_GENERATOR would not get recursive parallel make behavior.
This commit fixes that issue.
Fix niggly in ExternalProject that would inadvertently
create a log file of output when a command was explicitly
set to the empty string.
Also, increase the default value for showing context
around build errors to 10 lines, up from 6. Also add
the ability to customize this setting so that users
can put it back to 6 or even lower if they want, or
show more if they want.
Avoid issues with two external projects trying to extract two
separate trees at the same time into the same location. Should
fix the sporadically failing ExternalProject test on the dashboards
for parallel builds.
Optionally hide the output of each external project build step by
wrapping its command in a launcher script. Make output available in log
files instead. This simplifies the output of external project builds.
See http://public.kitware.com/Bug/view.php?id=10346.
The proposed patch for the issue could not be applied as is
because the SOURCE_DIR always exists for an ExternalProject_Add
call by the time we get to the place to emit the potential error.
The fix is to emit the error only if the source dir is empty.
By which, I mean devoid of files and subdirectories. If
SOURCE_DIR is used by itself, without any DOWNLOAD_COMMAND
or repository info, then it implies that the SOURCE_DIR is ready
to build as-is without need for a download step. Clearly, if it
is empty, then it is not ready to build as is. So complain if
the SOURCE_DIR is empty.
Use it from ExternalProject and the ExternalProject test's
CMakeLists file rather than having duplicate find_program calls.
Add logic so that we do not try to use *.cmd variants of git
programs when using the MSYS Makefiles generator. Should fix
the last remaining dashboard issue with the new ExternalProject
git support additions.
Also, correct minor problem regarding placement of the local git
repo during test execution. On clean builds, it was being placed
incorrectly because of the ../.. relative reference. Use an absolute
path to place the local git repo in the proper directory, and only
use the relative reference when referring to it.
Add archives of these file types and add to the test
cases covered in the ExternalProject test.
Also add an "Example" directory in the Tests/ExternalProject
directory containing the canonical simplest example of
ExternalProject usage.
Improve FILE(DOWNLOAD ...):
- Add percent complete progress output to the FILE DOWNLOAD
command. This progress output is off by default to
preserve existing behavior. To turn it on, pass
SHOW_PROGRESS as an argument.
- Add EXPECTED_MD5 argument. Verify that the downloaded
file has the expected md5 sum after download is complete.
- Add documentation for SHOW_PROGRESS and EXPECTED_MD5.
When the destination file exists already and has the
expected md5 sum, then do not bother re-downloading
the file. ("Short circuit" return.)
Also, add a test that checks for the status output
indicating that the short circuit behavior is actually
occurring. Use a binary file for the test so that the
md5 sum is guaranteed to be the same on all platforms
regardless of "shifting text file line ending" issues.
Improve ExternalProject:
- Add argument URL_MD5.
- Add verify step that compares md5 sum of .tar.gz file
before extracting it.
- Add md5 check to download step, too, to prevent
unnecessary downloads.
- Emit a warning message when a file is not verified.
Indicate that the file may be corrupt or that no
checksum was specified.
Fixes issue http://public.kitware.com/Bug/view.php?id=10258
Also, fix complaint that DOWNLOAD_COMMAND cannot contain arguments
consisting entirely of upper case letters. It validly does when,
for example, you construct a custom cvs command line and the module
name is all upper case, like VTK.
(I accidentially removed ExternalProject.cmake from git by doing
mv ExternalProject.cmake ExternalProject.cmake.save
git checkout master
which I hoped would basically do a revert as it does with svn and cvs, but it
deleted the file from git)
Alex
This adds copyright/license notification blocks CMake's non-find
modules. Most of the modules had no notices at all. Some had notices
referring to the BSD license already. This commit normalizes existing
notices and adds missing notices.
This teaches the ExternalProject module to check the download URL file
name. If it is not a tarball (.tar, .tgz, .tar.gz) it is an error
because UntarFile does not yet understand other archive formats.
This creates new module ExternalProject.cmake to replace the prototype
AddExternalProject.cmake module. The interface is more refined, more
flexible, and better documented than the prototype.
This also converts the ExternalProject test to use the new module. The
old module will be removed (it was never in a CMake release) after
projects using it have been converted to the new module.
Matt McCormick
Brad King
Kitware Robot
Bill Hoffman
David Cole
Peter Collingbourne
Marcus D. Hanwell
Todd Gamblin
Alex Neundorf