If multiple ExternalData_Target_Add calls generate the same output file
then we need to avoid calling add_custom_command multiple times with
that output. This was already done within a single target by setting a
variable in the local function scope. This will not be visible in other
calls though so we need to use a directory property instead to prevent
adding a custom command multiple times for one output in a directory.
Normally it is not safe to have multiple custom commands that produce
the same output file across multiple independent targets, but since we
use atomic replacement of outputs the resulting races should not be a
problem. For the convenience of projects, tolerate this instead of
diagnosing it. In particular, we previously allowed up to two copies
of the custom command in one directory because CMake has a fallback
from MAIN_DEPENDENCY to an `<output>.rule` file.
While at it, add a note to the documentation that typically only one
external data target should be needed for a project.
Reported-by: David Manthey <david.manthey@kitware.com>
Add an ExternalData_NO_SYMLINKS to enable use of copies instead of
symlinks to populate the real data files behind a DATA{} reference.
This will be useful on UNIX-like systems when the underlying filesystem
does not actually support symbolic links.
Suggested-by: Matt McCormick <matt.mccormick@kitware.com>
Extend the ``DATA{Dir/,...}`` syntax with a new ``RECURSE:`` option
to enable recursive matching of associated files. This will allow
an entire directory tree of data to be referenced at once.
Allow URL templates to contain a %(algo:<key>) placeholder that is
replaced by mapping the canonical hash algorithm name through a map
defined by the <key>.
Extend the Module.ExternalData test to cover the behavior.
Extend the RunCMake.ExternalData test to cover error cases.
Add support for a special URL template to map the fetch operation
to a project-specified .cmake script insead of using file(DOWNLOAD).
Extend the Module.ExternalData test to cover the behavior.
Extend the RunCMake.ExternalData test to cover error cases.
When the primary source tree path named by a DATA{} reference does not
exist, produce an AUTHOR_WARNING instead of a FATAL_ERROR. This is
useful when writing a new DATA{} reference to a test reference output
that has not been created yet. This way the developer can run the test,
manually verify the output, and then copy it into place to provide the
reference and eliminate the warning.
If the named source tree path is expected to be a file but exists as a
directory, we still need to produce a FATAL_ERROR.
Allow ExternalData_URL_TEMPLATES to be empty if a value for
ExternalData_OBJECT_STORES is provided. Assume in this use case that
the object stores will already contain all needed objects. Extend the
Module.ExternalData test to cover this case (all objects in stores).
Extend the RunCMake.ExternalData test to cover the non-failure message
case when stores are provided without URL templates.
The CMake language implicitly flattens lists so a ";" in a list element
must be escaped with a backslash. List expansion removes backslashes
escaping semicolons to leave raw semicolons in the values. Teach
ExternalData_Add_Test and ExternalData_Expand_Arguments to re-escape
semicolons found in list elements so the resulting argument lists work
as if constructed directly by the set() command.
For example:
ExternalData_Add_Test(Data NAME test1 COMMAND ... "a\\;b")
ExternalData_Expand_Arguments(Data args2 "c\\;d")
add_test(NAME test2 COMMAND ... ${args2})
should be equivalent to
set(args1 "a\\;b")
add_test(NAME test1 COMMAND ... ${args1})
set(args2 "c\\;d")
add_test(NAME test2 COMMAND ... ${args2})
which is equivalent to
add_test(NAME test1 COMMAND ... "a;b")
add_test(NAME test2 COMMAND ... "c;d")
Note that it is not possible to make ExternalData_Add_Test act exactly
like add_test when quoted arguments contain semicolons because the CMake
language flattens lists when constructing function ARGN values. This
re-escape approach at least allows test arguments to have semicolons.
While at it, teach ExternalData APIs to not transform "DATA{...;...}"
arguments because the contained semicolons are non-sensical.
Suggested-by: Jean-Christophe Fillion-Robin <jchris.fillionr@kitware.com>
Extend the Module.ExternalData test to cover a DATA{} reference whose
name contains a space. Skip the case when the native build tool does
not support spaces.
Use a trailing slash to reference a directory. Require that a list
of associated files be specified to select from within the directory.
One may simply use DATA{Dir/,REGEX:.*} to reference all files but
get a directory passed on the command line.
Refactor use of the ExternalData_SERIES_MATCH value to avoid assuming
that it has no ()-groups that interfere with group indexing.
Extend the Module.ExternalData test to cover this case.
Add a Module.ExternalData test to verify data retrieval and test
argument DATA{} references.
Add a RunCMake.ExternalData test to verify error handling and automatic
transformation of a raw data to a content link and staged object.
Brad King