Add an ExternalData_NO_SYMLINKS to enable use of copies instead of
symlinks to populate the real data files behind a DATA{} reference.
This will be useful on UNIX-like systems when the underlying filesystem
does not actually support symbolic links.
Suggested-by: Matt McCormick <matt.mccormick@kitware.com>
Extend the ``DATA{Dir/,...}`` syntax with a new ``RECURSE:`` option
to enable recursive matching of associated files. This will allow
an entire directory tree of data to be referenced at once.
Extend the _ExternalData_arg_find_files signature with an option to
specify the kind of file(GLOB) operation to be performed. Set
CMP0009 to NEW so that GLOB_RECURSE does not follow symlinks.
Allow URL templates to contain a %(algo:<key>) placeholder that is
replaced by mapping the canonical hash algorithm name through a map
defined by the <key>.
Extend the Module.ExternalData test to cover the behavior.
Extend the RunCMake.ExternalData test to cover error cases.
Add support for a special URL template to map the fetch operation
to a project-specified .cmake script insead of using file(DOWNLOAD).
Extend the Module.ExternalData test to cover the behavior.
Extend the RunCMake.ExternalData test to cover error cases.
Move the basic DATA{} description to a section just before the
file series description. Move all sections on referencing files
into subsections of a common "Referencing Files" section.
Subsume example usage into the introduction since it gives a
high-level starting point to understand the rest of the docs.
Let clients do their own research on the current strength of each
hash algorithm to choose what is best for their needs.
Suggested-by: Nico Schlömer <nico.schloemer@gmail.com>
When the primary source tree path named by a DATA{} reference does not
exist, produce an AUTHOR_WARNING instead of a FATAL_ERROR. This is
useful when writing a new DATA{} reference to a test reference output
that has not been created yet. This way the developer can run the test,
manually verify the output, and then copy it into place to provide the
reference and eliminate the warning.
If the named source tree path is expected to be a file but exists as a
directory, we still need to produce a FATAL_ERROR.
The matches have already been calculated and can simply be taken from
CMAKE_MATCH_n variables. This avoids multiple compilations of the same or very
similar regular expressions.
Allow ExternalData_URL_TEMPLATES to be empty if a value for
ExternalData_OBJECT_STORES is provided. Assume in this use case that
the object stores will already contain all needed objects. Extend the
Module.ExternalData test to cover this case (all objects in stores).
Extend the RunCMake.ExternalData test to cover the non-failure message
case when stores are provided without URL templates.
Manually revise the .rst format of the documentation. Use inline
literal quotes appropriately in paragraph text. Move the :: literal
block openers to the end of the preceding paragraphs. Convert the
command signature documentation and examples to cmake code-block
directives.
The ExternalData_LINK_CONTENT option tells ExternalData to convert real
data files it finds into content links and to "stage" the original
content in a ".ExternalData_<algo>_<hash>" file. However, after a data
object has been staged it is possible that a user-provided pattern in
the "REGEX:" option will later match the staged object file. We must
not process staged object files even when a user pattern matches them.
Fix the implementation to not match a staged object file as a normal
data file for conversion. Extend the RunCMake.ExternalData test to
cover this case.
The CMake language implicitly flattens lists so a ";" in a list element
must be escaped with a backslash. List expansion removes backslashes
escaping semicolons to leave raw semicolons in the values. Teach
ExternalData_Add_Test and ExternalData_Expand_Arguments to re-escape
semicolons found in list elements so the resulting argument lists work
as if constructed directly by the set() command.
For example:
ExternalData_Add_Test(Data NAME test1 COMMAND ... "a\\;b")
ExternalData_Expand_Arguments(Data args2 "c\\;d")
add_test(NAME test2 COMMAND ... ${args2})
should be equivalent to
set(args1 "a\\;b")
add_test(NAME test1 COMMAND ... ${args1})
set(args2 "c\\;d")
add_test(NAME test2 COMMAND ... ${args2})
which is equivalent to
add_test(NAME test1 COMMAND ... "a;b")
add_test(NAME test2 COMMAND ... "c;d")
Note that it is not possible to make ExternalData_Add_Test act exactly
like add_test when quoted arguments contain semicolons because the CMake
language flattens lists when constructing function ARGN values. This
re-escape approach at least allows test arguments to have semicolons.
While at it, teach ExternalData APIs to not transform "DATA{...;...}"
arguments because the contained semicolons are non-sensical.
Suggested-by: Jean-Christophe Fillion-Robin <jchris.fillionr@kitware.com>
Each data file to be created in the build tree corresponds one-to-one with
a raw file or content link in the source tree. Use the MAIN_DEPENDENCY of
add_custom_command to attach the build rule to the source tree file. This
looks much nicer in the IDE project file browser and avoids ".rule" files.
Use private global variables _ExternalData_REGEX_(ALGO|EXT) to match the
possible hash algorithm names and extensions in regular expressions.
Use "file(<algo>)" instead of "cmake -E md5sum" to compute hashes
without a child process and to support more hash algorithms.
Use a trailing slash to reference a directory. Require that a list
of associated files be specified to select from within the directory.
One may simply use DATA{Dir/,REGEX:.*} to reference all files but
get a directory passed on the command line.
Refactor use of the ExternalData_SERIES_MATCH value to avoid assuming
that it has no ()-groups that interfere with group indexing.
Extend the Module.ExternalData test to cover this case.
Automatic series recognition can generate false positives too easily
when the default series configuration is flexible enough to handle
common cases. Avoid false positives by requiring an explicit syntax to
activate series recognition. Choose the syntax DATA{<name>,:} to be
short, simple, and look like a vertical ellipsis.
This allows us to improve the default series match configuration. Allow
series references to contain one of the numbered file names. Allow '-'
as a separator in addition to '.' and '_'. Document what the default
configuration matches. Also provide more options to configure series
<name> parsing.
Take files we previously distributed in ITK 4.3.1:
CMake/ExternalData.cmake
CMake/ExternalData_config.cmake.in
and add them for distribution in upstream CMake. Update the copyright
notice block format to follow CMake conventions.
Daniel Pfeifer
Alex Turbov
Brad King
Rolf Eike Beer
Kitware Robot