Commit Graph

33 Commits

Author SHA1 Message Date
Brad King 0de47b05f7 ExternalData: Add option to disable use of symlinks
Add an ExternalData_NO_SYMLINKS to enable use of copies instead of
symlinks to populate the real data files behind a DATA{} reference.
This will be useful on UNIX-like systems when the underlying filesystem
does not actually support symbolic links.

Suggested-by: Matt McCormick <matt.mccormick@kitware.com>
2015-05-22 09:21:47 -04:00
Brad King 230f2d6e70 ExternalData: Add option to recursively match under directories
Extend the ``DATA{Dir/,...}`` syntax with a new ``RECURSE:`` option
to enable recursive matching of associated files.  This will allow
an entire directory tree of data to be referenced at once.
2015-03-27 14:44:54 -04:00
Brad King 564c07f7dd ExternalData: Parameterize internal file(GLOB) operation selection
Extend the _ExternalData_arg_find_files signature with an option to
specify the kind of file(GLOB) operation to be performed.  Set
CMP0009 to NEW so that GLOB_RECURSE does not follow symlinks.
2015-03-27 14:44:49 -04:00
Brad King f7f4ca55bd ExternalData: Add support for custom algorithm-to-URL mapping
Allow URL templates to contain a %(algo:<key>) placeholder that is
replaced by mapping the canonical hash algorithm name through a map
defined by the <key>.

Extend the Module.ExternalData test to cover the behavior.
Extend the RunCMake.ExternalData test to cover error cases.
2015-02-25 08:28:05 -05:00
Brad King 829fc5ad79 Help: Add notes for topic 'ExternalData-custom-download' 2015-01-15 11:03:33 -05:00
Brad King 0fe4d8bb3b ExternalData: Add support for custom download scripts
Add support for a special URL template to map the fetch operation
to a project-specified .cmake script insead of using file(DOWNLOAD).

Extend the Module.ExternalData test to cover the behavior.
Extend the RunCMake.ExternalData test to cover error cases.
2015-01-15 10:48:09 -05:00
Brad King 945571db74 ExternalData: Improve documentation organization
Move the basic DATA{} description to a section just before the
file series description.  Move all sections on referencing files
into subsections of a common "Referencing Files" section.

Subsume example usage into the introduction since it gives a
high-level starting point to understand the rest of the docs.
2015-01-15 10:48:09 -05:00
Brad King a32b2245ca ExternalData: Re-order documentation
Put example usage just after the intro.  Put hash algos last.
Put variables right after functions since they both cover the
module API.
2015-01-15 10:48:08 -05:00
Brad King 531e75e0b7 ExternalData: Document all variables defined by module
Extend the "Module Variables" section to list variables
previously mentioned only in text.
2015-01-15 10:48:08 -05:00
Brad King f3884b47ec ExternalData: Split documentation into sections
Also explicitly mark functions and variables.
2015-01-15 10:48:08 -05:00
Brad King 4ab5c652b6 ExternalData: Convert docs to a bracket comment
Use a bracket comment to hold the documentation instead of a block of
line comments.  This will make further updates easier.
2015-01-15 10:48:08 -05:00
Brad King f2ccc14b22 ExternalData: Drop 'not security software' disclaimer (#15060)
Let clients do their own research on the current strength of each
hash algorithm to choose what is best for their needs.

Suggested-by: Nico Schlömer <nico.schloemer@gmail.com>
2014-08-05 11:31:47 -04:00
Brad King ccd29b9af8 ExternalData: Warn on missing file instead of failing
When the primary source tree path named by a DATA{} reference does not
exist, produce an AUTHOR_WARNING instead of a FATAL_ERROR.  This is
useful when writing a new DATA{} reference to a test reference output
that has not been created yet.  This way the developer can run the test,
manually verify the output, and then copy it into place to provide the
reference and eliminate the warning.

If the named source tree path is expected to be a file but exists as a
directory, we still need to produce a FATAL_ERROR.
2014-04-17 09:45:09 -04:00
Rolf Eike Beer 2622bc3f65 Clean up usage of if(... MATCHES regex) followed string(REGEX REPLACE regex)
The matches have already been calculated and can simply be taken from
CMAKE_MATCH_n variables. This avoids multiple compilations of the same or very
similar regular expressions.
2014-04-14 18:16:58 +02:00
Brad King 8eb20eeabe ExternalData: Allow local stores without any URL templates
Allow ExternalData_URL_TEMPLATES to be empty if a value for
ExternalData_OBJECT_STORES is provided.  Assume in this use case that
the object stores will already contain all needed objects.  Extend the
Module.ExternalData test to cover this case (all objects in stores).
Extend the RunCMake.ExternalData test to cover the non-failure message
case when stores are provided without URL templates.
2013-11-13 10:03:36 -05:00
Brad King f3883f366d ExternalData: Show example network path URL template
In the ExternalData_URL_TEMPLATES example, add an entry showing how to
use the "file://" protocol with a network filesystem path.
2013-11-12 11:26:28 -05:00
Brad King ff6818bc0a ExternalData: Format module documentation
Manually revise the .rst format of the documentation.  Use inline
literal quotes appropriately in paragraph text.  Move the :: literal
block openers to the end of the preceding paragraphs.  Convert the
command signature documentation and examples to cmake code-block
directives.
2013-11-12 11:22:07 -05:00
Kitware Robot f051814ed0 Convert builtin help to reStructuredText source files
Run the convert-help.bash script to convert documentation:

 ./convert-help.bash "/path/to/CMake-build/bin"

Then remove it.
2013-10-15 14:12:03 -04:00
Rolf Eike Beer cd4451d1e7 replace string(... MATCHES "^const$) with string(... STREQUAL "const") 2013-06-02 22:18:19 +02:00
Brad King c35961b010 ExternalData: Do not re-stage staged object files
The ExternalData_LINK_CONTENT option tells ExternalData to convert real
data files it finds into content links and to "stage" the original
content in a ".ExternalData_<algo>_<hash>" file.  However, after a data
object has been staged it is possible that a user-provided pattern in
the "REGEX:" option will later match the staged object file.  We must
not process staged object files even when a user pattern matches them.

Fix the implementation to not match a staged object file as a normal
data file for conversion.  Extend the RunCMake.ExternalData test to
cover this case.
2013-05-24 15:45:38 -04:00
Brad King 1823ab4d76 ExternalData: Preserve escaped semicolons during argument expansion
The CMake language implicitly flattens lists so a ";" in a list element
must be escaped with a backslash.  List expansion removes backslashes
escaping semicolons to leave raw semicolons in the values.  Teach
ExternalData_Add_Test and ExternalData_Expand_Arguments to re-escape
semicolons found in list elements so the resulting argument lists work
as if constructed directly by the set() command.

For example:

  ExternalData_Add_Test(Data NAME test1 COMMAND ... "a\\;b")
  ExternalData_Expand_Arguments(Data args2 "c\\;d")
  add_test(NAME test2 COMMAND ... ${args2})

should be equivalent to

  set(args1 "a\\;b")
  add_test(NAME test1 COMMAND ... ${args1})
  set(args2 "c\\;d")
  add_test(NAME test2 COMMAND ... ${args2})

which is equivalent to

  add_test(NAME test1 COMMAND ... "a;b")
  add_test(NAME test2 COMMAND ... "c;d")

Note that it is not possible to make ExternalData_Add_Test act exactly
like add_test when quoted arguments contain semicolons because the CMake
language flattens lists when constructing function ARGN values.  This
re-escape approach at least allows test arguments to have semicolons.

While at it, teach ExternalData APIs to not transform "DATA{...;...}"
arguments because the contained semicolons are non-sensical.

Suggested-by: Jean-Christophe Fillion-Robin <jchris.fillionr@kitware.com>
2013-03-12 16:33:19 -04:00
Brad King 1fd8d017e6 ExternalData: Attach download rules to content links in IDEs
Each data file to be created in the build tree corresponds one-to-one with
a raw file or content link in the source tree.  Use the MAIN_DEPENDENCY of
add_custom_command to attach the build rule to the source tree file.  This
looks much nicer in the IDE project file browser and avoids ".rule" files.
2013-02-04 15:20:03 -05:00
Brad King e2e0d2e3c7 ExternalData: Collapse ../ components in DATA{} paths
Relative path components need to be normalized out even if they appear
in the middle of a caller-supplied string.
2013-01-30 14:55:12 -05:00
Brad King ee2abfdc89 ExternalData: Add support for SHA 1 and 2 hash algorithms
Update the Module.ExternalData and RunCMake.ExternalData tests to cover
some of them.
2013-01-30 10:05:07 -05:00
Brad King aa8b2288d3 ExternalData: Generalize hash algo/ext handling
Use private global variables _ExternalData_REGEX_(ALGO|EXT) to match the
possible hash algorithm names and extensions in regular expressions.
Use "file(<algo>)" instead of "cmake -E md5sum" to compute hashes
without a child process and to support more hash algorithms.
2013-01-30 10:05:07 -05:00
Brad King 9e518a8169 ExternalData: Allow DATA{} syntax to reference directories
Use a trailing slash to reference a directory.  Require that a list
of associated files be specified to select from within the directory.
One may simply use DATA{Dir/,REGEX:.*} to reference all files but
get a directory passed on the command line.
2013-01-30 10:05:07 -05:00
Brad King 175ed02207 ExternalData: Allow ()-groups in series match regex
Refactor use of the ExternalData_SERIES_MATCH value to avoid assuming
that it has no ()-groups that interfere with group indexing.

Extend the Module.ExternalData test to cover this case.
2013-01-30 10:05:07 -05:00
Brad King bcd25808ac ExternalData: Improve series matching using an explicit syntax
Automatic series recognition can generate false positives too easily
when the default series configuration is flexible enough to handle
common cases.  Avoid false positives by requiring an explicit syntax to
activate series recognition.  Choose the syntax DATA{<name>,:} to be
short, simple, and look like a vertical ellipsis.

This allows us to improve the default series match configuration.  Allow
series references to contain one of the numbered file names.  Allow '-'
as a separator in addition to '.' and '_'.  Document what the default
configuration matches.  Also provide more options to configure series
<name> parsing.
2013-01-29 15:23:00 -05:00
Brad King c0cebcb19a ExternalData: Remove unused private interface
The 'store' action was part of an older design and is no longer needed.
Remove it.
2013-01-28 15:31:31 -05:00
Brad King 52759930bd ExternalData: Cleanup stray TODO and typo in comments
Remove an outdated TODO line left from earlier designs.
Also fix a typo.
2013-01-28 15:31:31 -05:00
Brad King 7bb8344d50 ExternalData: Do not match directory names when resolving DATA{}
Our content link and object download infrastructure supports only
blobs (files), not trees (directories).  Do not allow references
to end in a slash.
2013-01-28 15:31:02 -05:00
Brad King 00d801fbfe ExternalData: Remove compatibility with CMake < 2.8.5
Now that the module comes in a sufficiently new CMake it does not need
to support older versions.
2013-01-24 13:34:12 -05:00
Brad King ab26249db1 Add ExternalData module
Take files we previously distributed in ITK 4.3.1:

  CMake/ExternalData.cmake
  CMake/ExternalData_config.cmake.in

and add them for distribution in upstream CMake.  Update the copyright
notice block format to follow CMake conventions.
2013-01-24 13:32:17 -05:00