kolan/CMake
1
0
Fork 0
Code Issues Pull Requests Releases Wiki Activity

Merge topic 'ExternalData-custom-download'

Browse Source 
829fc5ad Help: Add notes for topic 'ExternalData-custom-download'
0fe4d8bb ExternalData: Add support for custom download scripts
945571db ExternalData: Improve documentation organization
a32b2245 ExternalData: Re-order documentation
531e75e0 ExternalData: Document all variables defined by module
f3884b47 ExternalData: Split documentation into sections
4ab5c652 ExternalData: Convert docs to a bracket comment
This commit is contained in:
Brad King 2015-01-16 09:43:49 -05:00 committed by CMake Topic Stage
commit cc475e8129
20 changed files with 438 additions and 185 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

7
Help/release/dev/ExternalData-custom-download.rst Normal file
View File

@ -0,0 +1,7 @@
ExternalData-custom-download
----------------------------
* The :module:`ExternalData` module learned to support
:ref:`Custom Fetch Scripts <ExternalData Custom Fetch Scripts>`.
This allows projects to specify custom ``.cmake`` scripts for
fetching data objects during the build.

541
Modules/ExternalData.cmake
View File

@ -1,189 +1,301 @@
#.rst: #[=======================================================================[.rst:
# ExternalData ExternalData
# ------------ ------------
#
# Manage data files stored outside source tree .. only:: html
#
# Use this module to unambiguously reference data files stored outside .. contents::
# the source tree and fetch them at build time from arbitrary local and
# remote content-addressed locations. Functions provided by this module Manage data files stored outside source tree
# recognize arguments with the syntax ``DATA{<name>}`` as references to
# external data, replace them with full paths to local copies of those Introduction
# data, and create build rules to fetch and update the local copies. ^^^^^^^^^^^^
#
# The ``DATA{}`` syntax is literal and the ``<name>`` is a full or relative path Use this module to unambiguously reference data files stored outside
# within the source tree. The source tree must contain either a real the source tree and fetch them at build time from arbitrary local and
# data file at ``<name>`` or a "content link" at ``<name><ext>`` containing a remote content-addressed locations. Functions provided by this module
# hash of the real file using a hash algorithm corresponding to ``<ext>``. recognize arguments with the syntax ``DATA{<name>}`` as references to
# For example, the argument ``DATA{img.png}`` may be satisfied by either a external data, replace them with full paths to local copies of those
# real ``img.png`` file in the current source directory or a ``img.png.md5`` data, and create build rules to fetch and update the local copies.
# file containing its MD5 sum.
# For example:
# The ``ExternalData_Expand_Arguments`` function evaluates ``DATA{}``
# references in its arguments and constructs a new list of arguments: .. code-block:: cmake
#
# .. code-block:: cmake include(ExternalData)
# set(ExternalData_URL_TEMPLATES "file:///local/%(algo)/%(hash)"
# ExternalData_Expand_Arguments( "file:////host/share/%(algo)/%(hash)"
# <target> # Name of data management target "http://data.org/%(algo)/%(hash)")
# <outVar> # Output variable ExternalData_Add_Test(MyData
# [args...] # Input arguments, DATA{} allowed NAME MyTest
# ) COMMAND MyExe DATA{MyInput.png}
# )
# It replaces each ``DATA{}`` reference in an argument with the full path of ExternalData_Add_Target(MyData)
# a real data file on disk that will exist after the ``<target>`` builds.
# When test ``MyTest`` runs the ``DATA{MyInput.png}`` argument will be
# The ``ExternalData_Add_Test`` function wraps around the CMake replaced by the full path to a real instance of the data file
# :command:`add_test` command but supports ``DATA{}`` references in ``MyInput.png`` on disk. If the source tree contains a content link
# its arguments: such as ``MyInput.png.md5`` then the ``MyData`` target creates a real
# ``MyInput.png`` in the build tree.
# .. code-block:: cmake
# Module Functions
# ExternalData_Add_Test( ^^^^^^^^^^^^^^^^
# <target> # Name of data management target
# ... # Arguments of add_test(), DATA{} allowed .. command:: ExternalData_Expand_Arguments
# )
# The ``ExternalData_Expand_Arguments`` function evaluates ``DATA{}``
# It passes its arguments through ``ExternalData_Expand_Arguments`` and then references in its arguments and constructs a new list of arguments::
# invokes the :command:`add_test` command using the results.
# ExternalData_Expand_Arguments(
# The ``ExternalData_Add_Target`` function creates a custom target to <target> # Name of data management target
# manage local instances of data files stored externally: <outVar> # Output variable
# [args...] # Input arguments, DATA{} allowed
# .. code-block:: cmake )
#
# ExternalData_Add_Target( It replaces each ``DATA{}`` reference in an argument with the full path of
# <target> # Name of data management target a real data file on disk that will exist after the ``<target>`` builds.
# )
# .. command:: ExternalData_Add_Test
# It creates custom commands in the target as necessary to make data
# files available for each ``DATA{}`` reference previously evaluated by The ``ExternalData_Add_Test`` function wraps around the CMake
# other functions provided by this module. A list of URL templates may :command:`add_test` command but supports ``DATA{}`` references in
# be provided in the variable ``ExternalData_URL_TEMPLATES`` using the its arguments::
# placeholders ``%(algo)`` and ``%(hash)`` in each template. Data fetch
# rules try each URL template in order by substituting the hash ExternalData_Add_Test(
# algorithm name for ``%(algo)`` and the hash value for ``%(hash)``. <target> # Name of data management target
# ... # Arguments of add_test(), DATA{} allowed
# The following hash algorithms are supported:: )
#
# %(algo) <ext> Description It passes its arguments through ``ExternalData_Expand_Arguments`` and then
# ------- ----- ----------- invokes the :command:`add_test` command using the results.
# MD5 .md5 Message-Digest Algorithm 5, RFC 1321
# SHA1 .sha1 US Secure Hash Algorithm 1, RFC 3174 .. command:: ExternalData_Add_Target
# SHA224 .sha224 US Secure Hash Algorithms, RFC 4634
# SHA256 .sha256 US Secure Hash Algorithms, RFC 4634 The ``ExternalData_Add_Target`` function creates a custom target to
# SHA384 .sha384 US Secure Hash Algorithms, RFC 4634 manage local instances of data files stored externally::
# SHA512 .sha512 US Secure Hash Algorithms, RFC 4634
# ExternalData_Add_Target(
# Note that the hashes are used only for unique data identification and <target> # Name of data management target
# download verification. )
#
# Example usage: It creates custom commands in the target as necessary to make data
# files available for each ``DATA{}`` reference previously evaluated by
# .. code-block:: cmake other functions provided by this module.
# Data files may be fetched from one of the URL templates specified in
# include(ExternalData) the ``ExternalData_URL_TEMPLATES`` variable, or may be found locally
# set(ExternalData_URL_TEMPLATES "file:///local/%(algo)/%(hash)" in one of the paths specified in the ``ExternalData_OBJECT_STORES``
# "file:////host/share/%(algo)/%(hash)" variable.
# "http://data.org/%(algo)/%(hash)")
# ExternalData_Add_Test(MyData Module Variables
# NAME MyTest ^^^^^^^^^^^^^^^^
# COMMAND MyExe DATA{MyInput.png}
# ) The following variables configure behavior. They should be set before
# ExternalData_Add_Target(MyData) calling any of the functions provided by this module.
#
# When test ``MyTest`` runs the ``DATA{MyInput.png}`` argument will be .. variable:: ExternalData_BINARY_ROOT
# replaced by the full path to a real instance of the data file
# ``MyInput.png`` on disk. If the source tree contains a content link The ``ExternalData_BINARY_ROOT`` variable may be set to the directory to
# such as ``MyInput.png.md5`` then the ``MyData`` target creates a real hold the real data files named by expanded ``DATA{}`` references. The
# ``MyInput.png`` in the build tree. default is ``CMAKE_BINARY_DIR``. The directory layout will mirror that of
# content links under ``ExternalData_SOURCE_ROOT``.
# The ``DATA{}`` syntax can be told to fetch a file series using the form
# ``DATA{<name>,:}``, where the ``:`` is literal. If the source tree .. variable:: ExternalData_CUSTOM_SCRIPT_<key>
# contains a group of files or content links named like a series then a
# reference to one member adds rules to fetch all of them. Although all Specify a full path to a ``.cmake`` custom fetch script identified by
# members of a series are fetched, only the file originally named by the ``<key>`` in entries of the ``ExternalData_URL_TEMPLATES`` list.
# ``DATA{}`` argument is substituted for it. The default configuration See `Custom Fetch Scripts`_.
# recognizes file series names ending with ``#.ext``, ``_#.ext``, ``.#.ext``,
# or ``-#.ext`` where ``#`` is a sequence of decimal digits and ``.ext`` is .. variable:: ExternalData_LINK_CONTENT
# any single extension. Configure it with a regex that parses ``<number>``
# and ``<suffix>`` parts from the end of ``<name>``:: The ``ExternalData_LINK_CONTENT`` variable may be set to the name of a
# supported hash algorithm to enable automatic conversion of real data
# ExternalData_SERIES_PARSE = regex of the form (<number>)(<suffix>)$ files referenced by the ``DATA{}`` syntax into content links. For each
# such ``<file>`` a content link named ``<file><ext>`` is created. The
# For more complicated cases set:: original file is renamed to the form ``.ExternalData_<algo>_<hash>`` to
# stage it for future transmission to one of the locations in the list
# ExternalData_SERIES_PARSE = regex with at least two () groups of URL templates (by means outside the scope of this module). The
# ExternalData_SERIES_PARSE_PREFIX = <prefix> regex group number, if any data fetch rule created for the content link will use the staged
# ExternalData_SERIES_PARSE_NUMBER = <number> regex group number object if it cannot be found using any URL template.
# ExternalData_SERIES_PARSE_SUFFIX = <suffix> regex group number
# .. variable:: ExternalData_OBJECT_STORES
# Configure series number matching with a regex that matches the
# ``<number>`` part of series members named ``<prefix><number><suffix>``:: The ``ExternalData_OBJECT_STORES`` variable may be set to a list of local
# directories that store objects using the layout ``<dir>/%(algo)/%(hash)``.
# ExternalData_SERIES_MATCH = regex matching <number> in all series members These directories will be searched first for a needed object. If the
# object is not available in any store then it will be fetched remotely
# Note that the ``<suffix>`` of a series does not include a hash-algorithm using the URL templates and added to the first local store listed. If
# extension. no stores are specified the default is a location inside the build
# tree.
# The ``DATA{}`` syntax can alternatively match files associated with the
# named file and contained in the same directory. Associated files may .. variable:: ExternalData_SERIES_PARSE
# be specified by options using the syntax ExternalData_SERIES_PARSE_PREFIX
# ``DATA{<name>,<opt1>,<opt2>,...}``. Each option may specify one file by ExternalData_SERIES_PARSE_NUMBER
# name or specify a regular expression to match file names using the ExternalData_SERIES_PARSE_SUFFIX
# syntax ``REGEX:<regex>``. For example, the arguments:: ExternalData_SERIES_MATCH
#
# DATA{MyData/MyInput.mhd,MyInput.img} # File pair See `Referencing File Series`_.
# DATA{MyData/MyFrames00.png,REGEX:MyFrames[0-9]+\\.png} # Series
# .. variable:: ExternalData_SOURCE_ROOT
# will pass ``MyInput.mha`` and ``MyFrames00.png`` on the command line but
# ensure that the associated files are present next to them. The ``ExternalData_SOURCE_ROOT`` variable may be set to the highest source
# directory containing any path named by a ``DATA{}`` reference. The
# The ``DATA{}`` syntax may reference a directory using a trailing slash and default is ``CMAKE_SOURCE_DIR``. ``ExternalData_SOURCE_ROOT`` and
# a list of associated files. The form ``DATA{<name>/,<opt1>,<opt2>,...}`` ``CMAKE_SOURCE_DIR`` must refer to directories within a single source
# adds rules to fetch any files in the directory that match one of the distribution (e.g. they come together in one tarball).
# associated file options. For example, the argument
# ``DATA{MyDataDir/,REGEX:.*}`` will pass the full path to a ``MyDataDir`` .. variable:: ExternalData_TIMEOUT_ABSOLUTE
# directory on the command line and ensure that the directory contains
# files corresponding to every file or content link in the ``MyDataDir`` The ``ExternalData_TIMEOUT_ABSOLUTE`` variable sets the download
# source directory. absolute timeout, in seconds, with a default of ``300`` seconds.
# Set to ``0`` to disable enforcement.
# The variable ``ExternalData_LINK_CONTENT`` may be set to the name of a
# supported hash algorithm to enable automatic conversion of real data .. variable:: ExternalData_TIMEOUT_INACTIVITY
# files referenced by the ``DATA{}`` syntax into content links. For each
# such ``<file>`` a content link named ``<file><ext>`` is created. The The ``ExternalData_TIMEOUT_INACTIVITY`` variable sets the download
# original file is renamed to the form ``.ExternalData_<algo>_<hash>`` to inactivity timeout, in seconds, with a default of ``60`` seconds.
# stage it for future transmission to one of the locations in the list Set to ``0`` to disable enforcement.
# of URL templates (by means outside the scope of this module). The
# data fetch rule created for the content link will use the staged .. variable:: ExternalData_URL_TEMPLATES
# object if it cannot be found using any URL template.
# The ``ExternalData_URL_TEMPLATES`` may be set to provide a list of
# The variable ``ExternalData_OBJECT_STORES`` may be set to a list of local of URL templates using the placeholders ``%(algo)`` and ``%(hash)``
# directories that store objects using the layout ``<dir>/%(algo)/%(hash)``. in each template. Data fetch rules try each URL template in order
# These directories will be searched first for a needed object. If the by substituting the hash algorithm name for ``%(algo)`` and the hash
# object is not available in any store then it will be fetched remotely value for ``%(hash)``.
# using the URL templates and added to the first local store listed. If
# no stores are specified the default is a location inside the build Referencing Files
# tree. ^^^^^^^^^^^^^^^^^
#
# The variable ``ExternalData_SOURCE_ROOT`` may be set to the highest source Referencing Single Files
# directory containing any path named by a ``DATA{}`` reference. The """"""""""""""""""""""""
# default is ``CMAKE_SOURCE_DIR``. ``ExternalData_SOURCE_ROOT`` and
# ``CMAKE_SOURCE_DIR`` must refer to directories within a single source The ``DATA{}`` syntax is literal and the ``<name>`` is a full or relative path
# distribution (e.g. they come together in one tarball). within the source tree. The source tree must contain either a real
# data file at ``<name>`` or a "content link" at ``<name><ext>`` containing a
# The variable ``ExternalData_BINARY_ROOT`` may be set to the directory to hash of the real file using a hash algorithm corresponding to ``<ext>``.
# hold the real data files named by expanded ``DATA{}`` references. The For example, the argument ``DATA{img.png}`` may be satisfied by either a
# default is ``CMAKE_BINARY_DIR``. The directory layout will mirror that of real ``img.png`` file in the current source directory or a ``img.png.md5``
# content links under ``ExternalData_SOURCE_ROOT``. file containing its MD5 sum.
#
# Variables ``ExternalData_TIMEOUT_INACTIVITY`` and Referencing File Series
# ``ExternalData_TIMEOUT_ABSOLUTE`` set the download inactivity and absolute """""""""""""""""""""""
# timeouts, in seconds. The defaults are 60 seconds and 300 seconds,
# respectively. Set either timeout to 0 seconds to disable enforcement. The ``DATA{}`` syntax can be told to fetch a file series using the form
``DATA{<name>,:}``, where the ``:`` is literal. If the source tree
contains a group of files or content links named like a series then a
reference to one member adds rules to fetch all of them. Although all
members of a series are fetched, only the file originally named by the
``DATA{}`` argument is substituted for it. The default configuration
recognizes file series names ending with ``#.ext``, ``_#.ext``, ``.#.ext``,
or ``-#.ext`` where ``#`` is a sequence of decimal digits and ``.ext`` is
any single extension. Configure it with a regex that parses ``<number>``
and ``<suffix>`` parts from the end of ``<name>``::
ExternalData_SERIES_PARSE = regex of the form (<number>)(<suffix>)$
For more complicated cases set::
ExternalData_SERIES_PARSE = regex with at least two () groups
ExternalData_SERIES_PARSE_PREFIX = <prefix> regex group number, if any
ExternalData_SERIES_PARSE_NUMBER = <number> regex group number
ExternalData_SERIES_PARSE_SUFFIX = <suffix> regex group number
Configure series number matching with a regex that matches the
``<number>`` part of series members named ``<prefix><number><suffix>``::
ExternalData_SERIES_MATCH = regex matching <number> in all series members
Note that the ``<suffix>`` of a series does not include a hash-algorithm
extension.
Referencing Associated Files
""""""""""""""""""""""""""""
The ``DATA{}`` syntax can alternatively match files associated with the
named file and contained in the same directory. Associated files may
be specified by options using the syntax
``DATA{<name>,<opt1>,<opt2>,...}``. Each option may specify one file by
name or specify a regular expression to match file names using the
syntax ``REGEX:<regex>``. For example, the arguments::
DATA{MyData/MyInput.mhd,MyInput.img} # File pair
DATA{MyData/MyFrames00.png,REGEX:MyFrames[0-9]+\\.png} # Series
will pass ``MyInput.mha`` and ``MyFrames00.png`` on the command line but
ensure that the associated files are present next to them.
Referencing Directories
"""""""""""""""""""""""
The ``DATA{}`` syntax may reference a directory using a trailing slash and
a list of associated files. The form ``DATA{<name>/,<opt1>,<opt2>,...}``
adds rules to fetch any files in the directory that match one of the
associated file options. For example, the argument
``DATA{MyDataDir/,REGEX:.*}`` will pass the full path to a ``MyDataDir``
directory on the command line and ensure that the directory contains
files corresponding to every file or content link in the ``MyDataDir``
source directory.
Hash Algorithms
^^^^^^^^^^^^^^^
The following hash algorithms are supported::
%(algo) <ext> Description
------- ----- -----------
MD5 .md5 Message-Digest Algorithm 5, RFC 1321
SHA1 .sha1 US Secure Hash Algorithm 1, RFC 3174
SHA224 .sha224 US Secure Hash Algorithms, RFC 4634
SHA256 .sha256 US Secure Hash Algorithms, RFC 4634
SHA384 .sha384 US Secure Hash Algorithms, RFC 4634
SHA512 .sha512 US Secure Hash Algorithms, RFC 4634
Note that the hashes are used only for unique data identification and
download verification.
.. _`ExternalData Custom Fetch Scripts`:
Custom Fetch Scripts
^^^^^^^^^^^^^^^^^^^^
When a data file must be fetched from one of the URL templates
specified in the ``ExternalData_URL_TEMPLATES`` variable, it is
normally downloaded using the :command:`file(DOWNLOAD)` command.
One may specify usage of a custom fetch script by using a URL
template of the form ``ExternalDataCustomScript://<key>/<loc>``.
The ``<key>`` must be a C identifier, and the ``<loc>`` must
contain the ``%(algo)`` and ``%(hash)`` placeholders.
A variable corresponding to the key, ``ExternalData_CUSTOM_SCRIPT_<key>``,
must be set to the full path to a ``.cmake`` script file. The script
will be included to perform the actual fetch, and provided with
the following variables:
.. variable:: ExternalData_CUSTOM_LOCATION
When a custom fetch script is loaded, this variable is set to the
location part of the URL, which will contain the substituted hash
algorithm name and content hash value.
.. variable:: ExternalData_CUSTOM_FILE
When a custom fetch script is loaded, this variable is set to the
full path to a file in which the script must store the fetched
content. The name of the file is unspecified and should not be
interpreted in any way.
The custom fetch script is expected to store fetched content in the
file or set a variable:
.. variable:: ExternalData_CUSTOM_ERROR
When a custom fetch script fails to fetch the requested content,
it must set this variable to a short one-line message describing
the reason for failure.
#]=======================================================================]
#============================================================================= #=============================================================================
# Copyright 2010-2013 Kitware, Inc. # Copyright 2010-2015 Kitware, Inc.
# #
# Distributed under the OSI-approved BSD License (the "License"); # Distributed under the OSI-approved BSD License (the "License");
# see accompanying file Copyright.txt for details. # see accompanying file Copyright.txt for details.
@ -209,6 +321,37 @@ function(ExternalData_add_target target)
if(NOT ExternalData_OBJECT_STORES) if(NOT ExternalData_OBJECT_STORES)
set(ExternalData_OBJECT_STORES ${CMAKE_BINARY_DIR}/ExternalData/Objects) set(ExternalData_OBJECT_STORES ${CMAKE_BINARY_DIR}/ExternalData/Objects)
endif() endif()
set(_ExternalData_CONFIG_CODE "")
# Store custom script configuration.
foreach(url_template IN LISTS ExternalData_URL_TEMPLATES)
if("${url_template}" MATCHES "^ExternalDataCustomScript://([^/]*)/(.*)$")
set(key "${CMAKE_MATCH_1}")
if(key MATCHES "^[A-Za-z_][A-Za-z0-9_]*$")
if(ExternalData_CUSTOM_SCRIPT_${key})
if(IS_ABSOLUTE "${ExternalData_CUSTOM_SCRIPT_${key}}")
string(CONCAT _ExternalData_CONFIG_CODE "${_ExternalData_CONFIG_CODE}\n"
"set(ExternalData_CUSTOM_SCRIPT_${key} \"${ExternalData_CUSTOM_SCRIPT_${key}}\")")
else()
message(FATAL_ERROR
"No ExternalData_CUSTOM_SCRIPT_${key} is not set to a full path:\n"
" ${ExternalData_CUSTOM_SCRIPT_${key}}")
endif()
else()
message(FATAL_ERROR
"No ExternalData_CUSTOM_SCRIPT_${key} is set for URL template:\n"
" ${url_template}")
endif()
else()
message(FATAL_ERROR
"Bad ExternalDataCustomScript key '${key}' in URL template:\n"
" ${url_template}\n"
"The key must be a valid C identifier.")
endif()
endif()
endforeach()
# Store configuration for use by build-time script.
set(config ${CMAKE_CURRENT_BINARY_DIR}/${target}_config.cmake) set(config ${CMAKE_CURRENT_BINARY_DIR}/${target}_config.cmake)
configure_file(${_ExternalData_SELF_DIR}/ExternalData_config.cmake.in ${config} @ONLY) configure_file(${_ExternalData_SELF_DIR}/ExternalData_config.cmake.in ${config} @ONLY)
@ -715,6 +858,30 @@ function(_ExternalData_download_file url file err_var msg_var)
set("${msg_var}" "${msg}" PARENT_SCOPE) set("${msg_var}" "${msg}" PARENT_SCOPE)
endfunction() endfunction()
function(_ExternalData_custom_fetch key loc file err_var msg_var)
if(NOT ExternalData_CUSTOM_SCRIPT_${key})
set(err 1)
set(msg "No ExternalData_CUSTOM_SCRIPT_${key} set!")
elseif(NOT EXISTS "${ExternalData_CUSTOM_SCRIPT_${key}}")
set(err 1)
set(msg "No '${ExternalData_CUSTOM_SCRIPT_${key}}' exists!")
else()
set(ExternalData_CUSTOM_LOCATION "${loc}")
set(ExternalData_CUSTOM_FILE "${file}")
unset(ExternalData_CUSTOM_ERROR)
include("${ExternalData_CUSTOM_SCRIPT_${key}}")
if(DEFINED ExternalData_CUSTOM_ERROR)
set(err 1)
set(msg "${ExternalData_CUSTOM_ERROR}")
else()
set(err 0)
set(msg "no error")
endif()
endif()
set("${err_var}" "${err}" PARENT_SCOPE)
set("${msg_var}" "${msg}" PARENT_SCOPE)
endfunction()
function(_ExternalData_download_object name hash algo var_obj) function(_ExternalData_download_object name hash algo var_obj)
# Search all object stores for an existing object. # Search all object stores for an existing object.
foreach(dir ${ExternalData_OBJECT_STORES}) foreach(dir ${ExternalData_OBJECT_STORES})
@ -738,7 +905,11 @@ function(_ExternalData_download_object name hash algo var_obj)
string(REPLACE "%(hash)" "${hash}" url_tmp "${url_template}") string(REPLACE "%(hash)" "${hash}" url_tmp "${url_template}")
string(REPLACE "%(algo)" "${algo}" url "${url_tmp}") string(REPLACE "%(algo)" "${algo}" url "${url_tmp}")
message(STATUS "Fetching \"${url}\"") message(STATUS "Fetching \"${url}\"")
_ExternalData_download_file("${url}" "${tmp}" err errMsg) if(url MATCHES "^ExternalDataCustomScript://([A-Za-z_][A-Za-z0-9_]*)/(.*)$")
_ExternalData_custom_fetch("${CMAKE_MATCH_1}" "${CMAKE_MATCH_2}" "${tmp}" err errMsg)
else()
_ExternalData_download_file("${url}" "${tmp}" err errMsg)
endif()
set(tried "${tried}\n ${url}") set(tried "${tried}\n ${url}")
if(err) if(err)
set(tried "${tried} (${errMsg})") set(tried "${tried} (${errMsg})")

1
Modules/ExternalData_config.cmake.in
View File

@ -2,3 +2,4 @@ set(ExternalData_OBJECT_STORES "@ExternalData_OBJECT_STORES@")
set(ExternalData_URL_TEMPLATES "@ExternalData_URL_TEMPLATES@") set(ExternalData_URL_TEMPLATES "@ExternalData_URL_TEMPLATES@")
set(ExternalData_TIMEOUT_INACTIVITY "@ExternalData_TIMEOUT_INACTIVITY@") set(ExternalData_TIMEOUT_INACTIVITY "@ExternalData_TIMEOUT_INACTIVITY@")
set(ExternalData_TIMEOUT_ABSOLUTE "@ExternalData_TIMEOUT_ABSOLUTE@") set(ExternalData_TIMEOUT_ABSOLUTE "@ExternalData_TIMEOUT_ABSOLUTE@")
@_ExternalData_CONFIG_CODE@

3
Tests/Module/ExternalData/CMakeLists.txt
View File

@ -10,7 +10,9 @@ if(NOT "${CMAKE_CURRENT_SOURCE_DIR}" MATCHES "^/")
endif() endif()
set(ExternalData_URL_TEMPLATES set(ExternalData_URL_TEMPLATES
"file://${slash}${CMAKE_CURRENT_SOURCE_DIR}/%(algo)/%(hash)" "file://${slash}${CMAKE_CURRENT_SOURCE_DIR}/%(algo)/%(hash)"
"ExternalDataCustomScript://MyScript1/%(algo)/%(hash)"
) )
set(ExternalData_CUSTOM_SCRIPT_MyScript1 "${CMAKE_CURRENT_SOURCE_DIR}/MyScript1.cmake")
set(ExternalData_BINARY_ROOT "${CMAKE_CURRENT_BINARY_DIR}/ExternalData") set(ExternalData_BINARY_ROOT "${CMAKE_CURRENT_BINARY_DIR}/ExternalData")
file(REMOVE_RECURSE ${ExternalData_BINARY_ROOT}) # clean test file(REMOVE_RECURSE ${ExternalData_BINARY_ROOT}) # clean test
@ -23,6 +25,7 @@ ExternalData_Add_Test(Data1
COMMAND ${CMAKE_COMMAND} COMMAND ${CMAKE_COMMAND}
-D Data=DATA{Data.dat} -D Data=DATA{Data.dat}
${Data1CheckSpaces} ${Data1CheckSpaces}
-D DataScript=DATA{DataScript.dat}
-D DataMissing=DATA{DataMissing.dat} -D DataMissing=DATA{DataMissing.dat}
-D DataMissingWithAssociated=DATA{DataMissing.dat,Data.dat} -D DataMissingWithAssociated=DATA{DataMissing.dat,Data.dat}
-D SeriesA=DATA{SeriesA.dat,:} -D SeriesA=DATA{SeriesA.dat,:}

4
Tests/Module/ExternalData/Data1Check.cmake
View File

@ -8,6 +8,10 @@ if(DEFINED DataSpace)
message(SEND_ERROR "Input file:\n ${DataSpace}\ndoes not have expected content, but [[${lines}]]") message(SEND_ERROR "Input file:\n ${DataSpace}\ndoes not have expected content, but [[${lines}]]")
endif() endif()
endif() endif()
file(STRINGS "${DataScript}" lines LIMIT_INPUT 1024)
if(NOT "x${lines}" STREQUAL "xDataScript")
message(SEND_ERROR "Input file:\n ${DataScript}\ndoes not have expected content, but [[${lines}]]")
endif()
if(DataMissing) if(DataMissing)
if(EXISTS "${DataMissing}") if(EXISTS "${DataMissing}")
message(SEND_ERROR message(SEND_ERROR

1
Tests/Module/ExternalData/DataScript.dat.md5 Normal file
View File

@ -0,0 +1 @@
fd95c03719e8626c0d10a818f9996dc5

5
Tests/Module/ExternalData/MyScript1.cmake Normal file
View File

@ -0,0 +1,5 @@
if(ExternalData_CUSTOM_LOCATION STREQUAL "MD5/fd95c03719e8626c0d10a818f9996dc5")
file(WRITE "${ExternalData_CUSTOM_FILE}" "DataScript")
else()
set(ExternalData_CUSTOM_ERROR "no ${ExternalData_CUSTOM_LOCATION} known")
endif()

1
Tests/RunCMake/ExternalData/BadCustom1-result.txt Normal file
View File

@ -0,0 +1 @@
1

9
Tests/RunCMake/ExternalData/BadCustom1-stderr.txt Normal file
View File

@ -0,0 +1,9 @@
^CMake Error at .*/Modules/ExternalData.cmake:[0-9]+ \(message\):
Bad ExternalDataCustomScript key '0BadKey' in URL template:
ExternalDataCustomScript://0BadKey/%\(algo\)/%\(hash\)
The key must be a valid C identifier.
Call Stack \(most recent call first\):
BadCustom1.cmake:[0-9]+ \(ExternalData_Add_Target\)
CMakeLists.txt:[0-9]+ \(include\)$

5
Tests/RunCMake/ExternalData/BadCustom1.cmake Normal file
View File

@ -0,0 +1,5 @@
include(ExternalData)
set(ExternalData_URL_TEMPLATES
"ExternalDataCustomScript://0BadKey/%(algo)/%(hash)"
)
ExternalData_Add_Target(Data)

1
Tests/RunCMake/ExternalData/BadCustom2-result.txt Normal file
View File

@ -0,0 +1 @@
1

9
Tests/RunCMake/ExternalData/BadCustom2-stderr.txt Normal file
View File

@ -0,0 +1,9 @@
^CMake Error at .*/Modules/ExternalData.cmake:[0-9]+ \(message\):
Bad ExternalDataCustomScript key '' in URL template:
ExternalDataCustomScript:///%\(algo\)/%\(hash\)
The key must be a valid C identifier.
Call Stack \(most recent call first\):
BadCustom2.cmake:[0-9]+ \(ExternalData_Add_Target\)
CMakeLists.txt:[0-9]+ \(include\)$

5
Tests/RunCMake/ExternalData/BadCustom2.cmake Normal file
View File

@ -0,0 +1,5 @@
include(ExternalData)
set(ExternalData_URL_TEMPLATES
"ExternalDataCustomScript:///%(algo)/%(hash)"
)
ExternalData_Add_Target(Data)

1
Tests/RunCMake/ExternalData/BadCustom3-result.txt Normal file
View File

@ -0,0 +1 @@
1

7
Tests/RunCMake/ExternalData/BadCustom3-stderr.txt Normal file
View File

@ -0,0 +1,7 @@
^CMake Error at .*/Modules/ExternalData.cmake:[0-9]+ \(message\):
No ExternalData_CUSTOM_SCRIPT_MissingKey is set for URL template:
ExternalDataCustomScript://MissingKey/%\(algo\)/%\(hash\)
Call Stack \(most recent call first\):
BadCustom3.cmake:[0-9]+ \(ExternalData_Add_Target\)
CMakeLists.txt:[0-9]+ \(include\)$

5
Tests/RunCMake/ExternalData/BadCustom3.cmake Normal file
View File

@ -0,0 +1,5 @@
include(ExternalData)
set(ExternalData_URL_TEMPLATES
"ExternalDataCustomScript://MissingKey/%(algo)/%(hash)"
)
ExternalData_Add_Target(Data)

1
Tests/RunCMake/ExternalData/BadCustom4-result.txt Normal file
View File

@ -0,0 +1 @@
1

7
Tests/RunCMake/ExternalData/BadCustom4-stderr.txt Normal file
View File

@ -0,0 +1,7 @@
^CMake Error at .*/Modules/ExternalData.cmake:[0-9]+ \(message\):
No ExternalData_CUSTOM_SCRIPT_RelPathKey is not set to a full path:
RelPathScript.cmake
Call Stack \(most recent call first\):
BadCustom4.cmake:[0-9]+ \(ExternalData_Add_Target\)
CMakeLists.txt:[0-9]+ \(include\)$

6
Tests/RunCMake/ExternalData/BadCustom4.cmake Normal file
View File

@ -0,0 +1,6 @@
include(ExternalData)
set(ExternalData_URL_TEMPLATES
"ExternalDataCustomScript://RelPathKey/%(algo)/%(hash)"
)
set(ExternalData_CUSTOM_SCRIPT_RelPathKey "RelPathScript.cmake")
ExternalData_Add_Target(Data)

4
Tests/RunCMake/ExternalData/RunCMakeTest.cmake
View File

@ -1,5 +1,9 @@
include(RunCMake) include(RunCMake)
run_cmake(BadCustom1)
run_cmake(BadCustom2)
run_cmake(BadCustom3)
run_cmake(BadCustom4)
run_cmake(BadHashAlgo1) run_cmake(BadHashAlgo1)
run_cmake(BadOption1) run_cmake(BadOption1)
run_cmake(BadOption2) run_cmake(BadOption2)