A Data Flow connects two Endpoints so that data can be transferred between them in a single direction. For more details on Data Flows, see What is a Data Flow?
This article contains information about how you can work with Data Flows in CIC.
- Understanding the Data Flow Page
- Creating a Data Flow
- Editing a Data Flow
- Removing a Data Flow
- Disabling a Data Flow
- Filtering Source folder
- Advanced Commands
Understanding the Data Flow Page
The Data Flow page is displayed when you click
This page displays information about each of the Data Flows in your CIC system, including the following:
Name | The name of the Data Flow. Used to identify the Data Flow throughout your CIC system. |
From |
The source Endpoint for the Data Flow. |
To | The destination Endpoint for the Data Flow. |
Created by | The person who created the Data Flow. |
Modified |
The date and time the Data Flow was last modified. |
Modified By |
The person who last modified the Data Flow. |
Last Job |
Status, date, and time when the Data Flow was last run. |
Total Jobs |
The number of times the Data Flow has been run. |
Creating a Data Flow
- For FTP and SFTP Endpoints, click Select and then type in the path to the subfolder. There is no option to browse for a folder.
- (Optional) For Data Flows where the source Endpoint is a File System, SFTP, or FTP(S) Endpoint, you can run the Data Flow manually or schedule the Data Flow to run on a recurring basis. For more details, see Scheduled in Running a Data Flow in CIC Express.
- Click Save to save the Data Flow.
Editing a Data Flow
Removing a Data Flow
Disabling a Data Flow
When a Data Flow is disabled, it is greyed out on the Data Flows page.
You cannot disable a Data Flow directly. Instead, to disable a Data Flow, you disable one of its Endpoints.
Note: When you disable an Endpoint, all the Data Flows that use that Endpoint are also disabled.
See Enabling and Disabling Endpoints in the Managing Endpoints in CIC Express article.
Advanced Options
Filtering Source Folder
For File System Endpoints, once you have selected your source folder and clicked the Select button, you have the option to filter the source folder for a subset of files. This results in the Data Flow only sending the files that match the filter.
To specify a filter, click the filter button.
This displays a field where you can enter one or more filters. Enter one or more filters to apply in this field. Only files matching the pattern you enter are transferred for a given run of the Data Flow.
Filter Matching Rules and Examples
The following are the pattern matching rules for filtering path and file names.
-
The
*
character matches zero or more characters of a name without crossing directory boundaries. -
The
**
characters matches zero or more characters crossing directory boundaries. -
The
?
character matches exactly one character of a name. -
The backslash character (
\
) is used to escape characters that would otherwise be interpreted as special characters. The expression\\
matches a single backslash and "\{" matches a left brace for example. -
The
[ ]
characters are a bracket expression that match a single character of a name out of a set of characters. For example,[abc]
matches"a"
,"b"
, or"c"
. You can use the hyphen (-
) to specify a range. For example,[a-z]
specifies a range that matches from"a"
to"z"
(inclusive). You can mix these forms, so that [abce-g] matches"a"
,"b"
,"c"
,"e"
,"f"
or"g"
. -
If the character after the
[
is a!
, it is used for negation. For example,[!a-c]
matches any character except"a"
,"b"
, or"c"
.Within a bracket expression the
*
,?
and\
characters match themselves. The (-
) character matches itself if it is the first character within the brackets, or the first character after the!
if negating. -
The
{ }
characters are a group of subpatterns, where the group matches if any subpattern in the group matches. The","
character is used to separate the subpatterns. Groups cannot be nested. -
Leading period/dot characters in file name are treated as regular characters. For example, the
"*"
pattern matches file name".login"
. -
All other characters match themselves.
Here are some examples of common filters:
Advanced Commands
If your source Endpoint is FTP or SFTP, once you have selected your source Endpoint, you can enter advanced commands in the From Endpoint panel as shown below.
See Command Reference for a list of supported commands. When you specify advanced commands, you can use macro variables, wildcards, and regular expressions as described in Using Macro Variables in CIC and Using Wildcards and Regular Expressions in CIC, respectively.
IMPORTANT: Any Advanced Commands specified for the Data Flow override the Move action of the Data Flow. This means if you specify any advanced commands and you want to move data from the source Endpoint to the destination Endpoint, you must include commands to do that.