Running a Data Flow

When a Data Flow runs, data is transferred from one Endpoint to another. Data Flows run automatically, manually, or when scheduled.

Automatic

In some cases, Data Flows run without any direct intervention, for example, when the source Endpoint is a Partner Mailbox or an AS2 server. In these cases, the Data Flow runs when data is "dropped" on the Endpoint.

Manual

Click the Run Now button and select either Run with Source Endpoint Data or Run with Select Files. 

Run with Source Endpoint Data

For File System, email, FTP, SFTP, S3, AzureBlob, and GCS Endpoints, you can select Run with Source Endpoint Data. This transfers data from the source Endpoint as it is configured and the Data Flow runs immediately

Run with Select Files

When you select Run with select files, CIC displays a dialog box where you can upload files to CIC from your local system to be transferred using the Data Flow.

In this case, the Data Flow treats the uploaded files as if they were being transferred from the source Endpoint. After you upload files, the Run button is enabled and you can click it to run the Data Flow.

For a Partner Mailbox Endpoint, if you have multiple folders selected, you must select one of them to act as the "source" for the transfer. That is, the transfer will treat the files as though they came from the folder you selected. After you upload files and select a folder, the Run button is enabled and you can click it to run the Data Flow.

Scheduled

In cases where the source Endpoint type is File System, SFTP, FTP(S), S3, AzureBlob, Email, or OFTP(S), you can schedule a Data Flow to run at some point in the future. You can specify multiple schedules for a Data Flow. All schedules you set up are recurring.

View this short video to see how to schedule a Data Flow to run.

1. Go to Integrations > Data Flows and select a Data Flow. The Data Flow edit page displays.
2. Click Add Schedule.

   

   The Schedule dialog box displays.

   

3. Select a time zone. All the schedules you add will be relative to the time zone you select.
4. Select an interval type – Minutes, Hourly, Daily, Weekly or Monthly.

   Note: The smallest interval supported is 1 minute.

   Interval Type	Description
   Minutes	Choose an interval in minutes and a starting point in seconds.
   Hourly	Choose an interval in hours and a starting point in minutes and seconds
   Daily	Choose an interval in days and a starting point in hours, minutes, and seconds.
   Weekly	Select the days on which you want the Data Flow to run and specify a starting time.
   Monthly	Select from the following: A numerical day of the month (for example, the 15th of the month), an interval in months (for example, every 3 months), and a starting time. Should the day of the month fall on a non-working day, you can specify that the Data Flow should run on the weekday that falls closest to the scheduled start time. A day of the week and its order (for example, the second Tuesday of the month), an interval in months (for example, every 3 months), and a starting time.

5. Click Advanced to specify a schedule using Quartz Cron notation.  See Using Cron Expressions in CIC Scheduler for more information.

   Note: The smallest interval supported is 1 minute.

6. Click Add additional schedule to set another time to run the Data Flow.