Retrying Failed File Transfer Attempts

When file transfer send attempts fail, CIC retries them automatically. For outbound transfers, retries can be configured to continue on an extended basis should the initial automatic retry fail. You can also trigger retries manually.

About Automatic Retries

Failed transfer attempts, whether send or receive, are subject to Automatic Retry processing, where the transfer is attempted again immediately after it fails. 

Automatic Retry is configured on a per-Endpoint basis.

For each eligible Endpoint, you specify the number of retries that should be made during an attempt. An attempt consists of the number of transfer retries you specify over an interval of time you also specify. For example, if you specify 5 retries and 30 seconds, an attempt would consist of 5 retries occurring in the span of 150 seconds.

Note: Extended Retry uses the same retries-per-attempt configuration. 

About Extended Retry

If an initial transfer attempt fails, Extended Retry processing begins. The first Extended Retry is attempted immediately after the failed initial attempt. If that attempt fails, further attempts are made after 1, 2, 4, 8, 16, 32, and 60 minutes, and every 60 minutes thereafter. The intervals at which these attempts are made are not configurable. However, for each eligible Endpoint, you specify the period of time during which outbound retries are attempted.

Outbound retries will not be attempted after the end of the period specified. This period must be a minimum of 15 minutes and can be as long as 3 days. It starts when the first retry is attempted and expires when the time period you specify is over. 

The Extended Retries are no longer attempted when all retries have either succeeded or been manually canceled and marked as “failed.”  See Canceling a payload.

Manual Retries

Manual Retry occurs when a user selects a failed job and clicks the Retry button.

Manual retry is also available at any point during the Extended Retry cycle. If an extended retry attempt is already active at the time a manual retry is requested, the manual retry request is, in essence, ignored. However, this does reset the retry cycle to the beginning.

Retrying Failed Jobs Manually

Sometimes when sending a job to its destination, a failure may occur due to permission or storage issues, etc. Once the issue gets corrected, you can quickly resend the job from exactly where it failed using the Retry Job button, which displays at the top right corner of the Job Detail page when manual retry is available.

Not all jobs can be retried with this option. However, the Retry Job button only appears when a job can be.

How Manual Retries Work

1. Select failed inbound or outbound job to retry.
2. Click the Retry Job button.
3. Refresh your browser.
4. View the results.

In the simple example below we see that a job has failed and an Issue has been created.

You can check the Payloads tab to see details of the issue, and also the Log tab to see the events, which in this example, shows a SEND_FAILURE.

After troubleshooting and correcting the problem, you would click the Retry Job button. This results in the job being queued for retry.  Important: At this time, users must refresh their browser in order to see the outcome of the retry.

Once complete (and issues addressed), the Job Detail page displays the job as successful. 

You can also confirm this in the Payloads and Log tabs (shown here). Note that the SEND_FAILURE, as well as MANUAL_RETRY, have been logged.

Resending failed Jobs that contain multiple payloads and job steps

Failed jobs that contain multiple payloads and multiple job steps can also be retried using the Retry Job button. In this example, two of the five job steps have failures. Once the issues have been resolved, clicking the button would retry the failed job steps. Note that not all jobs can be retried with this option. However, the Retry Job button only appears when a job can be.

A successful run of these job steps would then display.

Understanding Retry Information on the Jobs List

Jobs that fail and are subsequently retried are displayed on the Jobs List screen. They have a new icon and new status - Retrying. If a job runs and exhausts its initial command retries, the job enters into an extended retry mode and it is displayed with the Retrying icon. 

Like other jobs on the list, you can drill into a job with a status of Retrying to see it on the Jobs Details screen. 

In addition, the Retrying status is available for you to use as criteria for filtering the Jobs list.

Understanding Retry Information on the Job Details screen

When you click a Job on the Job List to “drill into’ it, the Job Details screen displays. 

When you click into a “retrying job” (like the one above), the Job Progress bar shows the Job as “retrying” and the Issues tab displays.

Payloads Tab

The Payloads tab shows the failed payload as retrying. Retrying status is displayed at the payload level to show which payload is currently being retried.

If you are using AS2 and have receipts configured, you can click the Receipt icon to display the Receipt dialog box.

Log Tab

Click the Log tab to display log entries related to this job.
For a failed payload being automatically retried, you see a SEND_FAILURE event followed immediately by a RETRYING event. See Job Events.

If that retry attempt fails, we again see a SEND_FAILURE event, but followed immediately by a DELAYING_BEFORE_RETRY event in accordance with the extended retry schedule (see above).

Number of retries is displayed at the top of the Job Details page.

When you click the Retries icon, you see a short history of your retries and when the next retry is scheduled.

Also, when you hover over the Retrying icon on the Payloads page, information about the next retry is displayed.

Canceling a payload

1. Select a payload (1) and then click the Cancel button (2).
   
2. Refresh the page to see that the Status has changed to failed and that processing continues on next payload.
   

If all payloads are manually canceled and marked as failed, the retry cycle will stop, and the job will enter a failed Status. The manual Retry button will no longer be available on the job itself because the failure will be considered intentional, but you can still retry individual canceled payloads to start a new retry cycle.

Retrying a canceled payload

Select a payload previously marked as “failed”(1) and click the Retry button (2).