Sign in
< Back to Top

File System Endpoint

A File System Endpoint allows a folder or directory to be accessible from an Access Point server—a local file system, a mounted network file system, or on a Windows Access Point, a UNC path—to act as a source or destination for data transfer. See also What is an Access Point? The File System Endpoint has the following attributes.

Attribute Description
Access point

Select an Access Point for this Endpoint. If there are no Access Points available to you, you can create one. See Managing Access Points. 

After you select an Access Point, browse through the file system on that Access Point, and then select a root directory to act as a source or destination for a Data Flow. 
Note: By default, a File System Endpoint built on a given Access Point has recursive access to the files and folders under the Access Point's root directory. This has the following implications:

  • Any admin that has access to the Access Point can also browse files and folders under and including the Access Point's root directory.
  • All the files in all the folders under the selected directory are selected for transfer. You can restrict this behavior when you configure a Data Flow by selecting a subfolder as the Data Flow's Source and/or by specifying a filter pattern. See Managing Data Flows for more information.
Note: On Linux systems, the CIC Agent installation creates the user, cleoand the Agent service runs as this user. The cleo user must have write permissions to any directories and files you want this Access Point and Endpoint to have access to. Your Linux admin needs to grant access to this user as needed to open up access.
On Windows systems, by default, the service runs as System, which restricts access to local drives only. To use network mounted drives or UNC, change this service to run as a domain account. 
File System Type Choose the type of file system on which you want the Endpoint source or destination directory to reside:
  • Local - Selects the local system where the Access Point is installed as the source or destination of the endpoint. 
  • Shared - Exposes fields in the UI where you can specify attributes for a shared file system such as Share Path, Domain, Username, and Password. 

After you choose a local or shared file system, you select a directory to act as the Endpoint source or destination. See Selecting a Source or Destination Directory for more information.
Share Path The SMB root path. Valid patterns: [^smb://.*$, ^//:.*$, ^smb:\\.*$, ^\\:.*$]
Domain The Domain name.
Username Share account user username.
Password Share account user password.
Browse When you connect to a file system, this tab is populated with a directory listing from which you can choose a source or destination directory. See Selecting a Source or Destination Directory for more information.
Open PGP OpenPGP protects files being transferred through encryption and signing.

Inbound
Verify Signed Payloads When you select this check box, you are prompted to select the trading partner's signing PGP key. The PGP key named in this field is the same as the outbound encryption PGP key. If you change this value here, it is also changed for the outbound encryption PGP key. See Endpoint Security.
Accept Encrypted Payloads When you select this check box, you are prompted to select your decryption PGP key. The PGP key named in this field is the same as the outbound signing PGP key. If you change this value here, it is also changed for the outbound signing PGP key. See Endpoint Security
Outbound
Sign Select this check box to sign messages sent from this Endpoint.
When you select Sign, you are prompted to select your signing PGP key. The PGP key named in this field is the same as the inbound decryption PGP key. If you change this value here, it is also changed for the inbound decryption PGP key. See Endpoint Security.
Encrypt Apply encryption to messages sent from this Endpoint.
When you select Encrypt, you are prompted to select the trading partner's encryption certificate. The PGP key named in this field is the same as the inbound signing PGP key. If you change this value here, it is also changed for the inbound signing PGP key. See Endpoint Security.
Compress Apply ZLIB compression to messages sent from this Endpoint. Choose this option for large files to conserve bandwidth and improve efficiency and security.
Text Output (ASCII armor) Encase encrypted messages in ASCII for ease of sending using standard messaging formats.
Advanced This tab consists of several sections. The Connection and Transfer section is displayed only for Endpoints that use a Local File System. The Connection and Logging sections are displayed only for Endpoints that use a Shared File System. The OpenPGP section is displayed for both Shared and Local File Systems. 
Section Description

Connection and Transfer

(Local File System only)
Retrieve Directory Sort Specify the sorting order for files being transferred and processed. Choose from the following:
  • None  (Default value)
  • Alphabetical (ascending)
  • Alphabetical (descending)
  • Date/Time Modified (ascending)
  • Date/Time Modified (descending)
  • Size (ascending)
  • Size (descending)

Connection

(Shared File System Only)
Min SMB Version

The connector will attempt to use the highest dialect set by the Max SMB Version and will negotiate down to the Min SMB Version until a dialect is selected that both the client and the server support. The SMB Endpoint currently supports SMB 1 (NT LM 0.12), SMB2.02, SMB2.1, SMB3.0, SMB3.0.2, and SMB3.1.1.
Max SMB Version
Additional SMB Properties You can add a list of additional properties. Visit  https://jcifs.samba.org/src/docs/api/overview-summary.html#scp for more information about the properties. 
See Working with Additional SMB Properties 
Retrieve Directory Sort Specify the sorting order for files being transferred and processed. Choose from the following:
  • None  (Default value)
  • Alphabetical (ascending)
  • Alphabetical (descending)
  • Date/Time Modified (ascending)
  • Date/Time Modified (descending)
  • Size (ascending)
  • Size (descending)
Command Retries The number of times a command should be retried if an error or exception occurs during a command. Valid range: [0-5]
Command Retry Delay The amount of time (in seconds) before a retry should be attempted. Valid range: [0-120]

Logging

(Shared File System Only)
Enable Debug Select this check box to make debug logs available for this endpoint.
OpenPGP
OpenPGP Algorithms
Hash Algorithm Choose the signing method used when OpenPGP packaging (with signing) is requested from the following:
  • MD2
  • MD5
  • RIPE-MD-160
  • SHA-1
  • SHA-256
  • SHA-384
  • SHA-512
V3 Signature Select this check box to sign messages with Version
3 signatures
Encryption Algorithm Choose the algorithm you want to use to encrypt
messages. The remote host receiving the message
must be able to decrypt the message using the
algorithm you choose.
Compression Algorithm Choose the algorithm you want to use to compress
messages. The remote host receiving the message
must be able to compress the message using the
algorithm you choose.
Choose either ZIP or ZLIB.

Selecting a Source or Destination Directory 

As part of creating a File System Endpoint, you select an Access Point and specify a directory on the Access Point to be a source or destination for a Data Flow. The file system where the directory resides can be the local system where the Access Point is installed or a shared system you have access to. See File System Endpoint for more detailed information about Access Points as they relate to File System Endpoints.

Follow these steps to select an Access Point and directory for a File System Endpoint.

  1. On the Endpoint screen, select an Access Point.
  2. Specify whether to use a Local or Shared file system.
  3. If you select Local, click Connect and, with no further intervention, CIC displays a list of available directories.
  4. If you select Shared, provide the required connection information (see File System Endpoint for details) and then click Connect to see the list of available directories. 
  5. To select a directory, do one of the following:
    • Click Select to choose the root directory of the Access Point.
    • Navigate to a sub-directory and then click Select.

      CIC displays the selected directory.
    • Finally, you can manually type in a path, click Go to display its contents, and then click Select to make it the selected directory.
  • Click Save to save your File System Endpoint.

Working with Additional SMB Properties

To add SMB Properties:

  1. On the Advanced tab, click Edit near the Additional SMB Properties item.

    The Additional SMB Properties list displays.
  2. Enter a property name (see https://jcifs.samba.org/src/docs/api/overview-summary.html#scp) and a value, and click Add.
  3. The new property is added to the list.
  4. Click OK to close the list

To delete SMB Properties:

  1. On the Advanced tab, click Edit near the Additional SMB Properties item.
  2. On the Additional SMB Properties list, select the property to be deleted and then click the Delete button.
  3. Click OK to close the list.

File System Endpoint commands

The following commands are available for File System Endpoints.

CLEAR

Valid only for local File System Endpoints.

Clear a property string value. The cleared value only affects the commands that follow the CLEAR.

CLEAR property

property

 Property name with no embedded spaces.

GET

Valid for Source Endpoints.
Receive one or more files from the host.

GET [-DEL] [-REC] "source"
-DEL

Optional.

If GET is successful, delete remote file.
-REC

Optional.

Recursively retrieve nested subdirectories. 
When you use the REC option in conjunction with the DEL option, the retrieved files are deleted from the server, but the subdirectories remain.
source Remote source path. You can specify a path (folder/) or a path+filter (folder/*.txt)

LCOPY

Copy one or more files locally.

LCOPY [–DEL] [-REC] [–UNI|–APE] "source" "destination"
-DEL If the command is successful, delete the local file. If used with -REC, delete the file and the subfolder structure.
-REC Recursively search all subdirectories and copy the subfolder structure into the destination.
-UNI Ensure the copied filename is unique.
-APE Append copied file to existing destination file.
"source" Source path
  • Path can be to a filename or to a directory
  • You can use * and ?, or a regular expression when you specify a filename. See Using Wildcards and Regular Expressions in CIC for additional information.
  • If you specify a relative path, the command uses the Data Flow's configured subpath.
  • You can use macro variables. See Using Macro Variables in CIC (Source File context) for a list of the applicable macros.
  • If the path contains a space, dash (-), comma (,), or equal sign (=), it must be enclosed with double quotes ("...").
"destination" Destination path.
  • Path can be to a filename or to a directory.
  • If you specify a relative path, the command uses the Data Flow's configured subpath.
  • You can use macro variables. See Using Macro Variables in CIC (Source File context) for a list of the applicable macros.
  • When copying a file without the -APE option, or when copying a file with the -APE option where the destination file does not already exist, a temporary file name is used while the copy operation is taking place. This temporary file is placed in the destination directory. Its name begins with the product name and ends with .tmp. Once the copy completes successfully, the temporary file is renamed to the destination name.
  • If the path contains a space, dash (-), comma (,), or equal sign (=), it must be enclosed with double quotes ("...").

LDELETE

Delete one or more files locally.

LDELETE [-REC] "source"
-REC

Optional.

Recursively search all subdirectories and delete the subfolder structure from the destination.
source Source path
  • Path can be to a filename or to a directory
  • You can use * and ?, or a regular expression when you specify a filename. See Using Wildcards and Regular Expressions in CIC for additional information.
  • If you specify a relative path, the command uses the File System Endpoint subpath.
  • You can use macro variables. See Using Macro Variables in CIC (Source File context) for a list of the applicable macros.
  • If the path contains a space, dash (-), comma (,), or equal sign (=), it must be enclosed with double quotes ("...").

LREPLACE

Replace bytes in one or more files locally. 

LREPLACE "source" Replace="input bytes" With="output bytes"

For example, LREPLACE sourcePath/file.txt Replace=10,13 With=10

"source" Source path
  • Path can be to a filename or to a directory
  • You can use * and ?, or a regular expression when you specify a filename. See Using Wildcards and Regular Expressions in CIC for additional information.
  • If you specify a relative path, the command uses the File System Endpoint subpath.
  • You can use macro variables. See Using Macro Variables in CIC (Source File context) for a list of the applicable macros.
  • If the path contains a space, dash (-), comma (,), or equal sign (=), it must be enclosed with double quotes ("...").
"input bytes" List of bytes to be replaced.
  • Comma separated list of byte values (0-255).
  • All bytes in comma-separated list must be found in the file in listed sequence in order to be replaced.
"output bytes" List of bytes to be substituted for original input bytes.
  • Comma-separated list of byte values (0-255).
  • If With parameter is omitted, then the input bytes are deleted from the file.

PUT

Valid for Target Endpoints.

Send one or more files to the host.

PUT [-UNI|-APE] "source" "destination"
-UNI Ensure remote filename is unique
-APE Append to the existing destination file 
source

Source path 
destination Remote destination path. The use of macro variables is supported.  See Using Macro Variables in CIC (Destination context) for a list of the applicable macros.

SET

Change a property value. The new value only affects the commands that follow the SET.

SET property=value
property = value Property and new value
  • The property name must have no embedded spaces.
  • The value specified remains in effect until it is set again or until the end of the Data Flow.  
  • To reset the property back to default value, specify 
    SET property

    or

    SET property=
Valid properties for SET command are as follows:
PostProcessingCommand
PreProcessingCommand

Use the PreProcessingCommand property to run an operating system or local command before each file in a GET.
Use the PostProcessingCommand property to run an operating system or local command after each file in a PUT.

For example,

SET PostProcessingCommand=/bin/sh -c 'cp \"${file}\" \"${noext}-ppcopy${ext}\"'

You can also run the LCOPY, LDELETE, and LREPLACE local agent commands as Pre/PostProcessingCommands within a SET command by prepending the command name with a $.

For example,

SET PostProcessingCommand=$LCOPY "copyFromDirPath/myfile.txt" "copyToDirPath/"

SET PostProcessingCommand=$LDELETE "myfile.txt"

SET PostProcessingCommand=$LREPLACE sourcePath/file.txt Replace=10,13 With=10

PostProcessWorkingDirectory

 Specify a working directory for any commands that follow.

PostProcessTimeout(seconds)

Specify a time limit after which CIC stops attempting to run the command.

A value of 0 means there is no timeout.

GetNumberOfFilesLimit

Specify the maximum number of files pulled during a GET operation.

A value of 0 means there is no limit.

ExpandFileNameOnPut

When set to true, allows PUT command destination expressions for constructing filenames.

For example, to handle filenames with characters that the operating system cannot handle in filenames, use the ExpandFileNameOnPut property with the PUT command similar to the following:

SET ExpandFileNameOnPut=true

PUT * ${filename.replaceAll(':','_')}
TerminateOnFail

Valid for all Endpoints configured as source in a Data Flow.
Use this property to control command processing when errors occur.
Possible values:

  • True - Command processing stops when an error occurs.
  • False - Command processing continues even when an error occurs.

Default value is True.

WAIT

Pause execution.

WAIT seconds
seconds Number of seconds to pause.

Related articles

Comments

0 comments

Please sign in to leave a comment.