The AzureBlob connector is used to communicate with Microsoft Azure Blob Storage (Blob Storage). Azure Blob Storage is Microsoft's object storage solution for the cloud. Blob Storage is optimized for storing massive amounts of unstructured data, such as text or binary data. Blob Storage exposes three resources: the Azure storage account, the containers within the account, and the blobs within a container. Azure offers three types of blobs – block blobs, append blobs, and page blobs. The Cleo AzureBlob connector supports block and append blobs for upload operations and all three types for download operations. When creating a new blob through an upload operation, use the Blob Type property to designate the type you want. Block blobs are the default.
The blobs within a container have a flat structure, without the hierarchy you might see in a typical file system. However, for organizational simplicity, the Cleo AzureBlob connector supports the folder concept as a means of grouping objects. Therefore, you can treat your folder and file objects as you would in a standard hierarchical file system. For example:
PUT example.txt AzureTestFolder/example.txt
This command transfers the local file, example.txt, to the configured container within a "folder" object called AzureTestFolder. Note that AzureTestFolder must exist as an object in blob container.
Two different authentication methods are supported.
- The first method is to use the access key directly with the property 'Access Key'.
- The second is to use the access key stored inside of an Azure Key Vault secret.
To setup this second method, you need to have an Azure Key Vault with a secret that contains the access key. The key vault name and secret name must be supplied through the properties 'Key Vault Name' and 'Key Vault Secret Name', respectively. You also need an application service principal that has access to the Key Vault. The application clientId and clientSecret must be supplied through the properties 'Application Client ID' and 'Application Client Secret', respectively.
Notes:
- The
ATTRcommand currently supports a subset of all Azure blob attributes. - The Pseudo Folders property can be used to control whether actual folder objects are created and needed by the connector.
LCOPY -DELcommands that attempt to move files will use the connector RENAME, thereby avoiding download/upload Azure charges.LCOPY -DELcommands that attempt to move files between two different containers by overriding the destination container with a query parameter will also use the connector RENAME to avoid download/upload Azure charges. However, defining any destination query parameters other than the container will cause the file to be downloaded from one container and uploaded to the other container which will most likely incur an Azure charge.- If during an
LCOPY -DELthe source container is overridden with a query parameter the destination will also use the same container. If desired, both the source and destination containers can be overridden with query parameters to copy between containers.
AzureBlob Connector Properties
Each instance of the AzureBlob Connector can be configured using the following settings:
| Property | Description | Required |
|---|---|---|
| Storage Account Name | The Azure Storage account name. | Yes |
| Access Key | The Azure Storage account access key. If not supplied, then Azure Key Vault authentication will be used. | |
| Key vault Name | The Azure Key Vault name. | |
| Key Vault Secret Name | The name of the secret n the Azure Key Vault. | |
| Application Client ID | The ID of the client registered as an application in Azure Active Directory. Used to authenticate to Azure Key Vault. | |
| Application Client Secret | The secret for the client registered as an application n Azure Active Directory. Used to authenticate to Azure Key Vault. | |
| Endpoint Suffix | The Azure Storage endpoint suffix. | Yes |
| Container | The Azure Storage container name. Valid pattern: [^[a-z0-9]{1}[a-z0-9-]{1,62}] |
Yes |
| Btob Type | The Azure Blob Storage blob type. This setting controls the blob type for created blobs. | Yes |
| Pseudo Folders | Whether MKDIR/RMDIR should create/remove pseudo folder objects and where pseudo folder objects are required for DIR. | |
| Proxy Address | An optional HTTP proxy IP address or hostname. | |
| Proxy Port | An optional HTTP proxy IP port. Valid range: [0-65535] |
|
| Headers | A list of optional additional HTTP headers. | |
| Command Retries | The number of Ornes the command should be retried when an error or exception occurs. Valid range: [10-51] |
Yes |
| Command Retry Delay (seconds) | The number of seconds to wait between retries. Valid range: [0-120] |
Yes |
| Do Not Send Zero Length Fies | For PUT, a switch that indicates to not send a file if it is zero-length. | |
| Delete Received Zero Length Fies | For GET, a switch that indicates to remove a received file that is zero-length. | |
| Retrieve Directory Sort | For PUT, the sorting options for the list of outbound files. | |
| Enable Debug | A switch that indicates whether to perform debug logging. | |
| System Scheme Name | The URI scheme name used as a shortcut to this host Valid pattern: [[a-zA-Z]{1}{a-zA-Z0-9+\-\.]{1.23}]. |
|
| System Public | A switch that indicates whether the connector is public. |
Comments
0 comments
Please sign in to leave a comment.