What's new in version 5.8
- New REST API-based import for P12 certificates – Enhanced endpoint now allows importing certificates.
- New three-level support for nested ExecuteOn commands – Commands are now supported up to three levels.
- New Advanced Users property for Archive Nested Subdirectories – Archive file transfers to the user and system sent/received boxes.
- Enhanced change HTTP status code return – Ability to control the status return code when basic authentication is disabled
- Enhanced report generation – Go beyond the UI grid to access file path transfer information
- Enhanced Amazon S3 Connector – Enable cross-account access using AWS's AssumeRole in S3.
- Enhanced Google Cloud Platform Bucket Connector – Improved GCPBucket connections support for forward proxying; Support Pseudo Folders Property for GCPBuckets.
- Enhanced SMTP proxy configuration – Configure individual SMTP proxies to use start TLS property
Security Updates and Enhancements
- Upgraded to the latest version of log4j v2
- New SFTP algorithms and MQ SSL cipher specs
- New Admin-user level configuration to control accessibility to host visibility
- Enable Explicit AUTH Required setting for FTP
- Ensure paths in filenames on incoming requests are ignored for AS2, ebMS, RNIF, and SMTP protocols that do not support paths
- Removed the default OSGi HTTP listening port 8181
Additional Enhancements and Fixes
- New support for MySQL 8
- Enhanced ability to execute post processing commands after a file is written
- Enhanced ebMS to modify the format of the Content-Id header and new advanced property setting
- Enhanced Message Queuing support for MQ SSL cipher specs
- Enhanced ebMS (ebMXL) configuration and advanced property settings with new Allow Incoming Request With Missing Role Element property.
- Enhanced HTTP with new Save Error Response Content On Put Plus Get advanced property setting
- Updated support for multiple SFTP client and server-side algorithms.
- Updated outbound AS4 signed messages with multiple attachments now orders digest messages the same as the attachments.
- Improved performance of SSH FTP directory listings for Linux
- Improved performance of SMB connector
- Improved S3 UI performance on startup
Upgrading to version 5.8
When upgrading to Cleo Harmony version 5.8, Cleo recommends the following:
- Back up your configuration using the Export functionality. In the Web UI, go to Administration > System > Export. In the native UI, go to File > Export. Performing an Export will save your data in a format that you can import using the Cleo Harmony Import functionality should the need arise.
- Make sure your system meets the system requirements for Cleo Harmony version 5.8, as it requires greater resources than earlier versions. All new installs must be 64-bit. Visit Cleo Harmony 5.8 System Requirements to view the System Requirements for your product.
- Because this release of Cleo Harmony uses OpenJDK, if you are using the Web UI on a Unix system, you might need to install the latest fontconfig. The command is dependent on the flavor of Unix you are using. For example:
• Red Hat:
yum install fontconfig
apt-get install -y --no-install-recommends libfontconfig
- Run the Cleo Harmony 5.8 installer to perform an in-place upgrade. Your data and configuration remain intact from the previous version of the Cleo Harmony software.
Further Considerations for Upgrade
Cleo Harmony and Cleo VLTrader: The Unify file/folder share and anonymous download link options have been removed as Unify is no longer being offered within Portal and the VersaLex REST API. As part of this, the embedded Graph DB was also removed. If upgrading a system where Unify in Portal is in use, the product installer will block the upgrade until the system option "Disable Unify In Portal"is set to "True". (Go to find "Disable Unify in Portal".) The Portal Two-Factor Authentication (TFA) option, which was previously tied to Unify In Portal licensing, is still supported. But if upgrading, the product installer will also block the upgrade if TFA is in use and the last 5.7 patch has not been installed (which will migrate TFA away from the Graph DB).
Cleo Harmony and Cleo VLTrader: MySQL 8 is now supported. However, as part of this, the 'Recursive' column in the 'VLSLAKPI' database table had to be renamed to 'Recurse' as 'Recursive' is now a reserved keyword in MySQL 8. The column rename was applied to ALL databases and versions, not just MySQL 8. VersaLex will attempt to rename the column at startup after upgrade, but if the database user configured in VersaLex does not have DDL privilege or if the column rename otherwise fails, the column must be renamed outside of VersaLex. There is also a database view 'View_Checkpoints' where the 'Recursive' column was renamed. VersaLex will actually first attempt to drop this view, then rename the table column, and then recreate the view. For reference, use [Export Database Definition...] in the Transfers configuration window and find references to 'Recursive' in the CREATE TABLE VLSLAKPI and CREATE VIEW View_Checkpoints statements.
Cleo Harmony only: If you are upgrading from VersaLex 22.214.171.124 or older and you use Cleo Dashboards, you must upgrade Cleo Dashboards to v3.3.6 or newer because of database changes in VersaLex. In addition, Cleo Dashboards v3.3.6 requires Clarify v5.1.16 or newer.
The following sections contain descriptions of issues fixed in Version 5.8:
Fixed issues in version 5.8
Security - Framework
- Cleo Harmony and Cleo VLTrader only: Removed the default OSGi HTTP listening port 8181. This port was not necessary and was not locally bound.
- VersaLex now ensures that any paths in filenames on incoming requests are ignored for protocols that do not support paths, including AS2, ebMS, RNIF, and SMTP.
- For the main VersaLex process, upgraded log4j v1 to the latest version of log4j v2.
Security - Portal
- Cleo Harmony only: When logging into Portal, the session cookie is now changed after login to help prevent session fixation attacks.
- Cleo Harmony and Cleo VLTrader only: Increased web session cookie id length to be greater than 32 characters.
- Cleo Harmony and Cleo VLTrader only: Set-Cookie header now includes the secure flag when redirecting Portal from an http connection to an https connection.
Enhancements - Framework
- Added a new property called "Accessing raw payload from transfer reports requires Host permissions" to Administrator User configuration. Setting this property to "false" allows users with the ability to view transfer reports (but without the ability to view hosts) to view or email raw payload. By default, this is set to "true" to replicate current functionality.
- When sending bundled Database Payload, added the ability for each file to use additional properties only when explicitly set in the VLOutgoingProperties table. All other settings use the defaults from the host, mailbox, or action. To enable this, set 'Clear.Set.Properties' to 'True' in the VLOutgoingProperties table for each file.
- Cleo Harmony and Cleo VLTrader only: Added support for new %resttransferid% macro. This macro can be used wherever the traditional %transferid% macro can be used, but resolves to the REST API transfer id (also known as the document DB transfer id).
- Added the ability to change the HTTP status code returned when 'Disable Basic Access Authentication for REST API Requests' is turned on.
- Cleo Harmony and Cleo VLTrader only: Added support for using SAML with a custom authentication connector. By default, the user's SAML nameId assertion attribute is verified as included in the custom auth connector's set of usernames; however, the full set of assertions are available and this verification can be further customized in the auth connector implementation itself (by overriding the 'lookupUserByAssertions' method).
- Cleo Harmony and Cleo VLTrader only: Added new Users advanced property, 'Archive Nested Subdirs'. When set on, file transfers to subdirectories within the configured upload and download folders will also be archived both to the user's and the system sent/received boxes.
- Nested ExecuteOn... commands are now supported up to three levels. An example would be an ExecuteOnFail from a failure result of an ExeucteOnCheckConditionsMet (this would be two levels).
- The Generate Report option in the admin web UI Transfers page would previously include only the information viewable from the UI grid. Now all available transfer information, such as file path, is included in the generated report. Also, a report generated from classic mode specifically now includes the file path if it is enabled in the user's group.
- Cleo Harmony and Cleo VLTrader only: If a file share has already timed out, then any subsequent scheduler failures on the file share during the configured 'Wait Time For Nonresponsive File Systems' are now logged as warnings rather than errors. This helps to cut down on email-on-fail alerts for the same file share issue.
- Added the ability to import a P12 certificate through the REST API.
- Cleo Harmony and Cleo VLTrader only: Added virtual subfolder support in Users hosts at the sub-folder level. Previously virtual subfolders could only be specified at the root level. Also added LIST-DIRS and LIST-FILES permissions to allow directories or files to show in the directory listing.
- Improved performance renaming/moving files within the same connector.
Enhancements - AS4
- Cleo Harmony only: When sending signed messages with multiple attachments, the digest references are now ordered the same as the attachments.
Enhancements - FTP
- Cleo Harmony and Cleo VLTrader only: Added a "Before Login" option on the FTP/S Explicit AUTH Required setting. The option is located in the Local Listener |FTP| tab. With this new option turned on when AUTH is required, a user must issue the AUTH command before the USER and PASS commands.
Enhancements - HTTP
- Added a new HTTP 'Save Error Response Content On Put Plus Get' advanced property, which when set on causes the response content from a PUT+GET command request to be saved to the inbox even on error responses.
Enhancements - SSH FTP
- Added an option to SSHFTP Client host named 'Ignore STAT Errors' which will ignore any FXP_STAT errors when opening a directory.
- Added system options for limiting client-side SSH FTP cipher, key exchange, mac, and public key algorithms for all client connections. Go to Administration>System>Other in the admin web UI and filter on Protocols to configure regular expressions for each algorithm.
- Cleo Harmony and Cleo VLTrader only: Improved performance of SSHFTP directory listings when VersaLex is the server on Linux.
- All negotiated algorithms are now logged at the beginning of each SFTP client and server session.
- Added support for the following SFTP algorithms: Public Key: ecdsa-sha2-nistp256, ecdsa-sha2-nistp384, ecdsa- sha2-nistp521, Key Exchange: ecdh-sha2-nistp256, ecdh-sha2-nistp384, ecdh-sha2-nistp521, and MAC: hmac- sha2-512, hmac-sha2-256-96, hmac-sha2-512-96. The new Public Key algorithms are available on the client side only, while the new Key Exchange and MAC algorithms are available on both client and server side (although server side only applies to VLTrader and Harmony). The new Key Exchange algorithms are not available in FIPS mode.
Enhancements - ebMS
- Added an option to ebXML to modify the format of the Content-Id header.
- Added new ebMS "Allow Incoming Request With Missing Role Element" advanced property, which when enabled allows an incoming request without a role element value to be processed if it otherwise matches a configured ebMS mailbox.
Enhancements - OFTP
- Added a new OFTP host advanced property "Allow Duplicate SFIDs". Setting this property to True allows files with duplicate SFIDs to be accepted and simply log a message if a duplicate is received.
- Added support for configuring EERP timeouts and resends at the OFTP host level through two new advanced properties: 'Async EERP Timeout (minutes)' and 'Async EERP Resends'. If these values are changed from default, they override the values set in the Local Listener. The REST API has been updated with these new properties and the OFTP property 'outgoing.signEerp' was moved to 'outgoing.receipt.sign'.
Enhancements - MQ
- Added support for the following MQ SSL cipher specs: ECDHE_RSA_AES_128_GCM_SHA256 and TLS_AES_256_GCM_SHA384.
Enhancements - SMTP
- Added the ability to configure individual SMTP Proxies to use Start TLS via the property 'Use Start TLS' in the proxy configuration screen. This property defaults to 'True' to match existing functionality.
Enhancements - File
- Cleo Harmony and Cleo VLTrader only: Added ability to the File: connector to run a 'Post Processing Command' that can execute a script or command after a file has been written.
Enhancements - SMB
- Cleo Harmony and Cleo VLTrader only: Added an option 'Force Make Directories' to the SMB connector to enable the connector to create any parent folders that do not exist for the destination of a file.
- Cleo Harmony and Cleo VLTrader only: Improved performance of the SMB connector by caching file attributes for all files in a specific folder for two seconds if more than five files are accessed from that folder within ten seconds.
Enhancements - S3
- Cleo Harmony only: Added three new optional S3 connection properties: 1)'User Metadata' can be used to specify metadata key/value pairs which are added to new S3 objects, 2)'Put Object Key' is an expression used to name new S3 objects, 3)'Force Unique' forces all new S3 objects to be uniquely named. After upgrade, refer to the S3 connection |S3| and |Info| tabs for more information.
- Added support for cross-account access using AWS's AssumeRole feature in the S3 Connector.
- Added new S3 connector property, AccessControlList (ACL), for cross-account use. This new property applies the selected ACL permissions on objects PUT to a bucket.
Enhancements - GCPBucket
- Cleo Harmony only: Added support for being able to use CMEK keys in GCP buckets.
- Cleo Harmony and Cleo VLTrader only: Fixed an issue with the GCPBucket connector where not all traffic would be directed through the specified proxy. Also, introduced support for multiple proxies in the GCPBucket connector.
Enhancements - AzureBlob/GCPBucket
- Cleo Harmony only: Like the S3 connector, the AzureBlob and GCPBucket connectors now support the Pseudo Folders property which indicates whether actual folder objects are created and required for holding file objects.
Bug Fixes - Framework
- Fixed an issue where cloning a connector host that has a 'System Scheme Name' defined would break directories using the original connector's 'System Scheme Name'.
- Fixed an issue where failed DocumentDB events on disk could be attempted continuously. These events are now moved to a subfolder to allow for investigation and corrective action
- Cleo Harmony and Cleo VLTrader only: TLS v1.2 is now supported when in FIPS mode.
- Fixed an issue where api/resourceFolders endpoint would fail after a change was made to a host through the command line. This would impact the WebUI displaying the hosts.
- Cleo Harmony and Cleo VLTrader only: For the database payload feature, removed unnecessary table identifiers in a SQL UPDATE statement that was causing a syntax error on Postgres.
- Upgraded BouncyCastle library to 1.70 and upgraded JCIFS-NG library to 2.17.
- Fixed a bug where placing & or && after LREPLACE or LDELETE commands would cause the action to fail when run through the REST API.
- Cleo Harmony and Cleo VLTrader only: Fixed a bug where, when using SAML with a custom authentication connector and the email address could not be found, the mailbox name would be displayed in Portal instead. Now, the nameID is shown if email address is not found. Also, added some debug that can be turned on by enabling debug on the custom auth connector.
- Fixed an issue where generating a User certificate with a DSA key would fail.
- Fixed an issue, introduced in 126.96.36.199, where importing a User certificate with a DSA key would fail.
- Fixed an issue where connecting to the WebUI through a HTTP/s port with FIPS mode enabled would cause the web browser to report a cipher error and prevent the page from loading.
- Fixed an issue where updating a host's certificate through REST API would set the host to 'Not Ready' when the certificate is a PGP key-generated certificate.
- Cleo Harmony and Cleo VLTrader only: Fixed an issue where using a connector as the inbox/outbox for a connector would result in a NullPointerException.
Bug Fixes - AS2
- Fixed an issue where, if the AS2 Receipt-Delivery-Option header contained a username:password in the URL, it would fail to send the MDN to the trading partner.
Bug Fixes - AS4
- Cleo Harmony only: Fixed an issue where a delay in deleting files pulled through AS4 would allow the same file to be pulled multiple times.
- Cleo Harmony only: Fixed an issue where AS4 transfers would fail if schema validation was enabled. This was corrected by adding additional schemas to the Harmony AS4/schemas folder.
Bug Fixes - FTP
- Fixed a problem where, if the FTP AUTH TLS command (or variant) should throw an exception and command retries are in effect, the command would not be re-invoked.
- Fixed an FTPs Active mode issue introduced in 188.8.131.52 where, when the 'SSL Maximum Protocol Version' was set below the new maximum of 'TLS 1.3', it would fail to find an open data port in the specified range or it would fail in SSL negotiation.
Bug Fixes - HTTP
- Fixed an issue where SSL connections could fail with a NullPointerException when SSL Debug was enabled.
Bug Fixes - SSH FTP
- Cleo Harmony and Cleo VLTrader only: Fixed an issue where deleting file/folders from an SSH FTP server concurrently using SSH_FXP_REMOVE could result in the file/folder not being deleted and a ConcurrentModificationException logged to the console.
- Fixed a potential SFTP server problem where a file stat request would not return a response. This could occur after a file upload, if a file stat request from a client occurred at the same time that the file was deleted or moved by the server.
- Fixed an issue where, if the trading partner's SSH server prematurely closed a client connection during the initial protocol version negotiation, the result could be excessive CPU usage up to the configured connection timeout.
- Cleo Harmony and Cleo VLTrader only: Fixed an issue introduced in 184.108.40.206 where a zero-byte file uploaded through SFTP would not be written to disk.
Bug Fixes - SMTP
- Cleo Harmony and Cleo VLTrader only: Added SMTP server debug for inbound content type filtering. If the allowed inbound content types are being restricted and SMTP debug is turned on, the content type is logged for each file being checked.
- Cleo Harmony and Cleo VLTrader only: Fixed an issue where the VLMailc utility did not support TLS version 1.1 or higher when negotiating a secure connection.
Bug Fixes - Portal
- Cleo Harmony and Cleo VLTrader only: Fixed an issue with SAML authentication where IDP-initiated login would sometimes fail when using a Chromium-based browser.
- Cleo Harmony and Cleo VLTrader only: Fixed an issue where a user password change through Cleo Portal could be lost if an admin was updating the user's mailbox at the same time.
Bug Fixes - File
- Cleo Harmony and Cleo VLTrader only: In the File connector, for Windows, the DIR command no longer shows inaccessible directories.
Bug Fixes - SMB
- Cleo Harmony and Cleo VLTrader only: Fixed a small memory leak in the SMB connector when sending or receiving an SMB file.
- Cleo Harmony and Cleo VLTrader only: Fixed an issue where the SMB connector would fail when connecting to AS400 IFS SMB shares with the following error: "TreeID is invalid".
- Cleo Harmony and Cleo VLTrader only: Fixed a bug where VersaLex would not start up if FIPS was enabled due to an issue with the SMB connector. Also, fixed an issue with the SMB connector connecting to shares in FIPS mode.
Bug Fixes - S3
- Cleo Harmony only: Fixed a problem where an S3 directory listing would be truncated at 1000 objects.
- Cleo Harmony only: Fixed an issue where temp files could remain after a transfer when using the S3 connector on certain operating systems.
- Cleo Harmony only: Fixed an issue with the S3 connector that could prevent the UI from starting correctly.
- Cleo Harmony only: Fixed an issue where if a special character, such as a colon ':', were used in an S3 path, then a directory listing and subsequent wildcard GET would fail.
- Cleo Harmony only: Fixed a memory leak that occurred when transferring files with the S3 connector.
- S3 connectors can now be used as send/receive archive directories for local user hosts.
Bug Fixes - AzureBlob
- Fixed a bug where SFTP transfers would hang if the file was an AzureBlob and the client tried to set the file time.
- Fixed an issue where users would not be able to CD into a subdirectory of an Azure Blob connector when the Azure Blob container was set up as Data Lake Storage.
Fixed issues in version 220.127.116.11
Enhancements - Framework
- Added support for getting/setting all applicable connector host advanced properties through the REST API.
Enhancements - Portal
- Added a time picker to the Portal Transfers page so transfers can be filtered by date and time.
Bug Fixes - Framework
Fixed an issue where user mailboxes using LDAP connectors were sometimes counting an extra user against the license. This could potentially cause some licensed mailboxes to be automatically disabled.
Fixed an issue where the DocumentDB would not start correctly if the system did not have access to the internet. Also, fixed an issue where spaces in the directory path for VersaLex on Windows would cause the DocumentDB to not start.
Fixed a problem when generating an X509 certificate with or from an OpenPGP keyring where the master key expiration was not being set.
Fixed a problem when re-receiving a transfer that was locally packaged where the content would be locally packaged a second time (i.e. double encrypted).
Fixed an issue where including non-ASCII characters in the VLTransfers.ResultText database field could cause the value to be too large for the database. All entries are now truncated to the correct length regardless of included chars.
Bug Fixes - SSH FTP
- Fixed a problem during diffie-hellman-group-exchange-sha256 key exchange where VersaLex was incorrectly ignoring a reply message, causing the next message read to be unexpected and resulting in an InvalidMessageException.
Bug Fixes - S3
- Fixed an issue that could cause a BadDigest when uploading files from the S3 connector if the incoming buffer is not divisible by 1024 bytes.
Security - Framework
- Fixed an issue where clients were able to negotiate elliptical curve ciphers outside of the VersaLex Local Listener settings. Also removed deprecated named elliptical curves from the Local Listener according to RFC 8422. Lastly, VersaLex now honors the existing Local Listener advanced property "SSL Ignore Client Cipher Preference Order" for elliptical curve ciphers as well. Note: VLProxy 18.104.22.168 is required if using VLProxy.
Fixed issues in version 22.214.171.124
Major Enhancements - IBMMQ
- Added a new connector to allow integration with IBM MQ. Refer to the Info tab within the Templates > Generic > Generic IBMMQ host for more details.
Major Enhancements - SharePoint
- Added a new connector to allow integration with Microsoft SharePoint. Refer to the Info tab within the Templates > Generic > Generic SharePoint host for more details.
Enhancements - SSH FTP
- Added support for ECDSA and Ed25519 algorithms during SFTP key authentication for both client and server connections. ECDSA and Ed25519 keys can be imported or generated, but note that these can only be used with SFTP. Ed25519 is not supported in FIPS mode.
Note: VLProxy 126.96.36.199 is required if using Cleo VLProxy.
Enhancements - Kafka
- Added four properties, 'SASL Mechanism', 'SASL Security Protocol', 'Username' and 'Password' to the Kafka Connector used to support PLAIN, SCRAM-SHA-256 and SCRAM-SHA-512 SASL mechanisms. The Kafka Connector Receiver was also updated to properly start and stop based on connector settings. Updated Kafka library to version 3.3.1. Due to this upgrade, the previous 'Client Dns Lookup' default value of 'default' has been deprecated. If this value is currently configured, the setting must be changed to either 'use_all_dns_ips' or 'resolve_canonical_bootstrap_servers_only' in order for the Kafka connector to function.
Bug Fixes - Framework
- Fixed an issue where 'System Scheme Name' property on a connector host would be cleared when syncing to another node.
- Fixed an issue where, when the VLProxy Remote Read Timeout is set higher than 150 seconds, Cleo VLProxy reverse forward connections would error out on VLProxy after 150 seconds with an IOStreamConnector exception.
Note: VLProxy Remote Read Timeout should not be set higher than the Local Listener FTP Idle Timeout, as this can also cause IOStreamConnector exceptions on Cleo VLProxy.
- Fixed a problem where ExecuteOn for a specific mailbox was being limited to three concurrent execution threads (e.g. ExecuteOnSuccessfulReceive for a user mailbox).
Bug Fixes - SSH FTP
- Fixed a problem where the SFTP server was producing an error with each file left open on a session end. Now any open files on session end are just ignored.
Bug Fixes - AS4
- Fixed a problem that occurs when parsing an AS4 SOAP envelope. The following exception was logged when the problem occurred: "The matching wildcard is strict, but no declaration can be found for element 'ec:InclusiveNamespaces'"
Bug Fixes - Users
- When a user is cloned, the email address value is now cleared since an email address cannot be repeated.
Fixed issues in version 188.8.131.52
Bug Fixes - AS4
- Fixed a problem in the AS4 service where using the Subject Key Identifier Security Token Reference Type could result in false failures.