What's new in version 5.7
Technology Refresh
In Version 5.7, we have refreshed underlying technologies in Cleo Harmony. These upgrades include support for TLS 1.3 and replacing Oracle 8 JRE with OpenJDK 8.
AS4 Enhancements
AS4 support in Cleo Harmony has been enhanced in the following ways:
- Certificate exchange capability included.
- Cleo Harmony is certified as an AS4 eDelivery conformant solution.
- User Messages and Pull Request Signal (inbound and outbound) can now include
<wsse:UsernameToken>
elements for outbound messages and authenticate for inbound messages. - A GET command has been added to support the use of Pull Request Signals and User Messages.
- The advanced property, Use MIME Packaging For Signal Messages, has been added to allow Receipt Signals, Error Signals, and Pull Request Signals to be encapsulated within a MIME package.
- Other AS4 enhancements include Asynchronous Receipt Signals, automatice retries of User Messages that are awaiting an asynchronous Receipt Signal, and the ability to send and receive a payload within the
<eb:Body>
element of a User Message.
S3 Enhancements
S3 support in Cleo Harmony has been updated to include the following enhancements:
- Support for using AWS credentials that have been stored locally.
- The S3 connector now has a "Pseudo Folders" property, where Cleo Harmony creates actual folder objects when asked to make a directory and require actual folder objects to exist when asked to list a directory
- Added the ability to perform multipart multithreaded transfers to S3.
JMS Enhancements
JMS support has been enhanced in Cleo Harmony.
- Added support to the JMS URI to allow the filename to be determined by multiple properties.
- Added support for the TextMessage JMS message type in the JMS URI.
Connector Enhancements
Connectors that support directory structures, such as smb:
, can now be used as send/receive archive directories for local user hosts. In addition, SMB and AzureBlob support has been expanded.
SMB
- With this release, using an SMB Connector as send and receive archives for local user hosts is compatible with disabling the system-level property, Sent/Received Box Archive.
- SMB 3 now supported.
AzureBlob
- File movement to a different container with the same Azure storage account and file movement within the same Azure container are performed natively to avoid the extra overhead associated with performing these operations oustide the Azure network.
- The ability to authenticate using a key in the Azure Key Vault is available in this release.
- The maximum length of the common System Scheme Name connector property has been increased from 8 to 24 characters to match the maximum allowed for the actual connector scheme names.
SSH FTP
Cleo Harmony now allows a trading partner SFTP client to leave files or directories open on session end and now automatically close an open file or directory if the SFTP client removes a file or directory while it is open. In addition, performance of SSH FTP transfers when using AES ciphers has been improved.
Security Enhancements
Expired and retired trusted CA certificates will not be installed for new Cleo VersaLex installs. The expired certificates will remain intact for Cleo VersaLex upgrades. Some trusted CA certificates have been updated with new versions.
With this release, Cleo products have been enhanced to use Bouncy Castle libraries version 1.66.
Future-proofing your Cleo Harmony program
Deprecating features
The following features are being deprecated in future releases. You can future-proof your program by using newer features instead of the deprecating ones. A warning message has been added to each of the deprecated features' panels with a suggestion of a newer feature to use instead:
-
Win Unix/File System > CIFS Directories – use SMB hosts instead
-
Router – use Router hosts instead
-
VLPortal – use Portal instead
-
Options > LDAP / User Management > LDAP Settings – use LDAP hosts instead
-
Local FTP Users – use Users hosts instead
-
Local SSH FTP Users – use Users hosts instead
Upgrading to version 5.7
Recommendations for Upgrading
When upgrading to Cleo Harmony version 5.7, Cleo recommends the following:
- Back up your configuration using the Export functionality. In the Web UI, go to Administration > System > Export. In the native UI, go to File > Export. Performing an Export will save your data in a format that you can import using the Cleo Harmony Import functionality should the need arise.
- Make sure your system meets the system requirements for Cleo Harmony version 5.7, as it requires greater resources than earlier versions. All new installs must be 64-bit. Visit Cleo Harmony 5.7 System Requirements to view the System Requirements for your product.
- Because this release of Cleo Harmony uses OpenJDK, if you are using the Web UI on a Unix system, you might need to install the latest fontconfig. The command is dependent on the flavor of Unix you are using.
For example:-
- Red Hat:
yum install fontconfig
- Ubuntu:
apt-get install -y --no-install-recommends libfontconfig1
- Red Hat:
-
- Run the Cleo Harmony 5.7 installer to perform an in-place upgrade. Your data and configuration remain intact from the previous version of the Cleo Harmony software.
Update history
The following sections contain descriptions of issues fixed in Version 5.7 and subsequent patch releases:
Fixed issues in version 5.7
Enhancements - Framework
- Cleo Harmony and Cleo VLTrader only: Fixed an issue where Harmony/VLTrader would become unresponsive when the Document DB in-memory queue is full. Now, when the in-memory queue is full, events queued up for the Document DB are written to disk. When the Document DB becomes available again, events are read from disk and inserted into the database. The Replicate Event Log Queue Size is now the number of events that are in memory and on disk before the system logs an error and sends email notification to the administrator.
- Replaced Oracle 8 JRE with OpenJDK 8 due to licensing costs associated with using the Oracle JRE.
- Cleo Harmony and Cleo VLTrader only: Added finer control over certificate expiration notifications. Separated server certificate expiration notifications from the existing Email Local And Partner Certificate Expiration options into the Email Server Certificate Expiration options. Also added the ability to control the frequency of the certificate expiration notifications with the Email (Server/Local And Partner) Certificate Expiration Notification Frequency Days setting. For more information see the Cleo Harmony or Cleo VLTrader User Guide.
- Added support for Transport Layer Security (TLS) Protocol Version 1.3.
Note: When the
SSL Maximum Protocol Version
is left blank, the software will attempt to use TLS 1.3. - Added two new system-level other properties: Email And Execute On Resolution and Email Local And Partner Activation Notifications. Both of the new properties default to
true
, which matches previous behavior. If Email And Execute On Resolution istrue
and Email/Execute on Repetitive Failures is turned off, when the failure is resolved, an email is sent and/or execute on is performed. Email And Execute On Resolution applies to all three levels of Email/Execute On Repetitive Failures. If Email Local And Partner Activation Notifications is true, when a scheduled certificate is activated, an email is sent to the system administrator. - Cleo Harmony and Cleo VLTrader only: Cleo Harmony and Cleo VLTrader will no longer alert prematurely on event replication issues while it attempts to "self heal" the document DB. In other words, document DB errors and exceptions will not be emailed or executed on fail while VersaLex attempts to correct the problem. If the problems persist, the replicate event log queue full error will eventually result, which is still alerted on.
- Addressed an issue where
FileNotFoundExceptions
were being thrown because scheduled autosend files were temporarily unstable due to a slow underlying file system. Now, when running scheduled actions, these files are bypassed, avoiding unnecessary exceptions and Email-On-Fail emails. - Cleo Harmony and Cleo VLTrader only: Added a new Password Expirations standalone javascript action template. This action can be used to send emails to the set of users whose password is about to expire or is expired, where the email includes a link to Portal for the user to change their password. This action can also optionally send a summary email to the system administrator(s) listing the users who were emailed and their password status.
- Cleo Harmony and Cleo VLTrader only: Updated the static VLTransport database table such that LCOPY commands are considered to be transfers. This allows LCOPY records in the VLTransfers table to be displayed in Cleo Dashboards.
- Base release notes will be appended to the patch notes that are included within each patch, as stored in the conf/notes.txt file.
- Converted host-level Advanced property,
SSL Cipher
, to a regular expression field. Now, users can enter regular expressions (enclosed in brackets) or wildcard expressions to restrict the list of ciphers presented to the SSL server. The ability to specify only a single cipher is still possible, however, the UI has been improved to make this selection easier. Refer to the user's guide for detailed information on the usage of this property. - Improved wording for messages related to actions that are temporarily blocked from scheduler processing, especially as it relates to failed actions due to slow file systems.
- Added the ability to create and edit OFTP connections via the
/connections
REST endpoints. - An overrun of the data segment in the SSL/TLS handshake and a resulting failed inbound connection during SSL/TLS handshake could occur when both of the following are true: 1) any of the HTTP/s, FTP/s, SMTP/s or OFTP/s client authentication settings are enabled in the Local Listener and configured to
Accept all Certificate Manager Trusted CA certificates
; 2) the number of installed trusted CA certificates causes the byte length of the Distinguished Names of those trusted CA certificates to exceed 65535 bytes. To help diagnose this problem, a log message and email notification will now be sent when the byte count exceeds 65535 bytes. An example of that message would be: "There are 2300 trusted CA certificates with 65570 total bytes registered for HTTP/s client authentication. This exceeds the maximum threshold of 65535 bytes and may cause inbound connections to fail during the SSL/TLS handshake. You should remove unused trusted CA certificates to get that byte count below the maximum threshold." After you remove the unused CA certificates, you must restart the Local Listener for those changes to take effect. Once the byte count falls below the maximum threshold, another log message and email notification will be sent. An example of that message would be: "There are now 1200 trusted CA certificates registered for HTTP/s client authentication with 43456 total bytes and is below the maximum threshold. No further action is necessary at this time."
Enhancements - SSH FTP
- Cleo Harmony and Cleo VLTrader only: Harmony/VLTrader now allows a trading partner SFTP client to leave files or directories open on session end. In the past, this would result in errors at the end of the session, but it now results only in warnings. Also, Harmony/VLTrader now automatically closes an open file or directory if the SFTP client removes a file or directory while it is open.
- Improved performance of SSHFTP transfers when using AES ciphers.
Enhancements - OFTP
- Now block sending and receiving of already expired certificates through ACE.
- When an installed local or partner certificate is expired and there is a valid secondary certificate available that had previously been exchanged through ACE, the next secondary certificate for the specified usage(s) will be rolled over as the installed certificate before an OFTP message is either sent or received. In synced environments, certificates will be updated only on the node where the rollover has occurred to avoid syncing collisions. Each node will subsequently be updated during its own OFTP data exchange.
- When a new user certificate is sent through Automatic Certificate Exchange (ACE) in either a replacement or rollover scenario, attributes of the currently installed certificate are now included in the SFIDDESC field of the SFID message. These attributes can then be used by the receiver to implicitly trust the new certificate based on the trust of the currently installed certificate. Additionally, when an OFTP partner certificate is replaced through ACE, the certificate is archived and removed as long as it is no longer in-use by any other trading relationship.
- Today, the VersaLex SSL server automatically rejects expired client certificates. This capability has been expanded to the client side. To facilitate this capability, a new host Advanced property was added: "SSL Reject Expired Certificates". When set, if the client receives an expired certificate from the server, it will be rejected and the SSL handshake will be terminated.
- Added two new features to OFTP SSL client authentication, both facilitated through new switches under the client authentication configuration of the Local Listener OFTP tab: "Optional" and "Verify Key Usage". "Optional", when set, indicates that the SSL client certificate should be requested by the server, but should not be required. "Verify Key Usage", when set, indicates that the server should validate that the provided client certificate contains the "clientAuth" extended key usage setting. Note: VLProxy 3.9.0.0 is required if using VLProxy.
- Renamed OFTP Mailbox > V2 > Require Authentication to OFTP Mailbox > V2 > Secure Authentication. This better describes the purpose of this switch which now controls both initiator and responder action regarding secure authentication. This setting controls what is placed in the SSIDAUTH field (Y/N) when sending and responding. It also is used by the responder to enforce compliance with RFC 5024, which states the secure authentication must be set to the same value for both the initiator and responder.
- Added a new advanced property 'Validate String Characters For Inbound Message Fields'. When this property is enabled, VersaLex validates that the incoming values for SSID and SFID string fields only contain the following characters: 0-9 A-Z / - . & ( ). Note: VLProxy 3.9.0.0 is required if using VLProxy.
Enhancements - Portal
- Cleo Harmony and Cleo VLTrader only: Improved performance of Cleo Portal when there is a large number of files in a folder.
Enhancements - Connector
- Cleo Harmony and Cleo VLTrader only: Increased the maximum length of the common 'System Scheme Name' connector property from 8 to 24 characters to match the maximum allowed for the actual connector scheme names.
- Cleo Harmony only: Enhanced the AzureBlob connector so that when a file is moved to a different Azure container within the same Azure storage account, it is done natively so that we don't incur extra costs by leaving the Azure network.
- Cleo Harmony only: Added the ability in the Azure Key Blob Connector to authenticate using a key in the Azure Key Vault. See the Info tab on the connector for more details.
- Cleo Harmony only: Enhanced the AzureBlob connector so that when a file is moved within the same Azure container, it is done natively so that we don't incur extra costs by leaving the Azure network.
- Cleo Harmony and Cleo VLTrader only: Added SMB 3 support to the SMB connector.
Enhancements - ICAP
- Cleo Harmony and Cleo VLTrader only: Added support in the ICAP connector for the REQMOD method. Previously only the RESPMOD method (which is the default) was supported.
Enhancements - AS4
- Cleo Harmony only: Added certificate exchange capability to AS4. Similar to other protocols, this capability is initiated through the Exchange Certificates button that is located on many panels.
- Cleo Harmony only: Added capability for PING operations in accordance with the eDelivery "Test Service" feature.
- Cleo Harmony only: Added a new advanced property 'Use MIME Packaging For Signal Messages'. When enabled, all Signal Messages (that is, Receipt Signals, Error Signals, and Pull Request Signals) are encapsulated within a MIME package.
- Cleo Harmony only: Added the ability to include <wsse:UsernameToken> elements within outbound User Messages and Pull Request Signals. Also added the ability to authenticate inbound User Messages and Pull Request Signals that contain <wsse:UsernameToken> elements. These abilities are both governed by the PMode.Security.PModeAuthorize setting.
- Cleo Harmony only: Added support for sending/receiving a payload within the <eb:Body> element of a User Message.
- Cleo Harmony only: Added support for a GET command. A GET command will issue a Pull Request Signal to the trading partner. If the trading partner has a payload in its queue, it will respond with a User Message. Also added inbound support for Pull Request Signals. If a Pull Request Signal is received, and a payload is available in the outbox, the payload is returned to the trading partner, packaged within a User Message.
- Cleo Harmony only: Added automatic retries of User Messages that are awaiting an asynchronous Receipt Signal. This capability is managed through the PMode.ReceptionAwareness settings.
- Cleo Harmony only: Added support for asynchronous Receipt Signals. This ability can be initiated by setting PMode.Security.SendReceipt.ReplyPattern to 'Callback'. Also added support for asynchronous Error Signals. This ability can be initiated by deselecting PMode.ErrorHandling.Report.AsResponse.
- Cleo Harmony only: Added provisions to support eDelivery, version 1.13. Cleo Harmony is now officially certified as an AS4 eDelivery conformant solution. See https://ec.europa.eu/cefdigital/wiki/display/CEFDIGITAL/eDelivery+AS4+conformant+solutions#eDeliveryAS4conformantsolutions-CLEO.
Enhancements - S3
- Cleo Harmony only: The S3 connector now supports using AWS credentials that have been stored locally.
- Cleo Harmony only: Because S3 is a flat file system that does however support the concept of folders for grouping objects, the S3 connector now has a 'Pseudo Folders' property. Only when this property is set on (the default) will Harmony create actual folder objects when asked to make a directory and require actual folder objects to exist when asked to list a directory.
- Cleo Harmony and Cleo VLTrader only: Added the ability to perform multipart multithreaded transfers to S3.
- Cleo Harmony only: Added new S3 connector property, 'Enable Path Style Access', which enables path-style S3 URLs instead of virtual hosted-style S3 URLs. Path-style URLs are being deprecated by AWS, so this setting should be configured only for non-standard S3 connections.
Enhancements - JMS
- Cleo Harmony and Cleo VLTrader only: Added support to the JMS URI to allow the filename to be determined by multiple properties. This extends the 'filenameProp' URI property to allow a syntax of filenameProp=property1+property2+property3, which then produces a filename like value1+value2+value3. If desired, a filenameSeparator URI property can also be set, which overrides the use of + in the constructed filename. Note that the list of property names in filenameProperty always uses + since these property names must be Java identifiers and do not contain a +.
- Cleo Harmony and Cleo VLTrader only: Added support for the TextMessage JMS message type in the jms: URI. TextMessage can be specified by using a msgType=text parameter on the URI. Previously, only BytesMessage was supported.
Enhancements - General
- Cleo Harmony and Cleo VLTrader only: Added new local listener "Email And Execute On Unknown Trading Partner Failures" advanced property, which works in conjunction with the existing "Email On Fail" and "Execute On Fail" properties. If this new property is set to false (defaults to true), only inbound transfer failures associated with a known mailbox result in the configured email and/or execute on fail being invoked. This eliminates unnecessary emails/executions related to cyberattacks, and applies to all the local listener protocols and services.
Bug Fixes - Framework
- Cleo Harmony and Cleo VLTrader only: Fixed an issue where transfers with receipts would show duplicate events within the Transfers view.
- Removed unsupported PSK ciphers from SSL cipher suites.
- Fixed an issue where user certificate private keys exported with Base64 encoded PKCS #8 (.PEM) format had incorrect header and footer values.
- Cleo Harmony and Cleo VLTrader only: Fixed an issue where overlapping temporary actions off of the same base action would cause logged thread ids for the running actions to become corrupted. This would result in event and transfer logging being inaccurate. This problem could happen with triggers in particular.
- Cleo Harmony and Cleo VLTrader only: Fixed an issue where accessing the REST endpoint /api/connections/<connectionId>/transfers would sometimes return transfers not associated with the <connectionId> when used with certain protocols that have asynchronous receipts.
- Cleo Harmony and Cleo VLTrader only: Fixed an issue in the various Users host configuration pages where the ellipsis button [...] in the Archive Directories section would not display the correct folder.
- Cleo Harmony and Cleo VLTrader only: Fixed an issue where result text containing binary characters could overrun the size of the resultText field in the VLTransfers database table.
- Added support for the %transferid% macro in the destination filename field of the PUT and GET commands for FTP and SFTP. Also added support for %transferid% within the destination filename field of LCOPY commands.
- Added more detailed messaging around upgrading through our product. We are now specifying that you need to run the native UI as an admin user in Windows to upgrade through the product. Related incident #305910.
- Fixed an issue where certain upgrades to Cleo Harmony/VLTrader/LexiCom without updating Cleo VLProxy would cause VLProxy communications to become non-functional.
- Fixed an issue where passwords that start with "#" or "*" were not always handled correctly. Please note that passwords that begin with "#" or "*" should not be escaped by adding an extra "#" or "*". Rather, the passwords should be entered literally.
- Fixed an issue where certain functions (such as moving a host) would fail when accessing the WebUI through Cleo VLProxy.
- Fixed an issue where updating to an incorrect Local Signing or Encryption certificate in a running Local Listener would prevent SSHFTP and FTP users from logging in.
- Improved possible performance issues when querying Microsoft SQL Server databases through a Microsoft JDBC driver.
- Right clicking a host in the classic mode Web UI and selecting "Transfer Report" now automatically selects the host in the host list. Also, generating a transfer report from the mailbox level in the same manner will now show the mailbox-level selection view immediately.
- Fixed an issue where temporary actions were being written to top.xml unnecessarily causing delays in processing.
- Cleo Harmony and Cleo VLTrader only: Fixed an issue where the /connections endpoints would occasionally append '.0' to integer values.
- Cleo Harmony and Cleo VLTrader only: Fixed an issue where Sentbox Archive and ReceivedBox Archive in a Users host would write files to the install directory if they were set to the "%none%" macro.
- Cleo Harmony and Cleo VLTrader only: Fixed an issue where, if a user host has an SMB connector as the Home Directory and an Icap connector is scanning its incoming files, an exception would occur when a file was uploaded to the user.
- Fixed an issue where using the "All" button to select Trading Partner/CA Certificates in the Export window would improperly populate the list with duplicate entries, which would then produce an unusable export filter.
- Fixed a problem where the concatenated file size (for example, "10+20") was being reported for the %filesize% macro when placed in an 'Execute On Check Conditions Met' string for a multi-file result. Now the concatenated string is split apart for each file (for example, "10" and "20").
- Cleo Harmony and Cleo VLTrader only: Fixed an issue where VLProxy would not be sent updated configuration information if all the related hosts were disabled or not ready.
- Cleo Harmony and Cleo VLTrader only: Fixed a REST API issue where a connection's or user's partner or local packaging encryption certificate would be missing if the certificate was generated from an OpenPGP key.
Bug Fixes - FTP
- Fixed an issue where the "RESULT" log was missing when a FTP GET action failed on a CD command.
Bug Fixes - AS2/AS3
- Fixed an issue in the AS2 receiver where, if an asynchronous MDN was requested and the AS2 relationship was unknown, the MDN would not be sent and there was no Result logged.
- AS3 only: Fixed a problem where a NullPointerException would be logged if the AS3 partner included an "AS3-Version: 1.0" MIME header in their message payload.
Bug Fixes - SSH FTP
- Fixed an issue where, if an optional comment is returned from the SSH FTP server during version negotiation, SSH_MSG_KEX_INIT would fail with an Invalid Packet Size exception.
- Cleo Harmony and Cleo VLTrader only: Fixed an issue where, if a URI path was configured for the user home folder, that folder structure could be incorrectly created under the installation folder.
- Cleo Harmony and Cleo VLTrader only: Fixed issue where the SSH FTP server could prematurely close open directories in a user session under load.
- Fixed an issue where incorrect permissions were sent when retrieving a file from an SSHFTP server.
Bug Fixes - OFTP
- Fixed an issue where OFTP connections that use the option 'Send files when partner initiates connection' would cause a memory leak.
- Cleo Harmony and Cleo VLTrader only: Fixed an issue where the OFTP inbound transfer report showed the wrong host in the new Web UI.
- Cleo Harmony and Cleo VLTrader only: Fixed an issue with the OFTP property 'Validate String Characters For Inbound Message Fields' when performing certificate exchanges. A validation error is no longer thrown on CERTIFICATE_DELIVER, CERTIFICATE_REPLACE, and CERTIFICATE_REQUEST functions.
- Fixed an issue where a FileNotFoundException could occur on a temporary file while processing an incoming OFTP message.
- Fixed an issue where a temporary file wasn't being closed causing a two-minute delay while processing an OFTP message.
- Fixed a problem where individual entries in the ACE list for specified usages were not able to be properly deleted using the right-click ‘Remove’ option.
Bug Fixes - RNIF
- Fixed an issue where Inactive RNIF Message IDs were unnecessarily written to disk causing performance issues.
Bug Fixes - Portal
- Cleo Harmony and Cleo VLTrader only: Fixed an issue in Cleo Portal where, in Firefox only, after downloading a file by clicking on its name, the loading spinner on the page would become stuck and require a page refresh to continue using the application.
Bug Fixes - Connector
- Cleo Harmony: Storage connectors can now be cloned or deleted only if Unify in Portal is licensed and enabled.
- Cleo Harmony and Cleo VLTrader only: Fixed an issue where processing a TA1 Acknowledgment document through the Router connector would result in a NullPointer Exception.
- Fixed an issue where non-ASCII segment-terminating characters would cause the Router connector to throw an exception.
- Connectors that support directory structures, such as smb:, can now be used as send/receive archive directories for local user hosts.
- Cleo Harmony and Cleo VLTrader only: Fixed a problem where an SMB connector directory listing could fail if there was an invalid character found in one the filenames.
- Cleo Harmony and Cleo VLTrader only: Fixed an issue where actions using the Clarify connector would fail if they were scheduled for polling and the action was run on a synchronized system.
- SMB connectors used as send/receive archive directories for local user hosts no longer throw an error if the system-level other property "Sent/Received Box Archive" is disabled.
Bug Fixes - HSP
- Cleo Harmony and Cleo VLTrader only: Fixed an issue where HSP transfers would fail if additional cookies were present.
- Cleo Harmony and Cleo VLTrader only: Fixed an issue with the HSP protocol where the server would not fail a transfer that the client had failed resulting in errors when the client attempted to resend the file.
Bug Fixes - General
- Cleo Harmony and Cleo VLTrader only: Removed the logged result for low-level inbound connection failures where a trading partner has not been identified and now just log the exception. This keeps "Email/Execute On Fail" from being invoked unnecessarily and reduces alert noisiness.
Security - Framework
- Expired and retired trusted CA certificates will not be installed for new Cleo VersaLex installs. The expired certificates will remain intact for Cleo VersaLex upgrades. Some trusted CA certificates have been updated with new versions.
- Cleo Harmony and Cleo VLTrader only: Fixed a security issue where a partial web admin UI could be accessed over HTTP when a secure port is required to access the Web Admin UI. The user will now be redirected to the secure port. Note: VLProxy 3.8.2.2 is required if using VLProxy. Related incident #400392.
- Upgraded the Bouncy Castle libraries to version 1.66. This includes necessary updates to Cleo software.