September 19, 2019 – Alert: Possible Query, Compilation, and Data Access Issues with Unicode Character 223 (ß)

InterSystems has corrected a defect in applications that use Unicode character 223 (ß). This defect can result in incomplete query results, class compilation errors, and removal of custom SQL privileges.

This problem occurs on systems that are running or have previously run on:

• Caché and Ensemble 2018.1.0, 2018.1.1, and 2018.1.2
• HealthShare Health Connect (HSAP) 15.032 on Core versions 2018.1.0, 2018.1.1, and 2018.1.2
• HealthShare Health Connect 2019.1
• InterSystems IRIS data platform – all currently released versions
• InterSystems IRIS for Health – all currently released versions

The defect is triggered by data and component names containing Unicode character 223 (ß). In the versions listed above, an uppercase conversion incorrectly maps that character to Unicode character 7838 (ẞ).  Applications perform this uppercase conversion using features such as $ZCONVERT and %SQLUPPER.

Problems can occur when accessing data or classes created or modified on a product with a different uppercase conversion than the one currently in use.

Mitigating Issues with Incomplete Query Results

This issue can cause problems in applications that use uppercase conversion. Also, InterSystems SQL uses uppercase conversion by default for indices, so queries on a dataset containing this character can return incorrect results.

To address this defect, InterSystems recommends that you rebuild all indices after upgrading from an unaffected version to an affected version, and vice versa.

Mitigating Issues with Class Compilation and Data Access

For classes with names or components that contain the ß character, the defect can also cause class compilation to fail in one of several ways.

In one case, compilation can fail with a message such as:

ERROR #5503: Field name is invalid: Field (Prop1ö) is not unique within Table (ISC.ße). (Prop1ö)

In a second case, compilation can report success but actually delete the table. In this situation, there are messages during compilation such as:

Dropping orphaned table: ISC.ße

Dropping orphaned procedure: ISC.ẞE_EXTENT

These messages indicate that the table has been dropped, though its underlying data is not removed. In this case, any custom SQL permissions on that table will be lost, causing data access issues.

For this situation, InterSystems recommends the following solution: After upgrading to or from an affected version, but prior to compiling any affected classes, export any users and roles that have been granted SQL permissions on any affected tables. To do this, use either the ^SECURITY routine or the Export class methods of the Security.Users and Security.Roles classes. Make sure to include SQL privileges, which are not included by default in the export via either of these methods. (For more information, see the article “Tips on exporting and importing security settings” on the InterSystems Developer Community.)

If the table is dropped during compilation, you can re-import the users and roles after successfully compiling the class, either via ^SECURITY or the corresponding Import class method.

Obtaining an Update and More Information

The correction for this defect is identified as JLC2212, which will be included in all future product releases. It is also available via Ad hoc change file (patch) or full kit distribution from the WRC. If you have any questions regarding this alert, please contact the Worldwide Response Center (WRC).