September is always a busy month for iOS developers because this is when Apple traditionally releases a new version of iOS to the public. By this stage we’ll have spent the summer months testing against a series of beta releases in an effort to ensure our software is fully compatible on day one. Users of Apple’s mobile platform are famed for being quick to upgrade to the latest version of their operating system, so we have to be just as quick to make sure our apps are ready for it.
Unfortunately there’s no guarantee that iOS issues will be identified before release day given that Apple make changes right up to the last minute. Indeed this was the case last week, when it came to light that initial versions of iOS 11 suffer from a severe bug that affects hundreds of thousands of apps in the App Store.
The issue in question relates to Smart Punctuation, a feature new to iOS 11 which automatically inserts typographically correct quotes and dashes. So for example, if the user enters two dashes, iOS will conveniently convert them to a single en dash (–). While some have complained as to the exact characters Apple chooses to substitute, the feature seemed to be working well until the developer of SongSheet, Gabriel Hauber, noted that if the user entered three dashes in a row iOS not only converted the text to an em dash (—), but also inserted a null character afterward.
Not So Smart Punctuation
Why is this a problem? Well, a null character has special significance in the C programming language: it’s used to indicate the end of a series of text characters, known as a string. So if null characters are spuriously added to text input by a user, the operating system will terminate the text at the first null character it comes across.
On iOS, apps are at risk whenever NSStrings (Apple’s string container which doesn’t use null characters for termination) are converted into plain C strings (which do). Since Apple’s Core Data persistence framework does just that, any app which uses Core Data as its database is at risk of not capturing all of the user’s content should they tap that dash button three times. In fact, this can be seen in Apple’s own Notes app:
Since Bipsync uses Core Data for local data persistence we were naturally concerned, and immediately started work on a patch to prevent the problem from manifesting in our app. This patch was released within 24 hours of the problem coming to light, in version 1.22.3 of Bipsync Notes for iOS. We encourage all our users to upgrade to this version, or to at least disable the Smart Punctuation feature until Apple has addressed the issue themselves, which we’re still waiting for one week on.
Recovery and Back Ups
To date we’re aware of one user experiencing problems relating to this issue, and we were able to recover the affected note from a backup on the device in short time. Of course the best disaster recovery process is the one you never have to use, but it was gratifying to see that the backups we have in place were able to cope with a scenario which we could not have realistically foreseen, and which many other apps have no solution for.
The interesting thing about this issue is that because it only becomes a problem when a string is cast into a plain C string, any process which stores the pre-cast string in an alternate way (e.g. as a transformable property in Core Data, or as a JSON file on disk) will have access to the non-truncated version – any null characters are never given the opportunity to influence what is stored.
Bipsync uses both those alternate storage methods mentioned above. Before we write any note content to Core Data we store a temporary backup on disk in JSON format, just in case the write fails. If the write succeeds we remove the file, but if it doesn’t, we’re able to go back and restore the note content from the file at a later date. This is our first level of backup, which accounts for rare events like database corruption.
From the Archives
Our second level of backup, which is most relevant to this Smart Punctuation issue, stores the note data in a serialised format. We keep a version history of edits to notes in the Core Data database, and the entity that stores that data uses a property of the ‘transformable’ type – this uses NSCoding to convert an object (in this case an NSDictionary) into an archived format that can be later converted back to its original type.
It was from here that we were able to recover our user’s note following the iOS 11 issue. From an encrypted backup we inspected the data and determined that although null characters had been embedded into the note content, the remainder of the data was intact. From here it was a case of unarchiving the dictionary and removing all null characters from its string properties before using it to restore the note as a new version. A quick reload of the app later, and the note was recovered and available.
For our clients, operating in the highly regulated investment management environment where their research content is valuable IP, ensuring effective disaster recovery, back-up and business continuity strategies are in place is critical. It’s why compliance checks and effective due-diligence with technology suppliers take precedent. This is true whether it’s a zombie apocalypse or an iOS bug you’re facing.
While Apple is still yet to address this particular issue, or even acknowledge it officially, we must assume that a fix is imminent. We certainly hope that’s the case, as many third-party apps aren’t even aware of the danger that Smart Punctuation poses, let alone working to combat it. In the meantime our timely patch release and proven disaster recovery process should reassure all our users; we’ll continue to monitor for problems arising from iOS and deal with them accordingly.
