From the inception of the Bipsync Notes app one of the features we felt compelled to offer was the ability to fetch the latest content even when the app wasn’t being directly used. This would reduce the burden on the user to have to manually refresh the app — and possibly allow them to access new content despite a lack of connectivity when returning to the app, since the content would have already been delivered ahead of time.
At the core of this requirement is Background Fetch, a technique first introduced in iOS 7 which allows an application to periodically import new content despite it being in a closed — “backgrounded” — state. Applications like Bipsync Notes which synchronise data authored elsewhere to an iOS device can use this feature to download content automatically and notify the user when there’s something new for them to see. When using the advised configuration the system manages the fetch times, and based on the user’s usage patterns will determine when, and how often, content should be fetched. From a development perspective, opting in ostensibly requires the flipping of a configuration switch and the implementation of a single method. Like other Apple initiatives, it should “just work”.
Preparing to fetch
When trying to gauge the complexity of the task I followed my usual methods.
- Read any relevant Apple documentation.
- Watch any relevant WWDC videos.
- Search for blog posts or Stack Overflow entries which cover the topic and offer some insight into typical problems we might face.
After completing these tasks I felt reasonably confident about implementing Background Fetch in our app. Of course every app is different so specific implementations will vary, but resources like Apple’s WWDC ’13 “What’s New With Multitasking” video suggest an approach where a Background Fetch algorithm is incorporated into an app’s existing data update mechanism, so for all intents and purposes an app behaves the same whether in active or background mode. A side benefit of this is that post-fetch, the app UI always reflects the influx of data; iOS uses this to take a screenshot of the app to show in the app switcher, for example.
Getting Background Fetch working proved deceptively simple. We first added the mandatory application:performFetchWithCompletionHandler hook to our AppDelegate class. This is called when a fetch begins, and in turn calls an instance of a new ‘background sync’ class which decorates our regular ‘active sync’ class; this instance knows to invoke the completion handler once the synchronisation process finishes. iOS uses the result of this callback to inform the timing of subsequent fetches.
After a day’s worth of development we had the app retrieving new content despite it being closed. The best way to test this was by manually triggering a fetch via XCode’s “Debug > Simulate Background Fetch” option as it’s difficult to predict exactly when a device will decide to perform a fetch. At this point, I thought we were very close to a shipping feature. I should coco, as my nan used to say.
Things get messy
In our early testing, Background Fetch seemed to be working just fine. Then we pushed a build to TestFlight and left the app running on our devices for a couple of days before noticing something strange. When opening the app after a period of closure it would sometimes be blank, i.e. the note list would be empty, as if there were no notes on the device. Though this was quickly corrected by navigating back and forth another area of the app, we were very concerned – it could appear to the user that all their content had been deleted when it was in fact simply not displaying.
After adding and inspecting some logging (we use the excellent CocoaLumberjack framework which lets us get detailed application logs from a device in a number of ways) we realised that an exception was being thrown when a NSFetchedResultsController attempted to load notes for display after a Background Fetch had completed. It transpired that on occasion some files in the application directory couldn’t be read when the app was closed. The answer to why that was the case led us to our first major issue with Background Fetch.
Data Protection don’t come for free
iOS offers a neat feature called Data Protection, which like Background Fetch is easy to enable in an app’s capabilities manifest. Its aim is to make applications more secure by automatically encrypting the files they store, making it very difficult for an attacker to read those files’ contents should they be able to gain access to them (typical files stored by an application would include databases and images).
Data Protection performs encryption using a secure key based on the user’s passcode; this means the key is only available when the device is unlocked. In the default case, a file encrypted in this fashion cannot be read at all when the device is locked. Apple do offer some configuration here: via the Member Center, developers can choose between three permission types:
- Complete Protection, the default.
- Protected Unless Open, where the file is unreadable while the device is locked unless it was already open when locking occurred, which is possible if an app began working with it beforehand.
- Protected Until First User Authentication, where the file is readable given the device has been unlocked at least once, i.e. after the device was powered on, the user subsequently entered their passcode.
Bearing the above in mind, if I now reveal that our Notes app has Data Protection enabled (since we value the security of our users’ data very highly) you might guess why we were seeing problems when the app performed a Background Fetch: in some cases, not only was the app closed, but the device was also locked – which meant that some files weren’t readable, and the app couldn’t behave normally. The ‘missing notes’ issue I mentioned earlier was in fact caused by the app crashing when a NSFetchedResultsController failed to read one of its cache files. While our Core Data store was accessible because by default it is created with an attribute of NSFileProtectionCompleteUntilFirstUserAuthentication for its NSPersistentStoreFileProtectionKey option, other critical files were not.
I set out to address this. Seeking to maintain as strict an approach as possible, we elected to ‘whitelist’ the files we needed access to in background mode — these being the files created directly by our app, as well as certain folders under the Library folder (like those containing Core Data cache files) and also the tmp folder. This was achieved by scanning for files in these paths on app startup/shutdown and ensuring they had a value of NSFileProtectionCompleteUntilFirstUserAuthentication for their NSFileProtectionKey property. We also applied that property to files created by the app, like images downloaded from the API.
In addition to this we also configured the items we store in iOS’ Keychain to have the kSecAttrAccessibleAfterFirstUnlockThisDeviceOnly property, which is analagous to the property we used on the files. This gives us access to API tokens and the like when the app is backgrounded – we store all sensitive credentials in the Keychain in line with security best practice.
This approach worked reasonably well but wasn’t perfect. The Keychain data was fine, but there were two issues with whitelisting files on disk:
- We’d occasionally see errors when trying to set attributes on files that were in the process of being deleted as the app closed (not a serious issue as the error didn’t make it to the UI or crash the app, but any error report isn’t great).
- Any new set of files created by the app would have to potentially be added to the whitelist, which we were likely to miss or forget at some stage and so reintroduce the problem of the app not working correctly while backgrounded.
I continued to research and finally came across a post on Stack Overflow which hinted that there were configuration options for Data Protection which allowed a default protection level to be set. Rather than placing these in the project as I would have expected, Apple have them in the App ID configuration in the Member Center portal, which allows them to be baked in to the provisioning profile of the app from where it takes effect. No doubt this is the correct place for the config as far as implementation of Data Protection is concerned, but it’s not very intuitive and given the amout of time I’d spent trying to get around issues caused by files not being accessible, I’m surprised there’s not a mention of this in Apple’s docs on the Data Protection capability – I raised a radar so hopefully that’ll be addressed and perhaps save someone some time.
One final note. If a Background Fetch process runs when the device has been restarted but the user has yet to enter their passcode, we’re back in the situation where some of our files are unreadable. In this case we don’t bother attempting a fetch and execute the completion handler straight away. This “failed fetch” may have an impact on the frequency of future fetches but it’s reasonable to consider this an edge case, which therefore doesn’t concern us greatly.
In a nutshell: anyone wanting to add Background Fetch to an app which makes use of Data Protection should be aware of any files that the app or the OS needs to access while the app is backgrounded, and make the necessary configurations in order to avoid errors and/or crashes.
The impact on testing
Away from the code, we found that there was a cost to our testing efforts too. Before Background Fetch was introduced, testing a new feature was a straightforward process: we’d deploy to Testflight, and the team would put the feature through its paces, reporting anything amiss. We could be confident that if nobody saw any issues or crashes, it was quite likely the feature was ready for release.
Now that the app can run without the user being aware of it, we can’t rely on visual confirmation as a quality metric. Testing becomes a more drawn out affair where the tester has to be aware of the state of content in the app at all times, so as to confirm that new content was indeed delivered while the app was inactive. Since it’s hard to predict when a Background Fetch will run it’s hard to organise testing into discrete timeslots, if we’re to effectively target ‘natural usage’.
Add to this that the app behaves differently within the environments in which it can run – active, inactive with the device unlocked, inactive with the device locked but after at least one unlock event, and inactive with the device never having been unlocked (a fresh boot) – and one starts to see the challenge for ensuring each function of the app is working correctly. This isn’t limited to Background Fetch features either. Any feature that makes use of the filesystem is potentially at risk, such as background downloads of media items.
Don’t depend on JavaScript
Quite a niche problem this, but one that affected us significantly. We make use of the WKWebView component to power our rich text editor, and after much debugging found that when the app is backgrounded, the WebView’s JavaScript process is terminated. This means any app behaviour that relies on the execution of JavaScript isn’t going to work.
We were seeing errors being raised because when new content was delivered to the device, we attempted to refresh the editor if it was currently displaying a note. That way the user would have the latest content immediately visible to them on return to the app, as per Apple’s recommendation. Of course this required us to execute JavaScript to update the HTML in the editor, and as we now know the JavaScript process was no longer available.
To get around this we have to queue any UI updates that require JavaScript and execute them once the app has returned to the foreground. It’s a relatively easy change to make, but serves as another illustration that implementing Background Fetch will often require a change to an app’s workflow. Apple’s demonstrations and sample code show the fetch being hooked into the same workflow as the app takes when it is active, but in our experience concessions will sometimes need to be made.
Conclusion
Overall, Background Fetch has proved to be one of the more challenging features to add to Bipsync Notes. If I’m ever again involved with an app that plans to make use of Background Fetch I’ll strive to architect the app with the feature in mind, and so hopefully avoid the sort of integration issues we’ve run into this time. It’s certainly something that needs to be considered right from the start, given the impact it can have.
If you’re thinking of adding Background Fetch to your app, please don’t be dissuaded by anything I’ve written. It’s a really neat feature that is genuinely appreciated by our users. Just be aware that it will likely involve a lot more work than you may be expecting.
