Now that the iOS app is taking shape, we need to put it in the hands of our testers. Initially we’ll test in-house so this means distributing the app to our co-workers, but eventually we’ll expand the beta test program to include some of our enterprise customers too. There are a few ways we can achieve this.
The simplest approach, in the sense that it doesn’t involve a third party, is to send the app binary to each tester and ask them to install it manually through iTunes1. This is impractical on a number of counts – it’s a pain to distribute the build (usually by email) in the first place; there’s no formal template or system for passing information about the build on to the tester (like what bugs have been addressed in the build, what new functionality is present and should be tested, etc.); and installation is a cumbersome process for the tester, which may discourage them. If the binary is ineligible for their device or corrupt, we won’t discover that until the tester attempts to install it. And speaking of devices – each tester’s device has to be registered in Apple’s developer portal; again a finicky process that involves the user having to locate their UDID, and the developer having to import it into a finite list of test devices known to Apple. It’s just messy all round.
Until now our solution to this problem has been to use TestFlight, a service popular with many developers. TestFlight’s owner Burstly was acquired by Apple in February 2014 which wasn’t a total surprise; they have an impressively solid application which we’ve used successfully with testers of varied technical ability. The process is straightforward from the developer’s perspective too – a release can usually be achieved in a matter of minutes. An app .ipa file (which includes the binary) is uploaded to TestFlight; it prompts the developer for release notes; then it determines which of the testers are eligible to install the device based on their devices, and manages notifying them and delivering the app to their device.
I don’t know what the long term plans are for the original Testflight service, but for the time being it’s still operational. Apple recently launched a new version of the service which is directly integrated with their iTunes Connect service (more on that in a bit). The original Testflight approach would still require us to register our tester’s devices with Apple, although the service has a feature that harvests the UDID from each tester device and makes them available to us in a file that can then be imported in Apple’s Member Center, which streamlines the device registration process significantly. They even provide a library that makes crash reports available for inspection by developers, associated with the build that caused the crash.
So the original Testflight is a more than capable service; there are a few others we could also explore, such as HockeyApp, which was acquired by Microsoft a few days ago. Considering that Apple could retire the original Testflight app in deference to their new iTunes Connect integration at a moment’s notice, we thought it made sense to take advantage of the clean slate we’re currently enjoying and transition to Apple’s version while we have the time to do it.
There are some immediate benefits from doing so:
- Tester devices don’t have to be registered with our Apple developer program account – it’s all managed through iTunes Connect.
- Apple makes a distinction between testers that are members of our team, and testers that are not. This makes it easy to distribute more polished beta builds to our external users, while making more regular alpha builds in-house. Beta builds even go though App Review, which brings one of the most challenging hurdles in app publication forward, making launch less of a trial.
- Binary submission can be handled seamlessly within Xcode (albeit with the usual provisioning incantations and prayers).
We’ve been working this way for a week. Generally it’s been successful, though I do have a few gripes:
- It’s a little buggy. Right from the start, I was unable to distribute builds to our team; anything I uploaded wasn’t visible to them in the accompanying iOS app, and they weren’t notified when builds were available. I had to remove everyone from the team, disable beta testing, re-enable it, and add everyone again before it worked as expected. Sometimes pages fail to load (though this issue is endemic in iTunes Connect in general).
- It’s nice to not have to register UDIDs for test devices, but Apple’s alternative of associating Apple IDs with testers isn’t without its annoyances. We have dedicated Apple IDs for Bipsync work, and Apple allows only one ID to be associated with App Store / iTunes on each device. This means constantly signing in/out of various Apple IDs if we want to test on our personal devices.
- I mentioned that binary submission is performed via Xcode – this involves less manual processes but it does take longer, as Xcode checks “API usage” (which I assume means that it’s determining if any private APIs have been employed, which is prohibited by Apple) and then processes each build once it’s uploaded. This takes somewhere in the region of 10 minutes to complete, which is significantly longer than the original TestFlight’s equivalent process.
- There’s no way to delete builds, which is frustrating when a build is uploaded in error, or should one wish to make a build unavailable for some reason.
We’re satisfied that this is the right way to proceed; while it doesn’t feel like the service is mature, Apple may eventually address the above issues. Sadly we’re going to have to address the bug reports raised through beta testing ourselves…
1. At least, this used to be possible. Thankfully I haven’t needed to do such a thing in a long while.
