At WWDC 2015, Apple announced that they had acquired TestFlight and excited us developers by explaining how the woes of provisioning profiles will be no more, and the 100 device limit will be bolstered to a 1,000 user limit.
However, now that we are being ushered to use the iTunes Connect service exclusively after Apple announced an aggressive 1-month sunset period of TestFlightApp.com, it's not all sunshine and roses. Here are some of the concerns I have, which I have sent to Apple and TestFlight; any responses I've received are inline.
Concerns about Apple's TestFlight
Q. TestFlightApp.com currently supports iOS 7, but Apple's TestFlight does not - will it support iOS 7, or will we need to find an alternative?
TF: The all-new TestFlight Beta Testing in iTunes Connect requires iOS 8 or later. Unfortunately, there are no workarounds to this requirement.
Q. Will the upload APIs also be switched off? If so, are there any official recommendations for continuous integration/deployment?
TF: All services at TestFlightApp.com will be shut down. You can automate uploads to iTunes Connect using Transporter. You can find a guide located in the iTunes Connect Resources and Help section. I also recommend during research on open-source tools available that automates uploads to the App Store.
(Presumably they are referring to tools like shenzhen)
Q. Our testers use TestFlight to install old builds for regression testing. These builds are going to be unavailable - is there going to be a way to reference them later, download the IPAs now, or are they just going to be lost? At the moment, there is only an arduous manual process to do this.
Q. Distributing builds need App review, and is limited to only 2 builds per day - this would be moderately limiting.
Q. It seems that I cannot invite testers whose email address is already associated with iTunes connect.
TF: Unfortunately, you cannot be an iTunes Connect user on multiple iTunes Connect teams. If a user is already associated with another team, they will need to be invited under a different email address. Please note, to become an Internal tester, you must be an iTunes Connect user and be enabled for Internal testing.
Q. Each build expires after 30 days (so cannot have permalinks for regression testing).
TF: Only one beta build is made available for 30 days. Afterwards, the developer will need to submit a new build.
Q. Only one build of an app at any one time - so now there is no way to say "can you try this on the previous build to see if this new build broke the feature?"
Q. Cannot access crash logs if the user denies the OS level "send feedback to developers", which the previous version of TestFlight did not respect (because it's impossible for developers to see the user's answer to that question).
Q. Where we didn't need provisioning before (because an app was signed with an enterprise account), we must now upload a build and invite users to see it. Not a big issue, but if you want 200 of your company to test it, thats 200/1000 spaces gone, whereas before, it was unlimited.
TF: As an alternative, you can also use an MDM solution to distribute in-house Enterprise apps.
Given this current state of play, it's just not tenable to remain a TestFlight user right now[^1]. I don't mean for this list to sound like Apple owe it developers to fix my concerns - they don't. They don't owe any of us anything.
[^1]: Given Apple's unfair advantage here (e.g. removing the provisioning headaches), I fully expect to be back eventually.