When we added support for Gatekeeper in Fetch 5.7.1 last month, we caused Fetch to break on Mac OS X 10.5. Others ran into related compatibility problems; Daniel Jalkut and Ken Case proposed a solution. We chose a different approach from theirs.
A Mac OS X application can be digitally signed by its developer. The digital signature functions as a wax seal: the developer's identity is securely inscribed in the application, and it becomes possible to detect if the application has been tampered in any way.
This is a good idea for a variety of reasons; for example, detecting that an application has been tampered with is a measure of protection against malware.
With this mechanism in place, it becomes possible for users to give special rights to an application (for example, the right to access the address book) and be sure that this right will not be hijacked by an impostor. This, too, is a measure of protection against malware.
Central to this mechanism, therefore, is the question of app identity: if I have two apps that claim to be different versions of the same app, should I trust them? If they really are different versions of the same app, then they should be extended equal rights. If, however, one of them is an impostor, then the two should not be given equal rights.
The solution Apple took to this question is that every signed app, in addition to its developer's identity, embeds a statement that describes precisely what conditions have to be met by another app in order for the other app to be considered merely a different version of the same app. This statement is known as the app's designated requirement.
Where we run into problems is that the designated requirement embedded inside an application signed by Xcode 4.3 using an Apple Developer ID (which is required to support Gatekeeper in Mac OS X 10.8 Mountain Lion) is too complex for Mac OS X 10.5.x to understand. As a result, a Gatekeeper-enabled app running on Mac OS X 10.5.x always seems to be an impostor, which results in a variety of problems (such as the user always being prompted to allow the app to use passwords stored in the keychain, instead of being prompted only the first time — exactly the problem we ran into with Fetch 5.7.1).
Obviously, then, the question is whether it is possible to create a designated requirement that is compatible with Gatekeeper but not too complex for Mac OS X 10.5.x to understand, thus retaining Gatekeeper support, but keeping Mac OS X 10.5.x happy.
The answer is: yes, if you are willing to require Mac OS X 10.5.8.
Gatekeeper-enabled apps fail verification on Mac OS X 10.5.x for two different reasons:
- Understanding and verifying a Gatekeeper-compatible designated requirement requires support for some code signing features that were simply absent in Mac OS 10.5. There is nothing you can do about this, but since these features are present in Mac OS X 10.5.8, you can require your users to install the (free) update to Mac OS X 10.5.8. You are still left with a support burden, but it's not as bad as requiring a major upgrade. (We did not take the time to figure out exactly where in the 10.5.x release series this started working.)
- A designated requirement needs to be compiled before being built into an application. A Gatekeeper-compatible designated requirement compiled by Xcode 4.3.x on Mac OS X 10.7 is incompatible with Mac OS X 10.5.x and Mac OS X 10.6.x. However, the same designated requirement compiled on Mac OS X 10.5.8 is compatible with all later versions of Mac OS X. Therefore, if you compile your designated requirement on Mac OS X 10.5.8 and then embed it into your app using Xcode 4.3, you will produce a Gatekeeper-enabled app that is compatible with Mac OS X 10.5.8 and later.
Which all brings us to: why do you even care? Can't you just do what Daniel and Ken proposed and be done with it?
Well, you could. But if you do, you will expose yourself to the following two problems:
- When your Apple-issued developer certificate is renewed (its expiration time is no more than 5 years into the future), you will find that the app signed by the renewed certificate will not meet the designated requirement of the app signed by the current certificate. This is because Daniel and Ken used a designated requirement that is tied to a specific certificate, whereas our solution uses a designated requirement that is tied to our company identity.
- The Mac App Store version of your app and the Gatekeeper version of your app will not satisfy each other's designated requirement — again, because the designated requirement is tied to a specific certificate.
To understand the details of this, you need to look at what the designated requirement created looks like for an app distributed in the Mac App Store by a developer who has also been issued a Developer ID for Gatekeeper distribution:
[ 1] designated => ( [ 2] anchor apple generic [ 3] and certificate leaf[field.1.2.840.1136220.127.116.11.9] /* exists */ [ 4] or [ 5] anchor apple generic [ 6] and certificate 1[field.1.2.840.113618.104.22.168.6] /* exists */ [ 7] and certificate leaf[field.1.2.840.113622.214.171.124.13] /* exists */ [ 8] and certificate leaf[subject.OU] = some-developer-id [ 9] )  and identifier "some-bundle-id"
What this says is “The designated requirement (1) of this app is to have the bundle ID some-bundle-id (10) and either a) be trusted by Apple (2) and signed by Apple for App Store distribution (3) or b) be trusted by Apple (5) and signed by developer with ID some-developer-id (8) using their Gatekeeper certificate (6,7)”.
So, in short: you probably shouldn't use a trivial designated requirement tied to a specific certificate. Instead, you should:
- Extract the full Gatekeeper designated requirement. If your app is on the App Store, download it from the App Store and extract its designated requirement; otherwise, build for Gatekeeper distribution using Xcode 4.3 and extract that designated requirement. To extract the designated requirement from your app, run “codesign -d -r-” on it, and save the “designated =>…” part of it to a file.
- Compile the requirement on Mac OS X 10.5.8 (or the oldest version of Mac OS X that your app supports), using “csreq -r requirement.txt -b requirement.bin”.
- Embed that compiled requirement inside your Xcode 4.3 build, using “codesign -r requirement.bin”.
- Drop support for older minor versions of Mac OS X 10.5.
Much thanks to Ken and Daniel for giving me a starting point for figuring this out!