Since the day Snow Leopard came out, much has been said about creator codes, preferred applications, and Universal Type Identifiers. Regardless of whether you favor the Leopard behavior — in which a Mac OS 9-style creator code trumps a file‘s extension — or the Snow Leopard behavior — in which a Mac OS 9-style creator code is completely ignored — you, as a developer, may run into a case when you need to make sure that a particular file will open with a particular app when the user double-clicks it in the Finder.

We had the same need in Fetch; we allow Fetch users to specify which app a downloaded file will open with, because users sometimes want a downloaded file to open with a different application from a locally created file of the same type.

The general idea is to add a ‘usro’ resource with ID 0 to the file. This will take you about ten lines of code — then you will spend the next hundred lines of code doing things that make it all work well for the user.

void
FileRef::SetApplication(
  const string& inIdentifier,
  U::ApplicationRef inApplication)
{

This function takes two arguments (besides the implicit this, which represents the file you are changing): the Universal Type Identifier for the file’s type (such as ‘public.html’), and a reference to the application you want the file to open with.

An U::ApplicationRef is our wrapper around application bundles. It provides helpers for looking up bundle IDs, finding apps from a bundle ID, etc. Similarly, a FileRef is our wrapper around a Carbon FSRef. I am not including code for such wrappers here; they’s self-explanatory.

First, clean up any existing ‘usro’ resource in the file:

string path = inApplication.GetFile()->GetPath();
// Put a usro(0) resource in 
U::ResourceFileRef resFile(*this, fsRdWrPerm);
::Handle oldUsro = ::Get1Resource('usro', 0);
if (oldUsro) {
  ::RemoveResource(oldUsro);
}

Next, create a new ‘usro’ resource and add it to the file:

// Must be 1028 bytes long or Tiger freaks out
Handle newUsro(::NewHandleClear(PATH_MAX + sizeof(UInt32)));
if (path.length() > PATH_MAX - 1) {
  path.erase(path.begin() + PATH_MAX - 1, path.end());
}

**reinterpret_cast<UInt32**>(newUsro.Get()) = path.length();
copy(
  path.begin(), path.end(), 
  *reinterpret_cast<char**>(newUsro.Get()) + sizeof(UInt32)
);

::AddResource(newUsro.Get(), 'usro', 0, "\p");
ThrowIfResourceError();
newUsro.Abandon();

I think what that comment is trying to tell me is that Launch Services in some versions of Mac OS X 10.4 Tiger did not recognize a ‘usro’ resource unless it was exactly 1028 bytes long. Aside from that, this snippet contains the most salient information about the ‘usro’ resource: its first four bytes contain the length of the filesystem path to the application which will open when the file is double-clicked; the remaining 1024 bytes contain the path itself.

It’s rather interesting that we aren’t doing anything to make the 4 bytes of length at the front of the resource be of any particular endianness. According to our version history, this code predates the PPC-Intel transition, and therefore I can only assume that it used to work on PPC computers as well as it works on Intel computers today. This suggests that copying a file from a PPC computer to an Intel computer might render its ‘usro’ resource useless, depending on whether Launch Services knows to interpret ‘usro’ resources with non-native endianness.

If all you are looking for is a file that opens in the given app when double-clicked, you are all set. However, your users will probably be confused because the file’s icon will not reflect its new affiliation — you might now have a text file that opens in BBEdit, but the Finder will continue to show it with its default icon, which probably belongs to TextEdit. It would be nice to do something about that.

The app’s info dictionary holds information about which icon should be used:

// Track down the custom icon we should use for this file
string extension = GetFileNameExtension(this->GetName());

optional<string> iconFile;
U::CFDictionaryRef appInfo = inApplication.GetInfoDictionary();

// If the app has no info dictionary, it is probably an old Carbon or
// Classic app and may require the file creator to be set, so we set it
if (not appInfo) {
  this->SetCreatorAndType(inApplication.GetSignature(), this->GetType());
} else {
  this->SetCreatorAndType(kLSUnknownCreator, kLSUnknownType);
}

You may be inclined to summarily ignore old apps that don't have an info dictionary. If your customer support sometimes feels like archeology, you probably want to keep that section of the code.

Finally, extract the icons:

if (appInfo) {
  iconFile = IconFileInAppForType(appInfo, inIdentifier);
  if (not iconFile) {
     iconFile = IconFileInAppForExtension(appInfo, extension);
  }
}

I’ll elaborate on IconFileInAppForType and IconFileInAppForExtension later; for now, just assume you found the icon file name.

// Load the custom icon from app's resources or fall back on default
IconFamilyHandle iconFamily = 0;

if (iconFile) {
  // If the app is a bundle, use CFBundle to get at the icon
  if (not inApplication.GetFile()->IsFile()) {
    U::CFURLRef appURL(U::CFAdopt(::CFURLCreateFromFSRef(
      kCFAllocatorDefault,
      &inApplication.GetFile()->Get()
    )));
    
    U::CFURLRef icon = U::CFAdopt(::CFBundleCopyResourceURLInDirectory(
      appURL.get(),
      lexical_cast<U::CFStringRef>(iconFile.get()).get(),
      CFSTR("icns"),
      0
    ));

    // Try without extension if we can't find it with extension
    if (not icon) {
      icon = U::CFAdopt(::CFBundleCopyResourceURLInDirectory(
        appURL.get(),
        lexical_cast<U::CFStringRef>(iconFile.get()).get(),
        0,
        0
      ));
    }
    
    if (icon) {
      FSRef iconFile;
      if (::CFURLGetFSRef(icon.get(), &iconFile)) {
        ::ReadIconFromFSRef(&iconFile, &iconFamily);
      }
    }
  } else {
    SInt16 resID = lexical_cast<SInt16>(*iconFile);

    U::ResourceFileRef appResFile(*inApplication.GetFile(), fsRdPerm);
    
    // Try icns resource first
    iconFamily = reinterpret_cast<IconFamilyHandle>(
      ::Get1Resource(kIconFamilyType, resID)
    );
    if (iconFamily) {
      ::DetachResource(reinterpret_cast< ::Handle(iconFamily));
    }
    // Otherwise, use icon suite resources
    else {
      IconSuiteRef iconSuite = 0;
      ThrowIfOSError(::GetIconSuite(
        &iconSuite, resID, 
        kSelectorAllAvailableData
      ));
      ::IconSuiteToIconFamily(
        iconSuite, kSelectorAllAvailableData, 
        &iconFamily
      );
      ::DisposeIconSuite(iconSuite, true);
    }
  }
}

What just happened here? First off, an IconFamilyHandle is the Carbon way of representing the collection of several different resolutions of the same icon. I don’t know if there’s a newfangled way of managing those, without getting tangled in APIs from the last millennium, so IconFamilyHandle it is.

Second, an app might be a bundle (a maze of twisty little files and directories), or it might be a monolithic file (a maze of twisty little resources).

If it’s a bundle, then the icon filename you got earlier refers to an icon file somewhere inside the bundle — which CFBundle APIs will happily hand over to you. Except when they won’t, because some developers put the complete icon filename (including the extension) in their info dictionary, and some omit the extension, letting the OS tack on ‘icns’ as needed. Therefore, you need to look for the icon in two places.

If the app is not a bundle, then the icon filename you got from the info dictionary is not a filename at all — it’s a resource ID. It might refer to a single ‘icns’ resource (in modern apps) or it might be a collection of individual resources (of which there are approximately zillion, with such prosaic names as ‘t8mk’ and ‘ics#’). If you are looking at a single ‘icns’ resource, then a simple Get1Resource will fetch it. Otherwise, to load the myriad different separate resources, use GetIconSuite, and then convert the result to an icon family.

There's also the case when you find no icon at all — it’s better to give the document a generic icon (which will leave the user uninformed about which app will open the document) than leave it with no custom icon (which will probably suggest that the file will open in some app other than the one you just set it to open with):

if (not iconFamily) {
  static U::Icon genericIcon = U::Icon::FromTypeCreator(
    kSystemIconsCreator, kGenericDocumentIcon
  );
  ThrowIfOSError(::IconRefToIconFamily(
    genericIcon.Get(), kSelectorAllAvailableData, 
    &iconFamily
  ));
}

Alright! Now you have the icon. Remove the old icon, write the new one to the file, and let the Finder know that the file has a custom icon:

U::Handle newIcon(reinterpret_cast< ::Handle>(iconFamily));

// Put a custom icon resource in
::Handle oldIcon = ::Get1Resource(kIconFamilyType, kCustomIconResource);
if (oldIcon) {
  ::RemoveResource(oldIcon);
}

::AddResource(newIcon.Get(), kIconFamilyType, kCustomIconResource, "\p");
ThrowIfResourceError();
::SetResInfo(newIcon.Get(), kCustomIconResource, "\pBinding Override");
ThrowIfResourceError();
newIcon.Abandon();

// Set the custom icon flag
FSCatalogInfo info;
ThrowIfOSError(::FSGetCatalogInfo(
  &this->Get(), kFSCatInfoFinderInfo, 
  &info, 0, 0, 0
));
FileInfo& finderInfo = *reinterpret_cast<FileInfo*>(&info.finderInfo);
finderInfo.finderFlags |= kHasCustomIcon;
ThrowIfOSError(::FSSetCatalogInfo(&this->Get(), kFSCatInfoFinderInfo, &info));

And now you are done. Oh, wait… maybe some day you will want to undo this, and let the file open with its default application:

void
FileRef::ResetApplication()
{
  // Remove usro(0) resource, icns(-16455), and clear the custom icon bit
  U::ResourceFileRef resFile(*this, fsRdWrPerm);
  ::Handle oldUsro = ::Get1Resource('usro', 0);
  if (oldUsro) {
    ::RemoveResource(oldUsro);
  }
  ::Handle oldIcons = ::Get1Resource(kIconFamilyType, kCustomIconResource);
  if (oldIcons) {
    ::RemoveResource(oldIcons);
  }

  FSCatalogInfo info;
  ThrowIfOSError(::FSGetCatalogInfo(&this->Get(), kFSCatInfoFinderInfo, &info, 0, 0, 0));
  FileInfo& finderInfo = *reinterpret_cast<FileInfo*>(&info.finderInfo);
  finderInfo.finderFlags &= ~kHasCustomIcon;
  ThrowIfOSError(::FSSetCatalogInfo(&this->Get(), kFSCatInfoFinderInfo, &info));
}

If you made it this far, I suppose you probably want to know how to get the icon file name from an app’s info dictionary. It’s a simple matter of looking things up in CFBundleDocumentTypes (for apps that haven't adopted UTIs yet) or LSItemContentTypes (for those that have).

optional<string>
IconFileInAppForExtension(
  U::CFDictionaryRef inAppInfo,
  const std::string& inExtension)
{
  return IconFileInAppForKeyValue(
    inAppInfo, 
    CFSTR("CFBundleTypeExtensions"), inExtension
  );
}

optional<string>
IconFileInAppForType(
  U::CFDictionaryRef inAppInfo,
  const std::string& inExtension)
{
  return IconFileInAppForKeyValue(
    inAppInfo, 
    CFSTR("LSItemContentTypes"), inExtension
  );
}

optional<string>
IconFileInAppForKeyValue(
  U::CFDictionaryRef inAppInfo,
  CFStringRef inKey,
  const std::string& inValue)
{
  U::CFArrayRef docTypes = cf_cast<U::CFArrayRef>(::CFDictionaryGetValue(
    inAppInfo.get(), CFSTR("CFBundleDocumentTypes")
  ));

  if (docTypes) {
    U::CFStringRef valueStr = lexical_cast<U::CFStringRef>(inValue);
    
    for (
      CFIndex i = 0;
      i < ::CFArrayGetCount(docTypes.get());
      ++i
    ) {
      U::CFDictionaryRef docType = cf_cast<U::CFDictionaryRef>(
        ::CFArrayGetValueAtIndex(docTypes.get(), i)
      );

      U::CFArrayRef values = cf_cast<U::CFArrayRef>(
        ::CFDictionaryGetValue(docType.get(), inKey)
      );
      if (values) {
        for (
          CFIndex j = 0;
          j < ::CFArrayGetCount(values.get());
          ++j
        ) {
          U::CFStringRef appValue = cf_cast<U::CFStringRef>(
            ::CFArrayGetValueAtIndex(values.get(), j)
          );
          if (
            ::CFStringCompare(
              appValue.get(), valueStr.get(), 
              kCFCompareCaseInsensitive
            ) == kCFCompareEqualTo
          ) {
            U::CFStringRef iconFileName = cf_cast<U::CFStringRef>(
              ::CFDictionaryGetValue(docType.get(), CFSTR("CFBundleTypeIconFile"))
            );
            if (iconFileName) {
              return optional<string>(lexical_cast<string>(iconFileName));
            }
          }
        }
      }
    }
  }
  return optional<string>();
}

Of course, if you are only ever going to use this code to set the creator of a file to an application that you control, then you can skip most of the backwards compatibility cases. You won’t need the code for resource-fork based apps or apps without info dictionaries, and probably not even for apps that don’t use UTIs.