Today was going to be about so much else, but instead I found myself mêléeing with Apple’s build system.

I have a script I’ve been using very successfully for some years that automates the build process for my CLI apps. It builds a universal binary containing both ARM64 and x86-64 code, wraps it into a macOS installer package and submits the result to Apple’s notarisation service. One command in Terminal and it’s all done, or occasionally you get an error message.

But this week I embarked on a programme to extract a heap of common code from my various Swift-based macOS and Linux CLI apps and put the into a common location for each app’s codebase to access. It’s an obvious approach, I just had too few CLI apps to really make cutting and pasting the common code from one to another too much of a chore. But I’ve reached the stage where the change to one becomes a hassle to change them all, so it’s time to give the common code its own home.

That’s easy enough, and I now have a GitHub repo containing the common stuff. But how to apply it? My first thought was to use git submodules, and this worked, but felt kind of clunky — and not very Swift-y. So I turned to Swift Package Manager (SwiftPM). This has been around for some time, and I’ve already used it to allow some of my CLI apps to be built in Linux, for which the Swift toolchain is available but the Xcode development platform is not.

Xcode is handy because on the macOS side, you need it to embed the standard Mac app Info.plist property list file into the CLI app binary so that it can be read during notarisation. GUI apps keep the file within their broader bundle directory. Notarisation is the process by which executables gain important security verification on the platform, and it’s a Good Thing. The Swift build system doesn’t make it easy to perform this task, and in any case it’s not needed for binaries running under Linux.

It took me a little time to get the common code package’s Package.swift manifest file correct so that Xcode was happy to add it to my existing CLI apps, and then a little bit of extra work to ensure the packaged code was accessible to the app’s code. I got that done yesterday, and everything was hunky dory.

And I looked upon my work and saw that it was good.

And, lo, I rested…

Roll up your Sleeves, Get Ready to Rumble

Today, however, I discovered that building any of the CLI apps in Xcode to make sure they work is one thing; running the process via my scripts was an entirely different matter. Basically, the build process, which of necessity peeks into the Xcode project file to get what it needs — this is all done by Apple’s xcodebuild tool — would happily consume the common code but not in such a way that app code could use it. Searching for a solution among official and unofficial sources yielded some answers, but while they would build and link everything, it would no longer allow me to specify the output location of the built binary, which was also no longer universal.

I worked up a workaround: build the app using Xcode’s archive-and-export functionality to yield a release build, and then pass the output’s location to a modified form of my script, which now optionally takes that path as input for the packaging and notarisation work, bypassing the build component of the process.

It does the job, but using multiple tools where one was sufficient before just didn’t feel right, so it was time to go back to man xcodebuild.

Previously I had been specifying the target (ie. build product) from my Xcode project that I wanted the tool to build:

xcodebuild clean install -target "${app_name}" \
  -configuration Release || show_error_then_exit "Could not build app"

To get the build to handle the SwiftPM component correctly, I now have to specify the Xcode build scheme too with the -scheme option:

xcodebuild clean install -target "${app_name}" -scheme ${app_name} \
  -configuration Release || show_error_then_exit "Could not build app"

With a mixture of the manual and some solutions to pre-Swift build problems folk has posted on StackOverflow, I updated the line to focus on the project, not one target within it, switching -target to -project, and specified via -derivedDataPath a location for build intermediates. I also opted for the build command:

xcodebuild clean build -project "${app_name}.xcodeproj" \
  -scheme "${app_name}" -configuration Release \
  -derivedDataPath build/derivedData \
  -destination 'platform=macOS,arch=arm64' \
  || show_error_then_exit "Could not build app"

Finally, finally, it builds entirely successfully, and puts the output where I want it, in a consistent location, so that it can be easily consumed in the remainder of my build script.

But wait a minute. It’s building a non-universal binary. How do I get that back? The -destination option included above is there because otherwise xcodebuild will just pick the host’s architecture by default. By including that option, I can duplicate the line for an x86-64 build:

xcodebuild clean build -project "${app_name}.xcodeproj" \
  -scheme "${app_name}" -configuration Release \
  -derivedDataPath build/derivedData \
  -destination 'platform=macOS,arch=x86-64' \
  || show_error_then_exit "Could not build app"

This will overwrite the first binary of course, because the derived data paths are the same. So both lines need tweaking to use slightly different artefact output paths:

xcodebuild clean build -project "${app_name}.xcodeproj" \
  -scheme "${app_name}" -configuration Release \
  -derivedDataPath build/derivedDataIntel \
  -destination 'platform=macOS,arch=x86-64' \
  || show_error_then_exit "Could not build app"

xcodebuild clean build -project "${app_name}.xcodeproj" \
  -scheme "${app_name}" -configuration Release \
  -derivedDataPath build/derivedDataArm \
  -destination 'platform=macOS,arch=arm64' \
  || show_error_then_exit "Could not build app"

In fact, the finished script has one line here, in a function into which I pass suitable arguments, but you get the idea.

To combine these two binaries, which are automatically signed, by the way, into a single, universal binary, I pass the output locations to Xcode’s lipo tool:

lipo -create -output "${output_path}/${app_name}" \
  "${build_dir}/DerivedDataIntel/Build/Products/Release/${app_name}" \
  "${build_dir}/DerivedDataArm/Build/Products/Release/${app_name}" \
  || show_error_then_exit "Could not build universal binary"

And so I can now pass the binary at "${output_path}/${app_name}" into the remainder of the script, which uses pkgbuild to generate an installer package, notarytool to notarise it, and stapler to attach the notarisation record to the installer and contained binary.

Phew.

Except… no! Apple’s just thrown a weakened left-hook and it knocks you back for a moment. Notarisation fails. This time, at least, the issues can be identified and remedied. I strongly recommend you too do what Apple recommends and use the notarisation log download service to see what’s wrong:

xcrun notarytool log <your notarisation job ID> \
  --keychain-profile <your notary service app key> log.json

The log reveals signing errors on the binaries: both lack secure timestamps and “requests the com.apple.security.get-task-allow entitlement”. These can be adjusted in the Xcode build settings for the target, by adding respectively --timestamp as the value of the Other Code Signing Flags key, and setting the value of Code Signing Inject Base Entitlements to No.

Re-run the script and… Success!

Applying all this the Swift Way

So how would you do this without Xcode command line tools?

First, you have to insert the Info.plist record, which I mentioned earlier. Here’s how you fix that by editing the app’s Package.swift file and adding this into .executableTarget() right after path, if it’s present:

linkerSettings: [
    .unsafeFlags(
        ["-Xlinker", "-sectcreate",
         "-Xlinker", "__TEXT",
         "-Xlinker", "__info_plist",
         "-Xlinker", "cli-app/Info.plist"]
    )
],

This assumes your Info.plist (which you can make using Xcode or by editing an existing one) is in the same directory as your source code. Now you can run:

swift build 
codesign -s <Your cert of type Developer ID Application> -f \
  -o runtime .build/arm64-apple-macosx/debug/cli-app

The -s option is obvious; -f forces the addition of a new signing, and Apple recommends you always do this. The -o and its runtime argument is less obvious, but nonetheless essential: it’s how you harden the signed runtime. The app can’t be notarised without it. Next:

zip submission.zip cli-app
xcrun notarytool submit submission.zip -p <Your notary service app key> \
  --wait

Zipping the file is an easy way to check the process, but you’d probably want to use an installer package instead:

mkdir -p pkgroot/usr/local/bin
mv .build/arm64-apple-macosx/debug/cli-app pkgroot/usr/local/bin/cli-app
pkgbuild --root pkgroot --identifier <Your bundle ID>.pkg \
  --install-location "/" --sign <Your cert of type Developer ID Installer> \
  --version 1.0.0 app-cli-1.0.0.pkg

and then notarise:

xcrun notarytool submit app-cli-1.0.0.pkg -p <Your notary service app key> \
  --wait

On success:

xcrun stapler staple app-cli-1.0.0.pkg

And you can double-check it with:

spctl --assess -vvv --type install app-cli-1.0.0.pkg

Raise your glove, champ. You won.

Now it’s important to point out — yet again — it’s still not 100 per cent; there are issues here. Build the same thing under Linux, for example, and it will fail early on because swift build barfs at the additional linker settings. What’s required is conditionality on these, of the #if os(macOS)... #else... #endif type, but it’s not supported in Package.swift and failing that in the meantime you have … well… editing the Package.swift to exclude the Info.plist file from the build and comment out the linker passthroughs. Or a script to sample make those changes for you, according to the platform it’s run on…

Ah well, back to work…

• You can view my CLI build-and-package script on GitHub.