Offline Metal Shader Compilation and Validation in Cloud Mac CI

Graphics features may run correctly on a development machine yet fail with a Metal compilation error only near the end of a CI archive. This usually does not mean the machine lacks performance. More often, the shaders are not being managed as independent build inputs. A more reliable approach is to use the current Xcode toolchain on the cloud Mac to generate separate metallib files for devices and simulators before starting the full project build. Syntax errors, incorrect SDK selection, and missing functions then surface earlier, with much shorter failure logs.

Why separate shaders into their own gate

Xcode automatically processes .metal files added to a target, but errors are often buried in lengthy logs that also contain dependency resolution, resource copying, and code-signing steps. A dedicated gate does not replace the Xcode build. Instead, it answers three questions before that build begins:

1. Can the current toolchain compile every shader?
2. Can libraries be generated separately for device and simulator targets?
3. Can the application load the function names it actually references from those libraries?

Do not copy a metallib generated on a development machine directly into a long-lived artifact directory. After the toolchain, SDK, or compiler options change, the old file may remain in place even though it no longer represents the current source code.

The repository should contain only the .metal source files and compilation scripts. Store .air and .metallib files in a disposable build directory. If the team uses a cache, its key should include at least a source digest, the Xcode path, the SDK version, and the target platform.

Pin the toolchain for the current build

A cloud Mac may have multiple Xcode versions installed. Scripts should not depend on whichever version was most recently selected in the graphical interface. They should explicitly verify DEVELOPER_DIR. The path must reflect what is actually installed on the node; do not reuse an unverified value across jobs.

set -euo pipefail

: "${DEVELOPER_DIR:?DEVELOPER_DIR is required}"

xcodebuild -version
xcrun --sdk iphoneos --show-sdk-version
xcrun --sdk iphonesimulator --show-sdk-version
xcrun --sdk iphoneos --find metal
xcrun --sdk iphoneos metal --version

mkdir -p build/metal/metadata
{
  echo "developer_dir=$DEVELOPER_DIR"
  xcodebuild -version
  echo "iphoneos=$(xcrun --sdk iphoneos --show-sdk-version)"
  echo "iphonesimulator=$(xcrun --sdk iphonesimulator --show-sdk-version)"
  xcrun --sdk iphoneos metal --version
} > build/metal/metadata/toolchain.txt

Archive toolchain.txt with the failure logs. It makes it easy to distinguish a source regression from a runner switching to another Xcode version. Recording only xcodebuild -version is insufficient because the SDK actually used also affects the result.

Compile device and simulator artifacts separately

Compile each source file into an individual .air file first, then combine those files into a library. Compiling files individually lets errors point directly to the relevant source file instead of producing only a generic link failure.

set -euo pipefail

sources=(Shaders/*.metal)
if [ ! -e "${sources[0]}" ]; then
  echo "No Metal source files found" >&2
  exit 1
fi

for sdk in iphoneos iphonesimulator; do
  out="build/metal/$sdk"
  rm -rf "$out"
  mkdir -p "$out/air"

  for src in "${sources[@]}"; do
    name="$(basename "$src" .metal)"
    xcrun --sdk "$sdk" metal -c "$src" \
      -o "$out/air/$name.air"
  done

  xcrun --sdk "$sdk" metallib "$out"/air/*.air \
    -o "$out/AppShaders.metallib"

  test -s "$out/AppShaders.metallib"
  shasum -a 256 "$out/AppShaders.metallib" \
    > "$out/AppShaders.metallib.sha256"
done

The two artifact sets must be kept in separate directories so that the later job cannot overwrite the earlier one. If the source uses C/C++ preprocessor macros to distinguish platforms, define those macro options centrally in the script and ensure that the Xcode Build Phase uses the same definitions.

Validate the function contract with a loading test

A nonempty file proves only that the compiler produced output. It does not prove that the functions requested by the application still exist. After a kernel or fragment function is renamed, the corresponding string in the Swift code may not be updated. Load the library in a test target and query its required entry points.

import Metal
import XCTest

final class ShaderLibraryTests: XCTestCase {
    func testRequiredFunctionsExist() throws {
        let device = try XCTUnwrap(MTLCreateSystemDefaultDevice())
        let url = try XCTUnwrap(
            Bundle(for: Self.self).url(
                forResource: "AppShaders",
                withExtension: "metallib"
            )
        )
        let library = try device.makeLibrary(URL: url)

        for name in ["imageVertex", "imageFragment", "toneMapKernel"] {
            XCTAssertNotNil(
                library.makeFunction(name: name),
                "Missing Metal function: \(name)"
            )
        }
    }
}

The function list should come from the names the application actually uses when creating pipelines. The test bundle must also confirm that it copied the iphonesimulator artifact rather than a device file left behind in the workspace. At minimum, the smoke test should create an MTLDevice, load the library, and query the functions. If texture formats or threadgroup constraints are involved, add pipeline creation tests as well.

What evidence to preserve after a failure

When investigating a Metal CI failure, a small set of comparable data is more valuable than the entire working directory:

• DEVELOPER_DIR, the Xcode version, and both SDK versions;
• the actual compilation commands and standard error output;
• a source digest for each .metal file;
• the size and SHA-256 value of both metallib files;
• the function name that failed to load, the test target, and the run destination;
• the commit identifier used by the job.

Generate source digests with find Shaders -name '*.metal' -print0 | sort -z | xargs -0 shasum -a 256. A changed artifact hash should not cause a failure by itself because a toolchain upgrade may also change the binary. The correct strategy is to compare toolchain fingerprints first, then use the results of recompilation, library loading, and pipeline creation as the gate.

Even when these jobs run on dedicated physical MacMLab nodes, consecutive jobs can still contaminate the same workspace. Clear the target output directory before every build, and archive only the metadata, logs, and final libraries when the job finishes. This prevents old .air files from being mixed into a new library and gives the next failure a reproducible starting point.

Frequently asked questions