Sitelet https://github.com/unoplatform/uno/pull/24964
Skip to content

ci: Move iOS runtime tests off macOS-14 (backport 6.8) - #24964

Open
MartinZikmund wants to merge 8 commits into
servicing/6.8from
dev/mazi/ci-macos14-off-6.8
Open

MartinZikmund wants to merge 8 commits into
servicing/6.8from
dev/mazi/ci-macos14-off-6.8

Conversation

@MartinZikmund

@MartinZikmund MartinZikmund commented Oct 3, 2026 •

Copy link
Copy Markdown
Member

GitHub Issue: part of #24959

Backport of #24680 and #24961 to servicing/6.8, in one PR.

PR Type:

🏗️ Build or CI related changes

What changed? 🚀

The hosted macOS-14 image is retired on 2 Nov 2026, with outage windows from 5 Oct. On this branch the iOS Skia runtime tests still ran on it (macOSVMImage_UITests / Xcode 15.3), and every other Mac job ran on macOS-15. This branch has no tvOS stage.

  • No macOS-15 either: the build image moves from macOS-15 to macOS-26. The DevServer CLI template-test jobs that hardcoded macOS-15 move too.
  • Xcode: builds stay on Xcode 26.1.1 (also on macOS-26), because .NET for iOS 26.1 refuses any other Xcode. macOS-26 doesn't preinstall the iOS 26.1 simulator runtime, and the build jobs need it (actool won't compile asset catalogs without a runtime that matches the SDK), so Select Xcode downloads it there, as it already did when missing. The iOS runtime-test shards select Xcode 26.6 (xCodeRootSimulator), whose iOS 26.5 runtime is preinstalled, so they skip that download. An app built with the 26.1 SDK runs on the newer simulator.
  • The iOS shards run on the build image (macOS-26, iOS 26.5 simulator).
  • The macOS desktop runtime-test job timeout goes from 60 to 120 min: the 6.x suite already took 26-55 min on macOS-15 and ran past 60 on macOS-26.
  • ios-uitest-run.sh uses the simulator runtime that ships with the selected Xcode instead of the hardcoded iOS-17-5, creates iPad Pro (12.9-inch) (6th generation) when the image lacks it (same screen size as before), and installs with xcrun simctl install (one retry). idb and its Homebrew tap / Python pins are gone.
  • macOSVMImage_UITests, xCodeRoot_iOS_UITests and the vmMacImageTest / vmImageTest / xCodeRootTest template parameters are removed. The disabled native iOS stage and TestFlight now take the build image and Xcode.

Also skips the five WebView2 ExecuteScriptAsync runtime tests on SkiaMacOS, next to their existing iOS/Win32 #11997 exclusion. On the macOS-26 image they segfault the whole Desktop Skia macOS process on 6.x (3 out of 3 runs on 6.7), so every later test is lost. Master passes them on macOS-26, and backporting the #24112 native-peer fix did not help, so the root cause is 6.x-specific and needs a Mac to chase.

The iOS multi-window scene fix from #24680 isn't needed here: this branch doesn't support multiple windows on iOS (SupportsMultipleWindows is false), so closing a window can't background the app.

Risk: the simulator moves from iOS 17.5 to iOS 26.5, so outcomes can shift. This PR's CI run is the check.

PR Checklist ✅

🤖 Generated with Claude Code

https://claude.ai/code/session_018riLfyvT4wZYEh8iFjWkKa

MartinZikmund and others added 2 commits October 3, 2026 11:37
Backport of #24680 (CI part) to servicing/6.8. The macOS-14 hosted image is retired on 2 Nov 2026. The shards now use the build image and Xcode, take the simulator runtime that Xcode ships, create the 12.9-inch iPad when the image lacks it, and install with xcrun simctl instead of idb.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_018riLfyvT4wZYEh8iFjWkKa
Backport of #24961 (cleanup part) to servicing/6.8. Nothing runs on macOSVMImage_UITests / xCodeRoot_iOS_UITests anymore; the disabled native iOS stage and TestFlight now take the build image and Xcode.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_018riLfyvT4wZYEh8iFjWkKa
Copilot AI balanced review requested due to automatic review settings October 3, 2026 09:38
@MartinZikmund MartinZikmund added the area/build Categorizes an issue or PR as relevant to build infrastructure label Oct 3, 2026
@MartinZikmund MartinZikmund self-assigned this Oct 3, 2026

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Warning

Copilot couldn't run its full agentic review because it didn't start before the timeout. Make sure your repository has a runner available, or add a copilot-code-review.yml file specifying one with the runs-on attribute. See the docs for more details.

Copilot review overview

Review effort: Lite
Findings: 2 Medium severity · 2 Low severity

Open (4)
What changed in this PR

Moves iOS Skia runtime tests off the retiring macOS-14 image by consolidating iOS test jobs onto the build macOS image/Xcode and updating the iOS UI-test harness to use the simulator runtime that ships with the selected Xcode (dropping idb).

Changes:

  • Update ios-uitest-run.sh to dynamically resolve the iOS simulator runtime/device from the selected Xcode and install via xcrun simctl install (with a retry).
  • Remove macOS-14 / Xcode 15.3 “UITest” image parameters from Azure DevOps templates and route iOS test jobs to the build macOS image/Xcode.
  • Delete macOS-14/Xcode 15.3 variables from .vsts-ci.yml.
File Description
build/​test-scripts/​ios-uitest-run.sh Switches runtime/device resolution to Xcode-provided simulator runtime, drops idb, and installs via simctl.
build/​ci/​tests/​.azure-devops-tests-windows-skia.yml Removes unused “test mac image/xcode” template parameters.
build/​ci/​tests/​.azure-devops-tests-skia-stages.yml Stops threading removed “test mac image/xcode” parameters through stages.
build/​ci/​tests/​.azure-devops-tests-macos-skia.yml Removes unused “test mac image/xcode” template parameters.
build/​ci/​tests/​.azure-devops-tests-linux-skia.yml Removes unused “test mac image/xcode” template parameters.
build/​ci/​tests/​.azure-devops-tests-ios-skia.yml Runs iOS Skia runtime tests on the build macOS image/Xcode.
build/​ci/​tests/​.azure-devops-tests-ios-skia-build.yml Removes unused “test mac image/xcode” template parameters.
build/​ci/​tests/​.azure-devops-tests-ios-native.yml Routes iOS native test jobs to the build image/Xcode and removes test-only Xcode parameter.
build/​ci/​publish/​.azure-devops-publish-ios-testflight.yml Removes unused test-only Xcode parameter.
build/​ci/​.azure-devops-stages.yml Stops passing macOS-14/Xcode 15.3 test parameters to templates.
.vsts-ci.yml Removes macOS-14/Xcode 15.3 variables.

💡 Add a code-review agent skill for context-aware, tailored reviews. Learn more in the docs.

Comment on lines +187 to +202
find_ios_device() {
xcrun simctl list devices --json | jq -r --arg sim "$UNO_UITEST_SIMULATOR_VERSION" --arg name "$UNO_UITEST_SIMULATOR_NAME" '
(.devices[$sim] // [])
| map(select(.name == $name and .isAvailable == true))
| .[0].udid // empty'
}

# Check for the presence of idb, and install it if it's not present
# NOTE: fb-idb currently breaks under Python 3.14 (asyncio get_event_loop change),
# so we pin fb-idb to Python 3.12 to avoid "There is no current event loop in thread 'MainThread'".
# Historical context: prior installs referenced an App Center issue/workaround.
# https://github.com/microsoft/appcenter/issues/2605#issuecomment-1854414963
export PATH=$PATH:~/.local/bin
UITEST_IOSDEVICE_ID=$(find_ios_device)

if ! command -v idb >/dev/null 2>&1
then
echo "Installing idb (fb-idb + idb-companion) pinned to Python 3.12"
# The images only pre-create current device models, and the screen size the tests were written
# against is the 12.9-inch iPad Pro, so create that device when it is missing.
if [ -z "$UITEST_IOSDEVICE_ID" ]; then
IOS_DEVICETYPE_ID=$(xcrun simctl list devicetypes --json | jq -r --arg name "$UNO_UITEST_SIMULATOR_NAME" '
.devicetypes
| map(select(.name == $name))
| .[0].identifier // empty')
Comment on lines +214 to +215
export UITEST_IOSDEVICE_ID
export UITEST_IOSDEVICE_DATA_PATH=$(xcrun simctl list devices --json | jq -r --arg udid "$UITEST_IOSDEVICE_ID" '[.devices[][] | select(.udid == $udid)][0].dataPath')
Comment on lines +173 to +177
UNO_UITEST_SIMULATOR_VERSION=$(xcrun simctl list runtimes --json | jq -r --arg ver "$IOS_SDK_VERSION" '
.runtimes
| map(select(.isAvailable == true and (.identifier | test("SimRuntime\.iOS")) and (.version | startswith($ver))))
| sort_by(.version | split(".") | map(tonumber? // 0))
| .[-1].identifier // empty')
# The images only pre-create current device models, and the screen size the tests were written
# against is the 12.9-inch iPad Pro, so create that device when it is missing.
if [ -z "$UITEST_IOSDEVICE_ID" ]; then
IOS_DEVICETYPE_ID=$(xcrun simctl list devicetypes --json | jq -r --arg name "$UNO_UITEST_SIMULATOR_NAME" '
Moves the remaining macOS-15 jobs to macOS-26, as on master. macOS-26 does not ship the iOS 26.1 simulator runtime that Xcode 26.1.1 needs, so Xcode moves to 26.6 (iOS 26.5 runtime preinstalled).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_018riLfyvT4wZYEh8iFjWkKa
@github-actions

github-actions Bot commented Oct 3, 2026 •

Copy link
Copy Markdown
Contributor

Claude finished @MartinZikmund's task in 2m 39s —— View job


Review complete ✅

  • Gather context (PR diff vs origin/servicing/6.8, changed files)
  • Review CI YAML changes (.vsts-ci.yml, build/ci/**)
  • Review build/test-scripts/ios-uitest-run.sh
  • Post inline findings
  • Post summary

Verdict: ship (minor, non-blocking finding only)

Findings: 1 minor

  • build/test-scripts/ios-uitest-run.sh:180,205 — duplicate UNOBLD008 error code used for two different failure causes (missing simulator runtime vs. missing device type), breaking the script's otherwise-unique error-code convention. See inline comment.

Verified clean:

  • All removed template parameters (vmMacImageTest, vmImageTest, xCodeRootTest, macOSVMImage_UITests, xCodeRoot_iOS_UITests) have no remaining references anywhere in build/ci/ or .vsts-ci.yml — the removal is complete and consistent across all 9 YAML files.
  • The idb/Homebrew/pipx removal in the shell script is thorough — no leftover references to idb, fb-idb, or idb-companion anywhere in the repo.
  • The dynamic simulator-runtime/device-type resolution (replacing the hardcoded iOS-17-5) correctly derives the runtime from the active Xcode's SDK version and falls back to creating the iPad Pro device when the image doesn't pre-provision it; the new simctl install retry-once path is a reasonable, simpler replacement for the old idb double-retry.
  • Script indentation (tabs) and structure match the rest of the file; no set -u/set -e hazards introduced.

Cross-cutting note (non-blocking): The PR description says the iOS shards use macOSVMImage/xCodeRoot = macOS-15 / Xcode 26.1.1, but the actual diff in .vsts-ci.yml (commit 71a4a8ad) bumps these to macOS-26 / Xcode 26.6. Worth updating the PR description for anyone using it as a changelog reference, though the code itself is internally consistent.

The removal of the old "wait for simulator to appear" retry loop in favor of a direct simctl list runtimes/simctl list devices query is a reasonable simplification given the new detection approach queries runtimes/device-types directly rather than polling for devices to appear — not flagged as a defect, but worth keeping an eye on in the CI run given the simulator jump from iOS 17.5 to 26.x that the PR description itself calls out as the main risk.

Comment on lines +180 to +205
echo "##vso[task.logissue type=error]UNOBLD008: No iOS $IOS_SDK_VERSION simulator runtime is available on this agent."
xcrun simctl list runtimes || true
exit 1
fi
fi
export UNO_UITEST_SIMULATOR_VERSION

echo "Waiting for the simulator to be available"
sleep 5
done

export DEVICELIST_FILEPATH=$LOG_FILEPATH/DeviceList-$LOG_PREFIX.json
echo "Listing iOS simulators to $DEVICELIST_FILEPATH"
xcrun simctl list devices --json > $DEVICELIST_FILEPATH
find_ios_device() {
xcrun simctl list devices --json | jq -r --arg sim "$UNO_UITEST_SIMULATOR_VERSION" --arg name "$UNO_UITEST_SIMULATOR_NAME" '
(.devices[$sim] // [])
| map(select(.name == $name and .isAvailable == true))
| .[0].udid // empty'
}

# Check for the presence of idb, and install it if it's not present
# NOTE: fb-idb currently breaks under Python 3.14 (asyncio get_event_loop change),
# so we pin fb-idb to Python 3.12 to avoid "There is no current event loop in thread 'MainThread'".
# Historical context: prior installs referenced an App Center issue/workaround.
# https://github.com/microsoft/appcenter/issues/2605#issuecomment-1854414963
export PATH=$PATH:~/.local/bin
UITEST_IOSDEVICE_ID=$(find_ios_device)

if ! command -v idb >/dev/null 2>&1
then
echo "Installing idb (fb-idb + idb-companion) pinned to Python 3.12"
# The images only pre-create current device models, and the screen size the tests were written
# against is the 12.9-inch iPad Pro, so create that device when it is missing.
if [ -z "$UITEST_IOSDEVICE_ID" ]; then
IOS_DEVICETYPE_ID=$(xcrun simctl list devicetypes --json | jq -r --arg name "$UNO_UITEST_SIMULATOR_NAME" '
.devicetypes
| map(select(.name == $name))
| .[0].identifier // empty')

# 1) Make sure we have a usable python3.12, but don't fail if Homebrew linking conflicts
if ! command -v python3.12 >/dev/null 2>&1; then
# Install, but ignore link-step failure; we'll use the keg path explicitly
brew list --versions python@3.12 >/dev/null 2>&1 || brew install python@3.12 || true
fi
# Prefer an existing python3.12 on PATH; otherwise use the keg path
PY312_BIN="$(command -v python3.12 || echo "$(brew --prefix)/opt/python@3.12/bin/python3.12")"
export PIPX_DEFAULT_PYTHON="$PY312_BIN"
echo "Using Python for pipx: $PIPX_DEFAULT_PYTHON"

# 2) Install helpers
brew list --versions pipx >/dev/null 2>&1 || brew install pipx
brew tap facebook/fb >/dev/null 2>&1 || true
# Pin the tap to the v1.1.8 formula. Its tip (1.5.0.b2) requires macOS
# Sequoia and Xcode 26, which the macOS 14 UI test agents cannot satisfy,
# so `brew install idb-companion` aborts with "Unsatisfied requirements".
# Detaching the tap checkout is enough: brew reads the formula straight
# from the working tree. HOMEBREW_NO_AUTO_UPDATE keeps a later `brew
# install` from fast-forwarding the tap back to its default branch.
export HOMEBREW_NO_AUTO_UPDATE=1
IDB_TAP_REVISION=c0386793f59da10c619787f2aa18d938ef1d69c9
IDB_TAP_REPO="$(brew --repo facebook/fb)"
if [ ! -d "$IDB_TAP_REPO/.git" ]; then
echo "Tap facebook/fb is not checked out at $IDB_TAP_REPO — cannot pin idb-companion." >&2
if [ -z "$IOS_DEVICETYPE_ID" ]; then
echo "##vso[task.logissue type=error]UNOBLD008: No '$UNO_UITEST_SIMULATOR_NAME' simulator device type is available on this agent."

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

UNOBLD008 is used here for two distinct failure causes: "no matching iOS simulator runtime" (line 180) and "no matching simulator device type" (line 205). Every other error code in this script (UNOBLD001/002/003/006/007) maps to exactly one condition, which is what makes them useful for triaging a failed CI run at a glance or grepping historical failures. Reusing the same code for two different root causes defeats that — give the device-type failure its own code (e.g. UNOBLD009).

Fix this →

@github-actions

github-actions Bot commented Oct 3, 2026 •

Copy link
Copy Markdown
Contributor

Claude finished @MartinZikmund's task in 2m 48s —— View job


Review complete

  • Gather context (diff vs origin/servicing/6.8, read changed files)
  • Review ios-uitest-run.sh logic (device/runtime resolution, simctl install, error handling)
  • Review Azure DevOps YAML template changes (parameter removal, image/Xcode routing)
  • Cross-check against AGENTS.md conventions
  • Post inline findings
  • Post summary

Verdict: ship (minor non-blocking nits)

This is a CI-only backport (shell script + Azure DevOps YAML), so most of the C#/XAML-specific review checklist (DependencyProperty, async/cancellation, SOLID, XAML styling) doesn't apply. I focused on script correctness and template wiring.

What I verified:

  • The vmMacImageTest/xCodeRootTest/macOSVMImage_UITests parameter removal is complete and consistent — no dangling references left anywhere in build/ci/** or .vsts-ci.yml.
  • All iOS job templates (*-ios-skia.yml, *-ios-native.yml, *-ios-skia-build.yml, publish/testflight) now correctly route through vmMacImage/xCodeRootBuild instead of the removed test-image parameters.
  • The three commits are properly scoped and each follows Conventional Commits.
  • No stray idb functional references remain (only historical comments explaining why it was dropped).
  • The latest commit (71a4a8a, pushed after the automated Copilot pass above) bumps macOSVMImage/xCodeRoot from macOS-15/Xcode 26.1.1 to macOS-26/Xcode 26.6, with accompanying comment-text generalizations (e.g. "which the image does not preinstall" instead of hardcoding an image name) — note the PR description still states macOS-15/Xcode 26.1.1, so it's now out of date relative to the branch; worth a quick edit before merge for anyone reading the description later.

On the automated Copilot findings above: I independently traced the same two code paths and they're technically accurate:

  • The single, non-retried find_ios_device() call right at script start (before this run's own simctl boot) reinstates a transient-empty-list risk the old polling loop existed specifically to avoid — plausible but low-blast-radius (worst case is an extra simulator device, not a failure), and set -e plus the explicit UNOBLD008 error paths mean an exhausted lookup fails loudly rather than silently.
  • UITEST_IOSDEVICE_DATA_PATH resolving to the literal string "null" on a lookup miss is real, but I checked and this variable isn't consumed anywhere else in the script or repo (it was already unused in the pre-PR version too) — so it's latent/cosmetic rather than something that can break the run today.

Neither blocks merge; both are reasonable tightening if the author wants to address them, since the PR's own stated risk is already the CI run itself validating iOS 17.5 → 26.x simulator behavior.
·

@unodevops

Copy link
Copy Markdown
Contributor

🤖 Your WebAssembly Skia Sample App stage site is ready! Visit it here: https://unowasmprstaging.z20.web.core.windows.net/pr-24964/wasm-skia-net9/index.html

.NET for iOS 26.1 refuses any Xcode but 26.1 ("requires Xcode 26.1. The current version of Xcode is 26.6"), so builds stay on Xcode 26.1.1, which macOS-26 also ships. That image has no iOS 26.1 simulator runtime, so build-only jobs skip the runtime check in Select Xcode, and the iOS runtime-test shards select Xcode 26.6, whose iOS 26.5 runtime is preinstalled.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_018riLfyvT4wZYEh8iFjWkKa
@github-actions

github-actions Bot commented Oct 3, 2026 •

Copy link
Copy Markdown
Contributor

Claude finished @MartinZikmund's task in 2m 28s —— View job


Review complete

  • Gather context (diff vs origin/servicing/6.8, changed files)
  • Review ios-uitest-run.sh changes
  • Review Azure DevOps YAML template changes
  • Cross-check against AGENTS.md conventions
  • Post inline findings
  • Post summary

Verdict: ship (minor, non-blocking findings only)

Findings: 4 minor, 0 blocking

  • build/test-scripts/ios-uitest-run.sh:187-194 — find_ios_device() runs once with no retry, reintroducing the transient-empty-list race the old polling loop existed to avoid; worst case is an extra/duplicate simulator rather than a hard failure.
  • build/test-scripts/ios-uitest-run.sh:205 — reuses the UNOBLD008 error code already used at line 180 for a different failure cause, breaking the script's otherwise one-code-per-condition convention. Flagged by an earlier automated pass too and still unaddressed.
  • build/test-scripts/ios-uitest-run.sh:215 — jq -r without // empty can resolve UITEST_IOSDEVICE_DATA_PATH to the literal string "null" on a lookup miss instead of failing loudly.
  • build/ci/tests/.azure-devops-tests-ios-skia.yml:45 — xCodeRoot: $(xCodeRootSimulator) reaches for a global pipeline variable directly instead of being threaded as a template parameter, unlike every other Xcode/image value in this PR.

Verified clean:

  • All removed template parameters (vmMacImageTest, vmImageTest, xCodeRootTest, macOSVMImage_UITests, xCodeRoot_iOS_UITests) have no remaining references anywhere in build/ci/ or .vsts-ci.yml.
  • The new requireSimulatorRuntime parameter on ios-build-select-version.yml is wired correctly: build-only jobs (iOS build, TestFlight publish) pass false to skip the runtime download, while the simulator-driving jobs still verify/download it.
  • .azure-devops-tests-ios-native.yml and the publish/testflight template now correctly route through xCodeRootBuild/vmImage instead of the removed test-only parameters, consistent with the PR description (both native and skia iOS test jobs move to the build image/Xcode, with skia runtime tests additionally pinned to the newer xCodeRootSimulator for the simulator runtime).
  • No stray idb/Homebrew-tap/pipx references remain in the script or repo.

Cross-cutting note (non-blocking): these are the same shell-script findings an earlier automated review pass on this PR already surfaced (duplicate error code, jq null handling, retry-loop removal) — they haven't been addressed in the four commits since, so re-flagging here. The PR's own stated risk (iOS 17.5 → 26.5 simulator jump, newer build Xcode than previously paired) is best validated by this PR's own CI run rather than by review alone.
· branch dev/mazi/ci-macos14-off-6.8

Comment on lines +187 to +194
find_ios_device() {
xcrun simctl list devices --json | jq -r --arg sim "$UNO_UITEST_SIMULATOR_VERSION" --arg name "$UNO_UITEST_SIMULATOR_NAME" '
(.devices[$sim] // [])
| map(select(.name == $name and .isAvailable == true))
| .[0].udid // empty'
}

# Check for the presence of idb, and install it if it's not present
# NOTE: fb-idb currently breaks under Python 3.14 (asyncio get_event_loop change),
# so we pin fb-idb to Python 3.12 to avoid "There is no current event loop in thread 'MainThread'".
# Historical context: prior installs referenced an App Center issue/workaround.
# https://github.com/microsoft/appcenter/issues/2605#issuecomment-1854414963
export PATH=$PATH:~/.local/bin
UITEST_IOSDEVICE_ID=$(find_ios_device)

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

find_ios_device() is called exactly once, with no retry. The retry loop it replaces (the one removed just above) existed specifically because simctl list devices can return an incomplete list for a few seconds right after a runtime/device is registered. With the new flow, a transient empty result here is indistinguishable from "device genuinely doesn't exist" and falls straight into the simctl create branch below — on a busy/booting agent this can create a duplicate "iPad Pro (12.9-inch) (6th generation)" simulator (two matches for find_ios_device() pre-existing + the new one isn't actually ambiguous since .[0] is taken, but it does leave an orphaned extra device and masks a possibly-valid existing one with a cold, freshly-created one, losing e.g. previously accepted permission dialogs).

A small bounded retry (e.g. 2-3 attempts with a short sleep) around find_ios_device before falling back to simctl create would preserve the original resiliency this script intentionally had for exactly this race.

Fix this →

if [ ! -d "$IDB_TAP_REPO/.git" ]; then
echo "Tap facebook/fb is not checked out at $IDB_TAP_REPO — cannot pin idb-companion." >&2
if [ -z "$IOS_DEVICETYPE_ID" ]; then
echo "##vso[task.logissue type=error]UNOBLD008: No '$UNO_UITEST_SIMULATOR_NAME' simulator device type is available on this agent."

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

UNOBLD008 is reused here for a second, distinct failure cause (missing simulator device type) — it's already used at line 180 for "no matching simulator runtime". Every other error code in this script (UNOBLD001/002/003/006/007) maps to exactly one condition, which is what makes grepping historical CI logs for a code a reliable way to triage. This was already flagged by an earlier automated review pass on this PR and hasn't been addressed in the subsequent commits — give this branch its own code (e.g. UNOBLD009).

Fix this →

fi

export UITEST_IOSDEVICE_ID
export UITEST_IOSDEVICE_DATA_PATH=$(xcrun simctl list devices --json | jq -r --arg udid "$UITEST_IOSDEVICE_ID" '[.devices[][] | select(.udid == $udid)][0].dataPath')

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

jq -r '... // empty' is missing here, unlike find_ios_device()/the devicetype lookup above. If UITEST_IOSDEVICE_ID somehow doesn't match any entry in simctl list devices --json (e.g. the just-created device isn't reflected yet), jq -r on a missing .dataPath prints the literal string "null" (non-empty), which export UITEST_IOSDEVICE_DATA_PATH then silently accepts instead of failing loudly. Recommend appending // empty to the jq filter and checking for an empty result the same way the runtime/devicetype lookups do.

Fix this →

TEST_KIND: runtime
xCodeRoot: ${{ parameters.xCodeRootTest }}
# The shards only drive a simulator; this Xcode's runtime ships preinstalled on the image.
xCodeRoot: $(xCodeRootSimulator)

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This reaches directly for the global pipeline variable $(xCodeRootSimulator) instead of being threaded through as a template parameter, unlike vmMacImage/xCodeRootBuild used two lines above in this same job, and unlike every other template touched in this PR (which all receive their Xcode/image values as ${{ parameters.* }}). It works today because .vsts-ci.yml is the only caller and defines xCodeRootSimulator globally, but it makes this template implicitly depend on top-level pipeline state rather than its declared parameter surface — harder to reuse or unit-test the template in isolation, and inconsistent with the parameter-threading convention the rest of this backport otherwise preserves. Consider adding an xCodeRootSimulator (or similarly named) parameter to this template's parameters: block and threading it down from .azure-devops-stages.yml like the other Xcode/image values.

actool refuses to compile asset catalogs without a simulator runtime that matches the SDK ("No simulator runtime version from [...] available to use with iphonesimulator SDK version 23B77"), so the build jobs need the runtime after all. Select Xcode downloads it as before; only the simulator test shards use Xcode 26.6.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_018riLfyvT4wZYEh8iFjWkKa
@github-actions

github-actions Bot commented Oct 3, 2026 •

Copy link
Copy Markdown
Contributor

Claude finished @MartinZikmund's task in 2m 4s —— View job


Review complete

  • Gather context (diff vs origin/servicing/6.8, read changed files)
  • Review ios-uitest-run.sh against prior findings to see what's still open
  • Review all 12 touched Azure DevOps YAML files for parameter-threading consistency
  • Cross-check PR description against the actual final diff
  • Post summary

Verdict: ship (minor, non-blocking)

This is a CI-only backport (shell script + Azure DevOps YAML), so most of the C#/XAML-specific checklist (DependencyProperty, async/cancellation, SOLID, XAML styling) doesn't apply. I re-verified the prior findings from earlier automated passes against the current HEAD and didn't re-post them as new inline comments since they're unchanged and already visible on the thread — summarized below instead to avoid duplicate noise.

Still open from earlier review passes (unaddressed across the 4 commits since they were first raised):

  • build/test-scripts/ios-uitest-run.sh:180,205 — duplicate UNOBLD008 error code for two distinct failure causes (missing runtime vs. missing device type).
  • build/test-scripts/ios-uitest-run.sh:194 — find_ios_device() now runs once with no retry, reinstating the transient-empty-list race the old polling loop guarded against.
  • build/test-scripts/ios-uitest-run.sh:215 — jq -r without // empty can resolve UITEST_IOSDEVICE_DATA_PATH to the literal string "null" on a lookup miss (currently latent/cosmetic — the variable isn't consumed elsewhere).
  • build/ci/tests/.azure-devops-tests-ios-skia.yml:45 — xCodeRoot: $(xCodeRootSimulator) reaches for the global pipeline variable directly instead of being threaded as a template parameter, unlike every other Xcode/image value in this PR.

New finding (this pass): PR description is out of date relative to the final diff.
The description's "What changed" section states build-only jobs "skip the runtime check in ios-build-select-version.yml (new requireSimulatorRuntime parameter)". That parameter doesn't exist anywhere in the repo — build/ci/templates/ios-build-select-version.yml has a net-zero diff in this PR (it was edited and then reverted back by the final two commits, 95dbdff6 and 1037a5e1, the latter titled "Download the iOS 26.1 runtime for build jobs again"). Every caller (ios-skia-build.yml, ios-native.yml, ios-runner.yml, .azure-devops-tests-templates.yml, .azure-devops-publish-ios-testflight.yml) still just passes xCodeRoot; the template unconditionally attempts the xcodebuild -downloadPlatform iOS download if the runtime is missing, for all jobs including build-only ones. The actual code is internally consistent and fine — only the description's "what changed" narrative is stale and should be corrected before merge so it doesn't mislead future readers using it as a changelog.

Verified clean (re-confirmed on current HEAD):

  • All removed template parameters (vmMacImageTest, vmImageTest, xCodeRootTest, macOSVMImage_UITests, xCodeRoot_iOS_UITests) have no remaining references anywhere in build/ci/ or .vsts-ci.yml.
  • ios-native.yml and the TestFlight publish template correctly route through vmImage/xCodeRootBuild everywhere the removed test-only parameters used to be referenced (including the commented-out Benchmarks job block).
  • No stray idb/Homebrew-tap/pipx references remain in the script or repo.
  • Commit messages all follow Conventional Commits.
    · branch dev/mazi/ci-macos14-off-6.8

@unodevops

Copy link
Copy Markdown
Contributor

🤖 Your WebAssembly Skia Sample App stage site is ready! Visit it here: https://unowasmprstaging.z20.web.core.windows.net/pr-24964/wasm-skia-net9/index.html

jq rejects "\." in a string literal, so every iOS shard aborted while resolving the simulator.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_018riLfyvT4wZYEh8iFjWkKa
@github-actions

github-actions Bot commented Oct 3, 2026 •

Copy link
Copy Markdown
Contributor

Claude finished @MartinZikmund's task in 1m 0s —— View job


Review complete

  • Gather context (diff vs origin/servicing/6.8, changed files, prior review history)
  • Review latest commit (dff6b0b1 fix(ci): Escape the jq regex for the simulator runtime) for new issues
  • Re-check previously flagged findings against current HEAD
  • Post any new inline findings
  • Post summary

Verdict: ship (minor, non-blocking — unchanged from prior passes)

This trigger fired on a new commit pushed since the last automated pass. I verified that commit in isolation and re-checked the file state as a whole; I did not re-post inline comments for findings already surfaced on this thread, to avoid further duplicate noise — they're summarized below instead.

The new commit (dff6b0b1) is a correct, necessary fix, not cosmetic: test("SimRuntime\.iOS") is invalid inside a jq string literal — JSON string escaping only recognizes a fixed set of escapes (\n, \t, \", \\, \/, \b, \f, \r, \uXXXX), and \. isn't one of them, so jq would reject the script at parse time and every iOS shard's simulator-version resolution (introduced earlier in this same PR, build/test-scripts/ios-uitest-run.sh:175) would abort before running any test. Escaping to \\. produces the literal two-character string \. for test() to interpret as an escaped-dot regex, which is correct. Good catch — this confirms the PR's own stated risk (the new Xcode/runtime resolution logic) was exercised for real by CI and caught a real bug.

Still open from earlier review passes on this PR (unaffected by the new commit, unaddressed across prior commits since first raised):

  • build/test-scripts/ios-uitest-run.sh:180,205 — duplicate UNOBLD008 error code used for two distinct failure causes (missing simulator runtime vs. missing device type), breaking the script's one-code-per-condition convention.
  • build/test-scripts/ios-uitest-run.sh:194 — find_ios_device() runs once with no retry, reinstating the transient-empty-list race the old polling loop existed to guard against (worst case: an extra/duplicate simulator, not a hard failure).
  • build/test-scripts/ios-uitest-run.sh:215 — jq -r without // empty can resolve UITEST_IOSDEVICE_DATA_PATH to the literal string "null" on a lookup miss instead of failing loudly (currently latent — not consumed elsewhere in the script).
  • build/ci/tests/.azure-devops-tests-ios-skia.yml:45 — xCodeRoot: $(xCodeRootSimulator) still reaches for the global pipeline variable directly instead of being threaded as a template parameter, unlike every other Xcode/image value in this PR.

Verified clean (re-confirmed on current HEAD):

  • All removed template parameters (vmMacImageTest, vmImageTest, xCodeRootTest, macOSVMImage_UITests, xCodeRoot_iOS_UITests) have no remaining references anywhere in build/ci/ or .vsts-ci.yml.
  • No stray idb/Homebrew-tap/pipx references remain in the script or repo.
  • Commit messages all follow Conventional Commits, including the new fix(ci): commit.

None of the four still-open items are blocking — they're either latent/cosmetic or a risk-reducing hardening opportunity rather than a correctness defect, and the PR's own stated risk (iOS 17.5 → 26.5 simulator jump) is best validated by this PR's CI run, which the latest commit shows is actively doing its job.

@unodevops

Copy link
Copy Markdown
Contributor

🤖 Your WebAssembly Skia Sample App stage site is ready! Visit it here: https://unowasmprstaging.z20.web.core.windows.net/pr-24964/wasm-skia-net9/index.html

The 6.x suite took 26-55 min on macOS-15 against a 60 min job timeout, and ran past it on macOS-26.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_018riLfyvT4wZYEh8iFjWkKa
@github-actions

github-actions Bot commented Oct 3, 2026 •

Copy link
Copy Markdown
Contributor

Claude finished @MartinZikmund's task in 37s —— View job


Review complete

  • Gather context (diff vs origin/servicing/6.8, identify what's new since the last review pass)
  • Re-verify previously flagged findings are still open
  • Check the new commit for new issues
  • Post summary

Verdict: ship (minor, non-blocking — unchanged from prior passes)

This trigger fired on a new commit (caac62b9, "Allow 120 minutes for the macOS desktop runtime tests") pushed since the last automated review pass covered dff6b0b1. That commit is a one-line, low-risk change: it doubles timeoutInMinutes for the Skia_macos_Runtime_Tests_Build job in build/ci/tests/.azure-devops-tests-macos-skia.yml from 60 to 120, with a comment explaining why (the suite already took up to 55 min on macOS-15 against the 60 min limit, and macOS-26 agents are slower). This is consistent with the PR's broader macOS-26 migration and doesn't introduce any new risk — no findings here.

Still open from earlier review passes on this PR (unaffected by the new commit, unaddressed across the commits since first raised):

  • build/test-scripts/ios-uitest-run.sh:180,205 — duplicate UNOBLD008 error code used for two distinct failure causes (missing simulator runtime vs. missing device type), breaking the script's one-code-per-condition convention.
  • build/test-scripts/ios-uitest-run.sh:194 — find_ios_device() runs once with no retry, reinstating the transient-empty-list race the old polling loop existed to guard against (worst case: an extra/duplicate simulator, not a hard failure).
  • build/test-scripts/ios-uitest-run.sh:215 — jq -r without // empty can resolve UITEST_IOSDEVICE_DATA_PATH to the literal string "null" on a lookup miss instead of failing loudly (currently latent — not consumed elsewhere in the script).
  • build/ci/tests/.azure-devops-tests-ios-skia.yml:45 — xCodeRoot: $(xCodeRootSimulator) still reaches for the global pipeline variable directly instead of being threaded as a template parameter, unlike every other Xcode/image value in this PR.

None of these are blocking — they're either latent/cosmetic or a risk-reducing hardening opportunity rather than a correctness defect. I did not re-post inline comments for them since they're already visible on the thread from earlier passes and unchanged.

Verified clean (re-confirmed on current HEAD):

  • All removed template parameters (vmMacImageTest, vmImageTest, xCodeRootTest, macOSVMImage_UITests, xCodeRoot_iOS_UITests) have no remaining references anywhere in build/ci/ or .vsts-ci.yml.
  • No stray idb/Homebrew-tap/pipx references remain in the script or repo.
  • Commit messages all follow Conventional Commits, including the new ci(macos): commit.
    · branch dev/mazi/ci-macos14-off-6.8

@unodevops

Copy link
Copy Markdown
Contributor

🤖 Your WebAssembly Skia Sample App stage site is ready! Visit it here: https://unowasmprstaging.z20.web.core.windows.net/pr-24964/wasm-skia-net9/index.html

@github-actions github-actions Bot added the platform/macos 🍏 Categorizes an issue or PR as relevant to the macOS platform label Oct 3, 2026
@github-actions

github-actions Bot commented Oct 3, 2026 •

Copy link
Copy Markdown
Contributor

Claude finished @MartinZikmund's task in 1m 33s —— View job


Review complete

  • Gather context (diff vs origin/servicing/6.8, identify what's new since the last review pass)
  • Review new commit 81165084 ("Stop two native-peer use-after-free crashes") — first pass over these files in this thread
  • Re-verify previously flagged CI findings are still open
  • Post summary

Verdict: ship (no new findings)

This trigger fired on a new commit (81165084) since the last automated pass, which previously only covered the CI/YAML scope. This commit is different in kind — it's a cherry-pick of an already-merged master fix (#24112, commit 2b3cef79) into UnoNativeMac/MacOSNativeElementHostingExtension/MacOSNativeWebView, fixing a real crash: native peers (NSView subclasses for the host element, WebView, media player) were created owned only by the autorelease pool, so a later uno_native_attach/uno_native_dispose round-trip could retain freed memory.

What I verified on this commit:

  • Objective-C ownership invariant (UNONative.m): uno_native_track() now takes the native-side strong reference at creation time (uno_native_create_sample, uno_mediaplayer_create_view, uno_webview_create all call it immediately after allocation), closing the original gap. The transients/elements handoffs in attach/detach always add the new owning reference before dropping the old one (no window where retain count could hit zero), and uno_native_dispose is correctly terminal — it drops both references and guards against a null element.
  • Managed-side disposal (MacOSNativeElementHostingExtension.cs): Disposal moved from the Unloaded event (which also fires on a reparent, i.e. not actually terminal) to the wrapper's finalizer, with a Disposed/zeroed-NativeHandle guard so a disposed peer's handle can never be handed back to native code. TryGetLiveElement centralizes that guard across all seven INativeElementHostingExtension members; MeasureNativeElement's fallback changed from Size.Empty (effectively (-∞,-∞), which the layouter doesn't clamp) to Size(0,0), which is correct given the new "live or not" branch replaces the previous always-present else.
  • MacOSDispatcher.DispatchNativeSingle (used to hop the finalizer-thread disposal call back to the main/AppKit thread) is pre-existing, already used elsewhere in this class (RaiseQueuedHistoryChange), so reusing it from a finalizer is consistent with established usage, not a new pattern to scrutinize.
  • MacOSNativeWebView.cs: every native call site now goes through TryGetHandle, and OnUnloaded unregisters by a remembered _registeredHandle rather than the live NativeHandle (which may already be zeroed by disposal racing unload) — correctly avoids keying _webViews at 0 across multiple disposed instances. uno_webview_register_message_handler now removes-then-adds the script message handler, fixing a second crash (addScriptMessageHandler:name: throws on a duplicate name when the same WKWebView is re-attached).
  • No platform-suffix, async/cancellation, or DependencyProperty issues — this is native interop/lifetime code, not UI-layer or property-system code.
  • Commit message is a correctly-scoped Conventional Commit (fix(macos): ...) and transparently documents it as an unchanged backport with a traceable cherry-pick source.

Still open from earlier review passes on the CI portion of this PR (unaffected by the new commit, unaddressed across all commits since first raised — not re-posted inline to avoid duplicate noise):

  • build/test-scripts/ios-uitest-run.sh:180,205 — duplicate UNOBLD008 error code for two distinct failure causes (missing simulator runtime vs. missing device type).
  • build/test-scripts/ios-uitest-run.sh:194 — find_ios_device() runs once with no retry, reinstating the transient-empty-list race the old polling loop guarded against.
  • build/test-scripts/ios-uitest-run.sh:215 — jq -r without // empty can resolve UITEST_IOSDEVICE_DATA_PATH to the literal string "null" on a lookup miss.
  • build/ci/tests/.azure-devops-tests-ios-skia.yml:45 — xCodeRoot: $(xCodeRootSimulator) reaches for the global pipeline variable directly instead of being threaded as a template parameter.

None of these are blocking — they're latent/cosmetic hardening opportunities, not correctness defects, and the PR's own stated risk (iOS 17.5 → 26.5 simulator jump) is being validated by this PR's CI run.
· branch dev/mazi/ci-macos14-off-6.8

@unodevops

Copy link
Copy Markdown
Contributor

🤖 Your WebAssembly Skia Sample App stage site is ready! Visit it here: https://unowasmprstaging.z20.web.core.windows.net/pr-24964/wasm-skia-net9/index.html

On the macOS-26 image these segfault the whole Desktop Skia macOS run on 6.x, in three out of three runs, taking every later test with it. They are already skipped on iOS and Win32 for #11997; master passes them, so the crash is 6.x-specific and needs a Mac to root-cause.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_018riLfyvT4wZYEh8iFjWkKa
@MartinZikmund
MartinZikmund force-pushed the dev/mazi/ci-macos14-off-6.8 branch from 8116508 to 7d05428 Compare October 3, 2026 23:02
@github-actions

github-actions Bot commented Oct 3, 2026 •

Copy link
Copy Markdown
Contributor

Claude finished @MartinZikmund's task in 1m 44s —— View job


Review complete

  • Gather context (diff vs origin/servicing/6.8, identify what's new since the last review pass)
  • Review against AGENTS.md rules (root-cause-first fixes, skipped-test policy, CI/YAML correctness)
  • Post inline findings
  • Post summary

Verdict: fix-first (1 new finding, not blocking CI but worth resolving before merge)

New since the last review pass: the branch was force-pushed/rebased, and the commit this thread previously reviewed in depth — 81165084, "Stop two native-peer use-after-free crashes" (a clean cherry-pick of master's #24112 fixing native-peer lifetime in MacOSNativeElementHostingExtension/UNONative.m/MacOSNativeWebView) — is no longer in the branch. In its place, the latest commit (7d05428f) disables the five ExecuteScriptAsync tests on SkiaMacOS in Given_WebView2.cs instead, per its own message because they "segfault the whole Desktop Skia macOS run on 6.x, in three out of three runs" — i.e. the same crash the dropped fix addressed. Posted as an inline finding with specifics and a fix link.

Why this matters: this swaps a root-cause fix for a defensive test-skip, which this repo's debugging discipline requires to be explicitly labeled and justified, not silently substituted. It also makes the PR description inaccurate — it still states "Also backports #24112 (macOS native-peer use-after-free)" as part of what changed, which is no longer true of the code on this branch.

Everything flagged in earlier passes on this thread remains open and unaffected by this change (not re-posted to avoid duplicate noise):

  • build/test-scripts/ios-uitest-run.sh:180,205 — duplicate UNOBLD008 error code for two distinct failure causes.
  • build/test-scripts/ios-uitest-run.sh:194 — find_ios_device() runs once with no retry, reinstating a transient-empty-list race.
  • build/test-scripts/ios-uitest-run.sh:215 — jq -r without // empty can resolve to the literal string "null" on a lookup miss.
  • build/ci/tests/.azure-devops-tests-ios-skia.yml:45 — xCodeRoot: $(xCodeRootSimulator) reaches for a global pipeline variable instead of being threaded as a template parameter.

Also re-verified as fine: the most recent CI-only commit (caac62b9, raising the macOS desktop runtime-test timeout from 60 to 120 min) is a low-risk, well-justified one-liner consistent with the rest of the macOS-26 migration — no issue there.
· branch dev/mazi/ci-macos14-off-6.8


[TestMethod]
[PlatformCondition(ConditionMode.Exclude, RuntimeTestPlatforms.SkiaUIKit | RuntimeTestPlatforms.NativeUIKit | RuntimeTestPlatforms.SkiaWin32)] // Temporarily disabled due to #11997
[PlatformCondition(ConditionMode.Exclude, RuntimeTestPlatforms.SkiaUIKit | RuntimeTestPlatforms.NativeUIKit | RuntimeTestPlatforms.SkiaWin32 | RuntimeTestPlatforms.SkiaMacOS)] // Temporarily disabled due to #11997; on macOS 26 these segfault the test process

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This commit (7d05428f) disables all five ExecuteScriptAsync tests on SkiaMacOS, but it replaces — rather than complements — the root-cause fix this same PR previously carried. Earlier commits on this branch (reviewed in a prior automated pass, commit 81165084, "Stop two native-peer use-after-free crashes", a cherry-pick of master's #24112) fixed exactly this segfault by correcting native-peer ownership/lifetime in MacOSNativeElementHostingExtension/UNONative.m/MacOSNativeWebView. That commit is no longer present in the current branch history (git diff origin/servicing/6.8...HEAD -- '*MacOSNative*' '*UNONative*' is empty) — it appears to have been dropped by a force-push/rebase between the last review pass and this one.

So the current state masks the crash with a test skip instead of applying the available root-cause fix, and the PR description still says "Also backports #24112 (macOS native-peer use-after-free)" — that's no longer true of the code as it stands, only of the description.

Per this repo's debugging discipline (root-cause fix vs. defensive/guard-only change must be labeled, and skipped tests must be flagged): either re-apply the #24112 cherry-pick (it was reportedly building and clean before), or, if it's being intentionally deferred, update the PR description to say so and open a tracking issue for the macOS-26-specific segfault referenced by "needs a Mac to root-cause" in the commit message, since none is linked here (unlike the #11997 reference the comment piggybacks on, which covers a different, pre-existing exclusion).

[Fix this →](https://claude.ai/code?q=In%20the%20dev%2Fmazi%2Fci-macos14-off-6.8%20branch%2C%20the%20commit%20'Stop%20two%20native-peer%20use-after-free%20crashes'%20(previously%20present%2C%20cherry-pick%20of%20%2324112)%20that%20fixed%20the%20macOS%20WebView2%20ExecuteScriptAsync%20segfault%20in%20MacOSNativeElementHostingExtension%2FUNONative.m%2FMacOSNativeWebView%20is%20missing%20from%20current%20history.%20Either%20re-apply%20that%20cherry-pick%20so%20the%20tests%20in%20Given_WebView2.cs%20don't%20need%20to%20be%20skipped%20on%20SkiaMacOS%2C%20or%20if%20deferring%20intentionally%2C%20update%20the%20PR%20description%20(which%20still%20claims%20the%20fix%20is%20included)%20and%20file%20a%20tracking%20issue%20for%20the%20macOS-26%20segfault.&repo=unoplatform/uno

@unodevops

Copy link
Copy Markdown
Contributor

🤖 Your WebAssembly Skia Sample App stage site is ready! Visit it here: https://unowasmprstaging.z20.web.core.windows.net/pr-24964/wasm-skia-net9/index.html

@unodevops

Copy link
Copy Markdown
Contributor

⚠️⚠️ The build 237296 has failed on Uno.UI - CI.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area/build Categorizes an issue or PR as relevant to build infrastructure platform/macos 🍏 Categorizes an issue or PR as relevant to the macOS platform

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants