[dotnet] Fix `dotnet test` on macOS and Mac Catalyst templates by jonathanpeppers · Pull Request #25963 · dotnet/macios

jonathanpeppers · 2026-07-07T16:46:12Z

dotnet test on the macostest and maccatalysttest templates was failing with "Zero tests ran" and Microsoft.Testing.Platform (MTP) handshake failures.

Two bugs, both needed for the desktop platforms to work:

The desktop test templates' Main.cs was dropping the incoming args when calling TestApplication.CreateBuilderAsync. dotnet test invokes the app with MTP protocol arguments appended to the command line (the server type and the named-pipe path for the test host), so discarding args disables the MTP handshake. Forward the current process' command-line arguments into CreateBuilderAsync alongside the existing --results-directory / --report-trx args. iOS and tvOS aren't affected because mlaunch forwards MTP env vars into the simulator and MTP picks up the protocol config from those env vars.
On macOS and Mac Catalyst the default dotnet run/test launch path goes through open (LaunchServices), which does not propagate the parent process' environment variables and does not reliably forward trailing CLI arguments to the app. Default RunWithOpen to false in Xamarin.Shared.Sdk.MSTest.props so MSTest projects launch the app binary directly. This is a no-op for iOS/tvOS (RunWithOpen is only consulted by the desktop targets).

Re-enable the previously-commented-out [TestCase] entries for maccatalysttest and macostest in DotNetTestTest, and skip the simulator boot/shutdown steps for desktop platforms.

Verified locally

dotnet new macostest / dotnet new maccatalysttest + dotnet test, using the default template's built-in tests (1 passing, 1 skipped, 1 expected-to-fail):

Test run summary: Failed!
  total: 3
  failed: 1
  succeeded: 1
  skipped: 1

No handshake failures, per-test results reported via MTP protocol, exit code matches the expected failure.

Copilot

Pull request overview

Fixes dotnet test execution for macOS and Mac Catalyst test templates by ensuring Microsoft.Testing.Platform (MTP) handshake arguments and launch behavior are compatible with desktop app launching semantics.

Changes:

Forward MTP protocol command-line arguments from the running process into TestApplication.CreateBuilderAsync in all test templates.
Default MSTest desktop launches to execute the app binary directly (disable open/LaunchServices path) to preserve CLI args and environment for MTP.
Re-enable macOS/Mac Catalyst coverage in DotNetTestTest and avoid simulator boot/shutdown for desktop platforms.

Reviewed changes

Copilot reviewed 4 out of 4 changed files in this pull request and generated 2 comments.

Show a summary per file

File	Description
tests/dotnet/UnitTests/DotNetTestTest.cs	Re-enables macOS/Mac Catalyst template coverage and skips simulator management for desktop runs.
dotnet/Templates/Microsoft.tvOS.Templates/tvostest/csharp/Main.cs	Forwards MTP args into the test app builder.
dotnet/Templates/Microsoft.macOS.Templates/macostest/csharp/Main.cs	Forwards MTP args into the test app builder (macOS).
dotnet/Templates/Microsoft.MacCatalyst.Templates/maccatalysttest/csharp/Main.cs	Forwards MTP args into the test app builder (Mac Catalyst).
dotnet/Templates/Microsoft.iOS.Templates/iostest/csharp/Main.cs	Forwards MTP args into the test app builder.
dotnet/targets/Xamarin.Shared.Sdk.MSTest.props	Sets `RunWithOpen` default to `false` for MSTest projects to avoid LaunchServices argument/env propagation issues.

rolfbjarne · 2026-07-07T17:34:13Z

+				// Forward args from 'dotnet test', which sends the MTP protocol config
+				// (--server / --dotnet-test-pipe) via the command line.
+				string [] cliArgs = Environment.GetCommandLineArgs ();
+				string [] mtpArgs = cliArgs.Length > 1 ? cliArgs [1..] : [];


This seems to drop the first argument, but I can't see any explanation/comment of why this is necessary?

Added a comment noting that Environment.GetCommandLineArgs()[0] is the executable path, not an argument.

`dotnet test` on the `macostest` and `maccatalysttest` templates was failing with "Zero tests ran" and Microsoft.Testing.Platform (MTP) handshake failures. Two bugs, both needed for the desktop platforms to work: 1. The desktop test templates' `Main.cs` was dropping the incoming `args` when calling `TestApplication.CreateBuilderAsync`. `dotnet test` invokes the app with MTP protocol arguments appended to the command line (the server type and the named-pipe path for the test host), so discarding `args` disables the MTP handshake. Forward the current process' command-line arguments into `CreateBuilderAsync` alongside the existing `--results-directory` / `--report-trx` args. iOS and tvOS aren't affected because mlaunch forwards MTP env vars into the simulator and MTP picks up the protocol config from those env vars. 2. On macOS and Mac Catalyst the default `dotnet run/test` launch path goes through `open` (LaunchServices), which does not propagate the parent process' environment variables and does not reliably forward trailing CLI arguments to the app. Default `RunWithOpen` to `false` in `Xamarin.Shared.Sdk.MSTest.props` so MSTest projects launch the app binary directly. This is a no-op for iOS/tvOS (`RunWithOpen` is only consulted by the desktop targets). Re-enable the previously-commented-out `[TestCase]` entries for `maccatalysttest` and `macostest` in `DotNetTestTest`, and skip the simulator boot/shutdown steps for desktop platforms. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>

vs-mobiletools-engineering-service2 · 2026-07-07T18:02:42Z

✅ [CI Build #7e67820] Prepare .NET Release succeeded ✅

📦 Published NuGet packages (32 packages)

iOS

Microsoft.iOS.Ref.net11.0_26.5.26.5.11824-net11-p7.nupkg
Microsoft.iOS.Runtime.ios-arm64.net11.0_26.5.26.5.11824-net11-p7.nupkg
Microsoft.iOS.Runtime.ios.net11.0_26.5.26.5.11824-net11-p7.nupkg
Microsoft.iOS.Runtime.iossimulator-arm64.net11.0_26.5.26.5.11824-net11-p7.nupkg
Microsoft.iOS.Runtime.iossimulator-x64.net11.0_26.5.26.5.11824-net11-p7.nupkg
Microsoft.iOS.Sdk.net11.0_26.5.26.5.11824-net11-p7.nupkg
Microsoft.iOS.Templates.26.5.11824-net11-p7.nupkg
Microsoft.iOS.Windows.Sdk.net11.0_26.5.26.5.11824-net11-p7.nupkg
Microsoft.NET.Sdk.iOS.Manifest-11.0.100-preview.7.26.5.11824-net11-p7.nupkg

MacCatalyst

Microsoft.MacCatalyst.Ref.net11.0_26.5.26.5.11824-net11-p7.nupkg
Microsoft.MacCatalyst.Runtime.maccatalyst-arm64.net11.0_26.5.26.5.11824-net11-p7.nupkg
Microsoft.MacCatalyst.Runtime.maccatalyst-x64.net11.0_26.5.26.5.11824-net11-p7.nupkg
Microsoft.MacCatalyst.Runtime.maccatalyst.net11.0_26.5.26.5.11824-net11-p7.nupkg
Microsoft.MacCatalyst.Sdk.net11.0_26.5.26.5.11824-net11-p7.nupkg
Microsoft.MacCatalyst.Templates.26.5.11824-net11-p7.nupkg
Microsoft.NET.Sdk.MacCatalyst.Manifest-11.0.100-preview.7.26.5.11824-net11-p7.nupkg

macOS

Microsoft.macOS.Ref.net11.0_26.5.26.5.11824-net11-p7.nupkg
Microsoft.macOS.Runtime.osx-arm64.net11.0_26.5.26.5.11824-net11-p7.nupkg
Microsoft.macOS.Runtime.osx-x64.net11.0_26.5.26.5.11824-net11-p7.nupkg
Microsoft.macOS.Runtime.osx.net11.0_26.5.26.5.11824-net11-p7.nupkg
Microsoft.macOS.Sdk.net11.0_26.5.26.5.11824-net11-p7.nupkg
Microsoft.macOS.Templates.26.5.11824-net11-p7.nupkg
Microsoft.NET.Sdk.macOS.Manifest-11.0.100-preview.7.26.5.11824-net11-p7.nupkg

tvOS

Microsoft.NET.Sdk.tvOS.Manifest-11.0.100-preview.7.26.5.11824-net11-p7.nupkg
Microsoft.tvOS.Ref.net11.0_26.5.26.5.11824-net11-p7.nupkg
Microsoft.tvOS.Runtime.tvos-arm64.net11.0_26.5.26.5.11824-net11-p7.nupkg
Microsoft.tvOS.Runtime.tvos.net11.0_26.5.26.5.11824-net11-p7.nupkg
Microsoft.tvOS.Runtime.tvossimulator-arm64.net11.0_26.5.26.5.11824-net11-p7.nupkg
Microsoft.tvOS.Runtime.tvossimulator-x64.net11.0_26.5.26.5.11824-net11-p7.nupkg
Microsoft.tvOS.Sdk.net11.0_26.5.26.5.11824-net11-p7.nupkg
Microsoft.tvOS.Templates.26.5.11824-net11-p7.nupkg

Other

Sharpie.Bind.Tool.26.6.0.428-net11-p7.nupkg

Pipeline on Agent
Hash: 7e67820de4a03a0e56a559c56acd51c2cd7cd600 [PR build]

vs-mobiletools-engineering-service2 · 2026-07-07T20:16:40Z

✅ [PR Build #4da16b4] Build passed (Build packages) ✅

Pipeline on Agent
Hash: 4da16b4b645ad2298adcde6e0c7351cc1c26dc0c [PR build]

vs-mobiletools-engineering-service2 · 2026-07-07T21:51:22Z

🔥 [CI Build #4da16b4] Test results 🔥

Test results

❌ Tests failed on VSTS: test results

0 tests crashed, 3 tests failed, 253 tests passed.

Failures

❌ dotnettests tests (iOS)

1 tests failed, 0 tests passed.

Failed tests

DotNet tests: Failed (Execution failed with exit code 1)
- Xamarin.Tests.DotNetTestTest.DotNetTest(iOS,"iostest"): 'dotnet test' failed with exit code 1.
  Binlog: /Users/cloudtest/vss/_work/1/s/macios/tests/dotnet/UnitTests/bin/Debug/net11.0/...
- Xamarin.Tests.DotNetWatchTest.DotNetWatch(iOS,False,False): Timed out waiting for the app to start. Output:
  dotnet watch \ud83d\udd25 Hot reload enabled. For a list of supported edits, see...

Html Report (VSDrops) Download

❌ dotnettests tests (tvOS)

1 tests failed, 0 tests passed.

Failed tests

DotNet tests: Failed (Execution failed with exit code 1)
- Xamarin.Tests.DotNetTestTest.DotNetTest(TVOS,"tvostest"): 'dotnet test' failed with exit code 1.
  Binlog: /Users/cloudtest/vss/_work/1/s/macios/tests/dotnet/UnitTests/bin/Debug/net11.0/...

Html Report (VSDrops) Download

❌ monotouch tests (iOS)

1 tests failed, 22 tests passed.

Failed tests

monotouch-test/iOS - simulator/Release (all optimizations): Failed

Html Report (VSDrops) Download

Successes

✅ assembly-processing: All 1 tests passed. Html Report (VSDrops) Download
✅ cecil: All 1 tests passed. Html Report (VSDrops) Download
✅ dotnettests (MacCatalyst): All 1 tests passed. Html Report (VSDrops) Download
✅ dotnettests (macOS): All 1 tests passed. Html Report (VSDrops) Download
✅ dotnettests (Multiple platforms): All 1 tests passed. Html Report (VSDrops) Download
✅ framework: All 2 tests passed. Html Report (VSDrops) Download
✅ fsharp: All 4 tests passed. Html Report (VSDrops) Download
✅ generator: All 5 tests passed. Html Report (VSDrops) Download
✅ interdependent-binding-projects: All 4 tests passed. Html Report (VSDrops) Download
✅ introspection: All 7 tests passed. Html Report (VSDrops) Download
✅ linker (iOS): All 31 tests passed. Html Report (VSDrops) Download
✅ linker (MacCatalyst): All 31 tests passed. Html Report (VSDrops) Download
✅ linker (macOS): All 21 tests passed. Html Report (VSDrops) Download
✅ linker (tvOS): All 31 tests passed. Html Report (VSDrops) Download
✅ monotouch (MacCatalyst): All 23 tests passed. Html Report (VSDrops) Download
✅ monotouch (macOS): All 18 tests passed. Html Report (VSDrops) Download
✅ monotouch (tvOS): All 23 tests passed. Html Report (VSDrops) Download
✅ msbuild: All 2 tests passed. Html Report (VSDrops) Download
✅ sharpie: All 1 tests passed. Html Report (VSDrops) Download
✅ windows: All 3 tests passed. Html Report (VSDrops) Download
✅ xcframework: All 4 tests passed. Html Report (VSDrops) Download
✅ xtro: All 1 tests passed. Html Report (VSDrops) Download

macOS tests

✅ Tests on macOS Sonoma (14): All 5 tests passed. Html Report (VSDrops) Download
✅ Tests on macOS Sequoia (15): All 5 tests passed. Html Report (VSDrops) Download
✅ Tests on macOS Tahoe (26): All 5 tests passed. Html Report (VSDrops) Download

Linux Build Verification

✅ Linux build succeeded

Pipeline on Agent
Hash: 4da16b4b645ad2298adcde6e0c7351cc1c26dc0c [PR build]

vs-mobiletools-engineering-service2 · 2026-07-07T22:07:05Z

✅ [CI Build #4da16b4] Build passed (Build macOS tests) ✅

Pipeline on Agent
Hash: 4da16b4b645ad2298adcde6e0c7351cc1c26dc0c [PR build]

vs-mobiletools-engineering-service2 · 2026-07-07T22:26:38Z

🔥 [PR Build #4da16b4] Build failed (Detect API changes) 🔥

Build failed for the job 'Detect API changes' (with job status 'Failed')

Pipeline on Agent
Hash: 4da16b4b645ad2298adcde6e0c7351cc1c26dc0c [PR build]

vs-mobiletools-engineering-service2 · 2026-07-07T22:29:42Z

🔥 Unable to find the contents for the comment: D:\a\1\s\change-detection\results\gh-comment.md does not exist :fire

Pipeline on Agent
Hash: 4da16b4b645ad2298adcde6e0c7351cc1c26dc0c [PR build]

Copilot AI review requested due to automatic review settings July 7, 2026 16:46

jonathanpeppers requested review from dalexsoto, mauroa and rolfbjarne as code owners July 7, 2026 16:46

jonathanpeppers changed the base branch from main to net11.0 July 7, 2026 16:46

Copilot started reviewing on behalf of jonathanpeppers July 7, 2026 16:46 View session

jonathanpeppers force-pushed the jonathanpeppers-test-maccatalyst-macos-dotnet-test branch from 2f097c9 to b6903ad Compare July 7, 2026 16:49

Copilot AI reviewed Jul 7, 2026

View reviewed changes

Comment thread dotnet/Templates/Microsoft.macOS.Templates/macostest/csharp/Main.cs Outdated

Comment thread dotnet/Templates/Microsoft.MacCatalyst.Templates/maccatalysttest/csharp/Main.cs Outdated

jonathanpeppers force-pushed the jonathanpeppers-test-maccatalyst-macos-dotnet-test branch 2 times, most recently from cd320db to 46b0078 Compare July 7, 2026 16:55

This comment has been minimized.

Sign in to view

rolfbjarne requested changes Jul 7, 2026

View reviewed changes

jonathanpeppers force-pushed the jonathanpeppers-test-maccatalyst-macos-dotnet-test branch from 46b0078 to 4da16b4 Compare July 7, 2026 17:57

Uh oh!

Conversation

jonathanpeppers commented Jul 7, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Verified locally

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Reviewed changes

Uh oh!

Uh oh!

Uh oh!

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

Uh oh!

Uh oh!

rolfbjarne Jul 7, 2026

Choose a reason for hiding this comment

Uh oh!

jonathanpeppers Jul 7, 2026

Choose a reason for hiding this comment

Uh oh!

vs-mobiletools-engineering-service2 commented Jul 7, 2026

✅ [CI Build #7e67820] Prepare .NET Release succeeded ✅

iOS

MacCatalyst

macOS

tvOS

Other

Uh oh!

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

vs-mobiletools-engineering-service2 commented Jul 7, 2026

✅ [PR Build #4da16b4] Build passed (Build packages) ✅

Uh oh!

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

vs-mobiletools-engineering-service2 commented Jul 7, 2026

🔥 [CI Build #4da16b4] Test results 🔥

Test results

Failures

❌ dotnettests tests (iOS)

Failed tests

❌ dotnettests tests (tvOS)

Failed tests

❌ monotouch tests (iOS)

Failed tests

Successes

macOS tests

Linux Build Verification

Uh oh!

vs-mobiletools-engineering-service2 commented Jul 7, 2026

✅ [CI Build #4da16b4] Build passed (Build macOS tests) ✅

Uh oh!

vs-mobiletools-engineering-service2 commented Jul 7, 2026

🔥 [PR Build #4da16b4] Build failed (Detect API changes) 🔥

jonathanpeppers commented Jul 7, 2026 •

edited

Loading