Commit graph

17 commits

Author SHA1 Message Date
Lawrence Chen
a592ed126a
Add failing regression test for browser find focus (#1891)
* test: add browser find focus regression

* ci: run browser find focus regression on prs

* ci: merge pr ui regressions into one job

* fix: focus browser find after navigation

* test: rename ui regressions workflow guard

* Fix browser find focus after address bar navigation

* Fix test-e2e workflow manifest script

* Address PR review feedback

* Stabilize browser focus regression on CI

---------

Co-authored-by: Lawrence Chen <lawrencecchen@users.noreply.github.com>
2026-03-20 22:12:26 -07:00
Lawrence Chen
63e65a7f5c
Merge pull request #1724 from manaflow-ai/feat-display-harness-manifest-env
Fix display harness manifest handoff in UI test
2026-03-18 06:07:56 -07:00
Lawrence Chen
fbe209cb33
Read the prebuilt display helper path from the harness manifest 2026-03-18 03:30:32 -07:00
Lawrence Chen
7fb1f50966
Launch display helper from inside the UI test sandbox 2026-03-18 03:22:04 -07:00
Lawrence Chen
377d62e8fe
Fix empty env prefix in e2e workflow 2026-03-18 03:19:29 -07:00
Lawrence Chen
1c6d5568f7
Move display harness rendezvous files to /tmp 2026-03-18 03:13:23 -07:00
Lawrence Chen
95cd26c0b6
Restore stable display harness manifest path 2026-03-18 01:46:32 -07:00
Lawrence Chen
b1e2d1cb19
Pass display harness manifest path into UI test 2026-03-18 01:40:04 -07:00
Lawrence Chen
798c1fbc42
Fix laggy terminal sync during sidebar drags (#1598)
* Fix sidebar drag terminal resize lag

* Add display resolution churn regression

* Prelaunch display churn helper in e2e workflow

* Use manifest handoff for display churn UI test

* Fix e2e display churn harness startup

* Resolve display churn UI test socket path

* Use marker-based socket discovery in display UI test

* Add failing sidebar drag portal regression tests

* Fix sidebar drag terminal portal resize lag

* Add failing scoped resize regression tests

* Fix terminal portal resize scheduling lag

* Add failing zsh resize prompt regression test

* Fix zsh resize prompt duplication

* Fix Sequoia sidebar resize regression

* Guard display-resolution CI runner

* Run display-resolution CI on WarpBuild

* Allow backgrounded display regression app launch

* Launch display regression app directly

* Launch display regression app via NSWorkspace

* Load display regression launch env from manifest

* Write display regression manifest in runner temp dir

* Write display regression manifest in shared tmp

* Write display regression manifest in repo scratch dir

* Launch display regression app with explicit env

* Avoid xcodebuild broken pipe in compat CI

* Launch display regression via XCUIApplication

* Harden display regression socket readiness

* Trust display socket diagnostics path

* Replace display socket probe with render diagnostics

* Write display churn start marker atomically

* Move display churn harness out of /tmp

---------

Co-authored-by: Lawrence Chen <lawrencecchen@users.noreply.github.com>
2026-03-18 01:28:11 -07:00
Lawrence Chen
a561a272c1
Migrate CI/CD to WarpBuild, consolidate test jobs (#1501)
* Migrate CI/CD to WarpBuild, consolidate test jobs

Replace all macOS runner labels across workflows:
- depot-macos-latest → warp-macos-15-arm64-6x
- macos-15 → warp-macos-15-arm64-6x
- macos-14 → warp-macos-14-arm64-6x

Consolidates tests + tests-depot into a single tests job that runs
unit tests, regressions, UI tests, and lag tests sequentially on one
WarpBuild runner. Ubuntu jobs remain on ubuntu-latest.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* Upgrade stale zig on runners that have an outdated version pre-installed

WarpBuild macos-14 ships zig 0.15.1 but the project requires 0.15.2.
The install step skipped because zig was found, just outdated.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* Pin zig 0.15.2 via direct tarball instead of Homebrew

Homebrew's zig bottle for macOS 14 (Sonoma) is stuck at 0.15.1 but the
ghostty submodule requires 0.15.2. Download zig directly from
ziglang.org to guarantee the correct version on all runner images.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* Fix zig tarball URL: arch-os order is aarch64-macos, not macos-aarch64

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* Create /usr/local/bin and /usr/local/lib before copying zig

WarpBuild runners don't have /usr/local/lib by default.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* Add 20-min timeout to WarpBuild jobs

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* Fix UI test hang: stream output instead of variable capture, use GitHub runner for macOS 14

The OUTPUT=$(...) pattern buffers all xcodebuild output into a bash
variable. For the full cmux scheme (build + UI tests), this can be
hundreds of MB, causing the shell to hang. Replace with tee streaming.

macOS 14 on WarpBuild consistently hangs (unit tests timeout at 20min
vs 4min on macOS 15, same M4 Pro hardware). Use GitHub-hosted macos-14
runner for compat tests instead, which works on main today.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* Split UI tests to GitHub-hosted runner (WarpBuild can't activate GUI apps)

WarpBuild macOS VMs leave XCUIApplication stuck in "Running Background"
state, causing every UI test to burn ~62s waiting for activation and
timing out the job. Root cause: WarpBuild ephemeral VMs don't provide
a full GUI session for app activation.

Split CI into parallel jobs:
- tests: WarpBuild (unit tests + regressions, ~6 min)
- tests-ui: GitHub-hosted macos-15 (UI tests + lag regression)

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* Move tests-ui to WarpBuild with TCC permission grants

Grant accessibility, post-event, and screen capture TCC permissions
to Xcode and XCTest processes on WarpBuild ephemeral VMs. This should
fix "Failed to activate application (Running Background)" errors that
prevent XCUITests from bringing the app to foreground.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* Add GUI session diagnostics and DevToolsSecurity for WarpBuild UI tests

Add session diagnostics (who, console user, GUI domain, WindowServer,
loginwindow) to understand WarpBuild VM session state. Also enable
DevToolsSecurity and security authorizationdb for XCTest process
control. Try bootstrapping GUI session if missing.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* Fix TCC permissions: use Xcode-Helper + user DB (CircleCI approach)

Previous TCC grants used wrong client IDs (com.apple.dt.Xcode) and
only wrote to the system database. CircleCI's proven approach grants:
- kTCCServiceAccessibility to com.apple.dt.Xcode-Helper (not Xcode)
- kTCCServiceDeveloperTool to com.apple.Terminal
- Both system AND user-level TCC databases

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* Reduce UI test timeout to 15s for WarpBuild expected failures

WarpBuild Virtualization.framework VMs cannot activate macOS GUI apps
(XCUIApplication stuck "Running Background"). Tests still execute and
report expected failures. But the 62s per-test activation timeout
makes 30+ tests take 30+ minutes total.

Set per-test timeout to 15s so expected failures resolve quickly.
Full interactive UI test coverage runs via test-e2e.yml on
GitHub-hosted runners with proper display support.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* Replace XCUITest run with build + lag regression on WarpBuild

WarpBuild Virtualization.framework VMs cannot activate macOS GUI apps
(XCUIApplication stuck "Running Background" with 62s activation
timeout per test). Tried TCC permissions, DevToolsSecurity, virtual
display, reduced timeouts, nothing fixes the framework-level issue.

Replace tests-ui job with tests-build-and-lag:
- Build the full cmux scheme (verifies compilation)
- Run workspace churn typing-lag regression (socket-based, no GUI)
- XCUITests run via test-e2e.yml on GitHub-hosted runners

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* Move macOS 14 compat to WarpBuild (no GitHub-hosted runners)

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* Add diagnostic workflow to probe WarpBuild GUI activation

Tests multiple app activation approaches on WarpBuild VMs:
- open -a, NSWorkspace, NSRunningApplication.activate, osascript
- Virtual display state before/after CGVirtualDisplay
- TCC/accessibility permissions, Quartz session info
- VM type detection

This is a workflow_dispatch-only diagnostic to determine if
XCUITest can work on WarpBuild with the right configuration.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* Trigger GUI probe on branch push (workflow_dispatch needs main)

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* Rewrite GUI probe with Swift (Python lacks AppKit on WarpBuild)

v1 failed because WarpBuild's Python isn't a framework build and
can't import AppKit/Quartz. v2 uses a compiled Swift binary to test
NSRunningApplication.activate(), osascript, Quartz session state,
display info, and AX trust.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* GUI probe v3: try 5 approaches to unlock WarpBuild screen

1. defaults write (screensaver, loginwindow, pmset)
2. automationmodetool enable-automationmode-without-authentication
3. CGSSessionSetScreenLocked private API + System Events keystroke
4. sysadminctl -screenLock off + keychain unlock
5. CGEvent simulation (mouse move + Return key to dismiss lock)

Each approach is followed by an activation check to see if it worked.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* Test GUI activation on macOS 14, 15, and 26 (Tahoe)

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* Add DerivedData and GhosttyKit caching to CI workflows

Major caching improvements across ci.yml and ci-macos-compat.yml:

- Cache GhosttyKit.xcframework keyed on ghostty submodule SHA
  (skip download on cache hit)
- Cache DerivedData keyed on OS + Xcode version + Package.resolved +
  project.pbxproj (enables incremental builds across runs)
- Remove explicit DerivedData wipe (rely on cache key invalidation)
- Use download-prebuilt-ghosttykit.sh in compat workflow too

This should significantly speed up macOS 14 compat tests which were
taking 20+ min due to full recompilation every run.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* Bump macOS 14 compat timeout to 45 min for cold cache seeding

The DerivedData cache wasn't saved because the job timed out at 30 min,
causing the post-job cache save step to be skipped. 45 min gives enough
headroom for the first uncached run to complete and seed the cache.
Subsequent runs should be much faster with incremental builds.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* Use Depot runners for E2E tests (WarpBuild has screen lock on macOS 15/26)

WarpBuild VMs on macOS 15 and 26 have CGSSessionScreenIsLocked=1, which
prevents XCUIApplication activation. Depot runners have working GUI
activation. Can switch back to WarpBuild once they fix the VM images.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* Skip smoke test on macOS 14 compat, remove GUI diagnostic workflow

macOS 14 was slow because it built the full app (cmux scheme) on top of
unit tests (cmux-unit scheme). Unit tests are the real compat check;
smoke test runs on macOS 15 only. Also removes the temporary
test-warpbuild-gui.yml diagnostic workflow.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* Replace Sonoma with Tahoe in compat matrix, drop macOS 14

Swap macOS 14 (Sonoma) for macOS 26 (Tahoe). Smoke test runs on
macOS 15 only (WarpBuild screen lock blocks app activation on 26).

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* Drop macOS 26 from compat matrix (zig 0.15.2 linker failure)

Zig 0.15.2 can't link against the macOS 26 (Tahoe) SDK: undefined
symbols for basic libc functions (_abort, _free, _fork, etc.). The zig
toolchain needs an update to support Tahoe. Keep macOS 15 only for now.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: Lawrence Chen <lawrencecchen@users.noreply.github.com>
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-17 01:00:14 -07:00
Lawrence Chen
f576d2233f
Bundle Ghostty theme picker helper 2026-03-14 22:40:36 -07:00
Lawrence Chen
c37ace5deb
Cache Swift packages across CI runs (#998)
* Cache Swift packages across CI runs

Add actions/cache for the SPM cloned source packages directory so
subsequent runs skip fetching Sparkle, sentry-cocoa, swift-markdown-ui,
posthog-ios, and NetworkImage from GitHub each time.

- nightly/release: replace the no-op SwiftPM cache step with
  actions/cache + -clonedSourcePackagesDirPath on xcodebuild
- ci/ci-macos-compat/test-e2e: add actions/cache before the existing
  resolve step, stop deleting the cache dir each run

* Include runner in test-e2e cache key

Consistent with ci-macos-compat.yml which uses matrix.os in the key.
2026-03-05 23:26:05 -08:00
Lawrence Chen
37dc43a6de
Fix TCC dialog, trim black frames, add macos-26 runner option (#784)
- Grant kTCCServiceScreenCapture in system-level TCC database (sudo)
  and pre-date ScreenCaptureApprovals.plist to suppress Sequoia's
  private window picker dialog
- Move recording start to right before xcodebuild test (skip build time)
- Trim leading black frames from video using ffmpeg blackdetect
- Add runner input: macos-15 (Sequoia) or macos-26 (Tahoe)
2026-03-02 23:14:27 -08:00
Lawrence Chen
d77299c220
Clean up E2E ffmpeg device detection (#782)
* Install ffmpeg via brew for screen recording

macos-15 GitHub runners don't have ffmpeg pre-installed.

* Clean up ffmpeg device detection and add fallback

Suppress noisy device listing errors, add fallback to index 1 if
detected index fails, upgrade warning to error on total failure.
2026-03-02 22:55:03 -08:00
Lawrence Chen
63e7cc7faa
Install ffmpeg via brew for screen recording (#781)
macos-15 GitHub runners don't have ffmpeg pre-installed.
2026-03-02 22:45:22 -08:00
Lawrence Chen
3cb101f1c8
Switch screen recording from screencapture to ffmpeg AVFoundation (#780)
screencapture -v produces a 0-second file on GitHub Actions M-series
runners (IOServiceMatchingfailed for AppleM2ScalerParavirtDriver).
ffmpeg with avfoundation input handles virtual displays correctly.
2026-03-02 22:36:07 -08:00
Lawrence Chen
cd0c3cfa93
Add E2E test workflow with video recording (#778)
* Add E2E test workflow with video recording and issue posting

New workflow_dispatch workflow (test-e2e.yml) that runs XCUITests on
GitHub-hosted macos-15 runners, records the virtual display, uploads the
video as an artifact, and posts results as an issue on cmux-dev-artifacts.

Includes scripts/run-e2e.sh for convenient triggering from the terminal.

* Print issue URL in workflow annotation and run-e2e.sh output

- Capture gh issue create output URL, print as ::notice annotation
- Search issues by run ID instead of grabbing most recent
2026-03-02 22:16:38 -08:00