Fix frozen terminals after split churn (#12 )

* Fix blank terminal after split operations and add visual tests

## Blank Terminal Fix
- Add `needsRefreshAfterWindowChange` flag in GhosttyTerminalView
- Force terminal refresh when view is added to window, even if size unchanged
- Add `ghostty_surface_refresh()` call in attachToView for same-view reattachment
- Add debug logging for surface attachment lifecycle (DEBUG builds only)

## Bonsplit Migration
- Add bonsplit as local Swift package (vendor/bonsplit submodule)
- Replace custom SplitTree with BonsplitController
- Add Panel protocol with TerminalPanel and BrowserPanel implementations
- Add SidebarTab as main tab container with BonsplitController
- Remove old Splits/ directory (SplitTree, SplitView, TerminalSplitTreeView)

## Visual Screenshot Tests
- Add test_visual_screenshots.py for automated visual regression testing
- Uses in-app screenshot API (CGWindowListCreateImage) - no screen recording needed
- Generates HTML report with before/after comparisons
- Tests: splits, browser panels, focus switching, close operations, rapid cycles
- Includes annotation fields for easy feedback

## Browser Shortcut (⌘⇧B)
- Add keyboard shortcut to open browser panel in current pane
- Add openBrowser() method to TabManager
- Add shortcut configuration in KeyboardShortcutSettings

## Screenshot Command
- Add 'screenshot' command to TerminalController for in-app window capture
- Returns OK with screenshot ID and path

## Other
- Add tests/visual_output/ and tests/visual_report.html to .gitignore

* Add browser title subscription and set tab height to 30px

- Subscribe to BrowserPanel.$pageTitle changes to update bonsplit tabs
- Update tab titles in real-time as page navigation occurs
- Clean up subscriptions when panels are removed
- Set bonsplit tab bar and tab height to 30px (in submodule)

* Fix socket API regressions in list_surfaces, list_bonsplit_tabs, focus_pane

- list_surfaces: Remove [terminal]/[browser] suffix to keep UUID-only format
  that clients and tests expect for parsing
- list_bonsplit_tabs --pane: Properly look up pane by UUID instead of
  creating a new PaneID (requires bonsplit PaneID.id to be public)
- focus_pane: Accept both UUID strings and integer indices as documented

* Fix browser panel stability and keyboard shortcuts

- Prevent WKWebView focus lifecycle crashes during split/view reshuffles
- Match bracket shortcuts via keyCode (Cmd+Shift+[ / ], Cmd+Ctrl+[ / ])
- Support Ghostty config goto_split:* keybinds when WebView is focused
- Add focus_webview/is_webview_focused socket commands and regression tests
- Rename SidebarTab to Workspace and update docs

* Make ctrl+enter keybind test skippable

Skip when the Ghostty keybind isn't configured or when osascript can't send keystrokes (no Accessibility permission), so VM runs stay green.

* Auto-focus browser omnibar when blank

When a browser surface is focused but no URL is loaded yet, focus the address bar instead of the WKWebView.

* Stabilize socket surface indexing

* Focus browser omnibar escape; add webview keybind UI tests

- Escape in omnibar now returns focus to WKWebView\n- Add UI tests for Cmd+Ctrl+H pane navigation with WebKit focused (including Ghostty config)\n- Avoid flaky element screenshots in UpdatePillUITests on the UTM VM

* Fix browser drag-to-split blanks and socket parsing

* Fix webview-focused shortcuts and stabilize browser splits

- Match ctrl/shift shortcuts by keyCode where needed (Ctrl+H, bracket keys)
- Load Ghostty goto_split triggers reliably and refresh on config load
- Add debug socket helpers: set_shortcut + simulate_shortcut for tests
- Convert browser goto_split/keybind tests to socket-based injection (no osascript)
- Bump bonsplit for drag-to-split fixes

* Fix split layout collapse and harden socket pane APIs

* Stabilize OSC 99 notification test timing

* Fix terminal focus routing after split reparent

* Support simulate_shortcut enter for focus routing test

* Stabilize terminal focus routing test

* Fix frozen new terminal tabs after many splits

* Fix frozen new terminal tabs after splits

* Fix terminal freeze on launch/new tabs

* Update ghostty submodule

* Fix terminal focus/render stalls after split churn

* Fix nested split collapsing existing pane

* Fix nested split collapse + stabilize new-surface focus

* Update bonsplit submodule

* Fix SIGINT test flake

* Remove bonsplit tab-switch crossfade

* Remove PROJECTS.md

* Remove bonsplit tab selection animation

* Ignore generated test reports

* Middle click closes tab

* Revert unintended .gitignore change

* Fix build after main merge

* Revert "Fix build after main merge"

This reverts commit 16bf9816d0856b5385d52f886aa5eb50f3c9d9a4.

* Revert "Merge remote-tracking branch 'origin/main' into fix/blank-terminal-and-visual-tests"

This reverts commit 7c20fb53fd71fea7a19a3673f2dd73e5f0c783c4, reversing
changes made to 0aff107d787bc9d8bbc28220090b4ca7af72e040.

* Remove tab close fade animation

* Use terminal.fill icon

* Make terminal tab icon smaller

* Match browser globe tab icon size

* Bonsplit: tab min width 48 and tighter close button

* Bonsplit: smaller tab title font

* Show unread notification badge in bonsplit tabs and improve UI polish

Sync unread notification state to bonsplit tab badges (blue dot).
Improve EmptyPanelView with Terminal/Browser buttons and shortcut hints.
Add tooltips to close tab button and search overlay buttons.

* Fix reload.sh single-instance safety check on macOS

Replace GNU-only `ps -o etimes=` with portable `ps -o etime=` and
parse the dd-hh:mm:ss format manually for macOS compatibility.

* Centralize keyboard shortcut definitions into Action enum

Replace per-shortcut boilerplate with a single Action enum that holds
the label, defaults key, and default binding for each shortcut. All
call sites now use shortcut(for:). Settings UI is data-driven via
ForEach(Action.allCases). Titlebar tooltips update dynamically when
shortcuts are changed. Remove duplicate .keyboardShortcut() modifiers
from menu items that are already handled by the event monitor.

* Fix WKWebView consuming app menu shortcuts and close panel confirmation

Add CmuxWebView subclass that routes key equivalents through the main
menu before WebKit, so Cmd+N/Cmd+W/tab switching work when a browser
pane is focused. Fix Cmd+W close-panel path: bypass Bonsplit delegate
gating after the user confirms the running-process dialog by tracking
forceCloseTabIds. Add unit tests (CmuxWebViewKeyEquivalentTests) and
UI test scaffolding (MenuKeyEquivalentRoutingUITests) with a new
cmux-unit Xcode scheme.

* Update CLAUDE.md and PROJECTS.md with recent changes

CLAUDE.md: enforce --tag for reload commands, add cleanup safety rules.
PROJECTS.md: log notification badge, reload.sh fix, Cmd+W fix, WebView
key equiv fix, and centralized shortcuts work.

* Keep selection index stable on close

* Add concepts page documenting terminology hierarchy

New docs page explaining Window > Workspace > Pane > Surface > Panel
hierarchy with aligned ASCII diagram. Updated tabs.mdx and splits.mdx
to use consistent terminology (workspace instead of tab, surface
instead of panel) and corrected outdated CLI command references.

* Update bonsplit submodule

* WIP: improve split close stability and UI regressions

* Close terminal panel on child exit; hide terminal dirty dot

* Fix split close/focus regressions and stabilize UI tests

* Add unread Dock/Cmd+Tab badge with settings toggle

* Fix browser-surface shortcuts and Cmd+L browser opening

* Snapshot current workspace state before regression fixes

* Update bonsplit submodule snapshot

* Stabilize split-close regression capture and sidebar resize assertions

* Change default Show Notifications shortcut from Cmd+Shift+I to Cmd+I

* Fix update check readiness race, enable release update logging, and improve checking spinner

* Restore terminal file drop, fix browser omnibar click focus, and add panel workspace ID mutation for surface moves

* Add Cmd+digit workspace hints, titlebar shortcut pills, sidebar drag-reorder, and workspace placement settings

* Add v2 browser automation API, surface move/reorder commands, and short-handle ref system to TerminalController

* Add CLI browser command surface, --id-format flag, and move/reorder commands

* Extend test clients with move/reorder APIs, ref-handle support, and increased timeouts

* Harden test runner scripts with deterministic builds, retry logic, and robust socket readiness

* Stabilize existing test suites with focus-wait helpers, increased timeouts, and API shape updates

* Add terminal file drop e2e regression test

* Add v2 browser API, CLI ref resolution, and surface move/reorder test suites

* Add unit tests for shortcut hints, workspace reorder, drop planner, and update UI test stabilization

* Add cmux-debug-windows skill with snapshot script and agent config

* Update project docs: mark browser parity and move/reorder phases complete, add parallel agent workflow guidelines

* Update bonsplit submodule: re-entrant setPosition guard, tab shortcut hints, and moveTab/reorderTab API

* Add browser agent UX improvements: snapshot refs, placement reuse, diagnostics, and skill docs

- Upgrade browser.snapshot to emit accessibility tree text with element refs (eN)
- Add right-sibling pane reuse policy for browser.open_split placement
- Add rich not_found diagnostics with retry logic for selector actions
- Support --snapshot-after for post-action verification on mutating commands
- Allow browser fill with empty text for clearing inputs
- Default CLI --id-format to refs-first (UUIDs opt-in via --id-format uuids|both)
- Format legacy new-pane/new-surface output with short surface refs
- Add skills/cmuxterm-browser/ and skills/cmuxterm/ end-user skill docs
- Add regression tests for placement policy, snapshot refs, diagnostics, and ID defaults

* Update bonsplit submodule: keep raster favicons in color when inactive

2026-02-13 16:45:31 -08:00

4.3 KiB

Raw Blame History

name	description
cmux-browser	End-user browser automation with cmux. Use when you need to open sites, interact with pages, wait for state changes, and extract data from cmux browser surfaces.

Browser Automation with cmux

Use this skill for browser tasks inside cmux webviews.

Core Workflow

Open or target a browser surface.
Snapshot (--interactive) to get fresh element refs.
Act with refs (click, fill, type, select, press).
Wait for state changes.
Re-snapshot after DOM/navigation changes.

cmux browser open https://example.com --json
# use returned surface ref, for example: surface:7

cmux browser surface:7 snapshot --interactive
cmux browser surface:7 fill e1 "hello"
cmux browser surface:7 click e2 --snapshot-after --json
cmux browser surface:7 wait --load-state complete --timeout-ms 15000
cmux browser surface:7 snapshot --interactive

Surface Targeting

# identify current context
cmux identify --json

# open routed to a specific topology target
cmux browser open https://example.com --workspace workspace:2 --window window:1 --json

Notes:

CLI output defaults to short refs (surface:N, pane:N, workspace:N, window:N).
UUIDs are still accepted on input; only request UUID output when needed (--id-format uuids|both).
Keep using one surface:N per task unless you intentionally switch.

Wait Support

cmux supports wait patterns similar to agent-browser:

cmux browser <surface> wait --selector "#ready" --timeout-ms 10000
cmux browser <surface> wait --text "Success" --timeout-ms 10000
cmux browser <surface> wait --url-contains "/dashboard" --timeout-ms 10000
cmux browser <surface> wait --load-state complete --timeout-ms 15000
cmux browser <surface> wait --function "document.readyState === 'complete'" --timeout-ms 10000

Common Flows

Form Submit

cmux browser open https://example.com/signup --json
cmux browser surface:7 snapshot --interactive
cmux browser surface:7 fill e1 "Jane Doe"
cmux browser surface:7 fill e2 "jane@example.com"
cmux browser surface:7 click e3 --snapshot-after --json
cmux browser surface:7 wait --url-contains "/welcome" --timeout-ms 15000
cmux browser surface:7 snapshot --interactive

Clear an Input

cmux browser surface:7 fill e11 "" --snapshot-after --json
cmux browser surface:7 get value e11 --json

Stable Agent Loop (Recommended)

# snapshot -> action -> wait -> snapshot
cmux browser surface:7 snapshot --interactive
cmux browser surface:7 click e5 --snapshot-after --json
cmux browser surface:7 wait --load-state complete --timeout-ms 15000
cmux browser surface:7 snapshot --interactive

Deep-Dive References

Reference	When to Use
references/commands.md	Full browser command mapping and quick syntax
references/snapshot-refs.md	Ref lifecycle and stale-ref troubleshooting
references/authentication.md	Login/OAuth/2FA patterns and state save/load
references/authentication.md#saving-authentication-state	Save authenticated state right after login
references/session-management.md	Multi-surface isolation and state persistence patterns
references/video-recording.md	Current recording status and practical alternatives
references/proxy-support.md	Proxy behavior in WKWebView and workarounds

Ready-to-Use Templates

Template	Description
templates/form-automation.sh	Snapshot/ref form fill loop
templates/authenticated-session.sh	Login once, save/load state
templates/capture-workflow.sh	Navigate + capture snapshots/screenshots

Limits (WKWebView)

These commands currently return not_supported because they rely on Chrome/CDP-only APIs not exposed by WKWebView:

viewport emulation
offline emulation
trace/screencast recording
network route interception/mocking
low-level raw input injection

Use supported high-level commands (click, fill, press, scroll, wait, snapshot) instead.

4.3 KiB Raw Blame History