Commit graph

1065 commits

Author SHA1 Message Date
Jiayuan Zhang
486b090577 chore(benchmark): translate finance e2e case prompts to English
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-16 02:39:35 +08:00
Jiayuan Zhang
c38a576b8f feat(benchmark): parallelize finance e2e runs 2026-02-16 02:15:55 +08:00
Jiayuan Zhang
b897719775 fix(benchmark): support long-horizon timeout control 2026-02-16 01:54:20 +08:00
Jiayuan Zhang
edc55390cf feat(benchmark): add finance e2e case suite 2026-02-16 01:29:24 +08:00
Jiayuan Zhang
faa54fc671
Merge pull request #209 from multica-ai/forrestchang/subagent-redesign
feat(agent): replace sessions_spawn with synchronous delegate tool
2026-02-16 01:22:14 +08:00
Jiayuan Zhang
9c8be30d3d fix(test): increase timeout for summary fallback artifact extraction test
UC4 test times out in CI (5s default) because generateSummary's API
provider layer takes longer to fail on slow CI runners. Increase to 15s.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-16 01:10:18 +08:00
Jiayuan Zhang
aada2916f4 fix(agent): clear timeout timer in delegate tool to prevent unhandled rejection
The setTimeout in runSubagentTask was never cleared when childAgent.run()
completed before the timeout. The dangling timer would later reject an
unobserved promise, causing an unhandled promise rejection crash in Node.js
v15+. Capture the timer and clear it in a .finally() block.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-16 01:09:21 +08:00
Jiayuan Zhang
f60551195a chore(agent): remove old sessions_spawn/sessions_list tools and update references
Delete sessions-spawn.ts, sessions-list.ts and their tests. Update CLI
to remove waitForSubagents polling workaround (delegate is synchronous).
Update UI, desktop IPC, SWE-bench, and system prompt tests to use the
new delegate tool name.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-16 01:09:21 +08:00
Jiayuan Zhang
d3ef8ecc31 feat(agent): replace sessions_spawn with synchronous delegate tool
Replace the async sessions_spawn/sessions_list sub-agent system with a
single synchronous `delegate` tool. The new tool runs tasks in parallel
via Promise.all with per-task timeout, returning combined results directly
in the tool response. This eliminates the need for registry, announce queue,
persistence, and Hub involvement.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-16 01:09:21 +08:00
Jiayuan Zhang
94ae88ed8b
Merge pull request #208 from multica-ai/forrestchang/compaction-audit
Context window: 4-phase compaction improvements
2026-02-16 00:01:58 +08:00
Jiayuan Zhang
0bce493e10 fix(compaction): handle pi-agent-core toolResult format in truncation and pruning
The pre-emptive truncation, tool result pruning, and summary fallback
only checked for Anthropic-style `role: "user"` messages with
`type: "tool_result"` blocks. The actual runtime uses pi-agent-core
format with `role: "toolResult"`, `toolCallId`, and `toolName` on the
message itself. This caused truncation and pruning to silently skip
all tool results in real agent runs.

Add handlers for the pi-agent-core format in all four affected modules:
- session-manager.ts: check both "user" and "toolResult" roles
- tool-result-truncation.ts: new handler for toolResult format
- tool-result-pruning.ts: new processToolResultMessage() + updated loops
- summary-fallback.ts: include "toolResult" in artifact ref extraction

Verified via agent-driven E2E tests (5 test sessions, 6 artifacts).

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-15 23:39:52 +08:00
Jiayuan Zhang
b15e1eeb2a test(compaction): harden E2E integration tests for artifact pipeline
- Add real user messages for bootstrap protection in pruning tests
- Fix artifact directory path assertions (baseDir vs sessions/baseDir)
- Add cross-phase tests (Phase 1 truncation → Phase 2 pruning)
- Remove conditional assertion guards that could silently skip checks
- All 30 E2E integration tests now pass with mandatory assertions

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-15 23:13:12 +08:00
Jiayuan Zhang
58f02a2080 fix(compaction): match artifact refs from both soft trim and truncation markers
The extractArtifactRef regex only matched "Full result saved to" (from
pre-emptive truncation) but not "Full result available at" (from soft
trim). This caused hard clear to lose artifact references when preceded
by soft trim in the same pruning pass.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-15 23:13:05 +08:00
Jiayuan Zhang
a1ac250e2b feat(system-prompt): enhance report with truncation tracking and token estimates
SectionReport now includes truncated/originalChars fields for budget-controlled
sections. formatPromptReport shows estimated token count and truncation details.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-15 23:02:41 +08:00
Jiayuan Zhang
c3433871a6 feat(system-prompt): add bootstrap budget control for workspace and skills
Workspace.md content is capped at 20k chars and skills prompt at 12k chars.
Oversized content is intelligently truncated (head 70% + marker + tail 20%)
with newline-boundary snapping. Inspired by OpenClaw's bootstrap budget system.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-15 23:02:34 +08:00
Jiayuan Zhang
5aa8a52784 feat(compaction): make pruning and summary artifact-aware
Soft trim and hard clear now detect and preserve artifact references
in their markers. Summary instructions include guidance to note artifact
paths. Plain-text fallback extracts and lists all artifact references
in a "Saved Artifacts" section.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-15 23:02:27 +08:00
Jiayuan Zhang
3f9a30423d feat(session): add artifact storage and pre-emptive tool result truncation
Oversized tool results (>30% of context window) are now saved as artifacts
before being truncated in the session. The LLM sees a truncated version with
head+tail preservation and a marker pointing to the full artifact file,
which it can re-read on demand. This prevents information loss during
context window management.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-15 23:02:18 +08:00
Naiyuan Qing
fa040404d2
Merge pull request #207 from multica-ai/fix/onboarding-header-centering
fix(desktop): center onboarding step indicator
2026-02-15 22:10:50 +08:00
Naiyuan Qing
1e5174582b fix(desktop): center onboarding step indicator using absolute positioning
Use absolute positioning for the step indicator to achieve true centering
regardless of left/right element widths. This is the standard approach for
header layouts where the center element must be centered relative to the
entire container width, not the remaining flex space.

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-02-15 22:05:03 +08:00
Jiayuan Zhang
9e40bd5eac
Merge pull request #206 from multica-ai/forrestchang/tg-reply-optimize
feat(telegram): add reply context and instant feedback optimization
2026-02-15 21:56:10 +08:00
Jiayuan Zhang
b8fb671a4b feat(agent): add reply context and responsiveness guidance to channel system prompt
Add instructions for the agent to understand [Replying to: "..."] annotations
and to send brief acknowledgments before tool calls when messages come from
messaging channels.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-15 21:39:45 +08:00
Jiayuan Zhang
87a83fba1a feat(telegram): add reply context parsing and send new messages for tool narration
- Extract reply_to_message text from Telegram replies and prepend as
  [Replying to: "..."] context so the agent can reference specific messages
- Replace edit-message pattern with new messages for each tool narration,
  improving clarity in the chat flow
- Remove dead code: sendOrEditStatus, editFormatted, statusMessages map

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-15 21:39:45 +08:00
Jiayuan Zhang
b31a49c4d3
Merge pull request #205 from multica-ai/forrestchang/fix-orphan-tool-call
fix: prevent orphaned tool calls from session abort
2026-02-15 21:38:01 +08:00
Jiayuan Zhang
7a16df7c56 chore(compaction): add E2E compaction benchmark script
Multi-turn test script for compaction behavior with low context window.
Runs 4 turns of file reading to push context pressure and outputs
run-log analysis.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-15 21:37:36 +08:00
Jiayuan Zhang
b412ca902b refactor(compaction): remove dead code and legacy count mode
- Remove compactMessagesWithSummary (~100 lines, never called; only
  the Chunked variant was used)
- Remove compactMessagesByCount, findSafeCompactionPoint, and all
  count-mode references (~90 lines)
- Narrow CompactionResult.reason to "tokens" | "summary" | "pruning"
- Narrow compactionMode to "tokens" | "summary" (was "count" | ...)
- Simplify session-manager: remove maxMessages/keepLast params,
  enable tool result pruning by default

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-15 21:37:23 +08:00
Jiayuan Zhang
92cf312843 refactor(compaction): remove pre-flight tool result pruning
Pre-flight compaction runs in-memory only (not persisted), so tool
result pruning in this path was wasted work — results were thrown away
after the LLM call. Post-turn compaction still handles pruning and
persists the results. Only Phase 2 (emergency message drop) remains
as a safety net in pre-flight.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-15 21:37:15 +08:00
Jiayuan Zhang
fbb0b11c6e fix(compaction): fix system prompt token estimation and reduce safety margin
- estimateSystemPromptTokens now uses estimateTokens() (chars/4) instead
  of chars/2, eliminating the 2x overestimate that caused pre-flight
  compaction to fire on every LLM call at small context windows
- ESTIMATION_SAFETY_MARGIN reduced from 1.5 to 1.2, increasing usable
  context from ~53% to ~73% before compaction triggers

At 200k context, effective usable tokens before compaction improved from
~86k to ~120k message tokens (39% increase).

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-15 21:37:09 +08:00
Jiayuan Zhang
40a2e8ae55 fix(context-window): prioritize config over model for context window resolution
resolveContextWindowInfo now uses config > model > default priority so
explicit --context-window flag overrides model defaults. Also adds
--context-window CLI option to the run command.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-15 21:37:02 +08:00
Jiayuan Zhang
71f95d042a fix(agent): prevent double-save of assistant message on abort
Track the last assistant message saved by the message_end event handler
and skip saving it again in the abort handler. This prevents the
duplicate assistant entries in session.jsonl that caused the
"tool_call_id is not found" bug.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-15 21:31:36 +08:00
Jiayuan Zhang
fa2616c390 fix(session): drop duplicate assistant messages in transcript repair
When a session is aborted mid-tool-execution, the assistant message can
be persisted twice (once by message_end, once by the abort handler).
The repair logic failed to handle this: it generated a synthetic tool
result for the first copy but deduplicated the result for the second,
leaving an orphaned tool call that caused "tool_call_id is not found"
errors on all subsequent API calls.

Detect and remove duplicate assistant messages whose tool call IDs
have all already been paired with results from an earlier copy.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-15 21:31:30 +08:00
Jiayuan Zhang
08b8c71dcb
Merge pull request #204 from multica-ai/forrestchang/fix-onboarding-drag
fix(desktop): make login and onboarding windows draggable on macOS
2026-02-15 21:28:47 +08:00
Jiayuan Zhang
9f95566616 fix(desktop): make login and onboarding windows draggable on macOS
Login page: replace full-container drag region with a dedicated header
bar, matching the pattern used across the app. The previous approach
applied -webkit-app-region: drag to the entire content container which
could interfere with child element interactions.

Onboarding steps 1-4: move drag region from a tiny 80px left strip to
the entire header element, and mark the ModeToggle as no-drag so it
remains clickable.

Closes MUL-241

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-15 21:16:27 +08:00
Jiayuan Zhang
74c0ca0ddc
Merge pull request #203 from multica-ai/forrestchang/finance-benchmark
fix: agent stability, tooling, and sub-agent orchestration
2026-02-15 20:50:44 +08:00
Jiayuan Zhang
a443d3009d chore: remove leftover patch file from parallel execution revert
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-15 20:45:32 +08:00
Jiayuan Zhang
084657868f revert(agent): remove parallel tool execution patch, keep serial
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-15 20:43:37 +08:00
Jiayuan Zhang
e39f9a5dfe feat(cli): wait for sub-agents and output findings in run mode
In CLI mode, the parent Agent is not registered with the Hub, so the
normal announce flow can't deliver sub-agent results. Added polling
mechanism that waits for sub-agents to complete and prints their
findings directly to stdout.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-15 19:53:26 +08:00
Jiayuan Zhang
b394b0ccf9 fix(skills): use python3 and inject skill directory path into prompt
- SKILL.md: python → python3 (macOS has no `python` binary)
- skills/index.ts: inject skill directory path so agent can resolve
  relative paths like scripts/recalc.py to absolute paths

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-15 19:53:18 +08:00
Jiayuan Zhang
691e33e71e fix(tools): auto-create group when custom groupId is provided
LLM often invents custom groupId strings that don't exist in the
registry, causing "group not found" errors. Now auto-creates the
group instead, matching the behavior when `next` is provided.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-15 19:53:12 +08:00
Jiayuan Zhang
d162ba98a9 fix(agent): pass sessionId to tools for sub-agent session tracking
toolsOptions spread `options` which had sessionId undefined for
auto-generated sessions. This caused sessions_list and sessions_spawn
to fail with "No session ID available" — breaking sub-agent orchestration.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-15 19:53:03 +08:00
Jiayuan Zhang
a254daff01 feat(agent): enable parallel tool execution via pi-agent-core patch
Replace sequential for+await tool dispatch with Promise.allSettled for
parallel execution. All tool_execution_start events emit immediately,
tools run concurrently, results are processed in original order.

Also fix run-log toolStartTimes to key by toolCallId instead of toolName
to prevent collisions with parallel same-name tools.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-15 18:47:53 +08:00
Jiayuan Zhang
c012bff246 fix(tools): show findings and full runId in sessions_list list view
Grouped runs now display findings for completed sub-agents (up to 4000
chars). Ungrouped runs increased truncation from 200 to 4000 chars. All
status lines include full runId for subsequent API queries.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-15 18:47:44 +08:00
Jiayuan Zhang
02ed09b77b fix(tools): use boolean error flag in web_fetch and web_search error responses
Return error: true (boolean) with code field instead of error: "string_code"
to match ToolErrorPayload convention. Also update runner.ts formatRunLogToolSummary
to prefer details.code for error categorization.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-15 18:47:37 +08:00
Jiayuan Zhang
9cb6ee30a2
Merge pull request #202 from multica-ai/forrestchang/swe-bench-runner
feat: add SWE-bench runner for agent evaluation
2026-02-15 18:34:56 +08:00
Jiayuan Zhang
a0a837e76b docs: add testing & benchmarks section to README
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-15 18:33:11 +08:00
Jiayuan Zhang
45acb965ba docs: add SWE-bench section to CLAUDE.md
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-15 18:32:04 +08:00
Jiayuan Zhang
10c57c0f7a docs: add SWE-bench runner guide
Covers the full pipeline: dataset download, agent execution,
result analysis, and official Docker evaluation. Includes
runner options, output format, known limitations, and initial
benchmark results.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-15 18:30:58 +08:00
Jiayuan Zhang
b007ddffc8 feat(cli): initialize Hub in run mode for full agent capabilities
Always initialize Hub in CLI run mode to match the Desktop environment
where Hub is always active. This enables sessions_spawn (sub-agent
creation), cron tasks, channel plugins, and other Hub-dependent
features during E2E testing.

Hub constructor is non-blocking — gateway connection failures are
handled gracefully with auto-reconnect. hub.shutdown() in finally
block ensures clean teardown on exit.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-15 18:06:51 +08:00
Jiayuan Zhang
755ed5e9de feat(run-log): add result metadata to tool_end events
Enrich tool_end events with result_chars, result_summary, and
error_type fields. Since run-log.jsonl is append-only and never
compacted, this preserves tool result metadata that would otherwise
be lost when session.jsonl undergoes compaction.

New fields:
- result_chars: total character count of result content
- result_summary: short tool-specific summary (e.g. "10 results",
  "12.5KB", "finance/get_price_snapshot")
- error_type: error category for tool errors (e.g. "fetch_failed")

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-15 18:06:42 +08:00
Jiayuan Zhang
90d374ffd5 feat(scripts): add SWE-bench runner for Multica agent evaluation
- download-dataset.py: fetches SWE-bench Lite/Verified/Full from HuggingFace
- run.ts: core runner that clones repos, runs Agent, collects git diff patches
- evaluate.sh: wrapper for official SWE-bench Docker evaluation harness
- analyze.ts: summarizes run results with per-repo and timing breakdowns

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-15 18:05:17 +08:00
Jiayuan Zhang
1c24dd2885 fix(credentials): add fallback to ~/.super-multica for custom data dirs
When SMC_DATA_DIR is set (e.g., for E2E tests), the credentials lookup
now falls back to ~/.super-multica/credentials.json5 if the custom
data dir doesn't have its own credentials file. This mirrors the
existing fallback pattern in auth-store.ts and removes the need for
the SMC_CREDENTIALS_PATH workaround in E2E tests.

Lookup order:
1. SMC_CREDENTIALS_PATH env var (explicit override)
2. {DATA_DIR}/credentials.json5 (current data dir)
3. ~/.super-multica/credentials.json5 (default location fallback)

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-15 17:48:21 +08:00