Commit graph

588 commits

Author SHA1 Message Date
Naiyuan Qing
23905daaa1 Merge remote-tracking branch 'origin/main' into feat/telegram-channel
# Conflicts:
#	apps/desktop/electron/electron-env.d.ts
#	apps/desktop/electron/ipc/index.ts
#	apps/desktop/electron/preload.ts
#	apps/desktop/src/App.tsx
#	apps/desktop/src/pages/layout.tsx
#	src/agent/async-agent.ts
#	src/agent/runner.ts
#	src/hub/hub.ts
2026-02-09 13:44:08 +08:00
Naiyuan Qing
43d11a6e5d fix(channels): address code review issues
- Fix double useChannels() instantiation: call once in ChannelsPage,
  pass as props to TelegramCard
- Mask bot tokens in channels:getConfig before sending to renderer
- Add input validation (isValidId, token length) on all IPC handlers
- Fix stopAccount() to clean up typingTimer, lastRoute, aggregator,
  and debouncer when stopping the account they belong to
- Add try/catch to stopChannel/startChannel in useChannels hook
- Consistent return type { ok, error? } on channels:stop handler
- Add tooltip hint on disabled Remove button

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-09 13:05:34 +08:00
Naiyuan Qing
c99675b6e4 feat(desktop): add Channels configuration page with Telegram support
Add IPC handlers, preload API, useChannels hook, and Channels page UI.
Users can save/remove Telegram bot tokens and start/stop bots directly
from the desktop app with immediate effect and persistence across restarts.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-09 12:50:24 +08:00
Naiyuan Qing
0542c82fe6 feat(channels): add credential write and per-account lifecycle control
Add setChannelAccountConfig/removeChannelAccountConfig to CredentialManager
for persisting channel tokens. Make ChannelManager.startAccount public and
add stopAccount for individual account lifecycle control via IPC.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-09 12:50:15 +08:00
Naiyuan Qing
6a02fd29be fix(ui): adjust chat input padding, icon, and button size
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-09 11:43:49 +08:00
Naiyuan Qing
0500dc1d53 feat(channels): add inbound debouncer, ACK reactions, and sequentialize
- InboundDebouncer: batches rapid-fire messages from the same conversation
  into a single agent.write() call (500ms idle, 2s hard cap)
- ACK reactions: add 👀 emoji on message receipt, remove on completion
  (addReaction/removeReaction on ChannelOutboundAdapter interface)
- Grammy sequentialize middleware: ensures same-chat updates are processed
  in order, preventing race conditions on shared state

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-09 11:43:42 +08:00
Naiyuan Qing
614d2cfd88 fix(channels): address code review issues
Critical:
- describe-video: add mkdir for MEDIA_CACHE_DIR before ffmpeg write
- telegram: check bot ID (not is_bot) for reply-to detection in groups

Important:
- telegram: check @mention in caption for media messages in groups
- hub: add .catch() to channelManager.startAll()
- describe-image: add 20MB file size check to prevent OOM
- async-agent: remove dead writeWithImages, refactor with enqueue()
- manager: lazy agent subscription via ensureSubscribed() to handle
  late agent availability and agent replacement

Suggestions:
- telegram-format: escape quotes in link URLs to prevent HTML breakout
- transcribe: catch API errors and return null (match local fallback)

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-09 11:22:17 +08:00
Naiyuan Qing
49623b4779 docs(channels): add system overview and update media handling docs
- Create docs/channels/README.md: plugin architecture, adapters, lastRoute
  pattern, message flow, configuration, and new plugin guide
- Update media-handling.md: local whisper priority in tables, rewrite
  fallback section, remove completed items from future work
- Add @see doc references in types.ts, telegram.ts, manager.ts,
  transcribe.ts, describe-image.ts, describe-video.ts

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-09 11:03:49 +08:00
Naiyuan Qing
30e9041084 fix(chunker): add table awareness and increase Telegram chunk limit
- Add isInsideTable() to BlockChunker: prevents breaking Markdown tables
  in the middle (table rows lose header context when split across messages)
- Set Telegram chunkerConfig maxChars to 4000 (was default 2000; Telegram
  API limit is 4096, leaving room for HTML formatting overhead)

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-09 11:03:39 +08:00
Naiyuan Qing
db214b25ca feat(media): add image/video description and local whisper priority
- Add describe-image.ts: OpenAI Vision API (gpt-4o-mini) image description
- Add describe-video.ts: ffmpeg frame extraction + Vision API description
- Rewrite transcribe.ts: local whisper/whisper-cli → OpenAI API → null
- Update manager.ts routeMedia(): all media converted to text before agent
  - Image: describeImage() → text (was: raw ImageContent via writeWithImages)
  - Video: describeVideo() → text (was: file path info only)
  - Audio: unchanged (but underlying transcribeAudio now tries local first)

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-09 11:03:31 +08:00
Naiyuan Qing
4e5780692e feat(media): transcribe audio via Whisper API before reaching agent
Move audio transcription from agent-driven (exec + local whisper) to
Manager-layer processing via OpenAI Whisper API. Voice messages are
now transcribed automatically before the agent sees them, so the
agent only receives text. Local whisper skill remains as fallback
when API key is not configured. Also changed default model from
turbo to base for faster first-time experience.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-09 10:06:11 +08:00
Naiyuan Qing
922a3b2bb7 feat(skills): add whisper audio transcription skill
Bundled skill that enables the agent to transcribe audio files using
OpenAI Whisper CLI. Uses anyBins requirement so the skill is only
visible when whisper is installed. Includes brew and uv install specs.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-09 09:44:49 +08:00
Naiyuan Qing
23da5a35ff feat(channels): route media messages through agent
Add writeWithImages() to AsyncAgent for passing images directly to
the LLM via ImageContent. Extend Agent.run() to accept optional
images parameter. Update ChannelManager.routeIncoming() to download
media files and forward them: images as ImageContent to the LLM,
audio/video/document as file paths for agent-driven processing.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-09 09:44:39 +08:00
Naiyuan Qing
78738e89bf feat(telegram): detect and download media messages
Add handlers for voice, audio, photo, video, and document messages.
Each handler emits a ChannelMessage with media attachment metadata.
Implement downloadMedia() to fetch files from Telegram API and save
to the local media cache directory.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-09 09:44:30 +08:00
Naiyuan Qing
020d132260 feat(channels): add media attachment types and cache directory
Add ChannelMediaAttachment type with support for audio, image, video,
and document media types. Extend ChannelMessage with optional media
field and ChannelPlugin with optional downloadMedia method.
Add MEDIA_CACHE_DIR path for downloaded media files.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-09 09:44:21 +08:00
Naiyuan Qing
00c80b55c4 Revert "feat(telegram): handle voice and audio messages with <media:audio> placeholder"
This reverts commit 48f8302ebf.
2026-02-09 08:38:50 +08:00
Naiyuan Qing
48f8302ebf feat(telegram): handle voice and audio messages with <media:audio> placeholder
Forward voice messages and audio files to the agent as <media:audio>
placeholder text. In groups, only process voice/audio that replies to
the bot. Includes caption text if present.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-09 08:37:57 +08:00
Naiyuan Qing
ceb960c390 feat(channels): add typing indicators and Telegram HTML formatting
- Add sendTyping to ChannelOutboundAdapter (optional per platform)
- Implement typing lifecycle in ChannelManager (5s interval, cleanup on message_end/error/clear)
- Convert Markdown to Telegram HTML subset (bold, italic, code, links, blockquotes)
- Fallback to plain text on HTML parse errors

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-09 08:37:34 +08:00
Naiyuan Qing
54e2ac4a17 fix(desktop): open external links in system browser
Use setWindowOpenHandler to intercept window.open calls and route
them through shell.openExternal instead of navigating inside Electron.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-09 08:04:25 +08:00
Naiyuan Qing
54b3ebe9e9 feat(ui): render error messages with Markdown
Use MemoizedMarkdown for error messages so links are clickable
instead of plain text.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-09 08:04:19 +08:00
Naiyuan Qing
1819f4196d fix(sdk): add AgentErrorEvent to StreamPayload type
StreamPayload.event was missing the agent_error event type, causing
TypeScript errors in useGatewayChat and useLocalChat where the
comparison payload.event.type === "agent_error" had no type overlap.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-09 08:04:13 +08:00
Naiyuan Qing
56ebe613db fix(hooks): handle agent_error events in useGatewayChat
Desktop path already forwards agent_error to chat.setError() via
use-local-chat.ts, but the Web/Gateway path was missing this handling.
Add agent_error interception in the StreamAction branch so Web clients
render LLM errors the same way Desktop does.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-09 07:30:35 +08:00
LinYushen
0edc98142d
Merge pull request #102 from multica-ai/sessions-spawn-parent-trigger
feat(subagent): coalesce spawn announcements
2026-02-06 20:03:29 +08:00
yushen
c24fafadeb fix(tools): prevent AbortSignal listener leak in exec and process tools
Each tool call added an abort listener to the shared agent signal
without cleanup, exceeding the default 10-listener limit after 11+
exec calls. Fix by using { once: true } and removing the listener
on child process close (exec) to prevent accumulation.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-06 19:58:31 +08:00
yushen
a3f5a21561 fix(subagent): clear run records immediately after successful announcement
Instead of keeping announced runs in the registry for 60 minutes
(archive sweeper), delete them right after findings are delivered
to the parent. This prevents stale completed tasks from appearing
in sessions_list on subsequent parent turns.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-06 19:53:31 +08:00
yushen
a7db286530 Merge remote-tracking branch 'origin/main' into sessions-spawn-parent-trigger 2026-02-06 19:44:17 +08:00
yushen
a3acd732e0 fix(subagent): persist LLM summary after internal announce to parent context
After child subagents complete, the coalesced announcement runs as an
internal turn which rolls back all messages from the parent's in-memory
context. This causes the parent LLM to lose findings in subsequent turns.

Add persistResponse option to writeInternal that re-injects the LLM's
summary as a non-internal assistant message after the internal run
completes. The internal prompt stays hidden while the summary persists
in both memory and session JSONL for future turns.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-06 19:38:18 +08:00
Naiyuan Qing
be71614cf5 docs(channels): add message paths documentation
Document the three independent message paths (Desktop IPC, Web
WebSocket, Channel Bot API) including send/receive flows, error
handling, lastRoute pattern, and event filtering comparison.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-06 19:35:51 +08:00
Naiyuan Qing
e99600c356 fix(desktop): register ChatPage element in router
The /chat route was missing its element prop, causing React Router
to render an empty Outlet.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-06 19:35:46 +08:00
Naiyuan Qing
72598322d1 feat(desktop): handle agent_error events and clearLastRoute in IPC
Forward agent_error as passthrough event to renderer. Add
clearLastRoute() calls in hub:sendMessage and localChat:send
handlers so channel replies stop when desktop sends a message.
Handle agent_error in use-local-chat to show error UI.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-06 19:35:39 +08:00
Naiyuan Qing
4a503ecf4c feat(agent): add AgentErrorEvent for error propagation via subscribe
Add agent_error event type to MulticaEvent union so errors from
agent runs reach subscribe() consumers (Desktop IPC + Channel).
Make emitMulticaEvent public on Runner so AsyncAgent can emit errors.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-06 19:35:32 +08:00
Naiyuan Qing
9bb1fd6e12 refactor(channels): rewrite ChannelManager with lastRoute pattern
Replace per-conversation agent creation with single Hub agent model.
Messages from channels are routed to the existing Hub agent via
agent.write(), and replies are sent back through the lastRoute context.
Desktop and Gateway paths call clearLastRoute() so channel replies
stop when the user switches input surface.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-06 19:35:25 +08:00
yushen
a50239a9f5 fix(subagent): require coalesced announce to include all findings 2026-02-06 19:12:51 +08:00
yushen
12075b96f2 fix(subagent): capture latest non-empty findings from child runs 2026-02-06 19:10:44 +08:00
yushen
4d6017e782 fix(subagent): forward assistant stream events for internal announce 2026-02-06 19:04:40 +08:00
yushen
9687e7f2a6 fix(subagent): forward announce replies from internal runs 2026-02-06 18:58:54 +08:00
yushen
f7267f6698 fix(agent): prevent internal run leaks in async streams 2026-02-06 18:38:54 +08:00
yushen
d392238be3 feat(subagent): tag internal orchestration messages and filter from user-facing history
Announcement messages from subagent completion flows were persisted as
regular user messages, polluting conversation history and leaking
orchestration instructions to frontend/CLI.

- Add `internal?: boolean` to SessionEntry message variant
- SessionManager.saveMessage accepts { internal: true } option
- SessionManager.loadMessages filters internal by default
- Agent.runInternal() tags messages as internal, rolls back from memory
- Agent.withRunMutex() prevents concurrent run/runInternal mis-tagging
- AsyncAgent.writeInternal() suppresses event forwarding during internal runs
- Announce flows use writeInternal() instead of write()
- Desktop IPC getHistory reads from session storage (filtered)
- CLI session show parses SessionEntry, supports --show-internal flag

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-06 18:34:11 +08:00
LinYushen
f31439ee97
Merge pull request #104 from multica-ai/explore-approval-config
feat(exec-approval): relax defaults and support no-timeout
2026-02-06 18:13:09 +08:00
yushen
6ecdbc5783 fix(exec-approval): treat expiresAtMs=-1 as non-expiring 2026-02-06 18:09:01 +08:00
yushen
a36cbac3fd feat(exec-approval): default to full/off security and support no-timeout
- Change default security from "allowlist" to "full" (allow all commands)
- Change default ask from "on-miss" to "off" (never prompt)
- Change DEFAULT_APPROVAL_TIMEOUT_MS from 60s to -1 (no timeout)
- Support timeoutMs=-1 to wait indefinitely for user decision
- Update CLI and Hub approval flows to skip setTimeout when timeout<0

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-06 18:02:07 +08:00
yushen
9cc89cf297 feat(subagent): add sessions_list tool for viewing spawned sub-tasks
Adds a new `sessions_list` tool to the Subagent tool group, allowing
agents to query the status of their spawned sub-tasks. Supports both
list mode (all runs) and detail mode (specific runId).

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-06 17:58:22 +08:00
Bohan Jiang
dacf8894e9
Merge pull request #103 from multica-ai/feat/heartbeat-mechanism
feat(heartbeat): add heartbeat runner and integrate with hub/cron
2026-02-06 17:46:51 +08:00
Jiang Bohan
02838bba78 fix(desktop): remove heartbeat section from home page 2026-02-06 17:36:46 +08:00
yushen
7726079648 fix(subagent): recover delete cleanup after crash 2026-02-06 17:31:21 +08:00
Jiang Bohan
b9d9ae99b0 fix(heartbeat): keep manual trigger runs out of chat stream 2026-02-06 17:28:25 +08:00
Jiang Bohan
9f2a1c240f fix(hub): suppress heartbeat ack messages in streamed chat 2026-02-06 17:25:22 +08:00
yushen
7f9a0181c8 test(subagent): add coalescing announcement tests
Cover formatCoalescedAnnouncementMessage (single/multi record,
mixed outcomes, missing findings), registry coalescing state
transitions, shutdown capture, and new field serialization.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-06 17:24:44 +08:00
yushen
6786af4aa3 feat(subagent): coalesce announcements into single parent notification
When multiple children complete, buffer findings per-child and only
send one combined announcement to the parent after all unannounced
runs for the same requester have finished. This avoids N separate
LLM calls and gives the parent a complete picture of all results.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-06 17:24:39 +08:00
Naiyuan Qing
ee95102613 docs(channel): add OpenClaw channel system source code research
Comprehensive research document covering OpenClaw's channel architecture,
plugin system, message flow (inbound → agent → outbound), Telegram
integration details, routing/session management, and security model.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-06 17:13:43 +08:00