From 62aacf38b5749096c8e470fa65c79a4ca6978b52 Mon Sep 17 00:00:00 2001 From: Panniantong Date: Fri, 27 Feb 2026 08:17:51 +0100 Subject: [PATCH] feat: migrate Twitter backend from bird CLI to xreach CLI MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit bird CLI (@steipete/bird) is deprecated and no longer maintained. xreach CLI (xreach-cli on npm) is our maintained fork with: - Fixed SearchTimeline (POST + updated query ID) - Built-in proxy rotation support - Additional features (DMs, notifications, lists) Changes across 11 files: - channels/twitter.py: detect xreach instead of bird/birdx - cli.py: install/doctor/uninstall all reference xreach-cli - SKILL.md: updated command examples (bird read → xreach tweet) - guides/setup-twitter.md: rewritten for xreach - docs/troubleshooting.md: updated proxy guidance - README.md + README_en.md: all references updated - config.py: twitter_bird → twitter_xreach - core.py, mcp_server.py: comment updates npm package: https://www.npmjs.com/package/xreach-cli Source: https://github.com/Panniantong/xfetch --- README.md | 20 ++--- agent_reach/channels/twitter.py | 18 ++--- agent_reach/cli.py | 34 ++++---- agent_reach/config.py | 2 +- agent_reach/core.py | 2 +- agent_reach/guides/setup-twitter.md | 102 +++++++++++++----------- agent_reach/integrations/mcp_server.py | 2 +- agent_reach/skill/SKILL.md | 12 +-- docs/README_en.md | 20 ++--- docs/install.md | 10 +-- docs/troubleshooting.md | 103 ++++++++++--------------- 11 files changed, 155 insertions(+), 170 deletions(-) diff --git a/README.md b/README.md index 9d2af3a..c054962 100644 --- a/README.md +++ b/README.md @@ -50,7 +50,7 @@ AI Agent 已经能帮你写代码、改文档、管项目——但你让它去 |---|---| | 💰 **完全免费** | 所有工具开源、所有 API 免费。唯一可能花钱的是服务器代理（$1/月），本地电脑不需要 | | 🔒 **隐私安全** | Cookie 只存在你本地，不上传不外传。代码完全开源，随时可审查 | -| 🔄 **持续更新** | 底层工具（yt-dlp、bird、Jina Reader 等）定期追踪更新到最新版，你不用自己盯 | +| 🔄 **持续更新** | 底层工具（yt-dlp、xreach、Jina Reader 等）定期追踪更新到最新版，你不用自己盯 | | 🤖 **兼容所有 Agent** | Claude Code、OpenClaw、Cursor、Windsurf……任何能跑命令行的 Agent 都能用 | | 🩺 **自带诊断** | `agent-reach doctor` 一条命令告诉你哪个通、哪个不通、怎么修 | @@ -102,7 +102,7 @@ AI Agent 已经能帮你写代码、改文档、管项目——但你让它去 它会做什么？（点击展开） 1. **安装 CLI 工具** — `pip install` 装好 `agent-reach` 命令行 -2. **安装系统依赖** — 自动检测并安装 Node.js、gh CLI、mcporter、bird 等 +2. **安装系统依赖** — 自动检测并安装 Node.js、gh CLI、mcporter、xreach 等 3. **配置搜索引擎** — 通过 MCP 接入 Exa（免费，无需 API Key） 4. **检测环境** — 判断是本地电脑还是服务器，给出对应的配置建议 5. **注册 SKILL.md** — 在 Agent 的 skills 目录安装使用指南，以后 Agent 遇到"搜推特"、"看视频"这类需求，会自动知道该调哪个上游工具 @@ -119,7 +119,7 @@ AI Agent 已经能帮你写代码、改文档、管项目——但你让它去 - "帮我看看这个链接" → `curl https://r.jina.ai/URL` 读任意网页 - "这个 GitHub 仓库是做什么的" → `gh repo view owner/repo` - "这个视频讲了什么" → `yt-dlp --dump-json URL` 提取字幕 -- "帮我看看这条推文" → `bird read URL --json` +- "帮我看看这条推文" → `xreach tweet URL --json` - "订阅这个 RSS" → `feedparser` 解析 - "搜一下 GitHub 上有什么 LLM 框架" → `gh search repos "LLM framework"` @@ -135,7 +135,7 @@ AI Agent 已经能帮你写代码、改文档、管项目——但你让它去 Agent Reach 做的事情很简单：**帮你把这些选型和配置的活儿做完了。** -安装完成后，Agent 直接调用上游工具（bird CLI、yt-dlp、mcporter、gh CLI 等），不需要经过 Agent Reach 的包装层。 +安装完成后，Agent 直接调用上游工具（xreach CLI、yt-dlp、mcporter、gh CLI 等），不需要经过 Agent Reach 的包装层。 ### 🔌 每个渠道都是可插拔的 @@ -144,7 +144,7 @@ Agent Reach 做的事情很简单：**帮你把这些选型和配置的活儿做 ``` channels/ ├── web.py → Jina Reader ← 可以换成 Firecrawl、Crawl4AI…… -├── twitter.py → bird ← 可以换成 Nitter、官方 API…… +├── twitter.py → xreach ← 可以换成 Nitter、官方 API…… ├── youtube.py → yt-dlp ← 可以换成 YouTube API、Whisper…… ├── github.py → gh CLI ← 可以换成 REST API、PyGithub…… ├── bilibili.py → yt-dlp ← 可以换成 bilibili-api…… @@ -165,7 +165,7 @@ channels/ | 场景 | 选型 | 为什么选它 | |------|------|-----------| | 读网页 | [Jina Reader](https://github.com/jina-ai/reader) | 9.8K Star，免费，不需要 API Key | -| 读推特 | [bird](https://www.npmjs.com/package/@steipete/bird) | Cookie 登录，免费。官方 API 按量付费（读一条 $0.005） | +| 读推特 | [xreach](https://www.npmjs.com/package/xreach-cli) | Cookie 登录，免费。官方 API 按量付费（读一条 $0.005） | | 视频字幕 + 搜索 | [yt-dlp](https://github.com/yt-dlp/yt-dlp) | 148K Star，YouTube + B站 + 1800 站通吃 | | 搜全网 | [Exa](https://exa.ai) via [mcporter](https://github.com/steipete/mcporter) | AI 语义搜索，MCP 接入免 Key | | GitHub | [gh CLI](https://cli.github.com) | 官方工具，认证后完整 API 能力 | @@ -258,13 +258,13 @@ Star 一下，下次需要的时候能找到。⭐
AI Agent 怎么搜索 Twitter / X？不想付 API 费用 -Agent Reach 使用 [bird CLI](https://www.npmjs.com/package/@steipete/bird) 通过 Cookie 认证访问 Twitter，完全免费。安装 Agent Reach 后，用 Cookie-Editor 导出你的 Twitter Cookie，运行 `agent-reach configure twitter-cookies "your_cookies"` 即可。之后 Agent 就可以用 `bird search "关键词" --json` 搜索推文了。 +Agent Reach 使用 [xreach CLI](https://www.npmjs.com/package/xreach-cli) 通过 Cookie 认证访问 Twitter，完全免费。安装 Agent Reach 后，用 Cookie-Editor 导出你的 Twitter Cookie，运行 `agent-reach configure twitter-cookies "your_cookies"` 即可。之后 Agent 就可以用 `xreach search "关键词" --json` 搜索推文了。
How to search Twitter/X with AI agent for free (no API)? -Agent Reach uses the bird CLI with cookie auth — zero API fees. After installing, export your Twitter cookies with the Cookie-Editor extension, run `agent-reach configure twitter-cookies "your_cookies"`, then your agent can search with `bird search "query" --json`. +Agent Reach uses the xreach CLI with cookie auth — zero API fees. After installing, export your Twitter cookies with the Cookie-Editor extension, run `agent-reach configure twitter-cookies "your_cookies"`, then your agent can search with `xreach search "query" --json`.
@@ -300,14 +300,14 @@ Yes! Agent Reach is an installer + configuration tool — any AI coding agent th
Is this free? Any API costs? -100% free. All backends are open-source tools (bird CLI, yt-dlp, Jina Reader, Exa, etc.) that don't require paid API keys. The only optional cost is a residential proxy (~$1/month) if you need Reddit/Bilibili access from a server. +100% free. All backends are open-source tools (xreach CLI, yt-dlp, Jina Reader, Exa, etc.) that don't require paid API keys. The only optional cost is a residential proxy (~$1/month) if you need Reddit/Bilibili access from a server.
--- ## 致谢 -[Jina Reader](https://github.com/jina-ai/reader) · [yt-dlp](https://github.com/yt-dlp/yt-dlp) · [bird](https://www.npmjs.com/package/@steipete/bird) · [Exa](https://exa.ai) · [mcporter](https://github.com/steipete/mcporter) · [feedparser](https://github.com/kurtmckee/feedparser) · [xiaohongshu-mcp](https://github.com/xpzouying/xiaohongshu-mcp) · [douyin-mcp-server](https://github.com/yzfly/douyin-mcp-server) · [linkedin-scraper-mcp](https://github.com/stickerdaniel/linkedin-mcp-server) · [mcp-bosszp](https://github.com/mucsbr/mcp-bosszp) +[Jina Reader](https://github.com/jina-ai/reader) · [yt-dlp](https://github.com/yt-dlp/yt-dlp) · [xreach](https://www.npmjs.com/package/xreach-cli) · [Exa](https://exa.ai) · [mcporter](https://github.com/steipete/mcporter) · [feedparser](https://github.com/kurtmckee/feedparser) · [xiaohongshu-mcp](https://github.com/xpzouying/xiaohongshu-mcp) · [douyin-mcp-server](https://github.com/yzfly/douyin-mcp-server) · [linkedin-scraper-mcp](https://github.com/stickerdaniel/linkedin-mcp-server) · [mcp-bosszp](https://github.com/mucsbr/mcp-bosszp) ## License diff --git a/agent_reach/channels/twitter.py b/agent_reach/channels/twitter.py index cfe605b..8d08d4a 100644 --- a/agent_reach/channels/twitter.py +++ b/agent_reach/channels/twitter.py @@ -1,5 +1,5 @@ # -*- coding: utf-8 -*- -"""Twitter/X — check if bird CLI is available.""" +"""Twitter/X — check if xreach CLI is available.""" import shutil import subprocess @@ -9,7 +9,7 @@ from .base import Channel class TwitterChannel(Channel): name = "twitter" description = "Twitter/X 推文" - backends = ["bird CLI"] + backends = ["xreach CLI"] tier = 1 def can_handle(self, url: str) -> bool: @@ -18,21 +18,21 @@ class TwitterChannel(Channel): return "x.com" in d or "twitter.com" in d def check(self, config=None): - bird = shutil.which("bird") or shutil.which("birdx") - if not bird: + xreach = shutil.which("xreach") + if not xreach: return "warn", ( - "bird CLI 未安装。搜索可通过 Exa 替代。安装：\n" - " npm install -g @steipete/bird" + "xreach CLI 未安装。搜索可通过 Exa 替代。安装：\n" + " npm install -g xreach-cli" ) try: r = subprocess.run( - [bird, "whoami"], capture_output=True, text=True, timeout=10 + [xreach, "auth", "check"], capture_output=True, text=True, timeout=10 ) if r.returncode == 0: return "ok", "完整可用（读取、搜索推文）" return "warn", ( - "bird CLI 已安装但未配置 Cookie。运行：\n" + "xreach CLI 已安装但未配置 Cookie。运行：\n" " agent-reach configure twitter-cookies \"auth_token=xxx; ct0=yyy\"" ) except Exception: - return "warn", "bird CLI 已安装但连接失败" + return "warn", "xreach CLI 已安装但连接失败" diff --git a/agent_reach/cli.py b/agent_reach/cli.py index 17d2c51..b734130 100644 --- a/agent_reach/cli.py +++ b/agent_reach/cli.py @@ -387,24 +387,24 @@ def _install_system_deps(): except Exception: print(" ⚠️ Node.js install failed. Try: apt install nodejs npm, or nvm install 22, or download from https://nodejs.org") - # ── bird CLI (for Twitter search) ── - if shutil.which("bird") or shutil.which("birdx"): - print(" ✅ bird CLI already installed") + # ── xreach CLI (for Twitter search) ── + if shutil.which("xreach"): + print(" ✅ xreach CLI already installed") else: if shutil.which("npm"): try: subprocess.run( - ["npm", "install", "-g", "@steipete/bird"], + ["npm", "install", "-g", "xreach-cli"], capture_output=True, text=True, timeout=120, ) - if shutil.which("bird"): - print(" ✅ bird CLI installed (Twitter search + timeline)") + if shutil.which("xreach"): + print(" ✅ xreach CLI installed (Twitter search + timeline)") else: - print(" ⬜ bird CLI install failed (optional — Twitter reading still works via Jina)") + print(" ⬜ xreach CLI install failed (optional — Twitter reading still works via Jina)") except Exception: - print(" ⬜ bird CLI install failed (optional — Twitter reading still works via Jina)") + print(" ⬜ xreach CLI install failed (optional — Twitter reading still works via Jina)") else: - print(" ⬜ bird CLI requires Node.js (optional — Twitter reading still works via Jina)") + print(" ⬜ xreach CLI requires Node.js (optional — Twitter reading still works via Jina)") # ── undici (proxy support for Node.js fetch) ── if shutil.which("npm"): @@ -417,7 +417,7 @@ def _install_system_deps(): subprocess.run(["npm", "install", "-g", "undici"], capture_output=True, text=True, timeout=60) print(" ✅ undici installed (Node.js proxy support)") except Exception: - print(" ⬜ undici install failed (optional — bird may not work behind proxies)") + print(" ⬜ undici install failed (optional — xreach may not work behind proxies)") def _install_system_deps_safe(): @@ -429,7 +429,7 @@ def _install_system_deps_safe(): deps = [ ("gh", ["gh"], "GitHub CLI", "https://cli.github.com — or: apt install gh / brew install gh"), ("node", ["node", "npm"], "Node.js", "https://nodejs.org — or: apt install nodejs npm"), - ("bird", ["bird", "birdx"], "bird CLI (Twitter)", "npm install -g @steipete/bird"), + ("xreach", ["xreach"], "xreach CLI (Twitter)", "npm install -g xreach-cli"), ] missing = [] @@ -459,7 +459,7 @@ def _install_system_deps_dryrun(): checks = [ ("gh CLI", ["gh"], "apt install gh / brew install gh"), ("Node.js", ["node"], "curl NodeSource setup | bash + apt install nodejs"), - ("bird CLI", ["bird", "birdx"], "npm install -g @steipete/bird"), + ("xreach CLI", ["xreach"], "npm install -g xreach-cli"), ] for label, binaries, method in checks: @@ -690,16 +690,16 @@ def _cmd_configure(args): print("Testing Twitter access...", end=" ") try: import subprocess - bird = shutil.which("bird") or shutil.which("birdx") - if not bird: - print("⚠️ bird CLI not installed. Run: npm install -g @steipete/bird") + xreach = shutil.which("xreach") + if not xreach: + print("⚠️ xreach CLI not installed. Run: npm install -g xreach-cli") else: import os env = os.environ.copy() env["AUTH_TOKEN"] = auth_token env["CT0"] = ct0 result = subprocess.run( - [bird, "search", "test", "-n", "1"], + [xreach, "search", "test", "-n", "1"], capture_output=True, text=True, timeout=15, env=env, ) @@ -823,7 +823,7 @@ def _cmd_uninstall(args): print() print("Optional: remove tools installed by Agent Reach:") print(" npm uninstall -g mcporter") - print(" npm uninstall -g @steipete/bird") + print(" npm uninstall -g xreach-cli") print(" npm uninstall -g undici") diff --git a/agent_reach/config.py b/agent_reach/config.py index 49564b3..43b7a23 100644 --- a/agent_reach/config.py +++ b/agent_reach/config.py @@ -22,7 +22,7 @@ class Config: FEATURE_REQUIREMENTS = { "exa_search": ["exa_api_key"], "reddit_proxy": ["reddit_proxy"], - "twitter_bird": ["twitter_auth_token", "twitter_ct0"], + "twitter_xreach": ["twitter_auth_token", "twitter_ct0"], "groq_whisper": ["groq_api_key"], "github_token": ["github_token"], } diff --git a/agent_reach/core.py b/agent_reach/core.py index fd8caae..1d009e4 100644 --- a/agent_reach/core.py +++ b/agent_reach/core.py @@ -3,7 +3,7 @@ AgentReach — installer, doctor, and configuration tool. Agent Reach helps AI agents install and configure upstream platform tools -(bird CLI, yt-dlp, mcporter, gh CLI, etc.). After installation, agents +(xreach CLI, yt-dlp, mcporter, gh CLI, etc.). After installation, agents call the upstream tools directly — no wrapper layer needed. Usage: diff --git a/agent_reach/guides/setup-twitter.md b/agent_reach/guides/setup-twitter.md index f9da9de..95b939b 100644 --- a/agent_reach/guides/setup-twitter.md +++ b/agent_reach/guides/setup-twitter.md @@ -1,67 +1,77 @@ -# Twitter 高级功能配置指南（bird CLI） +# Twitter 高级功能配置指南（xreach CLI） -## 功能说明 -基础 Twitter 功能（搜索+读单条推文）无需配置，开箱即用。 +Twitter 基础阅读通过 Jina Reader 免费可用，无需配置。 -高级功能需要 bird CLI： -- 查看用户时间线 -- 深度搜索（更精确、更多结果） -- 读取完整线程（thread） -- 查看关注列表推文 +高级功能需要 xreach CLI： -bird 是免费开源工具（npm 包 @steipete/bird），但需要你的 Twitter 账号 cookie。 +- 搜索推文（`xreach search`） +- 读取完整推文和对话链（`xreach tweet`、`xreach thread`） +- 用户时间线（`xreach tweets`） -## Agent 可自动完成的步骤 +xreach 是免费开源工具（npm 包 xreach-cli），但需要你的 Twitter 账号 cookie。 + +## 快速配置 + +1. 检查 xreach 是否安装： -1. 检查 bird 是否安装： ```bash -which bird && echo "installed" || echo "not installed" +which xreach && echo "installed" || echo "not installed" ``` -2. 安装 bird： +2. 安装 xreach： + ```bash -npm install -g @steipete/bird +npm install -g xreach-cli ``` -3. 测试（需要先配置 cookie）： +3. 测试是否配置好： + ```bash -AUTH_TOKEN="xxx" CT0="yyy" bird search "test" -n 1 +AUTH_TOKEN="xxx" CT0="yyy" xreach search "test" -n 1 ``` -4. 用 agent-reach 配置 cookie（推荐）： +## 获取 Cookie（Cookie-Editor 方式，推荐） + +1. 安装 [Cookie-Editor](https://cookie-editor.com/) 浏览器扩展 +2. 登录 x.com +3. 点击 Cookie-Editor 图标 → Export → 复制全部 +4. 运行配置命令： + ```bash -agent-reach configure twitter-cookies "auth_token=xxx; ct0=yyy" +agent-reach configure twitter-cookies "粘贴的 cookie JSON" ``` -## 需要用户手动做的步骤 +这会自动提取 `auth_token` 和 `ct0`，并写入环境变量。 -请告诉用户： +## 手动设置 Cookie -> Twitter 高级功能需要你的 Twitter 账号 cookie（完全免费）。 -> -> **最简单的方式：** -> 1. 安装 Chrome 插件 [Cookie-Editor](https://chromewebstore.google.com/detail/cookie-editor/hlkenndednhfkekhgcdicdfddnkalmdm) -> 2. 打开 https://x.com 并确保已登录 -> 3. 点击 Cookie-Editor 插件图标 → Export → Header String -> 4. 把导出的内容发给我 -> -> **手动方式：** -> 1. 用 Chrome 打开 https://x.com 并确保你已登录 -> 2. 按 **F12** 打开开发者工具（Mac 按 Cmd+Option+I） -> 3. 点击顶部的 **Application**（应用）标签 -> 4. 左侧找到 **Cookies** → **https://x.com** -> 5. 在列表中找到以下两个值，双击复制： -> - **auth_token** — 一串字母数字 -> - **ct0** — 一串字母数字 -> 6. 把这两个值发给我 -> -> ⚠️ 这些 cookie 让我能以你的身份读取推文（只读）。我不会发推、点赞或做任何操作。 -> ⚠️ cookie 大约 1-3 个月会过期，届时需要重新导出。 +如果你已经知道 `auth_token` 和 `ct0`： -## Agent 收到 cookie 后的操作 +1. 安装 xreach（如果没装）：`npm install -g xreach-cli` -1. 安装 bird（如果没装）：`npm install -g @steipete/bird` -2. 配置 cookie：`agent-reach configure twitter-cookies "粘贴的内容"` -3. 测试：运行 `agent-reach doctor` 确认 Twitter 状态 -4. 反馈："✅ Twitter 高级功能已开启！现在可以搜索推文、查看时间线了。" -5. 如果失败："❌ Cookie 无效或已过期，请重新导出。" +2. 设置环境变量： + +```bash +export AUTH_TOKEN="你的auth_token" +export CT0="你的ct0" +``` + +3. 测试： + +```bash +xreach search "test" --auth-token "$AUTH_TOKEN" --ct0 "$CT0" -n 1 +``` + +## 代理配置 + +> xreach CLI 内置代理支持，通过 `--proxy` 参数传入： + +```bash +xreach search "test" --auth-token "$AUTH_TOKEN" --ct0 "$CT0" --proxy "http://user:pass@host:port" +``` + +也支持代理轮换文件： + +```bash +xreach search "test" --auth-token "$AUTH_TOKEN" --ct0 "$CT0" --proxy-file proxies.txt +``` diff --git a/agent_reach/integrations/mcp_server.py b/agent_reach/integrations/mcp_server.py index 636e4ee..8a09758 100644 --- a/agent_reach/integrations/mcp_server.py +++ b/agent_reach/integrations/mcp_server.py @@ -5,7 +5,7 @@ Agent Reach MCP Server — expose doctor/status as MCP tool. Run: python -m agent_reach.integrations.mcp_server Agent Reach is an installer + doctor tool. For actual reading/searching, -agents should call upstream tools directly (bird, yt-dlp, mcporter, etc.). +agents should call upstream tools directly (xreach, yt-dlp, mcporter, etc.). """ import asyncio diff --git a/agent_reach/skill/SKILL.md b/agent_reach/skill/SKILL.md index 7616e6c..8799945 100644 --- a/agent_reach/skill/SKILL.md +++ b/agent_reach/skill/SKILL.md @@ -23,7 +23,7 @@ agent-reach install --env=auto agent-reach doctor ``` -`install` auto-detects your environment and installs core dependencies (Node.js, mcporter, bird CLI, gh CLI, yt-dlp, feedparser). Run `doctor` to see what's active. +`install` auto-detects your environment and installs core dependencies (Node.js, mcporter, xreach CLI, gh CLI, yt-dlp, feedparser). Run `doctor` to see what's active. ## Management @@ -78,17 +78,17 @@ When a user asks to configure/enable any channel: After `agent-reach install`, call the upstream tools directly. No need for `agent-reach read` or `agent-reach search`. -### Twitter/X (bird CLI) +### Twitter/X (xreach CLI) ```bash # Search tweets -bird search "query" --json -n 10 +xreach search "query" --json -n 10 # Read a specific tweet -bird read https://x.com/user/status/123 --json +xreach tweet https://x.com/user/status/123 --json # Read a user's timeline -bird timeline @username --json -n 20 +xreach tweets @username --json -n 20 ``` ### YouTube (yt-dlp) @@ -248,7 +248,7 @@ for e in d.entries[:5]: ### Twitter "fetch failed" -bird CLI uses Node.js native `fetch()`, which doesn't respect `HTTP_PROXY`. Solutions: +xreach CLI uses Node.js `undici`, which doesn't respect `HTTP_PROXY`. Solutions: 1. Ensure `undici` is installed: `npm install -g undici` 2. Configure proxy: `agent-reach configure proxy http://user:pass@ip:port` 3. If still failing, use transparent proxy (Clash TUN, Proxifier) diff --git a/docs/README_en.md b/docs/README_en.md index aa0f5d2..46402ff 100644 --- a/docs/README_en.md +++ b/docs/README_en.md @@ -45,7 +45,7 @@ Copy that to your Agent. A few minutes later, it can read tweets, search Reddit, |---|---| | 💰 **Completely free** | All tools are open source, all APIs are free. The only possible cost is a server proxy ($1/month) — local computers don't need one | | 🔒 **Privacy safe** | Cookies stay local. Never uploaded. Fully open source — audit anytime | -| 🔄 **Kept up to date** | Upstream tools (yt-dlp, bird, Jina Reader, etc.) are tracked and updated regularly | +| 🔄 **Kept up to date** | Upstream tools (yt-dlp, xreach, Jina Reader, etc.) are tracked and updated regularly | | 🤖 **Works with any Agent** | Claude Code, OpenClaw, Cursor, Windsurf… any Agent that can run commands | | 🩺 **Built-in diagnostics** | `agent-reach doctor` — one command shows what works, what doesn't, and how to fix it | @@ -56,7 +56,7 @@ Copy that to your Agent. A few minutes later, it can read tweets, search Reddit, | Platform | Capabilities | Setup | Notes | |----------|-------------|:-----:|-------| | 🌐 **Web** | Read | Zero config | Any URL → clean Markdown ([Jina Reader](https://github.com/jina-ai/reader) ⭐9.8K) | -| 🐦 **Twitter/X** | Read · Search | Zero config / Cookie | Single tweets readable out of the box. Cookie unlocks search, timeline, posting ([bird](https://github.com/steipete/bird)) | +| 🐦 **Twitter/X** | Read · Search | Zero config / Cookie | Single tweets readable out of the box. Cookie unlocks search, timeline, posting ([xreach](https://www.npmjs.com/package/xreach-cli)) | | 📕 **XiaoHongShu** | Read · Search · **Post · Comment · Like** | mcporter | Via [xiaohongshu-mcp](https://github.com/user/xiaohongshu-mcp) internal API, install and go | | 🎵 **Douyin** | Video parsing · Watermark-free download | mcporter | Via [douyin-mcp-server](https://github.com/yzfly/douyin-mcp-server), no login needed | | 💼 **LinkedIn** | Jina Reader (public pages) | Full profiles, companies, job search | Tell your Agent "help me set up LinkedIn" | @@ -112,7 +112,7 @@ No configuration needed — just tell your Agent: - "Read this link" → `curl https://r.jina.ai/URL` for any web page - "What's this GitHub repo about?" → `gh repo view owner/repo` - "What does this video cover?" → `yt-dlp --dump-json URL` for subtitles -- "Read this tweet" → `bird read URL --json` +- "Read this tweet" → `xreach tweet URL --json` - "Subscribe to this RSS" → `feedparser` to parse feeds - "Search GitHub for LLM frameworks" → `gh search repos "LLM framework"` @@ -172,7 +172,7 @@ Every time you spin up a new Agent, you spend time finding tools, installing dep Agent Reach does one simple thing: **it makes those tool selection and configuration decisions for you.** -After installation, your Agent calls the upstream tools directly (bird CLI, yt-dlp, mcporter, gh CLI, etc.) — no wrapper layer in between. +After installation, your Agent calls the upstream tools directly (xreach CLI, yt-dlp, mcporter, gh CLI, etc.) — no wrapper layer in between. ### 🔌 Every Channel is Pluggable @@ -181,7 +181,7 @@ Each platform maps to an upstream tool. **Don't like one? Swap it out.** ``` channels/ ├── web.py → Jina Reader ← swap to Firecrawl, Crawl4AI… -├── twitter.py → bird ← swap to Nitter, official API… +├── twitter.py → xreach ← swap to Nitter, official API… ├── youtube.py → yt-dlp ← swap to YouTube API, Whisper… ├── github.py → gh CLI ← swap to REST API, PyGithub… ├── bilibili.py → yt-dlp ← swap to bilibili-api… @@ -202,7 +202,7 @@ Each channel file only checks whether its upstream tool is installed and working | Scenario | Tool | Why | |----------|------|-----| | Read web pages | [Jina Reader](https://github.com/jina-ai/reader) | 9.8K stars, free, no API key needed | -| Read tweets | [bird](https://github.com/steipete/bird) | Cookie auth, free. Official API is pay-per-use ($0.005/post read) | +| Read tweets | [xreach](https://www.npmjs.com/package/xreach-cli) | Cookie auth, free. Official API is pay-per-use ($0.005/post read) | | Video subtitles + search | [yt-dlp](https://github.com/yt-dlp/yt-dlp) | 148K stars, YouTube + Bilibili + 1800 sites | | Search the web | [Exa](https://exa.ai) via [mcporter](https://github.com/nicepkg/mcporter) | AI semantic search, MCP integration, no API key | | GitHub | [gh CLI](https://cli.github.com) | Official tool, full API after auth | @@ -233,7 +233,7 @@ This project was entirely vibe-coded 🎸 There might be rough edges here and th
How to search Twitter/X with AI agent without paying for API? -Agent Reach uses the [bird CLI](https://www.npmjs.com/package/@steipete/bird) with cookie-based authentication — completely free, no Twitter API subscription needed. After installing Agent Reach, export your Twitter cookies using the Cookie-Editor Chrome extension, run `agent-reach configure twitter-cookies "your_cookies"`, and your agent can search with `bird search "query" --json`. +Agent Reach uses the [xreach CLI](https://www.npmjs.com/package/xreach-cli) with cookie-based authentication — completely free, no Twitter API subscription needed. After installing Agent Reach, export your Twitter cookies using the Cookie-Editor Chrome extension, run `agent-reach configure twitter-cookies "your_cookies"`, and your agent can search with `xreach search "query" --json`.
@@ -257,13 +257,13 @@ Yes! Agent Reach is an installer + configuration tool. Any AI coding agent that
Is Agent Reach free? Any API costs? -100% free and open source. All backends (bird CLI, yt-dlp, Jina Reader, Exa) are free tools that don't require paid API keys. The only optional cost is a residential proxy (~$1/month) if you need Reddit/Bilibili access from a server. +100% free and open source. All backends (xreach CLI, yt-dlp, Jina Reader, Exa) are free tools that don't require paid API keys. The only optional cost is a residential proxy (~$1/month) if you need Reddit/Bilibili access from a server.
Free alternative to Twitter API for web scraping? -Agent Reach uses bird CLI which accesses Twitter via cookie auth — same as your browser session. No API fees, no rate limit tiers, no developer account needed. Supports search, read tweets, read profiles, and timelines. +Agent Reach uses xreach CLI which accesses Twitter via cookie auth — same as your browser session. No API fees, no rate limit tiers, no developer account needed. Supports search, read tweets, read profiles, and timelines.
@@ -282,7 +282,7 @@ Install douyin-mcp-server, then your agent can use `mcporter call 'douyin.parse_ ## Credits -[Jina Reader](https://github.com/jina-ai/reader) · [yt-dlp](https://github.com/yt-dlp/yt-dlp) · [bird](https://github.com/steipete/bird) · [Exa](https://exa.ai) · [feedparser](https://github.com/kurtmckee/feedparser) · [douyin-mcp-server](https://github.com/yzfly/douyin-mcp-server) · [linkedin-scraper-mcp](https://github.com/stickerdaniel/linkedin-mcp-server) · [mcp-bosszp](https://github.com/mucsbr/mcp-bosszp) +[Jina Reader](https://github.com/jina-ai/reader) · [yt-dlp](https://github.com/yt-dlp/yt-dlp) · [xreach](https://www.npmjs.com/package/xreach-cli) · [Exa](https://exa.ai) · [feedparser](https://github.com/kurtmckee/feedparser) · [douyin-mcp-server](https://github.com/yzfly/douyin-mcp-server) · [linkedin-scraper-mcp](https://github.com/stickerdaniel/linkedin-mcp-server) · [mcp-bosszp](https://github.com/mucsbr/mcp-bosszp) ## License diff --git a/docs/install.md b/docs/install.md index 2c7916a..51a4350 100644 --- a/docs/install.md +++ b/docs/install.md @@ -20,7 +20,7 @@ Copy this to your AI Agent: ### Goal -Install Agent Reach and all upstream tools so your user's agent has full internet access. After installation, you'll use the upstream tools directly (bird CLI, yt-dlp, mcporter, gh CLI, etc.) — Agent Reach is just the installer and health checker. +Install Agent Reach and all upstream tools so your user's agent has full internet access. After installation, you'll use the upstream tools directly (xreach CLI, yt-dlp, mcporter, gh CLI, etc.) — Agent Reach is just the installer and health checker. ### ⚠️ Boundaries @@ -41,7 +41,7 @@ pip install https://github.com/Panniantong/agent-reach/archive/main.zip agent-reach install --env=auto ``` -This auto-installs system dependencies (gh CLI, Node.js, mcporter, bird), configures Exa search, detects environment, and tests all channels. +This auto-installs system dependencies (gh CLI, Node.js, mcporter, xreach), configures Exa search, detects environment, and tests all channels. **Safe mode (no automatic system changes):** @@ -103,9 +103,9 @@ agent-reach configure twitter-cookies "PASTED_STRING" > **代理说明（中国大陆等需要翻墙的网络环境）：** > -> bird CLI 使用 Node.js 原生 `fetch()`，默认不走系统代理（`HTTP_PROXY`/`HTTPS_PROXY`）。 +> xreach CLI 使用 Node.js 原生 `fetch()`，默认不走系统代理（`HTTP_PROXY`/`HTTPS_PROXY`）。 > Agent Reach 已内置解决方案：安装时会自动安装 `undici`（`npm install -g undici`）， -> 运行 bird 时自动注入 `EnvHttpProxyAgent`，让 fetch 走用户配置的代理。 +> 运行 xreach 时自动注入 `EnvHttpProxyAgent`，让 fetch 走用户配置的代理。 > > **你（Agent）需要做的：** > 1. 确认 `undici` 已全局安装：`npm list -g undici`，没有就装：`npm install -g undici` @@ -277,7 +277,7 @@ After installation, use upstream tools directly. See SKILL.md for the full comma | Platform | Upstream Tool | Example | |----------|--------------|---------| -| Twitter/X | `bird` | `bird search "query" --json` | +| Twitter/X | `xreach` | `xreach search "query" --json` | | YouTube | `yt-dlp` | `yt-dlp --dump-json URL` | | Bilibili | `yt-dlp` | `yt-dlp --dump-json URL` | | Reddit | `curl` | `curl -s "https://reddit.com/r/xxx.json"` | diff --git a/docs/troubleshooting.md b/docs/troubleshooting.md index b6a934e..976ba10 100644 --- a/docs/troubleshooting.md +++ b/docs/troubleshooting.md @@ -1,69 +1,44 @@ -# Troubleshooting / 常见问题 +# 常见问题排查 -## Twitter/X: bird CLI "fetch failed" +## Twitter/X: xreach CLI "fetch failed" -**症状：** `bird whoami` 或 `bird search` 返回 "fetch failed" +**症状：** `xreach search` 或其他命令返回 "fetch failed" -**原因：** bird CLI 使用 Node.js 原生 `fetch()` 发请求，而 Node.js 的 fetch **不走系统代理**（不读取 `HTTP_PROXY`/`HTTPS_PROXY` 环境变量）。如果你的网络环境需要代理才能访问 x.com，bird 就连不上。 - -**解决方案（按推荐顺序）：** - -### 方案 1：使用透明代理 / TUN 模式（推荐） - -让代理工具接管所有网络流量，这样 bird 的 fetch 也会走代理： - -- **Clash Verge / Clash for Windows：** 开启 TUN 模式或系统代理 -- **Proxifier（Windows）：** 添加规则让 Node.js 进程走代理 -- **macOS：** 在 Surge/ClashX Pro 中开启增强模式 - -### 方案 2：验证 Cookie 有效性 - -确认 Cookie 没过期： - -1. 在浏览器里正常登录 x.com -2. 用 [Cookie-Editor](https://chromewebstore.google.com/detail/cookie-editor/hlkenndednhfkekhgcdicdfddnkalmdm) 重新导出 Header String -3. 重新配置：`agent-reach configure twitter-cookies "新的Cookie"` - -### 方案 3：不用 bird，用 Exa 搜索替代 - -bird 不可用时，可以直接用 Exa 搜索 Twitter 内容： - -```bash -mcporter call 'exa.web_search_exa(query: "site:x.com query", numResults: 10)' -``` - -### 方案 4：配置 Node.js 全局代理（高级） - -安装 `global-agent` 让 Node.js 的 fetch 走代理： - -```bash -npm install -g global-agent -``` - -然后在运行 bird 前设置环境变量： - -```bash -# Linux / macOS -export GLOBAL_AGENT_HTTP_PROXY=http://127.0.0.1:7890 -export NODE_OPTIONS="--require global-agent/bootstrap" -bird search "test" - -# Windows (PowerShell) -$env:GLOBAL_AGENT_HTTP_PROXY = "http://127.0.0.1:7890" -$env:NODE_OPTIONS = "--require global-agent/bootstrap" -bird search "test" -``` - -> ⚠️ 注意：这个方案需要每次运行 bird 前都设置环境变量，不太方便。推荐用方案 1。 - ---- - -## Boss直聘: "访问行为异常" - -**症状：** mcp-bosszp 登录成功，但 API 请求返回"您的访问行为异常" - -**原因：** Boss直聘的反爬机制会检测请求指纹（不只是 IP），Python requests 库的特征与真实浏览器不同。 +**原因：** xreach CLI 使用 Node.js 的 `undici` 库发请求。如果你的网络环境需要代理才能访问 x.com，需要明确传入代理参数。 **解决方案：** -- **本地电脑：** 正常使用，一般不会被拦 -- **服务器：** 使用 Jina Reader 读取职位页面 + Exa 搜索职位信息作为替代 + +### 方案 1：使用 --proxy 参数 + +```bash +xreach search "test" --auth-token "$AUTH_TOKEN" --ct0 "$CT0" --proxy "http://user:pass@host:port" +``` + +### 方案 2：使用全局代理工具 + +让代理工具接管所有网络流量，这样 xreach 的请求也会走代理： + +```bash +# macOS — ClashX / Surge 开启"增强模式" +# Linux — proxychains 或 tun2socks +proxychains xreach search "test" -n 1 +``` + +### 方案 3：不用 xreach，用 Exa 搜索替代 + +xreach 不可用时，可以直接用 Exa 搜索 Twitter 内容： + +```bash +mcporter call 'exa.web_search_exa(query: "site:x.com 搜索词", numResults: 5)' +``` + +### 方案 4：设置 HTTP_PROXY 环境变量 + +```bash +export HTTP_PROXY="http://127.0.0.1:7890" +export HTTPS_PROXY="http://127.0.0.1:7890" + +xreach search "test" +``` + +> ⚠️ 注意：Node.js 原生 fetch 不一定读取这些环境变量，推荐用方案 1 的 --proxy 参数。