release: v3.37.1 - threat-db v2.8.0, CC releases v2.1.78, 19 skills update, doc fixes
- threat-db v2.8.0: GhostClaw campaign, Fake OpenClaw Installer, CVE-2026-24910 (Bun), T017 Shadow MCP, T018 AI Search Poisoning, Jozu Agent Guard, MCP Sentinel - Claude Code releases tracked to v2.1.78 (StopFailure hook, plugin state, security fixes) - 19 skill descriptions improved (PR #9 selective merge, @popey/Tessl) - MCP vs CLI token overhead corrected (lazy loading, 85% reduction benchmark) - Agent Adoption Curve self-assessment (7-level maturity scale, Martignole framework) - ctx7 CLI section §5.5 + resource evals #079 #080 #081 Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
This commit is contained in:
Florian BRUNIAUX
parent
eea5662a65
commit
44818a3f04
19 changed files with 785 additions and 87 deletions
141
docs/resource-evaluations/079-fabro-workflow-orchestration.md
Normal file
141
docs/resource-evaluations/079-fabro-workflow-orchestration.md
Normal file
|
|
@ -0,0 +1,141 @@
|
|||
r# Resource Evaluation #079 — Fabro: Graph-Based Workflow Orchestrator for AI Coding Agents
|
||||
|
||||
**Source:** [fabro.sh](https://fabro.sh) / [github.com/fabro-sh/fabro](https://github.com/fabro-sh/fabro)
|
||||
**Type:** Open source tool (MIT) — standalone workflow orchestrator for AI coding agents
|
||||
**Author:** Bryan from qlty.sh (bryan@qlty.sh)
|
||||
**Evaluated:** 2026-03-17
|
||||
**Maturity at evaluation:** Created 2026-03-13 — 4 days old, 28 stars, 1 fork
|
||||
|
||||
---
|
||||
|
||||
## Summary
|
||||
|
||||
- **Workflow orchestrator for AI coding agents**: define pipelines as Graphviz DOT graphs with branching, loops, parallelism, and human approval gates — diffable and version-controlled
|
||||
- **Multi-model routing**: CSS-like stylesheets assign different LLM models (Claude, OpenAI, Gemini) to specific workflow nodes, with automatic fallback chains
|
||||
- **Git checkpointing per stage**: every stage commits code changes and execution metadata to a dedicated Git branch — unique feature with no direct equivalent found in the landscape
|
||||
- **Cloud sandboxes**: isolated Daytona VMs with snapshot-based setup, network controls, SSH access, and automatic cleanup
|
||||
- **Automatic retrospectives**: each run generates a cost/duration/narrative retrospective for continuous improvement
|
||||
- **Direct Claude Code integration**: `curl -fsSL https://fabro.sh/install.md | claude` (security note: pipes directly into Claude, no intermediate review step)
|
||||
- **Single Rust binary, zero runtime dependencies**: no Python, no Node.js, no Docker required
|
||||
- **REST API + SSE streaming + React web UI**: run workflows programmatically or as a service
|
||||
|
||||
---
|
||||
|
||||
## Relevance Score
|
||||
|
||||
| Score | Meaning |
|
||||
|-------|---------|
|
||||
| 5 | Essential — Major gap in the guide |
|
||||
| 4 | Very relevant — Significant improvement |
|
||||
| **3** | **Pertinent — Useful complement, Watch status** |
|
||||
| 2 | Marginal — Secondary information |
|
||||
| 1 | Out of scope — Not relevant |
|
||||
|
||||
**Final score: 3/5 (Watch)**
|
||||
|
||||
**Justification:** Fabro falls directly in the "External Orchestration Frameworks" category already documented in the guide (`third-party-tools.md`). Its DOT graph approach is architecturally distinct from all three existing entries (Ruflo = swarms, Athena Flow = hooks layer, Pipelex = DSL). Git checkpointing per stage is a genuine differentiator — no equivalent found in the landscape. Direct Claude Code integration via `claude` pipeline is legitimate. However: 28 stars at 4 days old is the same immaturity profile as Athena Flow (#073, score 2/5). Raised to 3/5 vs Athena Flow because of stronger architectural clarity, a wider feature set with more evidence, and a genuinely unique Git checkpointing angle.
|
||||
|
||||
---
|
||||
|
||||
## Comparison
|
||||
|
||||
| Aspect | Fabro | Guide (current state) |
|
||||
|--------|-------|-----------------------|
|
||||
| DOT graph pipeline definition | Unique approach | Not covered |
|
||||
| Multi-model routing per node | CSS-like stylesheets | Not covered |
|
||||
| Git checkpointing per stage | Concrete differentiator | Not covered anywhere |
|
||||
| Cloud sandboxes (Daytona) | Declared, unverified in prod | Not covered |
|
||||
| Human-in-the-loop approval gates | Hexagon nodes in DOT | Partially covered via Ruflo |
|
||||
| External orchestration frameworks | External layer over Claude Code | Ruflo + Athena Flow + Pipelex |
|
||||
| Maturity / community traction | 28 stars, 4 days | Ruflo at 18.9k stars |
|
||||
| `curl \| claude` install security | Risk: no review step | Guide warns against `curl \| bash` |
|
||||
|
||||
---
|
||||
|
||||
## Competitive Landscape (Perplexity research, 2026-03-17)
|
||||
|
||||
Full competitive analysis conducted. Key findings:
|
||||
|
||||
| Tool | Category | Stars | Key difference from Fabro |
|
||||
|------|----------|-------|--------------------------|
|
||||
| **LangGraph** (LangChain) | Graph-based pipelines | ~34k | Python library (not standalone binary), general purpose (not coding-agent specific), no Git checkpointing |
|
||||
| **Goose** (Block) | Coding agent orchestration | ~15k | Recipe-based (not graph), conversational architecture, no Git checkpointing — but much more mature |
|
||||
| **OpenHands** | Coding agent platform | ~48k | Event-stream architecture, Docker sandboxes, research-oriented — no DOT graph, no Git checkpointing per stage |
|
||||
| **Ruflo** | External orchestration (guide) | 18.9k | Swarm-based (queen + workers), npm, SQLite memory — no DOT graph, no Git checkpointing |
|
||||
| **Athena Flow** | Hook-layer runtime (guide) | Watch | Hook → UDS → Node.js — entirely different architecture layer |
|
||||
| **Pipelex** | MTHDS DSL (guide) | Watch | Declarative DSL for multi-LLM pipelines — different abstraction |
|
||||
| **AutoGen** (Microsoft) | Multi-agent conversations | ~47.9k | General purpose, conversation-loop model, no coding-agent specifics |
|
||||
| **CrewAI** | Role-based agent crews | ~34.7k | Role assignment model, no graph definition, no Git checkpointing |
|
||||
|
||||
**Fabro's unique combination** (no competitor does all three):
|
||||
1. DOT graph workflow definition as a standalone binary
|
||||
2. Git checkpointing per stage (code + metadata committed to branches)
|
||||
3. Native Claude Code integration via `claude`
|
||||
|
||||
**Most relevant alternative for guide readers today**: LangGraph for graph-based workflows (much more mature, Python), Goose (Block) for coding agent orchestration (better traction, MIT).
|
||||
|
||||
---
|
||||
|
||||
## Recommendations
|
||||
|
||||
**When to integrate:** Add as Watch entry now. Promote to full entry in `guide/ecosystem/third-party-tools.md` under "External Orchestration Frameworks" when trigger is reached.
|
||||
|
||||
**Where:** After Athena Flow in `third-party-tools.md` External Orchestration Frameworks section.
|
||||
|
||||
**How:** Short entry (same format as Athena Flow) with:
|
||||
- Architectural distinction (DOT graph — distinct from all three existing entries)
|
||||
- Git checkpointing differentiator
|
||||
- Security note on `curl | claude` install (same pattern as Ruflo's `curl | bash` warning)
|
||||
- Status: "Published March 2026, not yet audited"
|
||||
|
||||
**Do NOT do:**
|
||||
- Recommend the `curl | claude` install without a security note
|
||||
- Present any feature as production-validated (no community evidence yet)
|
||||
- Add before the traction trigger is reached
|
||||
|
||||
**Secondary discovery:** Goose (Block, github.com/block/goose) warrants its own evaluation (#080). 15k stars, MIT, recipes + subagents + 20 LLM providers — potentially more immediately relevant to the guide's audience.
|
||||
|
||||
---
|
||||
|
||||
## Challenge (technical-writer agent)
|
||||
|
||||
**Initial proposed score:** 4/5
|
||||
**Score after challenge:** 3/5 (lowered)
|
||||
|
||||
Points raised:
|
||||
|
||||
- **Immaturity flag**: 28 stars / 4 days = same pattern as Athena Flow (score 2/5). Applying this inconsistently undermines the scoring framework. Compromise: 3/5 because Fabro shows more architectural evidence than Athena Flow at equivalent age.
|
||||
- **`curl | claude` security**: more dangerous than typical `curl | bash` — pipes directly into the codebase with no review step. Guide's own security section would flag this. Must be noted in any future integration.
|
||||
- **Ambitious unverified feature set**: cloud sandboxes, DOT routing, retrospectives — none validated by community use at evaluation time.
|
||||
- **Risk of not integrating at this stage**: near zero. Category already covered by 3 entries.
|
||||
- **What is genuinely novel**: DOT graph definition as diffable text (distinct from Ruflo/Pipelex/Athena) + Git checkpointing per stage = angles worth tracking.
|
||||
|
||||
---
|
||||
|
||||
## Fact-Check
|
||||
|
||||
| Claim | Verified | Source |
|
||||
|-------|----------|--------|
|
||||
| 28 GitHub stars | Verified | GitHub API direct |
|
||||
| Created 2026-03-13 | Verified | GitHub API `created_at` |
|
||||
| Rust, single binary, zero deps | Verified | README |
|
||||
| MIT license | Verified | GitHub API + README badge |
|
||||
| `curl \| claude` install | Verified | README + fabro.sh landing |
|
||||
| DOT graph workflows | Verified | README + example code |
|
||||
| Daytona cloud sandboxes | Declared | README feature table (unaudited) |
|
||||
| Supports Claude/OpenAI/Gemini | Declared | WebFetch landing (unaudited) |
|
||||
| Git checkpointing per stage | Declared | README feature table (unaudited) |
|
||||
| Automatic retrospectives | Declared | README feature table (unaudited) |
|
||||
| Bryan from qlty.sh | Verified | `bryan@qlty.sh` in README |
|
||||
|
||||
**No corrections needed:** all claims traced to primary sources. Features marked "Declared" are present in README but not community-validated.
|
||||
|
||||
---
|
||||
|
||||
## Final Decision
|
||||
|
||||
- **Final score**: 3/5
|
||||
- **Action**: Watch — add to `watch-list.md`, revisit when trigger reached
|
||||
- **Re-eval trigger**: >200 GitHub stars OR practitioner write-up from production use
|
||||
- **Confidence**: High on score, medium on features (project too recent for full audit)
|
||||
- **Next action**: Evaluate Goose (Block) as #080 — more immediately relevant to guide's audience
|
||||
120
docs/resource-evaluations/080-goose-block-coding-agent.md
Normal file
120
docs/resource-evaluations/080-goose-block-coding-agent.md
Normal file
|
|
@ -0,0 +1,120 @@
|
|||
# Resource Evaluation #080 — Goose (Block): Open-Source AI Coding Agent
|
||||
|
||||
**Source:** [block.github.io/goose](https://block.github.io/goose) / [github.com/block/goose](https://github.com/block/goose)
|
||||
**Type:** Open source tool (Apache 2.0) — on-machine AI coding agent
|
||||
**Author:** Block (formerly Square) — maintained by Block's engineering team
|
||||
**Evaluated:** 2026-03-17
|
||||
**Maturity at evaluation:** Launched officially January 2025, 33,166 stars, 3,058 forks
|
||||
|
||||
---
|
||||
|
||||
## Summary
|
||||
|
||||
- **On-machine AI coding agent**: local-first CLI + desktop app, not cloud. Automates complex engineering tasks end-to-end
|
||||
- **Model-agnostic**: works with Claude (recommended for tool calling), GPT-4o, Gemini, Groq, local models (Ollama) — 20+ providers
|
||||
- **Recipes**: versionable, shareable, parameterized multi-step workflows. Distinct from "rules files" — recipes define what agents do, not how they behave
|
||||
- **Subagent orchestration**: spawn specialized agents autonomously or via sub-recipes, with dynamic model switching per task/cost
|
||||
- **1,700+ MCP servers** supported (first open source agent to support MCP, January 2025)
|
||||
- **Goose Grant Program**: Block funds developers building Goose extensions (launched July 2025)
|
||||
- **Custom Distributions**: teams can build branded Goose distros with preconfigured providers, extensions, and branding
|
||||
- **Backed by Block** (Square, Cash App) — institutional engineering resources, not a solo project
|
||||
|
||||
---
|
||||
|
||||
## Status in the Guide
|
||||
|
||||
**Already documented**: `guide/ecosystem/ai-ecosystem.md` §11.1 "Goose: Open-Source Alternative (Block)"
|
||||
|
||||
**The entry exists and is structurally sound.** The issue is outdated data and two missing feature callouts.
|
||||
|
||||
---
|
||||
|
||||
## Relevance Score
|
||||
|
||||
| Score | Meaning |
|
||||
|-------|---------|
|
||||
| 5 | Essential — Major gap in the guide |
|
||||
| **4** | **Very relevant — Significant improvement needed** |
|
||||
| 3 | Pertinent — Useful complement |
|
||||
| 2 | Marginal — Secondary information |
|
||||
| 1 | Out of scope — Not relevant |
|
||||
|
||||
**Final score: 4/5 (Update existing entry)**
|
||||
|
||||
**Justification:** Goose is already documented. Score reflects the importance of keeping the entry current — at 33k stars (2x what the guide says), Goose is clearly not a niche alternative. The missing Recipes and subagent orchestration paragraphs are also directly relevant to a guide that extensively documents Claude Code's equivalent patterns (skills, slash commands, multi-agent).
|
||||
|
||||
---
|
||||
|
||||
## What Needs Updating in §11.1
|
||||
|
||||
### 1. Stats (outdated)
|
||||
|
||||
| Field | Current guide (Jan 2026) | Actual (Mar 2026) |
|
||||
|-------|--------------------------|-------------------|
|
||||
| GitHub Stars | 15,400+ | 33,166 |
|
||||
| MCP servers | 3,000+ (table) vs 1,700+ (inconsistency) | 1,700+ (per Goose docs) |
|
||||
| Releases | 100+ | ~175+ (estimated, fast release cadence) |
|
||||
|
||||
**Fix**: Update the metrics table and resolve the 3,000 vs 1,700 MCP inconsistency.
|
||||
|
||||
### 2. Recipes — missing
|
||||
|
||||
Recipes are Goose's equivalent of Claude Code slash commands + skills combined. They are:
|
||||
- Versionable, shareable as standalone workflows
|
||||
- Importable via deeplinks
|
||||
- Parameterized (reusable across contexts)
|
||||
- Can be shared across teams
|
||||
|
||||
This is directly relevant to a guide section that extensively documents commands and skills. One paragraph with a cross-reference to §3 (commands) and §4 (skills) is warranted.
|
||||
|
||||
### 3. Subagent orchestration — missing
|
||||
|
||||
Goose's July 2025 roadmap introduced subagent orchestration: spawn specialized sub-agents (Planner, Architect, Frontend Dev, Backend Dev) with dynamic model switching per agent. Example from Berkeley Agentic AI Summit: 7 agents collaboratively built a full-stack app in under an hour.
|
||||
|
||||
This overlaps with Claude Code's own multi-agent patterns (§9). A one-paragraph callout with a comparison to Claude Code's Agent tool would help readers understand the architectural difference (Claude Code: single agent + Tool spawning vs Goose: recipe-defined multi-agent subflows).
|
||||
|
||||
### 4. agentskills.io — verify live status
|
||||
|
||||
The "Skill Portability" paragraph references agentskills.io. **Verified live** (2026-03-17). No change needed.
|
||||
|
||||
---
|
||||
|
||||
## Challenge (technical-writer agent)
|
||||
|
||||
**Score confirmed: 4/5 (update pass)**
|
||||
|
||||
Key points:
|
||||
- 15.4k → 33k stars: 2x undercount signals the section hasn't been maintained. Trust erosion, not just a metric miss.
|
||||
- MCP discrepancy (3,000 vs 1,700): one of these is wrong. Fix before any update goes live.
|
||||
- Recipes and subagents absent: closest Goose analogy to Claude Code's skills + multi-agent. Should be documented.
|
||||
- Risk of not updating: low urgency for readers, moderate for guide credibility as a current reference.
|
||||
- Scope: 30-minute update pass, not a restructure.
|
||||
|
||||
---
|
||||
|
||||
## Fact-Check
|
||||
|
||||
| Claim | Verified | Source |
|
||||
|-------|----------|--------|
|
||||
| 33,166 GitHub stars | Verified | GitHub API (2026-03-17) |
|
||||
| 3,058 forks | Verified | GitHub API |
|
||||
| Apache 2.0 license | Verified | GitHub API + README |
|
||||
| Rust (primary language) | Verified | GitHub API (`language: "Rust"`) |
|
||||
| Created August 2024, launched Jan 2025 | Verified | GitHub API + "1 Year of goose" discussion (Jan 2026) |
|
||||
| First open source agent to support MCP | Claimed | "1 Year of goose" GitHub discussion |
|
||||
| 1,700+ MCP servers | Per Perplexity (sourced from Goose docs) | Cross-check recommended |
|
||||
| Claude 3.5 Sonnet recommended for tool calling | Claimed | Perplexity search citing Goose docs |
|
||||
| Goose Grant Program (July 2025) | Verified | block.xyz/inside/introducing-the-goose-grant-program |
|
||||
| agentskills.io live | Verified | HTTP fetch (2026-03-17) |
|
||||
| Dynamic model switching per subagent | Claimed | GitHub roadmap discussion #3319 |
|
||||
|
||||
**MCP server count discrepancy**: Guide says 3,000+ (comparison table), Perplexity reports 1,700+ from Goose docs. Need to check Goose documentation directly before updating. Use the more conservative figure if unsure.
|
||||
|
||||
---
|
||||
|
||||
## Final Decision
|
||||
|
||||
- **Final score**: 4/5
|
||||
- **Action**: Update existing `guide/ecosystem/ai-ecosystem.md` §11.1 — stats refresh + Recipes paragraph + subagent orchestration paragraph
|
||||
- **Confidence**: High on stats, medium on MCP server count (needs direct doc check)
|
||||
- **Priority**: Medium — not urgent, but a 2x star count delta is worth fixing promptly
|
||||
|
|
@ -0,0 +1,43 @@
|
|||
# Resource Evaluation: Martignole Agent Adoption Maturity Levels
|
||||
|
||||
**Date**: 2026-03-17
|
||||
**Evaluator**: Claude Code Ultimate Guide team
|
||||
**Score**: 3/5 — Integrated
|
||||
|
||||
---
|
||||
|
||||
## Resource
|
||||
|
||||
- **URL**: https://www.touilleur-express.fr/2026/03/17/decouvrir-les-niveaux-de-maturite-de-ladoption-des-coding-agents
|
||||
- **Author**: Nicolas Martignole, Principal Engineer at Back Market, Le Touilleur Express blog
|
||||
- **Type**: Practitioner framework / blog post
|
||||
- **Language**: French
|
||||
|
||||
## Summary
|
||||
|
||||
6-level maturity framework (0-5 in the original, extended to 6 here) for individual developers adopting coding agents. The real contribution is the Level 3-5 arc: basic user → stage delegator → context engineer → orchestrator. Maps cleanly onto Claude Code concepts (Plan mode, sub-agents, MCP servers, context engineering).
|
||||
|
||||
## Score Justification
|
||||
|
||||
**3/5 — Pertinent complement**
|
||||
|
||||
- Fills a real gap: no individual self-placement scale existed in the guide
|
||||
- Practitioner source (production engineering context), not a blogger opinion piece
|
||||
- Zero empirical data behind the levels — one engineer's taxonomy
|
||||
- Levels 0-2 are noise for the guide's audience (already using Claude Code)
|
||||
- ThoughtWorks occupies the "maturity model" reference slot; Martignole's upper levels are more Claude Code-specific
|
||||
|
||||
## Integration
|
||||
|
||||
Adapted into `guide/roles/learning-with-ai.md` as a "Where Are You on the Agent Adoption Curve?" section (inserted before the 30-Day Progression Plan). The level descriptions were extended (6 levels vs 5 in the original) and diagnostic questions added. Attribution and source link included.
|
||||
|
||||
## Fact-Check Notes
|
||||
|
||||
- Author identity and role: confirmed via LinkedIn
|
||||
- "~5% manual coding at Level 3+": practitioner estimate, not empirical — used as illustrative signal
|
||||
- "Free Claude Code Architect certification by Anthropic": not verifiable, not reproduced in the guide
|
||||
- Article date 2026-03-17: confirmed from URL
|
||||
|
||||
## Decision
|
||||
|
||||
**Integrated** — adapted content added to `learning-with-ai.md`. Not a verbatim reproduction; the framework was restructured and extended for the guide's English-speaking, already-technical audience.
|
||||
Loading…
Add table
Add a link
Reference in a new issue