From 257f2ff65dd45cb20bb91a2efa043f5c35077f84 Mon Sep 17 00:00:00 2001 From: Florian BRUNIAUX Date: Sun, 25 Jan 2026 18:47:29 +0100 Subject: [PATCH] docs: add search tools guides and ast-grep patterns New documentation: - guide/search-tools-cheatsheet.md: Quick reference for Grep/Glob/Read tools - guide/workflows/README.md: Workflows directory index - examples/skills/ast-grep-patterns.md: AST-based code search patterns Updated: - guide/architecture.md: MCP architecture visual diagram section Co-Authored-By: Claude --- examples/skills/ast-grep-patterns.md | 395 +++++++++++++++++++++++++++ guide/architecture.md | 46 +++- guide/search-tools-cheatsheet.md | 113 ++++++++ guide/workflows/README.md | 122 +++++++++ 4 files changed, 675 insertions(+), 1 deletion(-) create mode 100644 examples/skills/ast-grep-patterns.md create mode 100644 guide/search-tools-cheatsheet.md create mode 100644 guide/workflows/README.md diff --git a/examples/skills/ast-grep-patterns.md b/examples/skills/ast-grep-patterns.md new file mode 100644 index 0000000..02e5abb --- /dev/null +++ b/examples/skills/ast-grep-patterns.md @@ -0,0 +1,395 @@ +# ast-grep Patterns for Claude Code + +> **Skill**: Teach Claude when and how to use ast-grep for structural code searches + +## Installation + +```bash +# Install ast-grep skill +npx skills add ast-grep/agent-skill + +# Or via plugin marketplace +/plugin marketplace add +``` + +## When to Suggest ast-grep + +Suggest ast-grep to the user when: + +| Scenario | Signal Words | Why ast-grep? | +|----------|--------------|---------------| +| **Large refactoring** | "rename all calls to...", "change API signature", ">50k lines" | Structural precision needed | +| **Framework migration** | "React 17→18", "Vue 2→3", "upgrade dependencies" | AST-aware transformations | +| **Pattern detection** | "find functions without...", "locate unused...", "identify anti-patterns" | Structural rules | +| **Codebase analysis** | "which components depend on...", "find tightly coupled..." | Dependency graphs | + +**Don't suggest for**: +- Simple string searches (function names, imports) → Use Grep +- Small projects (<10k lines) → Grep is sufficient +- One-off searches → Grep is faster +- Semantic searches → Use Serena MCP or grepai + +## Decision Tree + +``` +User request analysis: +├─ "Find string/text" → Grep (native) +├─ "Find by meaning" → Serena MCP or grepai +├─ "Find by structure" → ast-grep (plugin) +└─ Mixed requirements → Start with Grep, escalate if needed +``` + +## Common Patterns + +### 1. Async Functions Without Error Handling + +**Use case**: Find async functions missing try/catch blocks + +```yaml +# ast-grep rule +rule: + pattern: | + async function $FUNC($$$PARAMS) { + $$$BODY + } + not: + has: + pattern: try { $$$TRY } catch +``` + +**When to use**: Security audits, production readiness checks + +### 2. React Components with Specific Hooks + +**Use case**: Find all components using `useEffect` without cleanup + +```yaml +rule: + pattern: | + useEffect(() => { + $$$BODY + }) + not: + has: + pattern: return () => { $$$CLEANUP } +``` + +**When to use**: Memory leak detection, React best practices audit + +### 3. Functions Exceeding Parameter Threshold + +**Use case**: Find functions with >5 parameters (complexity smell) + +```yaml +rule: + pattern: function $NAME($P1, $P2, $P3, $P4, $P5, $P6, $$$REST) { $$$BODY } +``` + +**When to use**: Code quality improvement, refactoring candidates + +### 4. Console.log in Production Code + +**Use case**: Remove debug logging from production files + +```yaml +rule: + pattern: console.log($$$ARGS) + inside: + pattern: | + class $CLASS { + $$$METHODS + } +``` + +**When to use**: Production cleanup, pre-release audits + +### 5. Unused React Props + +**Use case**: Detect props passed but never used + +```yaml +rule: + pattern: | + function $COMP({ $PROP, $$$OTHER }) { + $$$BODY + } + not: + has: + pattern: $PROP + inside: $$$BODY +``` + +**When to use**: Dead code elimination, performance optimization + +### 6. Deprecated API Usage + +**Use case**: Find usage of old API methods + +```yaml +rule: + any: + - pattern: React.Component + - pattern: componentWillMount + - pattern: componentWillReceiveProps +``` + +**When to use**: Framework migrations, deprecation cleanup + +### 7. SQL Injection Risk Patterns + +**Use case**: Find potential SQL injection vulnerabilities + +```yaml +rule: + pattern: | + db.query($TEMPLATE_LITERAL) + where: + $TEMPLATE_LITERAL: + kind: template_string +``` + +**When to use**: Security audits, vulnerability scanning + +### 8. Missing TypeScript Return Types + +**Use case**: Enforce explicit return types + +```yaml +rule: + pattern: | + function $NAME($$$PARAMS) { + $$$BODY + } + not: + has: + pattern: ': $TYPE' +``` + +**When to use**: TypeScript best practices, type safety improvements + +### 9. Large Switch Statements (Refactoring Candidates) + +**Use case**: Find switch statements with >10 cases + +```yaml +rule: + pattern: | + switch ($EXPR) { + $C1: $$$B1 + $C2: $$$B2 + $C3: $$$B3 + $C4: $$$B4 + $C5: $$$B5 + $C6: $$$B6 + $C7: $$$B7 + $C8: $$$B8 + $C9: $$$B9 + $C10: $$$B10 + $C11: $$$B11 + } +``` + +**When to use**: Complexity reduction, polymorphism refactoring + +### 10. Empty Catch Blocks (Swallowed Errors) + +**Use case**: Find error handling that silently fails + +```yaml +rule: + pattern: | + try { + $$$TRY + } catch ($ERR) { + // empty or only comment + } +``` + +**When to use**: Debugging mysterious failures, error handling audit + +## Setup Complexity vs. Value + +| Codebase Size | Setup Worth It? | Alternative | +|---------------|-----------------|-------------| +| <10k lines | ❌ No | Use Grep | +| 10k-50k lines | ⚠️ Maybe | Start with Grep, escalate if needed | +| 50k-200k lines | ✅ Yes | ast-grep for structural, Grep for text | +| >200k lines | ✅ Definitely | ast-grep + Serena MCP combo | + +## Troubleshooting + +### ast-grep Not Found + +```bash +# Verify installation +npx ast-grep --version + +# Reinstall skill +npx skills add ast-grep/agent-skill --force +``` + +### Claude Not Using ast-grep + +**Problem**: Claude uses Grep instead of ast-grep + +**Solution**: Be explicit in your request +- ❌ "Find async functions" +- ✅ "Use ast-grep to find async functions" + +### Performance Issues + +**Problem**: ast-grep slow on large codebase + +**Solutions**: +1. Narrow search scope: `ast-grep --path src/components/` +2. Use file filters: `ast-grep --lang tsx` +3. Cache results for iterative refinement + +### Pattern Not Matching + +**Problem**: ast-grep pattern doesn't match expected code + +**Debug steps**: +1. Test pattern in isolation: `ast-grep -p 'your-pattern' file.js` +2. Check AST structure: `ast-grep --debug-query` +3. Simplify pattern incrementally +4. Verify language syntax (JS vs TS vs JSX) + +## Integration Examples + +### Workflow: Pre-Commit Hook with ast-grep + +```bash +#!/bin/bash +# .git/hooks/pre-commit + +# Check for console.log in staged files +if ast-grep -p 'console.log($$$)' $(git diff --cached --name-only); then + echo "❌ Found console.log statements" + exit 1 +fi +``` + +### Workflow: Migration Script + +```bash +#!/bin/bash +# Migrate React class components to hooks + +# Find all class components +ast-grep -p 'class $C extends React.Component' --json > components.json + +# Process each component +jq -r '.[] | .file' components.json | while read file; do + echo "Migrating: $file" + # ... transformation logic +done +``` + +### Workflow: Security Audit + +```bash +#!/bin/bash +# security-audit.sh + +echo "=== Security Audit ===" + +# SQL injection risks +ast-grep -p 'db.query(`${$VAR}`)' --lang ts + +# XSS risks +ast-grep -p 'innerHTML = $VAR' --lang js + +# Hardcoded secrets +ast-grep -p 'password: "$PASSWORD"' --lang ts +``` + +## Claude Prompt Templates + +### Template 1: Large Refactoring + +``` +I need to refactor [FEATURE] across our codebase (~[SIZE] lines). + +Use ast-grep to: +1. Find all instances of [OLD_PATTERN] +2. Identify which files will be affected +3. Suggest transformation strategy +4. Create a phased migration plan + +Start with analysis only, wait for my approval before making changes. +``` + +### Template 2: Framework Migration + +``` +We're migrating from [OLD_FRAMEWORK v1] to [NEW_FRAMEWORK v2]. + +Use ast-grep to: +1. Find all deprecated API usage +2. Map to new API equivalents +3. Estimate migration effort (files affected) +4. Identify high-risk changes + +Provide a dependency graph showing migration order. +``` + +### Template 3: Code Quality Audit + +``` +Run a code quality audit on [DIRECTORY] using ast-grep. + +Focus on: +- Functions with >5 parameters +- Async functions without error handling +- Empty catch blocks +- Unused function parameters + +Rank issues by severity and provide refactoring suggestions. +``` + +## Advanced: Combining ast-grep with Other Tools + +### ast-grep + Serena MCP + +```bash +# 1. ast-grep finds structural patterns +ast-grep -p 'async function $F' --json > async-funcs.json + +# 2. Serena finds symbols and dependencies +claude mcp call serena find_symbol --name "authenticate" + +# 3. Combine insights for full context +# ast-grep: "This is an async function" +# Serena: "Called by 12 other functions" +``` + +### ast-grep + grepai + +```bash +# 1. grepai for semantic search +# "Find authentication-related code" + +# 2. ast-grep for structural refinement +# "Among those results, which are async without error handling?" +``` + +## Best Practices + +1. **Start Simple**: Begin with Grep, escalate to ast-grep when needed +2. **Test Patterns**: Verify on small files before running on entire codebase +3. **Document Patterns**: Save successful patterns as reusable rules +4. **Explicit Requests**: Always tell Claude explicitly when to use ast-grep +5. **Combine Tools**: Use ast-grep for structure, Grep for text, Serena for symbols + +## Resources + +- [ast-grep Documentation](https://ast-grep.github.io/) +- [Pattern Playground](https://ast-grep.github.io/playground.html) +- [ast-grep GitHub](https://github.com/ast-grep/ast-grep) +- [Claude Skill](https://github.com/ast-grep/claude-skill) + +--- + +**Last updated**: January 2026 +**Compatible with**: Claude Code 2.1.7+ diff --git a/guide/architecture.md b/guide/architecture.md index aef7fdd..cf44bd5 100644 --- a/guide/architecture.md +++ b/guide/architecture.md @@ -30,6 +30,10 @@ Each claim is marked with its confidence level. **Always prefer official documen 2. **Eight Core Tools**: Bash (universal adapter), Read, Edit, Write, Grep, Glob, Task (sub-agents), TodoWrite. That's the entire arsenal. + **Search Strategy Evolution**: Early Claude Code versions experimented with RAG using Voyage embeddings for semantic code search. Anthropic switched to grep-based (ripgrep) agentic search after internal benchmarks showed superior performance with lower operational complexity — no index sync required, no security liabilities from external embedding providers. This "Search, Don't Index" philosophy trades latency/tokens for simplicity/security. Community plugins (ast-grep for AST patterns) and MCP servers (Serena for symbols, grepai for RAG) available for specialized needs. + + *Source*: [Latent Space podcast](https://www.latent.space/p/claude-code) (May 2025), ast-grep documentation + 3. **200K Token Budget**: Context window shared between system prompt, history, tool results, and response buffer. Auto-compacts at ~75-92% capacity. 4. **Sub-agents = Isolation**: The `Task` tool spawns sub-agents with their own context. They cannot spawn more sub-agents (depth=1). Only their summary returns. @@ -165,7 +169,7 @@ Claude Code has exactly 8 core tools: | `Read` | Read file contents | Max 2000 lines, handles truncation | High for large files | | `Edit` | Modify existing files | Diff-based, requires exact match | Medium | | `Write` | Create/overwrite files | Must read first if file exists | Medium | -| `Grep` | Search file contents | Ripgrep-based, replaces RAG | Low | +| `Grep` | Search file contents | Ripgrep-based (regex), replaced RAG/embedding approach. For structural code search (AST-based), see ast-grep plugin. Trade-off: Grep (fast, simple) vs ast-grep (precise, setup required) vs Serena MCP (semantic, symbol-aware) | Low | | `Glob` | Find files by pattern | Path matching, sorted by mtime | Low | | `Task` | Spawn sub-agents | Isolated context, depth=1 limit | High (new context) | | `TodoWrite` | Track progress | Structured task management | Low | @@ -203,6 +207,46 @@ Claude decides which tool to use based on the task. There's no hardcoded routing └─────────────────────────────────────────────────────┘ ``` +### Extended Tool Ecosystem + +Beyond the 8 core tools, Claude Code can leverage: + +**MCP Servers** (Model Context Protocol): +- **Serena**: Symbol-aware code navigation + session memory +- **grepai**: Semantic search + call graph analysis (Ollama-based) +- **Context7**: Official library documentation lookup +- **Sequential**: Structured multi-step reasoning +- **Playwright**: Browser automation and E2E testing + +**Community Plugins**: +- **ast-grep**: AST-based structural code search (explicit invocation) + +### Search Tool Selection Matrix + +Claude Code offers multiple ways to search code, each with specific strengths: + +| Search Need | Native Tool | MCP/Plugin Alternative | When to Escalate | +|-------------|-------------|----------------------|------------------| +| Exact text | `Grep` (ripgrep) | - | Never (fastest) | +| Function name | `Grep` | Serena `find_symbol` | Multi-file refactoring | +| By meaning | - | grepai `search` | Don't know exact text | +| Call graph | - | grepai `trace_callers` | Dependency analysis | +| Structural pattern | - | ast-grep | Large migrations (>50k lines) | +| File structure | - | Serena `get_symbols_overview` | Need symbol context | + +**Performance Comparison**: + +| Tool | Speed | Setup | Use Case | +|------|-------|-------|----------| +| Grep (ripgrep) | ⚡ ~20ms | ✅ None | 90% of searches | +| Serena | ⚡ ~100ms | ⚠️ MCP | Refactoring, symbols | +| grepai | 🐢 ~500ms | ⚠️ Ollama + MCP | Semantic, call graph | +| ast-grep | 🕐 ~200ms | ⚠️ Plugin | AST patterns, migrations | + +**Decision principle**: Start with Grep (fastest), escalate to specialized tools only when needed. + +> **📖 Deep Dive**: See [Search Tools Mastery](../workflows/search-tools-mastery.md) for comprehensive workflows combining all search tools. + --- ## 3. Context Management Internals diff --git a/guide/search-tools-cheatsheet.md b/guide/search-tools-cheatsheet.md new file mode 100644 index 0000000..73bc909 --- /dev/null +++ b/guide/search-tools-cheatsheet.md @@ -0,0 +1,113 @@ +# Search Tools Cheatsheet (1-Page Reference) + +> **Quick reference for choosing between rg, grepai, Serena & ast-grep** + +--- + +## ⚡ Quick Decision (5 Seconds) + +``` +Know exact text? → rg +Know exact name? → rg or Serena +Know concept? → grepai +Know structure? → ast-grep +Need dependencies? → grepai trace +``` + +--- + +## 📊 Speed vs Intelligence + +``` +Fast ←─────────────────────────────────→ Smart + +rg Serena ast-grep grepai +~20ms ~100ms ~200ms ~500ms + +Exact Symbols Structure Meaning +``` + +--- + +## 🎯 Use Cases (Choose the Right Tool) + +| Task | Tool | Command | +|------|------|---------| +| "Find TODO comments" | `rg` | `rg "TODO"` | +| "Find login function" | `rg`/`Serena` | `rg "login"` or `serena find_symbol` | +| "Find auth code" | `grepai` | `grepai search "authentication"` | +| "Who calls login?" | `grepai` | `grepai trace callers "login"` | +| "Get file structure" | `Serena` | `serena get_symbols_overview` | +| "Async without try/catch" | `ast-grep` | `ast-grep "async function $F"` | +| "React class → hooks" | `ast-grep` | Migration pattern | +| "Remember decision" | `Serena` | `serena write_memory` | + +--- + +## 🔍 Tool Capabilities Matrix + +| Feature | rg | grepai | Serena | ast-grep | +|---------|:--:|:------:|:------:|:--------:| +| **Exact match** | ✅ | ❌ | ✅ | ✅ | +| **Semantic** | ❌ | ✅ | ❌ | ❌ | +| **Call graph** | ❌ | ✅ | ❌ | ❌ | +| **Symbols** | ❌ | ❌ | ✅ | ❌ | +| **AST patterns** | ❌ | ❌ | ❌ | ✅ | +| **Memory** | ❌ | ❌ | ✅ | ❌ | +| **Setup** | ✅ | ⚠️ | ⚠️ | ⚠️ | + +--- + +## 🚀 Combined Workflow (5 Steps) + +**Example: Refactor authentication** + +```bash +# 1. DISCOVER (grepai - semantic) +grepai search "authentication flow" + +# 2. STRUCTURE (Serena - symbols) +serena get_symbols_overview --file auth.service.ts + +# 3. DEPENDENCIES (grepai - call graph) +grepai trace callers "login" + +# 4. PATTERNS (ast-grep - structure) +ast-grep "async function login" + +# 5. VERIFY (rg - exact) +rg "validateSession" --type ts +``` + +--- + +## ⚠️ Common Mistakes + +| ❌ Wrong | ✅ Right | +|---------|---------| +| `grepai search "createSession"` | `rg "createSession"` | +| `rg "auth.*login.*session"` | `grepai search "auth flow"` | +| `rg + sed` for refactoring | `Serena find_symbol` | +| Refactor without checking callers | `grepai trace callers` first | + +--- + +## 📈 When to Escalate + +``` +Start: rg (90% of searches) + ↓ + Need meaning? → grepai + Need symbols? → Serena + Need AST? → ast-grep +``` + +--- + +## 🎓 Full Guide + +📖 [Search Tools Mastery](./workflows/search-tools-mastery.md) - Complete workflows, scenarios, benchmarks + +--- + +**Print this page for quick reference** | Last updated: January 2026 diff --git a/guide/workflows/README.md b/guide/workflows/README.md new file mode 100644 index 0000000..886a8eb --- /dev/null +++ b/guide/workflows/README.md @@ -0,0 +1,122 @@ +# Claude Code Workflows + +Step-by-step guides for common development patterns with Claude Code. + +--- + +## 🔍 Search & Discovery + +### [Search Tools Mastery](./search-tools-mastery.md) ⭐ NEW + +**Master the art of code search by combining rg, grepai, Serena & ast-grep** + +Learn when to use each tool, how to combine them for maximum efficiency, and real-world workflows including: +- Exploring unknown codebases +- Large-scale refactoring +- Security audits +- Framework migrations +- Performance optimization + +**Key Topics**: +- Quick decision matrix +- Complete feature comparison +- 5 combined workflows +- Performance benchmarks +- Common pitfalls +- Tool selection cheatsheet + +--- + +## 🎯 Development Workflows + +### [Plan-Driven Development](./plan-driven.md) + +Structure complex tasks with planning mode before execution. + +**When to use**: Multi-step features, architectural changes, uncertainty about approach + +### [TDD with Claude](./tdd-with-claude.md) + +Test-Driven Development workflow: write tests first, implement after. + +**When to use**: Critical functionality, regression prevention, API design + +### [Spec-First Development](./spec-first.md) + +Write specifications before code for better requirements clarity. + +**When to use**: Team collaboration, complex features, documentation-first projects + +### [Iterative Refinement](./iterative-refinement.md) + +Improve code through multiple refinement cycles. + +**When to use**: Quality improvements, performance optimization, code cleanup + +--- + +## 🎨 Design & Content + +### [Design to Code](./design-to-code.md) + +Convert design mockups (Figma, wireframes) into working code. + +**When to use**: Frontend development, UI implementation, design system work + +### [PDF Generation](./pdf-generation.md) + +Generate professional PDFs using Quarto/Typst with Claude Code. + +**When to use**: Reports, documentation, whitepapers, technical documents + +### [TTS Setup](./tts-setup.md) + +Configure Text-to-Speech for Claude Code responses (Agent Vibes integration). + +**When to use**: Audio feedback, accessibility, hands-free coding + +--- + +## 🔬 Code Exploration + +### [Exploration Workflow](./exploration-workflow.md) + +Systematically explore and understand unfamiliar codebases. + +**When to use**: New projects, legacy code, documentation gaps + +**Related**: See [Search Tools Mastery](./search-tools-mastery.md) for advanced multi-tool exploration strategies. + +--- + +## Quick Selection Guide + +| Your Situation | Recommended Workflow | +|----------------|---------------------| +| **New to codebase** | [Exploration Workflow](./exploration-workflow.md) + [Search Tools Mastery](./search-tools-mastery.md) | +| **Complex feature** | [Plan-Driven](./plan-driven.md) or [Spec-First](./spec-first.md) | +| **Need reliability** | [TDD with Claude](./tdd-with-claude.md) | +| **Large refactoring** | [Search Tools Mastery](./search-tools-mastery.md) | +| **UI implementation** | [Design to Code](./design-to-code.md) | +| **Code quality** | [Iterative Refinement](./iterative-refinement.md) | +| **Documentation** | [PDF Generation](./pdf-generation.md) | +| **Audio feedback** | [TTS Setup](./tts-setup.md) | + +--- + +## Contributing + +New workflow ideas? Open an issue or PR in the main repository. + +**Workflow Template Structure**: +1. Title & Purpose +2. When to Use +3. Prerequisites +4. Step-by-Step Guide +5. Real-World Examples +6. Common Pitfalls +7. Related Workflows + +--- + +**Last updated**: January 2026