docs: add Kairn memory MCP + resource evaluations + guide updates
- guide/ultimate-guide.md §10.2: Add Kairn (knowledge graph memory with biological decay) - Typed relationships (depends-on, resolves, causes), 18 MCP tools - Updated comparison table: Serena / grepai / doobidoo / Kairn - Added decision routing for long-term memory + causality tracking - guide/ultimate-guide.md §5.1: Add real-world CLAUDE.md migration example (Avo, 600-line → 15 path-scoped files) - guide/ai-ecosystem.md: Minor update - machine-readable/reference.yaml: Add Kairn entries - examples/config/mcp.json: Add Kairn MCP config - docs/resource-evaluations/: Add 2 new evaluations (context-evaluator-packmind, kairn-memory-mcp) - docs/resource-evaluations/agents-md-empirical-study: Add community reception section - docs/resource-evaluations/2026-02-23-agentsview: Minor fix Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
This commit is contained in:
parent
a6b0a0084a
commit
97f9167a61
8 changed files with 220 additions and 11 deletions
|
|
@ -143,4 +143,4 @@ The challenge agent recommended **3/5** (vs initial 4/5) for the following reaso
|
|||
- **Final score**: **3/5**
|
||||
- **Action**: **Integrate** — `observability.md` + mention in `third-party-tools.md`
|
||||
- **Timing**: 2-4 weeks (wait for repo to reach ~200+ stars)
|
||||
- **Confidence**: **Medium** — real gap confirmed, tool verified functional, but adoption too recent for high confidence. Wes McKinney's credibility (pandas) is a strong positive signal.
|
||||
- **Confidence**: **Medium** — real gap confirmed, tool verified functional, but adoption too recent for high confidence. Wes McKinney's credibility (pandas) is a strong positive signal.
|
||||
|
|
@ -0,0 +1,85 @@
|
|||
# Evaluation: Context-Evaluator (context-evaluator.ai)
|
||||
|
||||
**Date**: 2026-02-25
|
||||
**Type**: URL + texte (LinkedIn post de Cédric Teyton, CTO Packmind)
|
||||
**Source**: https://context-evaluator.ai/
|
||||
|
||||
---
|
||||
|
||||
## Resume du contenu
|
||||
|
||||
- **Outil open-source** (Apache-2.0) par Packmind (même équipe que coding-agents-matrix déjà dans le guide)
|
||||
- **Scanner de fichiers CLAUDE.md, AGENTS.md, copilot-instructions.md** avec 17 évaluateurs spécialisés (13 détecteurs d'erreurs + 4 générateurs de suggestions)
|
||||
- **Multi-agent**: Claude Code, Cursor, GitHub Copilot, OpenCode, OpenAI Codex
|
||||
- **Dual interface**: CLI local + web UI (context-evaluator.ai)
|
||||
- **Stack**: Bun, React, Tailwind CSS, TypeScript
|
||||
- **GitHub**: PackmindHub/context-evaluator | 8 stars | 2 contributeurs | v0.3.0 (23 fev 2026)
|
||||
|
||||
---
|
||||
|
||||
## Score de pertinence: 3/5
|
||||
|
||||
**Justification**: Premier outil qui formalise des critères de qualité pour les fichiers CLAUDE.md avec des checks reproductibles. Valeur pédagogique des 17 critères même indépendamment de l'outil. Mais v0.3.0 avec 8 stars = side project expérimental, risque d'abandon réel.
|
||||
|
||||
---
|
||||
|
||||
## Comparatif
|
||||
|
||||
| Aspect | Context-Evaluator | Notre guide |
|
||||
|--------|------------------|-------------|
|
||||
| Audit CLAUDE.md | 17 évaluateurs automatisés (déterministe) | audit-prompt.md (LLM-dependent, subjectif) |
|
||||
| Scope | Multi-agent (5 outils) | Claude Code spécifique |
|
||||
| Maintenance context files | Détection erreurs + suggestions | Documentation des bonnes pratiques (section 3.1) |
|
||||
| Maturité | v0.3.0, 8 stars, expérimental | 20K lignes, 175 templates, établi |
|
||||
| Remediation | Auto-fix via AI intégré | Templates + checklists manuelles |
|
||||
| Context drift detection | Oui (mismatch code/docs) | Mentionné en concept (iterative-refinement.md:253) |
|
||||
|
||||
---
|
||||
|
||||
## Recommandations
|
||||
|
||||
**Action**: Intégrer comme mention légère (pas de section dédiée)
|
||||
|
||||
1. **`guide/ai-ecosystem.md` ligne ~2064** (section Packmind Related Resources): Ajouter context-evaluator à côté de coding-agents-matrix. 3-5 lignes max.
|
||||
2. **`machine-readable/reference.yaml`**: Ajouter entrée `context_evaluator` pointant vers ai-ecosystem.md
|
||||
3. **Ne PAS créer de section dédiée** à 8 stars et v0.3.0
|
||||
4. **Optionnel**: Lien depuis section maintenance CLAUDE.md vers ai-ecosystem pour les curieux
|
||||
|
||||
**Priorité**: Basse (intégrer quand opportun)
|
||||
|
||||
---
|
||||
|
||||
## Challenge (technical-writer)
|
||||
|
||||
- **Score ajusté**: 3/5 confirmé, justification retravaillée
|
||||
- **Points manqués**: Les 17 critères concrets (le vrai trésor), le modèle de scoring, le dogfooding potentiel sur notre propre CLAUDE.md
|
||||
- **Risques de non-intégration**: Quasi nuls. 8 stars = personne ne reprochera l'absence. Le risque inverse (contenu mort si outil abandonné) est plus réel.
|
||||
- **Vraie valeur ajoutée**: Les critères formels d'évaluation, pas l'outil lui-même
|
||||
- **Suggestion pertinente**: Faire tourner l'outil sur notre propre CLAUDE.md pour data factuelle
|
||||
|
||||
---
|
||||
|
||||
## Fact-Check
|
||||
|
||||
| Affirmation | Vérifiée | Source |
|
||||
|-------------|----------|--------|
|
||||
| Open-source Apache-2.0 | OK | GitHub repo confirmé |
|
||||
| Par Packmind (Cédric Teyton) | OK | GitHub + ai-ecosystem.md:2012 déjà référencé |
|
||||
| 17 évaluateurs (13+4) | OK | Page web + README GitHub |
|
||||
| Supporte Claude Code, Cursor, Copilot, OpenCode, Codex | OK | Page web |
|
||||
| v0.3.0, release 23 fev 2026 | OK | GitHub releases |
|
||||
| 8 stars, 2 contributeurs | OK | GitHub repo (au 25 fev 2026) |
|
||||
| Gratuit ($0) | OK | Page web |
|
||||
| "An AI coding agent is only as smart as the last time your context was reviewed" | Claim marketing, non vérifiable | N/A |
|
||||
|
||||
**Perplexity**: Aucun résultat spécifique trouvé sur context-evaluator.ai. Outil trop récent/niche pour avoir de la couverture presse.
|
||||
|
||||
**Corrections**: Aucune hallucination détectée. Le texte LinkedIn est fidèle aux faits du repo.
|
||||
|
||||
---
|
||||
|
||||
## Decision finale
|
||||
|
||||
- **Score final**: 3/5
|
||||
- **Action**: Intégrer comme mention dans ai-ecosystem.md section Packmind + entrée reference.yaml
|
||||
- **Confiance**: Haute (facts vérifiés, repo existe, même auteur que ressource déjà intégrée)
|
||||
47
docs/resource-evaluations/2026-02-25-kairn-memory-mcp.md
Normal file
47
docs/resource-evaluations/2026-02-25-kairn-memory-mcp.md
Normal file
|
|
@ -0,0 +1,47 @@
|
|||
# Evaluation: Kairn — Knowledge Graph Memory MCP
|
||||
|
||||
**Date**: 2026-02-25
|
||||
**Evaluator**: Claude (eval-resource skill)
|
||||
**Score**: 4/5 — Intégré
|
||||
|
||||
---
|
||||
|
||||
## Sources évaluées
|
||||
|
||||
1. **LinkedIn post** — Robin Lorenz, "Context Engineering for Claude Code", 24 fév. 2026
|
||||
- Score: 2/5 — Watch list (claims invérifiables, pas de code source)
|
||||
2. **GitHub repo** — [kairn-ai/kairn](https://github.com/kairn-ai/kairn) (MIT, Python 100%)
|
||||
- Score: 4/5 — **Intégré**
|
||||
|
||||
---
|
||||
|
||||
## Résumé Kairn
|
||||
|
||||
MCP server Python offrant une mémoire organisée en knowledge graph avec 18 outils.
|
||||
|
||||
**Différenciateurs clés vs Serena/doobidoo**:
|
||||
- Typed relationships (`depends-on`, `resolves`, `causes`)
|
||||
- Biological decay model : solutions ~200j, workarounds ~50j (auto-pruning)
|
||||
- Full-text search + confidence routing
|
||||
- Cross-session et cross-IDE
|
||||
|
||||
---
|
||||
|
||||
## Décision d'intégration
|
||||
|
||||
| Élément modifié | Description |
|
||||
|-----------------|-------------|
|
||||
| `guide/ultimate-guide.md` ~10233 | Nouvelle section "Kairn: Knowledge Graph Memory with Biological Decay" |
|
||||
| `guide/ultimate-guide.md` | Comparison Matrix + "When to use" table — colonne Kairn ajoutée |
|
||||
| `guide/ultimate-guide.md` | Doobidoo limitation "No expiration" → pointe vers Kairn |
|
||||
| `examples/config/mcp.json` | Entrée `kairn` ajoutée |
|
||||
|
||||
---
|
||||
|
||||
## Watch list — LinkedIn Lorenz
|
||||
|
||||
**Score: 2/5** — Réévaluer si un repo/article technique accompagne le post.
|
||||
|
||||
Raison du rejet : post LinkedIn sans code source, stats non vérifiables (93% réduction tokens), Perplexity n'a trouvé aucune trace indépendante. La philosophie "infrastructure outlives prompts" est déjà couverte dans `guide/methodologies.md`.
|
||||
|
||||
**Réévaluation trigger** : publication d'un repo GitHub accompagnant le post.
|
||||
|
|
@ -127,4 +127,16 @@ Ajouter une mise en garde :
|
|||
|
||||
---
|
||||
|
||||
---
|
||||
|
||||
## 📣 Réception communautaire
|
||||
|
||||
La communauté a simplifié les résultats en "delete your CLAUDE.md" (cf. [Charly Wargnier, LinkedIn](https://www.linkedin.com/posts/charlywargnier_everyone-is-screaming-delete-your-claudemd-activity-7431988275193622528-pfBW)).
|
||||
|
||||
**Nuance importante à intégrer dans le guide** : la commande `/init` génère un context file LLM-generated → c'est ce type de fichier qui dégrade les performances (-3%). Les fichiers écrits manuellement restent bénéfiques (+4%).
|
||||
|
||||
D'autres posts (ex: [Daniel Vikulov, LinkedIn](https://www.linkedin.com/)) paraphrasent fidèlement l'étude sans valeur ajoutée — ne pas les citer comme sources indépendantes.
|
||||
|
||||
---
|
||||
|
||||
*Évaluation effectuée le 2026-02-19 | Méthode: WebFetch + Perplexity + grepai_search + agent challenge*
|
||||
|
|
@ -38,6 +38,11 @@
|
|||
"command": "memory",
|
||||
"args": ["server"],
|
||||
"description": "Semantic memory with cross-session search (complements Serena)"
|
||||
},
|
||||
"kairn": {
|
||||
"command": "python",
|
||||
"args": ["-m", "kairn", "serve"],
|
||||
"description": "Knowledge graph memory with biological decay — auto-expires stale info (complements doobidoo/Serena)"
|
||||
}
|
||||
}
|
||||
}
|
||||
|
|
|
|||
|
|
@ -2063,6 +2063,7 @@ An **interactive comparison matrix** of 23 AI coding agents across 11 technical
|
|||
|
||||
- [Packmind](https://packmind.com): Context engineering & governance for AI coding agents
|
||||
- [Packmind OSS](https://github.com/PackmindHub/packmind): Framework for versioning AI coding context
|
||||
- [Context-Evaluator](https://context-evaluator.ai) ([GitHub](https://github.com/PackmindHub/context-evaluator)): Open-source scanner for CLAUDE.md / AGENTS.md / copilot-instructions.md — 17 evaluators (13 error detectors + 4 suggestion generators), supports Claude Code, Cursor, Copilot, OpenCode, Codex. CLI + web UI. Apache-2.0, v0.3.0 (Feb 2026).
|
||||
- [Claude Code Templates](https://github.com/davila7/claude-code-templates): 200+ templates for Claude Code (17k⭐)
|
||||
- [Awesome Claude Code](https://github.com/hesreallyhim/awesome-claude-code): Curated tool library
|
||||
|
||||
|
|
|
|||
|
|
@ -5171,7 +5171,7 @@ These rules only apply when working with API files:
|
|||
- Include rate limiting middleware
|
||||
```
|
||||
|
||||
This enables progressive context loading—rules only appear when Claude works with matching files.
|
||||
This enables progressive context loading—rules only appear when Claude works with matching files. Real-world example: Avo migrated a 600-line CLAUDE.md to ~15 path-scoped files, reporting sharper responses and easier maintenance across domains. ([Björn Jóhannsson](https://www.linkedin.com/posts/bj%C3%B6rn-j%C3%B3hannsson-72435083_your-claudemd-is-eating-your-context-window-activity-7431750526729338881-ODSs))
|
||||
|
||||
**How matching works**:
|
||||
- Patterns use glob syntax (same as `.gitignore`)
|
||||
|
|
@ -10231,6 +10231,50 @@ Device C ──┘
|
|||
|
||||
> **Source**: [doobidoo/mcp-memory-service GitHub](https://github.com/doobidoo/mcp-memory-service) (791 stars, v10.0.2)
|
||||
|
||||
### Kairn: Knowledge Graph Memory with Biological Decay
|
||||
|
||||
> **⚠️ Status: Under Testing** - Evaluated Feb 2026. MIT licensed, Python 100%. Feedback welcome!
|
||||
|
||||
**Purpose**: Long-term project memory organized as a knowledge graph with automatic decay — stale information expires on its own, preventing context pollution.
|
||||
|
||||
**Key differentiators vs doobidoo/Serena**:
|
||||
- **Typed relationships**: `depends-on`, `resolves`, `causes` — captures causality, not just content
|
||||
- **Biological decay model**: solutions persist ~200 days, workarounds ~50 days — auto-pruning without `delete_memory` calls
|
||||
- **18 MCP tools**: graph ops, project tracking, experience management, intelligence layer (full-text search, confidence routing, cross-workspace patterns)
|
||||
|
||||
| Feature | Serena | doobidoo | Kairn |
|
||||
|---------|--------|----------|-------|
|
||||
| Storage model | Key-value | Semantic embeddings | Knowledge graph |
|
||||
| Memory decay / auto-expiry | No | No | Yes (biological) |
|
||||
| Typed relationships | No | Tags only | depends-on / resolves / causes |
|
||||
| Full-text search | No | Yes | Yes |
|
||||
| Auto-pruning stale info | No | No | Yes |
|
||||
|
||||
**When Kairn makes sense**:
|
||||
- Long-running projects where workarounds from months ago become noise
|
||||
- When causality matters: "this breaks *because* of that", "this fix *resolves* that bug"
|
||||
- Teams wanting automatic knowledge hygiene without manual cleanup
|
||||
|
||||
**MCP Config**:
|
||||
|
||||
```json
|
||||
"kairn": {
|
||||
"command": "python",
|
||||
"args": ["-m", "kairn", "serve"],
|
||||
"description": "Knowledge graph memory with biological decay"
|
||||
}
|
||||
```
|
||||
|
||||
**Install**:
|
||||
|
||||
```bash
|
||||
pip install kairn
|
||||
# or from source:
|
||||
git clone https://github.com/kairn-ai/kairn && cd kairn && pip install -e .
|
||||
```
|
||||
|
||||
> **Source**: [kairn-ai/kairn GitHub](https://github.com/kairn-ai/kairn) (MIT, Python 100%)
|
||||
|
||||
### MCP Memory Stack: Complementarity Patterns
|
||||
|
||||
> **⚠️ Experimental** - These patterns combine multiple MCP servers. Test in your workflow before relying on them.
|
||||
|
|
@ -10257,14 +10301,15 @@ Device C ──┘
|
|||
|
||||
**Comparison Matrix**:
|
||||
|
||||
| Capability | Serena | grepai | doobidoo |
|
||||
|------------|--------|--------|----------|
|
||||
| Cross-session memory | Key-value | No | Semantic |
|
||||
| Cross-IDE memory | No | No | Yes |
|
||||
| Cross-device sync | No | No | Yes (Cloudflare) |
|
||||
| Knowledge Graph | No | Call graph | Decision graph |
|
||||
| Fuzzy search | No | Code | Memory |
|
||||
| Tags/categories | No | No | Yes |
|
||||
| Capability | Serena | grepai | doobidoo | Kairn |
|
||||
|------------|--------|--------|----------|-------|
|
||||
| Cross-session memory | Key-value | No | Semantic | Knowledge graph |
|
||||
| Cross-IDE memory | No | No | Yes | Yes |
|
||||
| Cross-device sync | No | No | Yes (Cloudflare) | No |
|
||||
| Knowledge Graph | No | Call graph | Decision graph | Typed relationships |
|
||||
| Fuzzy search | No | Code | Memory | Full-text + semantic |
|
||||
| Tags/categories | No | No | Yes | Yes |
|
||||
| Memory decay / auto-expiry | No | No | No | Yes (biological) |
|
||||
|
||||
**Usage Patterns**:
|
||||
|
||||
|
|
@ -10336,6 +10381,8 @@ retrieve_memory("work in progress?")
|
|||
| "Share across devices" | doobidoo + Cloudflare | Cloud sync |
|
||||
| "Code symbol location" | Serena `find_symbol()` | Code indexation |
|
||||
| "Code by intent" | grepai `search()` | Semantic code search |
|
||||
| "Long-term project memory, auto-expiry" | Kairn | Biological decay model |
|
||||
| "Why did X break / what resolved Y?" | Kairn | Typed relationships (resolves, causes) |
|
||||
|
||||
**Current Limitations** (doobidoo):
|
||||
|
||||
|
|
@ -10344,7 +10391,7 @@ retrieve_memory("work in progress?")
|
|||
| No versioning | Can't see decision history | Include dates in content |
|
||||
| No permissions | Anyone can modify | Use separate DBs per team |
|
||||
| No source linking | No link to file/line | Include file refs in content |
|
||||
| No expiration | Stale memories persist | Manual cleanup with `delete_memory` |
|
||||
| No expiration | Stale memories persist | Manual cleanup with `delete_memory` OR use Kairn (auto-decay) |
|
||||
| No git integration | No branch-aware memory | Tag with branch name |
|
||||
|
||||
</details>
|
||||
|
|
|
|||
|
|
@ -1269,6 +1269,18 @@ ecosystem:
|
|||
agents_snapshot: 23 # As of 2026-01-19 (evolves over time)
|
||||
positioning: "Discovery tool - Use Matrix to find/compare agents, use this guide to master Claude Code"
|
||||
note: "External resource - verify data freshness as agents/criteria evolve"
|
||||
context_evaluator:
|
||||
url: "context-evaluator.ai"
|
||||
github: "github.com/PackmindHub/context-evaluator"
|
||||
maintainer: "Packmind (Cédric Teyton)"
|
||||
license: "Apache-2.0"
|
||||
focus: "Scanner for CLAUDE.md/AGENTS.md/copilot-instructions.md — 17 evaluators (13 error detectors + 4 suggestion generators)"
|
||||
updated: "2026-02-23"
|
||||
tech_stack: "Bun, React, Tailwind CSS, TypeScript"
|
||||
supports: ["Claude Code", "Cursor", "GitHub Copilot", "OpenCode", "OpenAI Codex"]
|
||||
interface: ["CLI", "Web UI"]
|
||||
guide_section: "ai-ecosystem.md#related-resources"
|
||||
note: "v0.3.0, experimental (8 stars as of 2026-02-25) — mention only, no dedicated section"
|
||||
practitioner_insights:
|
||||
dave_van_veen:
|
||||
url: "davevanveen.com/blog/agentic_coding/"
|
||||
|
|
|
|||
Loading…
Add table
Add a link
Reference in a new issue