docs: add Kairn memory MCP + resource evaluations + guide updates

- guide/ultimate-guide.md §10.2: Add Kairn (knowledge graph memory with biological decay)
  - Typed relationships (depends-on, resolves, causes), 18 MCP tools
  - Updated comparison table: Serena / grepai / doobidoo / Kairn
  - Added decision routing for long-term memory + causality tracking
- guide/ultimate-guide.md §5.1: Add real-world CLAUDE.md migration example (Avo, 600-line → 15 path-scoped files)
- guide/ai-ecosystem.md: Minor update
- machine-readable/reference.yaml: Add Kairn entries
- examples/config/mcp.json: Add Kairn MCP config
- docs/resource-evaluations/: Add 2 new evaluations (context-evaluator-packmind, kairn-memory-mcp)
- docs/resource-evaluations/agents-md-empirical-study: Add community reception section
- docs/resource-evaluations/2026-02-23-agentsview: Minor fix

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
This commit is contained in:
Florian BRUNIAUX 2026-02-25 17:39:20 +01:00
parent a6b0a0084a
commit 97f9167a61
8 changed files with 220 additions and 11 deletions

View file

@ -143,4 +143,4 @@ The challenge agent recommended **3/5** (vs initial 4/5) for the following reaso
- **Final score**: **3/5**
- **Action**: **Integrate**`observability.md` + mention in `third-party-tools.md`
- **Timing**: 2-4 weeks (wait for repo to reach ~200+ stars)
- **Confidence**: **Medium** — real gap confirmed, tool verified functional, but adoption too recent for high confidence. Wes McKinney's credibility (pandas) is a strong positive signal.
- **Confidence**: **Medium** — real gap confirmed, tool verified functional, but adoption too recent for high confidence. Wes McKinney's credibility (pandas) is a strong positive signal.

View file

@ -0,0 +1,85 @@
# Evaluation: Context-Evaluator (context-evaluator.ai)
**Date**: 2026-02-25
**Type**: URL + texte (LinkedIn post de Cédric Teyton, CTO Packmind)
**Source**: https://context-evaluator.ai/
---
## Resume du contenu
- **Outil open-source** (Apache-2.0) par Packmind (même équipe que coding-agents-matrix déjà dans le guide)
- **Scanner de fichiers CLAUDE.md, AGENTS.md, copilot-instructions.md** avec 17 évaluateurs spécialisés (13 détecteurs d'erreurs + 4 générateurs de suggestions)
- **Multi-agent**: Claude Code, Cursor, GitHub Copilot, OpenCode, OpenAI Codex
- **Dual interface**: CLI local + web UI (context-evaluator.ai)
- **Stack**: Bun, React, Tailwind CSS, TypeScript
- **GitHub**: PackmindHub/context-evaluator | 8 stars | 2 contributeurs | v0.3.0 (23 fev 2026)
---
## Score de pertinence: 3/5
**Justification**: Premier outil qui formalise des critères de qualité pour les fichiers CLAUDE.md avec des checks reproductibles. Valeur pédagogique des 17 critères même indépendamment de l'outil. Mais v0.3.0 avec 8 stars = side project expérimental, risque d'abandon réel.
---
## Comparatif
| Aspect | Context-Evaluator | Notre guide |
|--------|------------------|-------------|
| Audit CLAUDE.md | 17 évaluateurs automatisés (déterministe) | audit-prompt.md (LLM-dependent, subjectif) |
| Scope | Multi-agent (5 outils) | Claude Code spécifique |
| Maintenance context files | Détection erreurs + suggestions | Documentation des bonnes pratiques (section 3.1) |
| Maturité | v0.3.0, 8 stars, expérimental | 20K lignes, 175 templates, établi |
| Remediation | Auto-fix via AI intégré | Templates + checklists manuelles |
| Context drift detection | Oui (mismatch code/docs) | Mentionné en concept (iterative-refinement.md:253) |
---
## Recommandations
**Action**: Intégrer comme mention légère (pas de section dédiée)
1. **`guide/ai-ecosystem.md` ligne ~2064** (section Packmind Related Resources): Ajouter context-evaluator à côté de coding-agents-matrix. 3-5 lignes max.
2. **`machine-readable/reference.yaml`**: Ajouter entrée `context_evaluator` pointant vers ai-ecosystem.md
3. **Ne PAS créer de section dédiée** à 8 stars et v0.3.0
4. **Optionnel**: Lien depuis section maintenance CLAUDE.md vers ai-ecosystem pour les curieux
**Priorité**: Basse (intégrer quand opportun)
---
## Challenge (technical-writer)
- **Score ajusté**: 3/5 confirmé, justification retravaillée
- **Points manqués**: Les 17 critères concrets (le vrai trésor), le modèle de scoring, le dogfooding potentiel sur notre propre CLAUDE.md
- **Risques de non-intégration**: Quasi nuls. 8 stars = personne ne reprochera l'absence. Le risque inverse (contenu mort si outil abandonné) est plus réel.
- **Vraie valeur ajoutée**: Les critères formels d'évaluation, pas l'outil lui-même
- **Suggestion pertinente**: Faire tourner l'outil sur notre propre CLAUDE.md pour data factuelle
---
## Fact-Check
| Affirmation | Vérifiée | Source |
|-------------|----------|--------|
| Open-source Apache-2.0 | OK | GitHub repo confirmé |
| Par Packmind (Cédric Teyton) | OK | GitHub + ai-ecosystem.md:2012 déjà référencé |
| 17 évaluateurs (13+4) | OK | Page web + README GitHub |
| Supporte Claude Code, Cursor, Copilot, OpenCode, Codex | OK | Page web |
| v0.3.0, release 23 fev 2026 | OK | GitHub releases |
| 8 stars, 2 contributeurs | OK | GitHub repo (au 25 fev 2026) |
| Gratuit ($0) | OK | Page web |
| "An AI coding agent is only as smart as the last time your context was reviewed" | Claim marketing, non vérifiable | N/A |
**Perplexity**: Aucun résultat spécifique trouvé sur context-evaluator.ai. Outil trop récent/niche pour avoir de la couverture presse.
**Corrections**: Aucune hallucination détectée. Le texte LinkedIn est fidèle aux faits du repo.
---
## Decision finale
- **Score final**: 3/5
- **Action**: Intégrer comme mention dans ai-ecosystem.md section Packmind + entrée reference.yaml
- **Confiance**: Haute (facts vérifiés, repo existe, même auteur que ressource déjà intégrée)

View file

@ -0,0 +1,47 @@
# Evaluation: Kairn — Knowledge Graph Memory MCP
**Date**: 2026-02-25
**Evaluator**: Claude (eval-resource skill)
**Score**: 4/5 — Intégré
---
## Sources évaluées
1. **LinkedIn post** — Robin Lorenz, "Context Engineering for Claude Code", 24 fév. 2026
- Score: 2/5 — Watch list (claims invérifiables, pas de code source)
2. **GitHub repo** — [kairn-ai/kairn](https://github.com/kairn-ai/kairn) (MIT, Python 100%)
- Score: 4/5 — **Intégré**
---
## Résumé Kairn
MCP server Python offrant une mémoire organisée en knowledge graph avec 18 outils.
**Différenciateurs clés vs Serena/doobidoo**:
- Typed relationships (`depends-on`, `resolves`, `causes`)
- Biological decay model : solutions ~200j, workarounds ~50j (auto-pruning)
- Full-text search + confidence routing
- Cross-session et cross-IDE
---
## Décision d'intégration
| Élément modifié | Description |
|-----------------|-------------|
| `guide/ultimate-guide.md` ~10233 | Nouvelle section "Kairn: Knowledge Graph Memory with Biological Decay" |
| `guide/ultimate-guide.md` | Comparison Matrix + "When to use" table — colonne Kairn ajoutée |
| `guide/ultimate-guide.md` | Doobidoo limitation "No expiration" → pointe vers Kairn |
| `examples/config/mcp.json` | Entrée `kairn` ajoutée |
---
## Watch list — LinkedIn Lorenz
**Score: 2/5** — Réévaluer si un repo/article technique accompagne le post.
Raison du rejet : post LinkedIn sans code source, stats non vérifiables (93% réduction tokens), Perplexity n'a trouvé aucune trace indépendante. La philosophie "infrastructure outlives prompts" est déjà couverte dans `guide/methodologies.md`.
**Réévaluation trigger** : publication d'un repo GitHub accompagnant le post.

View file

@ -127,4 +127,16 @@ Ajouter une mise en garde :
---
---
## 📣 Réception communautaire
La communauté a simplifié les résultats en "delete your CLAUDE.md" (cf. [Charly Wargnier, LinkedIn](https://www.linkedin.com/posts/charlywargnier_everyone-is-screaming-delete-your-claudemd-activity-7431988275193622528-pfBW)).
**Nuance importante à intégrer dans le guide** : la commande `/init` génère un context file LLM-generated → c'est ce type de fichier qui dégrade les performances (-3%). Les fichiers écrits manuellement restent bénéfiques (+4%).
D'autres posts (ex: [Daniel Vikulov, LinkedIn](https://www.linkedin.com/)) paraphrasent fidèlement l'étude sans valeur ajoutée — ne pas les citer comme sources indépendantes.
---
*Évaluation effectuée le 2026-02-19 | Méthode: WebFetch + Perplexity + grepai_search + agent challenge*

View file

@ -38,6 +38,11 @@
"command": "memory",
"args": ["server"],
"description": "Semantic memory with cross-session search (complements Serena)"
},
"kairn": {
"command": "python",
"args": ["-m", "kairn", "serve"],
"description": "Knowledge graph memory with biological decay — auto-expires stale info (complements doobidoo/Serena)"
}
}
}

View file

@ -2063,6 +2063,7 @@ An **interactive comparison matrix** of 23 AI coding agents across 11 technical
- [Packmind](https://packmind.com): Context engineering & governance for AI coding agents
- [Packmind OSS](https://github.com/PackmindHub/packmind): Framework for versioning AI coding context
- [Context-Evaluator](https://context-evaluator.ai) ([GitHub](https://github.com/PackmindHub/context-evaluator)): Open-source scanner for CLAUDE.md / AGENTS.md / copilot-instructions.md — 17 evaluators (13 error detectors + 4 suggestion generators), supports Claude Code, Cursor, Copilot, OpenCode, Codex. CLI + web UI. Apache-2.0, v0.3.0 (Feb 2026).
- [Claude Code Templates](https://github.com/davila7/claude-code-templates): 200+ templates for Claude Code (17k⭐)
- [Awesome Claude Code](https://github.com/hesreallyhim/awesome-claude-code): Curated tool library

View file

@ -5171,7 +5171,7 @@ These rules only apply when working with API files:
- Include rate limiting middleware
```
This enables progressive context loading—rules only appear when Claude works with matching files.
This enables progressive context loading—rules only appear when Claude works with matching files. Real-world example: Avo migrated a 600-line CLAUDE.md to ~15 path-scoped files, reporting sharper responses and easier maintenance across domains. ([Björn Jóhannsson](https://www.linkedin.com/posts/bj%C3%B6rn-j%C3%B3hannsson-72435083_your-claudemd-is-eating-your-context-window-activity-7431750526729338881-ODSs))
**How matching works**:
- Patterns use glob syntax (same as `.gitignore`)
@ -10231,6 +10231,50 @@ Device C ──┘
> **Source**: [doobidoo/mcp-memory-service GitHub](https://github.com/doobidoo/mcp-memory-service) (791 stars, v10.0.2)
### Kairn: Knowledge Graph Memory with Biological Decay
> **⚠️ Status: Under Testing** - Evaluated Feb 2026. MIT licensed, Python 100%. Feedback welcome!
**Purpose**: Long-term project memory organized as a knowledge graph with automatic decay — stale information expires on its own, preventing context pollution.
**Key differentiators vs doobidoo/Serena**:
- **Typed relationships**: `depends-on`, `resolves`, `causes` — captures causality, not just content
- **Biological decay model**: solutions persist ~200 days, workarounds ~50 days — auto-pruning without `delete_memory` calls
- **18 MCP tools**: graph ops, project tracking, experience management, intelligence layer (full-text search, confidence routing, cross-workspace patterns)
| Feature | Serena | doobidoo | Kairn |
|---------|--------|----------|-------|
| Storage model | Key-value | Semantic embeddings | Knowledge graph |
| Memory decay / auto-expiry | No | No | Yes (biological) |
| Typed relationships | No | Tags only | depends-on / resolves / causes |
| Full-text search | No | Yes | Yes |
| Auto-pruning stale info | No | No | Yes |
**When Kairn makes sense**:
- Long-running projects where workarounds from months ago become noise
- When causality matters: "this breaks *because* of that", "this fix *resolves* that bug"
- Teams wanting automatic knowledge hygiene without manual cleanup
**MCP Config**:
```json
"kairn": {
"command": "python",
"args": ["-m", "kairn", "serve"],
"description": "Knowledge graph memory with biological decay"
}
```
**Install**:
```bash
pip install kairn
# or from source:
git clone https://github.com/kairn-ai/kairn && cd kairn && pip install -e .
```
> **Source**: [kairn-ai/kairn GitHub](https://github.com/kairn-ai/kairn) (MIT, Python 100%)
### MCP Memory Stack: Complementarity Patterns
> **⚠️ Experimental** - These patterns combine multiple MCP servers. Test in your workflow before relying on them.
@ -10257,14 +10301,15 @@ Device C ──┘
**Comparison Matrix**:
| Capability | Serena | grepai | doobidoo |
|------------|--------|--------|----------|
| Cross-session memory | Key-value | No | Semantic |
| Cross-IDE memory | No | No | Yes |
| Cross-device sync | No | No | Yes (Cloudflare) |
| Knowledge Graph | No | Call graph | Decision graph |
| Fuzzy search | No | Code | Memory |
| Tags/categories | No | No | Yes |
| Capability | Serena | grepai | doobidoo | Kairn |
|------------|--------|--------|----------|-------|
| Cross-session memory | Key-value | No | Semantic | Knowledge graph |
| Cross-IDE memory | No | No | Yes | Yes |
| Cross-device sync | No | No | Yes (Cloudflare) | No |
| Knowledge Graph | No | Call graph | Decision graph | Typed relationships |
| Fuzzy search | No | Code | Memory | Full-text + semantic |
| Tags/categories | No | No | Yes | Yes |
| Memory decay / auto-expiry | No | No | No | Yes (biological) |
**Usage Patterns**:
@ -10336,6 +10381,8 @@ retrieve_memory("work in progress?")
| "Share across devices" | doobidoo + Cloudflare | Cloud sync |
| "Code symbol location" | Serena `find_symbol()` | Code indexation |
| "Code by intent" | grepai `search()` | Semantic code search |
| "Long-term project memory, auto-expiry" | Kairn | Biological decay model |
| "Why did X break / what resolved Y?" | Kairn | Typed relationships (resolves, causes) |
**Current Limitations** (doobidoo):
@ -10344,7 +10391,7 @@ retrieve_memory("work in progress?")
| No versioning | Can't see decision history | Include dates in content |
| No permissions | Anyone can modify | Use separate DBs per team |
| No source linking | No link to file/line | Include file refs in content |
| No expiration | Stale memories persist | Manual cleanup with `delete_memory` |
| No expiration | Stale memories persist | Manual cleanup with `delete_memory` OR use Kairn (auto-decay) |
| No git integration | No branch-aware memory | Tag with branch name |
</details>

View file

@ -1269,6 +1269,18 @@ ecosystem:
agents_snapshot: 23 # As of 2026-01-19 (evolves over time)
positioning: "Discovery tool - Use Matrix to find/compare agents, use this guide to master Claude Code"
note: "External resource - verify data freshness as agents/criteria evolve"
context_evaluator:
url: "context-evaluator.ai"
github: "github.com/PackmindHub/context-evaluator"
maintainer: "Packmind (Cédric Teyton)"
license: "Apache-2.0"
focus: "Scanner for CLAUDE.md/AGENTS.md/copilot-instructions.md — 17 evaluators (13 error detectors + 4 suggestion generators)"
updated: "2026-02-23"
tech_stack: "Bun, React, Tailwind CSS, TypeScript"
supports: ["Claude Code", "Cursor", "GitHub Copilot", "OpenCode", "OpenAI Codex"]
interface: ["CLI", "Web UI"]
guide_section: "ai-ecosystem.md#related-resources"
note: "v0.3.0, experimental (8 stars as of 2026-02-25) — mention only, no dedicated section"
practitioner_insights:
dave_van_veen:
url: "davevanveen.com/blog/agentic_coding/"