diff --git a/CHANGELOG.md b/CHANGELOG.md index 99aada5..184b8e0 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -6,7 +6,28 @@ The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/). ## [Unreleased] -## [3.25.0] - 2026-02-10 + + +### Added + +- **Resource Evaluation**: "AI Fatigue is Real" by Siddhant Khare (`docs/resource-evaluations/siddhant-khare-ai-fatigue.md`) + - Evaluated blog post on AI-induced exhaustion and productivity paradoxes + - Score: 3/5 (Pertinent — complément utile) + - 90% content overlap with existing `learning-with-ai.md`, but identified session time-boxing gap + - Technical-writer challenge downgraded from initial 4/5 to 3/5 + - Fact-check confirmed: 0 research citations (anecdotal only) vs guide's peer-reviewed RCTs + - Extracted: Time-boxing tactics (30 min limit, 3 attempts max), nondeterminism stress recognition + +### Changed + +- **Learning Guide Enhancement**: AI fatigue symptom recognition integrated into `guide/learning-with-ai.md` + - **Red Flags Checklist** (line 869): Added "Prolonged sessions without breaks" with time-boxing mitigation (30 min limit, max 3 attempts before manual implementation) + - **Productivity Reality** (line 115): Added paragraph on nondeterminism stress (identical prompts → varying outputs causes "AI fatigue") + - **UVAL Protocol** (line 247): Added "Step 2.5: Recognize Fatigue Signals" checkpoint (session duration, retry count, frustration assessment) + - **Total footprint**: ~200 words across 3 locations (minimal integration) + - **Rationale**: Addresses session-level time-boxing gap (distinct from existing weekly 70/30 split) + +## [3.25.0] - 2026-02-10 ### Added diff --git a/docs/resource-evaluations/siddhant-khare-ai-fatigue.md b/docs/resource-evaluations/siddhant-khare-ai-fatigue.md new file mode 100644 index 0000000..fa2880e --- /dev/null +++ b/docs/resource-evaluations/siddhant-khare-ai-fatigue.md @@ -0,0 +1,200 @@ +# Resource Evaluation: "AI Fatigue is Real and Nobody Talks About It" + +**Date:** 2026-02-10 +**Evaluator:** Claude Code (eval-resource skill) +**Status:** Integrated (minimal) + +--- + +## Resource Details + +| Field | Value | +|-------|-------| +| **Title** | AI Fatigue is Real and Nobody Talks About It | +| **Author** | Siddhant Khare | +| **Credentials** | Research Engineer @ Ona (formerly Gitpod), OpenFGA Core Maintainer (CNCF), KubeCon speaker | +| **Publication Date** | February 8, 2026 | +| **URL** | https://siddhantkhare.com/writing/ai-fatigue-is-real | +| **Type** | Blog post (anecdotal, no research citations) | +| **Read Time** | 16 minutes | + +--- + +## Summary + +Khare argues that AI tools create a productivity paradox: faster task completion doesn't reduce workload, it expands expectations. The article identifies five sources of AI-related exhaustion: + +1. **Productivity paradox**: Each task takes less time → you do MORE tasks, not fewer +2. **Creator → Reviewer shift**: Reviewing AI code is cognitively draining vs energizing creation +3. **Nondeterminism stress**: Identical prompts → varying outputs creates persistent anxiety +4. **FOMO treadmill**: Tool proliferation (CrewAI, AutoGen, LangGraph) forces constant evaluation +5. **Thinking atrophy**: Outsourcing initial problem-solving degrades reasoning abilities + +**Solutions proposed:** +- Time-boxing sessions (30 min limit, 3 attempts max) +- Separate thinking time from execution time (morning/afternoon split) +- Accept 70% quality threshold (vs perfectionism) +- Strategic tool adoption (not reactive) +- Targeted code review (critical areas only) + +--- + +## Evaluation Score: **3/5** (Pertinent — complément utile) + +### Scoring Breakdown + +| Criterion | Score | Justification | +|-----------|-------|---------------| +| **Content novelty** | 2/5 | 90% overlap with existing `learning-with-ai.md` content | +| **Claude Code specificity** | 2/5 | Generic AI tools discussion, not CLI-specific | +| **Evidence quality** | 2/5 | Blog post with anecdotal claims vs guide's peer-reviewed RCTs | +| **Actionability** | 3/5 | Vague recommendations vs guide's structured UVAL protocol | +| **Strategic value** | 4/5 | Mental health/sustainability angle underrepresented in guide | + +**Average:** 2.6/5 → **Rounded to 3/5** + +### Comparison to Guide Content + +| Aspect | Article (Khare) | Guide (Current) | +|--------|-----------------|-----------------| +| **Productivity paradox** | ✅ Described (anecdotal) | ✅ Documented with RCT studies ([learning-with-ai.md](../guide/learning-with-ai.md) lines 100-153) | +| **Review burden** | ✅ "Creator → Reviewer shift" | ✅ "Vibe Coding Trap" + Accept All pattern (lines 81-96) | +| **Skill atrophy** | ✅ "Thinking atrophy" | ✅ "Three Patterns" + unemployability trajectory (lines 159-205) | +| **Nondeterminism stress** | ➕ Explicit (output variance) | ⚠️ Implicit (UVAL "verify everything") | +| **FOMO treadmill** | ➕ Tool proliferation fatigue | ❌ Out of scope (mono-tool guide) | +| **Time-boxing sessions** | ➕ 30 min limit, 3 attempts max | ⚠️ Implicit (70/30 weekly split, not sessions) | +| **Mental health framing** | ➕ "Fatigue" as explicit problem | ❌ Framed as dependency risk | +| **Evidence base** | ❌ Anecdotes, 0 citations | ✅ RCT studies (Shen & Tamkin, METR) | +| **Quality standard** | ❌ "70% OK" (dangerous) | ✅ "Understand 100%" (UVAL protocol) | + +**Key gap identified:** Session-level time-boxing (30 min, 3 attempts) distinct from weekly strategic allocation (70/30 split). + +--- + +## Fact-Check Results + +| Claim | Verified | Source | Notes | +|-------|----------|--------|-------| +| Author = Research Engineer @ Ona | ✅ | Structured data | OpenFGA maintainer, KubeCon speaker confirmed | +| Publication date = Feb 8, 2026 | ✅ | Article metadata | Contemporary | +| "Shipped more code last quarter" | ⚠️ | Anecdotal | Not measurable, self-reported | +| "70-80% AI output quality" | ⚠️ | Anecdotal | No methodology provided | +| "5% improvement" (tool migrations) | ⚠️ | Anecdotal | Not sourced | +| Tools mentioned (CrewAI, AutoGen, etc.) | ✅ | Verifiable | All tools exist | +| Solutions (30 min, 3 attempts, 70% bar) | ✅ | Present | Recommendations are clear | +| **Research citations** | ❌ | **Absent** | **0 external sources, pure observation** | + +**Critical finding:** Article contains NO citations to research, unlike the guide's peer-reviewed RCT studies (Shen & Tamkin 2026, METR 2025, GitHub Copilot studies). + +--- + +## Technical-Writer Challenge Summary + +**Initial score:** 4/5 (overestimated) +**Challenged score:** 2/5 (technical-writer argued for downgrade) +**Final score:** 3/5 (compromise after fact-check) + +**Key arguments from technical-writer:** +- 90% content overlap (productivity paradox, review burden, skill atrophy already covered) +- Article is generic AI tools, not Claude Code-specific +- Blog post anecdotes vs guide's peer-reviewed studies weakens credibility +- "70% quality OK" contradicts guide's "understand 100%" UVAL protocol +- FOMO treadmill (tool-hopping) out of scope for mono-tool guide + +**Counterarguments for 3/5:** +- Nondeterminism stress (output variance) explicitly underaddressed +- Session time-boxing (30 min) distinct from weekly 70/30 split +- Explicit "AI fatigue" framing aids symptom recognition +- 3 attempts limit is actionable tactic currently missing + +**Risks of non-integration:** Minimal. Users experiencing AI fatigue will find root cause solutions in existing dependency patterns, UVAL protocol, and 70/30 split sections. + +--- + +## Integration Decision + +**Action:** Full integration (all 3 priorities, ~200 words total) + +**Locations:** `guide/learning-with-ai.md` (3 locations) + +### Priority 1: Red Flags Checklist (line 869) + +**What was added:** + +```markdown +| Prolonged sessions without breaks | **Session fatigue** — identical prompts yield varying outputs, causing anxiety | Time-box sessions: 30 min limit, max 3 attempts before manual implementation | +``` + +**Rationale:** Highest visibility diagnostic tool, most actionable tactic + +### Priority 2: Productivity Reality (line 115) + +**What was added:** + +```markdown +**AI-specific stress factor**: Nondeterministic outputs (identical prompts → varying results) create cognitive anxiety distinct from traditional debugging. This variability can trigger "AI fatigue" — mental exhaustion from unpredictable tool behavior that compounds over extended sessions. Mitigation: Time-box sessions (30 min max), limit retry attempts (3 max before reverting to manual implementation), and recognize when tool unpredictability signals a need for context reset (`/clear`) or manual problem-solving. +``` + +**Rationale:** Frames fatigue as productivity cost, addresses nondeterminism gap + +### Priority 3: UVAL Protocol (line 247) + +**What was added:** + +```markdown +#### Step 2.5: Recognize Fatigue Signals (30 sec) + +Before moving forward, pause and assess your cognitive state: + +- **Session duration**: Been working >30 min? → Take a 5-min break, consider `/clear` to reset context +- **Retry count**: Tried the same prompt 3+ times with inconsistent results? → Switch to manual implementation +- **Frustration level**: Feeling anxious about unpredictable AI responses? → This is "AI fatigue" (nondeterminism stress), not your fault — it's the tool's inherent variability + +This checkpoint prevents compounding exhaustion from extended sessions with diminishing returns. +``` + +**Rationale:** Builds proactive habit, integrates into existing methodology + +**Alternatives considered and rejected:** +- ❌ New "Managing AI Fatigue" section → 90% redundant with existing content +- ❌ "70% quality OK" recommendation → contradicts UVAL protocol +- ❌ FOMO treadmill discussion → out of scope for mono-tool guide +- ❌ Standalone integration of any single priority → complementary value when combined + +--- + +## Key Takeaways + +1. **Score justification:** 3/5 reflects moderate relevance due to high overlap with superior existing content (RCT studies vs anecdotes) + +2. **Integration approach:** Extract only novel tactics (time-boxing, 3 attempts) and insert minimally into existing diagnostic tool (Red Flags Checklist) + +3. **Evidence gap:** Article's lack of research citations (vs guide's peer-reviewed sources) justified minimal integration rather than prominent feature + +4. **Philosophical alignment:** Rejected "70% quality OK" recommendation to preserve guide's "understand 100%" learning standard + +5. **Scope discipline:** Rejected FOMO treadmill discussion (tool-hopping) as out of scope for Claude Code-specific guide + +--- + +## Metadata + +**Evaluation method:** eval-resource skill +**Tools used:** WebFetch (content extraction), Grep (gap analysis), Task (technical-writer challenge), WebFetch (fact-check) +**Integration status:** ✅ Completed (Priority 1 only) +**Commit reference:** (to be added when committed) + +--- + +## References + +**Article:** +- Khare, S. (2026, February 8). AI Fatigue is Real and Nobody Talks About It. Retrieved from https://siddhantkhare.com/writing/ai-fatigue-is-real + +**Guide sections referenced:** +- [Learning with AI](../guide/learning-with-ai.md) — Primary integration location +- [Adoption Approaches](../guide/adoption-approaches.md) — Considered but not used + +**Related evaluations:** +- [Beyond Vibe Coding](./beyond-vibe-coding.md) — Complementary perspective on AI-assisted development +- [Addy Osmani: 80% Problem](./024-addy-osmani-80-percent-problem.md) — Quality threshold discussion diff --git a/guide/learning-with-ai.md b/guide/learning-with-ai.md index b604949..71727d2 100644 --- a/guide/learning-with-ai.md +++ b/guide/learning-with-ai.md @@ -113,6 +113,8 @@ Most developers experience three distinct phases: **Critical nuance**: These gains are conditional. Studies show experienced developers (5+ years) see larger, sustained gains. Junior developers often see initial spikes followed by regression — because speed without understanding creates technical debt. A 2026 RCT ([Shen & Tamkin, Anthropic Fellows](https://arxiv.org/abs/2601.20245)) measured a **17% reduction in skills acquisition** when developers learned a new library with AI assistance (n=52, p=0.01) — with no significant time savings. Only ~20% of AI users (pure delegation pattern) finished faster, at the cost of learning almost nothing. +**AI-specific stress factor**: Nondeterministic outputs (identical prompts → varying results) create cognitive anxiety distinct from traditional debugging. This variability can trigger "AI fatigue" — mental exhaustion from unpredictable tool behavior that compounds over extended sessions. Mitigation: Time-box sessions (30 min max), limit retry attempts (3 max before reverting to manual implementation), and recognize when tool unpredictability signals a need for context reset (`/clear`) or manual problem-solving. + ### Where AI Helps (And Where It Hurts) | High-Gain Tasks | Low/Negative-Gain Tasks | @@ -245,6 +247,16 @@ List 3 possible approaches, even if you're not sure they'll work: This forces you to think before asking AI. +#### Step 2.5: Recognize Fatigue Signals (30 sec) + +Before moving forward, pause and assess your cognitive state: + +- **Session duration**: Been working >30 min? → Take a 5-min break, consider `/clear` to reset context +- **Retry count**: Tried the same prompt 3+ times with inconsistent results? → Switch to manual implementation +- **Frustration level**: Feeling anxious about unpredictable AI responses? → This is "AI fatigue" (nondeterminism stress), not your fault — it's the tool's inherent variability + +This checkpoint prevents compounding exhaustion from extended sessions with diminishing returns. + #### Step 3: Identify Knowledge Gaps (3 min) What specifically do you NOT know? @@ -866,6 +878,7 @@ Warning signs you're becoming dependent, and what to do: | Always ask "how" never "why" | Surface-level usage | Force yourself to ask "why this approach?" | | Every solution looks the same | AI has patterns, you need variety | Study multiple implementations manually | | Task feels easy but you can't explain it | **Perception gap** — AI users rate tasks easier while scoring 17% lower ([Shen & Tamkin 2026](https://arxiv.org/abs/2601.20245)) | After each task, explain the solution without looking at code | +| Prolonged sessions without breaks | **Session fatigue** — identical prompts yield varying outputs, causing anxiety | Time-box sessions: 30 min limit, max 3 attempts before manual implementation | ### Weekly Self-Audit