docs: integrate Contribution Metrics blog (4/5) - Anthropic Jan 2026 data

New subsection in ultimate-guide.md with +67% PRs merged and 70-90%
AI-assisted code metrics. Separate from Aug 2025 study (different
methodology: PR-based vs self-reported). ROI cross-reference added.

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
This commit is contained in:
Florian BRUNIAUX 2026-01-30 23:34:15 +01:00
parent 26ee4ef894
commit 22f2b91b83
4 changed files with 99 additions and 1 deletions

View file

@ -6,6 +6,15 @@ The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/).
## [Unreleased]
- **Contribution Metrics (Anthropic blog, Jan 29 2026)**`guide/ultimate-guide.md`, `machine-readable/reference.yaml`
- Score: 4/5 (High Value — official source with harder metrics superseding Aug 2025 data)
- Source: [claude.com/blog/contribution-metrics](https://claude.com/blog/contribution-metrics)
- New subsection after Anthropic Internal Study: +67% PRs merged/engineer/day, 70-90% AI-assisted code
- Contribution Metrics dashboard: public beta, Team & Enterprise plans (GitHub integration)
- Methodological note: PR-based measurement vs Aug 2025 self-reported surveys
- ROI cross-reference added in cost optimization section
- Evaluation: `docs/resource-evaluations/026-contribution-metrics-blog.md`
- **Learning guide: Shen & Tamkin RCT integration**`guide/learning-with-ai.md`
- Source: [arXiv:2601.20245](https://arxiv.org/abs/2601.20245) (Shen & Tamkin, Anthropic Fellows, Jan 2026)
- Score: 3/5 (Pertinent - Complément utile, high overlap with existing content)

View file

@ -0,0 +1,61 @@
# Resource Evaluation: Contribution Metrics (Anthropic Blog)
| Field | Value |
|-------|-------|
| **Resource** | [Contribution Metrics](https://claude.com/blog/contribution-metrics) |
| **Type** | Official Anthropic blog post (product announcement) |
| **Published** | 2026-01-29 |
| **Evaluated** | 2026-01-30 |
| **Score** | **4/5** (High Value) |
| **Action** | Integrated |
## Summary
Anthropic announces "Contribution Metrics" for Claude Code — a GitHub-integrated analytics dashboard in public beta for Team and Enterprise plans. The post includes updated internal productivity data:
- **+67% PRs merged** per engineer per day at Anthropic
- **70-90% of code** written with Claude Code assistance across teams
- Dashboard tracks PRs and lines of code committed with/without Claude Code attribution
- Conservative measurement: only high-confidence Claude Code involvement counted
- Positioned as complement to DORA metrics and sprint velocity, not replacement
## Gap Analysis
| Aspect | Resource | Guide (before integration) |
|--------|----------|---------------------------|
| Productivity metrics | +67% PRs (PR-based, Jan 2026) | +50% self-reported (Aug 2025) |
| % AI-assisted code | 70-90% | Not documented |
| Analytics dashboard | Full feature description | Not documented |
| Methodology | PR/commit-based (GitHub) | Survey-based (132 engineers) |
## Integration Decision
**Score justification (4/5):**
- Official first-party source with harder metrics than existing guide content
- Supersedes Aug 2025 self-reported +50% with PR-based +67%
- New product feature (dashboard) worth documenting
- Score not 5/5 because: methodology vague, internal data only, Team/Enterprise scope limits audience
**Challenge (technical-writer agent):**
- Recommended score increase from 3 to 4 (accepted)
- Identified 3 blind spots: methodology comparison, "conservative" claim scrutiny, competitive context
- Corrected integration plan: separate subsection (not merged with Aug 2025 study)
## Fact-Check
| Claim | Status | Note |
|-------|--------|------|
| +67% PRs merged/engineer/day | Verified in article | No baseline timeframe specified |
| 70-90% AI-written code | Verified in article | No team breakdown |
| Public beta Team & Enterprise | Verified in article | Exact add-on requirements unconfirmed |
| "Conservative" measurement | Unverifiable | Marketing claim, no technical detail |
| Publication date Jan 29, 2026 | Verified | Page metadata |
**Confidence**: Medium — official source, but internal metrics without external validation or detailed methodology.
## Integration Applied
1. **New subsection** in `guide/ultimate-guide.md` after Anthropic Internal Study (Aug 2025) — separate section with own source, methodology note, and caveats
2. **Reference.yaml** entry with source, date, availability, and key stats
3. **ROI cross-reference** in cost optimization section (~line 10984)
4. **Not added** to CLI releases tracking (platform feature, not CLI version)

View file

@ -10983,7 +10983,7 @@ Smart escalation (Haiku → Sonnet for 10% of PRs):
**Perspective on ROI:**
Time savings from effective Claude Code usage typically far outweigh API costs for most development tasks. Rather than calculating precise ROI (which depends heavily on your specific context, hourly rate, and task complexity), focus on whether the tool is genuinely helping you ship faster.
Time savings from effective Claude Code usage typically far outweigh API costs for most development tasks. Rather than calculating precise ROI (which depends heavily on your specific context, hourly rate, and task complexity), focus on whether the tool is genuinely helping you ship faster. For team-level measurement, see [Contribution Metrics](#contribution-metrics-january-2026) — Anthropic's GitHub-integrated dashboard for tracking PR and code attribution (Team/Enterprise plans, public beta).
**When to optimize aggressively:**
- High-volume operations (>1000 requests/day)
@ -11624,6 +11624,28 @@ Anthropic studied how their own engineers use Claude Code, providing empirical d
---
### Contribution Metrics (January 2026)
Five months after the internal study, Anthropic published updated productivity data alongside a new analytics feature for Team and Enterprise customers.
**Updated metrics (Anthropic internal)**:
- **+67%** PRs merged per engineer per day (vs Aug 2025 self-reported +50%)
- **70-90%** of code now written with Claude Code assistance across teams
**Methodological note**: These figures are PR/commit-based (measured via GitHub integration), not self-reported surveys as in the Aug 2025 study. However, Anthropic discloses no baseline period, no team breakdown, and defines measurement only as "conservative — only code where we have high confidence in Claude Code's involvement." Treat as directional indicators, not rigorous benchmarks.
**Product feature — Contribution Metrics dashboard**:
- **Status**: Public beta (January 2026)
- **Availability**: Claude Team and Enterprise plans (exact add-on requirements unconfirmed)
- **Tracks**: PRs merged and lines of code committed, with/without Claude Code attribution
- **Access**: Workspace admins and owners only
- **Setup**: Install Claude GitHub App → Enable GitHub Analytics in Admin settings → Authenticate GitHub organization
- **Positioning**: Complement to existing engineering KPIs (DORA metrics, sprint velocity), not a replacement
**Source**: [Anthropic — Contribution Metrics (Jan 2026)](https://claude.com/blog/contribution-metrics)
---
### Cost-Benefit Analysis
Multi-instance workflows have hard costs and soft overhead (coordination, supervision, merge conflicts).

View file

@ -330,6 +330,12 @@ deep_dive:
multi_instance_workflows: 9583
boris_cherny_case_study: 9617
anthropic_study_metrics: 9721
# Contribution Metrics (Jan 2026 - platform feature, not CLI release)
contribution_metrics: 11625
contribution_metrics_source: "https://claude.com/blog/contribution-metrics"
contribution_metrics_date: "2026-01-29"
contribution_metrics_availability: "Public beta - Team and Enterprise plans"
contribution_metrics_stats: "+67% PRs merged/engineer/day, 70-90% AI-assisted code (Anthropic internal)"
git_worktrees_multi_instance: 10634
advanced_worktree_tooling: 10748
worktree_tooling_self_assessment: 10762