docs: integrate Contribution Metrics blog (4/5) - Anthropic Jan 2026 data
New subsection in ultimate-guide.md with +67% PRs merged and 70-90% AI-assisted code metrics. Separate from Aug 2025 study (different methodology: PR-based vs self-reported). ROI cross-reference added. Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
This commit is contained in:
parent
26ee4ef894
commit
22f2b91b83
4 changed files with 99 additions and 1 deletions
|
|
@ -6,6 +6,15 @@ The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/).
|
|||
|
||||
## [Unreleased]
|
||||
|
||||
- **Contribution Metrics (Anthropic blog, Jan 29 2026)** — `guide/ultimate-guide.md`, `machine-readable/reference.yaml`
|
||||
- Score: 4/5 (High Value — official source with harder metrics superseding Aug 2025 data)
|
||||
- Source: [claude.com/blog/contribution-metrics](https://claude.com/blog/contribution-metrics)
|
||||
- New subsection after Anthropic Internal Study: +67% PRs merged/engineer/day, 70-90% AI-assisted code
|
||||
- Contribution Metrics dashboard: public beta, Team & Enterprise plans (GitHub integration)
|
||||
- Methodological note: PR-based measurement vs Aug 2025 self-reported surveys
|
||||
- ROI cross-reference added in cost optimization section
|
||||
- Evaluation: `docs/resource-evaluations/026-contribution-metrics-blog.md`
|
||||
|
||||
- **Learning guide: Shen & Tamkin RCT integration** — `guide/learning-with-ai.md`
|
||||
- Source: [arXiv:2601.20245](https://arxiv.org/abs/2601.20245) (Shen & Tamkin, Anthropic Fellows, Jan 2026)
|
||||
- Score: 3/5 (Pertinent - Complément utile, high overlap with existing content)
|
||||
|
|
|
|||
61
docs/resource-evaluations/026-contribution-metrics-blog.md
Normal file
61
docs/resource-evaluations/026-contribution-metrics-blog.md
Normal file
|
|
@ -0,0 +1,61 @@
|
|||
# Resource Evaluation: Contribution Metrics (Anthropic Blog)
|
||||
|
||||
| Field | Value |
|
||||
|-------|-------|
|
||||
| **Resource** | [Contribution Metrics](https://claude.com/blog/contribution-metrics) |
|
||||
| **Type** | Official Anthropic blog post (product announcement) |
|
||||
| **Published** | 2026-01-29 |
|
||||
| **Evaluated** | 2026-01-30 |
|
||||
| **Score** | **4/5** (High Value) |
|
||||
| **Action** | Integrated |
|
||||
|
||||
## Summary
|
||||
|
||||
Anthropic announces "Contribution Metrics" for Claude Code — a GitHub-integrated analytics dashboard in public beta for Team and Enterprise plans. The post includes updated internal productivity data:
|
||||
|
||||
- **+67% PRs merged** per engineer per day at Anthropic
|
||||
- **70-90% of code** written with Claude Code assistance across teams
|
||||
- Dashboard tracks PRs and lines of code committed with/without Claude Code attribution
|
||||
- Conservative measurement: only high-confidence Claude Code involvement counted
|
||||
- Positioned as complement to DORA metrics and sprint velocity, not replacement
|
||||
|
||||
## Gap Analysis
|
||||
|
||||
| Aspect | Resource | Guide (before integration) |
|
||||
|--------|----------|---------------------------|
|
||||
| Productivity metrics | +67% PRs (PR-based, Jan 2026) | +50% self-reported (Aug 2025) |
|
||||
| % AI-assisted code | 70-90% | Not documented |
|
||||
| Analytics dashboard | Full feature description | Not documented |
|
||||
| Methodology | PR/commit-based (GitHub) | Survey-based (132 engineers) |
|
||||
|
||||
## Integration Decision
|
||||
|
||||
**Score justification (4/5):**
|
||||
- Official first-party source with harder metrics than existing guide content
|
||||
- Supersedes Aug 2025 self-reported +50% with PR-based +67%
|
||||
- New product feature (dashboard) worth documenting
|
||||
- Score not 5/5 because: methodology vague, internal data only, Team/Enterprise scope limits audience
|
||||
|
||||
**Challenge (technical-writer agent):**
|
||||
- Recommended score increase from 3 to 4 (accepted)
|
||||
- Identified 3 blind spots: methodology comparison, "conservative" claim scrutiny, competitive context
|
||||
- Corrected integration plan: separate subsection (not merged with Aug 2025 study)
|
||||
|
||||
## Fact-Check
|
||||
|
||||
| Claim | Status | Note |
|
||||
|-------|--------|------|
|
||||
| +67% PRs merged/engineer/day | Verified in article | No baseline timeframe specified |
|
||||
| 70-90% AI-written code | Verified in article | No team breakdown |
|
||||
| Public beta Team & Enterprise | Verified in article | Exact add-on requirements unconfirmed |
|
||||
| "Conservative" measurement | Unverifiable | Marketing claim, no technical detail |
|
||||
| Publication date Jan 29, 2026 | Verified | Page metadata |
|
||||
|
||||
**Confidence**: Medium — official source, but internal metrics without external validation or detailed methodology.
|
||||
|
||||
## Integration Applied
|
||||
|
||||
1. **New subsection** in `guide/ultimate-guide.md` after Anthropic Internal Study (Aug 2025) — separate section with own source, methodology note, and caveats
|
||||
2. **Reference.yaml** entry with source, date, availability, and key stats
|
||||
3. **ROI cross-reference** in cost optimization section (~line 10984)
|
||||
4. **Not added** to CLI releases tracking (platform feature, not CLI version)
|
||||
|
|
@ -10983,7 +10983,7 @@ Smart escalation (Haiku → Sonnet for 10% of PRs):
|
|||
|
||||
**Perspective on ROI:**
|
||||
|
||||
Time savings from effective Claude Code usage typically far outweigh API costs for most development tasks. Rather than calculating precise ROI (which depends heavily on your specific context, hourly rate, and task complexity), focus on whether the tool is genuinely helping you ship faster.
|
||||
Time savings from effective Claude Code usage typically far outweigh API costs for most development tasks. Rather than calculating precise ROI (which depends heavily on your specific context, hourly rate, and task complexity), focus on whether the tool is genuinely helping you ship faster. For team-level measurement, see [Contribution Metrics](#contribution-metrics-january-2026) — Anthropic's GitHub-integrated dashboard for tracking PR and code attribution (Team/Enterprise plans, public beta).
|
||||
|
||||
**When to optimize aggressively:**
|
||||
- High-volume operations (>1000 requests/day)
|
||||
|
|
@ -11624,6 +11624,28 @@ Anthropic studied how their own engineers use Claude Code, providing empirical d
|
|||
|
||||
---
|
||||
|
||||
### Contribution Metrics (January 2026)
|
||||
|
||||
Five months after the internal study, Anthropic published updated productivity data alongside a new analytics feature for Team and Enterprise customers.
|
||||
|
||||
**Updated metrics (Anthropic internal)**:
|
||||
- **+67%** PRs merged per engineer per day (vs Aug 2025 self-reported +50%)
|
||||
- **70-90%** of code now written with Claude Code assistance across teams
|
||||
|
||||
**Methodological note**: These figures are PR/commit-based (measured via GitHub integration), not self-reported surveys as in the Aug 2025 study. However, Anthropic discloses no baseline period, no team breakdown, and defines measurement only as "conservative — only code where we have high confidence in Claude Code's involvement." Treat as directional indicators, not rigorous benchmarks.
|
||||
|
||||
**Product feature — Contribution Metrics dashboard**:
|
||||
- **Status**: Public beta (January 2026)
|
||||
- **Availability**: Claude Team and Enterprise plans (exact add-on requirements unconfirmed)
|
||||
- **Tracks**: PRs merged and lines of code committed, with/without Claude Code attribution
|
||||
- **Access**: Workspace admins and owners only
|
||||
- **Setup**: Install Claude GitHub App → Enable GitHub Analytics in Admin settings → Authenticate GitHub organization
|
||||
- **Positioning**: Complement to existing engineering KPIs (DORA metrics, sprint velocity), not a replacement
|
||||
|
||||
**Source**: [Anthropic — Contribution Metrics (Jan 2026)](https://claude.com/blog/contribution-metrics)
|
||||
|
||||
---
|
||||
|
||||
### Cost-Benefit Analysis
|
||||
|
||||
Multi-instance workflows have hard costs and soft overhead (coordination, supervision, merge conflicts).
|
||||
|
|
|
|||
|
|
@ -330,6 +330,12 @@ deep_dive:
|
|||
multi_instance_workflows: 9583
|
||||
boris_cherny_case_study: 9617
|
||||
anthropic_study_metrics: 9721
|
||||
# Contribution Metrics (Jan 2026 - platform feature, not CLI release)
|
||||
contribution_metrics: 11625
|
||||
contribution_metrics_source: "https://claude.com/blog/contribution-metrics"
|
||||
contribution_metrics_date: "2026-01-29"
|
||||
contribution_metrics_availability: "Public beta - Team and Enterprise plans"
|
||||
contribution_metrics_stats: "+67% PRs merged/engineer/day, 70-90% AI-assisted code (Anthropic internal)"
|
||||
git_worktrees_multi_instance: 10634
|
||||
advanced_worktree_tooling: 10748
|
||||
worktree_tooling_self_assessment: 10762
|
||||
|
|
|
|||
Loading…
Add table
Add a link
Reference in a new issue