feat(roles): add MLOps Engineer, AI Developer Advocate, AI Orchestration Engineer
3 new full role profiles (§14-16) based on Perplexity market research (March 2026): - MLOps Engineer: CI/CD for models, drift monitoring, operational lifecycle - AI Developer Advocate: +35-40% YoY, $120K-$300K, active hiring at AI platforms - AI Orchestration Engineer: real job postings confirmed (Vista Equity, Zapier, Heidi, Adobe) Sections 14-17 renumbered to 17-20. Removed "Orchestration engineer" from "What's Not a Role Yet" — now a real distinct category. Career Decision Matrix and Salary Benchmarks updated with all 3 roles. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
This commit is contained in:
Florian BRUNIAUX
parent
1bbf346687
commit
c2a642dabe
2 changed files with 167 additions and 25 deletions
34
CHANGELOG.md
34
CHANGELOG.md
|
|
@ -8,6 +8,10 @@ The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/).
|
|||
|
||||
### Documentation
|
||||
|
||||
- **Resource Evaluation #076**: Addy Osmani — "Stop Using /init for AGENTS.md" (Feb 23, 2026). Score 3/5. Secondary synthesis of ETH Zürich paper (already evaluated). Verified: ETH Zürich claims confirmed. Unverified: Lulla et al. (ICSE JAWs 2026) and ACE framework (ICLR 2026) — no findable academic source. Arize AI concept verified, specific numbers uncorroborated. Integration: added discoverability filter + anchoring risk concepts to §3.1, added research note (ETH Zürich), added `/init` warning in commands table.
|
||||
|
||||
- **Resource Evaluation #077 + integration** (`guide/roles/learning-with-ai.md`): "Comprehension Debt" article (LinkedIn, March 14, 2026). Score 3/5. Integrated: (1) "comprehension debt" as emerging term after Vibe Coding section, (2) review bottleneck inversion framing — juniors can now generate code faster than seniors can audit, (3) new "Regulatory Exposure" subsection for tech leads covering EU AI Act active dates (GPAI Aug 2025, high-risk Aug 2026) and FDA AI guidance (Jan + Jun 2025). Confirmed by Perplexity research.
|
||||
|
||||
- **Claude Code Releases**: Updated tracking to v2.1.77
|
||||
- Opus 4.6 default max output raised to 64k tokens; upper bound for Opus 4.6 and Sonnet 4.6 raised to 128k tokens
|
||||
- Security fix: `PreToolUse` hooks returning `"allow"` could bypass enterprise `deny` permission rules
|
||||
|
|
@ -15,6 +19,19 @@ The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/).
|
|||
- Breaking: `Agent` tool `resume` parameter removed — use `SendMessage({to: agentId})` instead
|
||||
- Fixed auto-updater GBs memory leak; fixed `--resume` truncating recent history
|
||||
|
||||
- **Claude Code Releases**: Updated tracking to v2.1.76
|
||||
- MCP elicitation support — servers request structured input mid-task via interactive dialog
|
||||
- New hooks: `Elicitation`, `ElicitationResult`, `PostCompact`
|
||||
- `-n`/`--name` CLI flag for session display name; `worktree.sparsePaths` for monorepo sparse checkout
|
||||
- `/effort` slash command; fixed ToolSearch deferred tools losing schemas after compaction
|
||||
- Auto-compact circuit breaker (stops after 3 failures); fixed `Bash(cmd:*)` rules with `#` in args
|
||||
|
||||
- **Resource evaluation** (rejected, no file): LinkedIn post "Five Levels of Context Engineering" by Matthew Alverson (via Addy Osmani) — score 1/5, rejected. Content is a pedagogical reformulation of concepts already covered with more rigor in `guide/core/context-engineering.md`. Alverson's 5-level taxonomy is not empirically grounded and not widely cited in the literature. Evaluation surfaced 3 real gaps now addressed (see Added section). Better primary sources identified: Anthropic Engineering Blog (Sept 2025), MCP Maturity Model (Mitra, Nov 2025).
|
||||
|
||||
- **Resource evaluation** (no file — text digest): Anthropic weekly recap March 9-15, 2026 (5 Claude Code releases, Code Review launch, 1M GA, Spring Break promo, corporate news) — score 4/5. Two gaps actioned: (1) Code Review product feature added as `guide/workflows/code-review.md`; (2) 1M context status updated from beta to GA in `guide/ultimate-guide.md` lines 2021-2070. Source reliability note: digest incorrectly attributes Claude Code changelog to `anthropics/anthropic-sdk-python` (correct repo: `anthropics/claude-code`); Code Review pricing ($15-25/PR) verified against official docs.
|
||||
|
||||
- **Resource evaluation** (`docs/resource-evaluations/eval-claude-1m-context-window-jp-caparas.md`): JP Caparas article on 1M token context window — score 2/5, do not integrate. Central claim (flat pricing, no surcharge above 200K tokens) is factually wrong; invalidates the competitive pricing analysis. Fact-check table, comparative analysis vs guide, and independent action items (verify 1M GA status, potential update to guide lines 2028-2070 on beta/GA status).
|
||||
|
||||
### Changed
|
||||
|
||||
- **External support section — clarified positioning** (`docs/for-cto.md`, `docs/for-tech-leads.md`, `docs/for-cio-ceo.md`): Brown Bag Lunch, talks, and speaker/panelist slots (1-3h) explicitly marked as free and done for networking/challenge purposes. Training/consulting missions framed as open-but-not-actively-sought with "contact for availability and potentially pricing" wording. Contact link updated to `florian.bruniaux.com` across all three files.
|
||||
|
|
@ -25,6 +42,8 @@ The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/).
|
|||
|
||||
### Added
|
||||
|
||||
- **Resource evaluation #078: claude-swarm-monitor** (`docs/resource-evaluations/078-claude-swarm-monitor.md`): TUI dashboard (Rust + Ratatui) for monitoring multi-agent Claude Code workflows across git worktrees. Score 3/5 — watch-list. Unique angles: JSONL-native session file monitoring (distinct from agent-chat's SSE approach) and Docker stack visibility per worktree. Not integrated into guide yet — 10 stars, Linux-only, sub-agent tracking claim unverified. Re-evaluate at 50+ stars or confirmed macOS production use.
|
||||
|
||||
- **Packmind — Engineering Standards Distribution** (`guide/ecosystem/third-party-tools.md`, `guide/ultimate-guide.md`, `guide/ecosystem/mcp-servers-ecosystem.md`): Added Packmind (score 4/5, eval #076) as a new "Engineering Standards Distribution" section in third-party-tools. Tool distributes CLAUDE.md + slash commands + skills across repos and agents (Claude Code, Cursor, Copilot, Windsurf) from a single playbook, ships an MCP server, Apache-2.0 CLI self-hostable. Added cross-reference paragraph at end of ultimate-guide.md §3.5 (Team Configuration at Scale) linking the per-project `.claude/rules/` pattern to org-scale tooling. Added Packmind MCP server entry in mcp-servers-ecosystem.md Orchestration section.
|
||||
|
||||
- **3 new AI roles** (`guide/roles/ai-roles.md`): Added MLOps Engineer (§14), AI Developer Advocate (§15), and AI Orchestration Engineer (§16) as full role profiles. Includes responsibilities, required skills, salary benchmarks, entry paths, and key distinctions from adjacent roles. Sections 14→17 renumbered accordingly. Removed "Orchestration engineer" from "What's Not a Role Yet" — job postings at Vista Equity, Zapier, Heidi Health, and Adobe confirm it's now a real title. Career Decision Matrix and Salary Benchmarks updated with all 3 roles. Based on Perplexity market research (March 2026).
|
||||
|
|
@ -54,21 +73,6 @@ The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/).
|
|||
- **ICM (Infinite Context Memory)**: New MCP memory server section after Kairn (~line 11365) — Rust single binary, zero deps, Homebrew install, dual architecture (episodic decay Memories + permanent knowledge graph Memoirs), 9 typed relation types, auto-extraction 3 layers, 14 editor clients. Score 3/5 — recommended as Rust-native alternative when Python dependency management is a friction point. Includes explicit license callout (Source-Available, free ≤20 people) and vendor-reported benchmark flags.
|
||||
- **Comparison matrix update**: Added ICM column to MCP memory stack matrix (Runtime + License rows added for all tools)
|
||||
|
||||
### Documentation
|
||||
|
||||
- **Resource evaluation** (rejected, no file): LinkedIn post "Five Levels of Context Engineering" by Matthew Alverson (via Addy Osmani) — score 1/5, rejected. Content is a pedagogical reformulation of concepts already covered with more rigor in `guide/core/context-engineering.md`. Alverson's 5-level taxonomy is not empirically grounded and not widely cited in the literature. Evaluation surfaced 3 real gaps now addressed (see Added section above). Better primary sources identified: Anthropic Engineering Blog (Sept 2025), MCP Maturity Model (Mitra, Nov 2025).
|
||||
|
||||
- **Resource evaluation** (no file — text digest): Anthropic weekly recap March 9-15, 2026 (5 Claude Code releases, Code Review launch, 1M GA, Spring Break promo, corporate news) — score 4/5. Two gaps actioned: (1) Code Review product feature added as `guide/workflows/code-review.md`; (2) 1M context status updated from beta to GA in `guide/ultimate-guide.md` lines 2021-2070. Source reliability note: digest incorrectly attributes Claude Code changelog to `anthropics/anthropic-sdk-python` (correct repo: `anthropics/claude-code`); Code Review pricing ($15-25/PR) verified against official docs.
|
||||
|
||||
- **Resource evaluation** (`docs/resource-evaluations/eval-claude-1m-context-window-jp-caparas.md`): JP Caparas article on 1M token context window — score 2/5, do not integrate. Central claim (flat pricing, no surcharge above 200K tokens) is factually wrong; invalidates the competitive pricing analysis. Fact-check table, comparative analysis vs guide, and independent action items (verify 1M GA status, potential update to guide lines 2028-2070 on beta/GA status).
|
||||
|
||||
- **Claude Code Releases**: Updated tracking to v2.1.76
|
||||
- MCP elicitation support — servers request structured input mid-task via interactive dialog
|
||||
- New hooks: `Elicitation`, `ElicitationResult`, `PostCompact`
|
||||
- `-n`/`--name` CLI flag for session display name; `worktree.sparsePaths` for monorepo sparse checkout
|
||||
- `/effort` slash command; fixed ToolSearch deferred tools losing schemas after compaction
|
||||
- Auto-compact circuit breaker (stops after 3 failures); fixed `Bash(cmd:*)` rules with `#` in args
|
||||
|
||||
## [3.35.0] - 2026-03-13
|
||||
|
||||
### Added
|
||||
|
|
|
|||
|
|
@ -31,10 +31,13 @@ The AI wave didn't just create new tools. It created new jobs that didn't exist
|
|||
11. [AI Product Manager](#11-ai-product-manager)
|
||||
12. [AI Safety & Eval Engineer](#12-ai-safety--eval-engineer)
|
||||
13. [ML Engineer](#13-ml-engineer)
|
||||
14. [Career Decision Matrix](#14-career-decision-matrix)
|
||||
15. [Salary Benchmarks (2025-2026)](#15-salary-benchmarks-2025-2026)
|
||||
16. [What's Not a Role (Yet)](#16-whats-not-a-role-yet)
|
||||
17. [Job Listings](#17-job-listings)
|
||||
14. [MLOps Engineer](#14-mlops-engineer)
|
||||
15. [AI Developer Advocate](#15-ai-developer-advocate)
|
||||
16. [AI Orchestration Engineer](#16-ai-orchestration-engineer)
|
||||
17. [Career Decision Matrix](#17-career-decision-matrix)
|
||||
18. [Salary Benchmarks (2025-2026)](#18-salary-benchmarks-2025-2026)
|
||||
19. [What's Not a Role (Yet)](#19-whats-not-a-role-yet)
|
||||
20. [Job Listings](#20-job-listings)
|
||||
|
||||
---
|
||||
|
||||
|
|
@ -472,7 +475,138 @@ Python (fluent), PyTorch or TensorFlow, distributed computing, data pipeline too
|
|||
|
||||
---
|
||||
|
||||
## 14. Career Decision Matrix
|
||||
## 14. MLOps Engineer
|
||||
|
||||
**Status**: Established — distinct from ML Engineer, growing in enterprises deploying models at scale.
|
||||
|
||||
### What they do
|
||||
|
||||
Bridge the gap between model development and production infrastructure. While ML engineers build and fine-tune models and AI engineers build applications, MLOps engineers own the operational layer: CI/CD pipelines for models, deployment infrastructure, monitoring for drift and degradation, and the systems that keep models reliable in production over time.
|
||||
|
||||
### Responsibilities
|
||||
|
||||
- Build and maintain CI/CD pipelines for model training, evaluation, and deployment
|
||||
- Monitor production models for performance drift, data drift, and prediction quality degradation
|
||||
- Design feature stores and model registries
|
||||
- Implement A/B testing and canary deployments for new model versions
|
||||
- Manage compute infrastructure for training and inference (cost optimization)
|
||||
- Build observability tooling: metrics, logging, alerting for model behavior in production
|
||||
- Establish model versioning and rollback procedures
|
||||
|
||||
### Required skills
|
||||
|
||||
| Technical | Soft |
|
||||
|-----------|------|
|
||||
| Python (fluent) | Infrastructure mindset |
|
||||
| Cloud ML platforms (SageMaker, Vertex AI, Azure ML) | Cross-team collaboration (ML + Infra) |
|
||||
| Kubernetes, Docker, infrastructure as code | Reliability engineering instinct |
|
||||
| MLflow, Weights & Biases, or similar experiment tracking | Incident response discipline |
|
||||
| Data pipeline tools (Airflow, Prefect, dbt) | |
|
||||
| Monitoring and observability (Prometheus, Grafana) | |
|
||||
|
||||
### The distinction that matters
|
||||
|
||||
ML engineers ask: "Does the model work?" MLOps engineers ask: "Does the model keep working?" The operational lifecycle of a model — monitoring, retraining triggers, rollback procedures, cost per inference — is entirely separate from building it. Companies that skip this role discover it when a model silently degrades in production and nobody notices until user complaints spike.
|
||||
|
||||
### Entry paths
|
||||
|
||||
DevOps/platform engineer adding ML knowledge, ML engineer who gravitates toward infrastructure, data engineer moving toward model operations.
|
||||
|
||||
---
|
||||
|
||||
## 15. AI Developer Advocate
|
||||
|
||||
**Status**: High growth — actively hiring at all major AI companies in 2025-2026.
|
||||
|
||||
### What they do
|
||||
|
||||
Build the bridge between an AI platform and the developers who use it. Part engineer, part educator, part community builder. They go deep enough technically to build real things with the platform, then turn that knowledge into tutorials, documentation, sample projects, and public presence that helps other developers succeed.
|
||||
|
||||
### Responsibilities
|
||||
|
||||
- Build technical demos, sample projects, and integrations using the platform's APIs
|
||||
- Create developer content: tutorials, blog posts, video walkthroughs, conference talks
|
||||
- Represent developer needs and pain points to the product and engineering teams
|
||||
- Engage with developer communities (Discord, GitHub, forums, social)
|
||||
- Speak at conferences and run workshops
|
||||
- Onboard strategic partners and enterprise developers
|
||||
- Gather and synthesize developer feedback into product improvements
|
||||
|
||||
### Required skills
|
||||
|
||||
| Technical | Soft |
|
||||
|-----------|------|
|
||||
| Solid software engineering foundations | Clear technical writing |
|
||||
| Deep familiarity with the platform/API | Public speaking confidence |
|
||||
| Ability to build quick, illustrative prototypes | Community instinct |
|
||||
| Understanding of developer experience (DX) | Empathy for confused users |
|
||||
| Familiarity with AI concepts (prompting, RAG, agents) | Curiosity and continuous learning |
|
||||
|
||||
### What makes this role different
|
||||
|
||||
The audience is other developers, not end users. DevRel success measures developer activation (do developers try the product?), retention (do they keep using it?), and advocacy (do they tell others?). Credibility is the core asset — which means you have to actually build things, not just talk about them. A DevRel who hasn't shipped real production code with the platform has no credibility with the audience they're trying to reach.
|
||||
|
||||
### Salary context
|
||||
|
||||
$120K-$180K base (US), senior/lead roles $150K-$250K+. Total compensation includes equity at most AI companies.
|
||||
|
||||
### Where these roles are
|
||||
|
||||
Actively hiring: Anthropic, OpenAI, Together AI, Mistral, Cohere, Hugging Face, LangChain, and any company building developer-facing AI products. The role is expanding beyond AI labs as enterprise software companies add AI capabilities and need someone to help developers adopt them.
|
||||
|
||||
### Entry paths
|
||||
|
||||
Software engineer with a public presence (blog, open source, conference talks), technical writer with engineering background, early AI community member who builds in public.
|
||||
|
||||
---
|
||||
|
||||
## 16. AI Orchestration Engineer
|
||||
|
||||
**Status**: Emerging — real job postings in 2025, distinct from AI Agent Engineer in scope.
|
||||
|
||||
### What they do
|
||||
|
||||
Design and build intelligent workflows that connect AI capabilities with existing systems, data sources, and business processes. Where AI agent engineers build autonomous reasoning systems, AI orchestration engineers focus on the integration layer: connecting AI to enterprise tools, designing multi-step automation flows, and making AI reliably operable within existing infrastructure.
|
||||
|
||||
### Responsibilities
|
||||
|
||||
- Design end-to-end automation architectures using orchestration tools (n8n, LangChain, Power Automate, Zapier)
|
||||
- Integrate AI capabilities with CRMs, ERPs, data warehouses, and communication platforms
|
||||
- Build retrieval and synthesis stacks (RAG + answer grounding) for enterprise knowledge systems
|
||||
- Define workflow reliability patterns: retries, fallbacks, human escalation triggers
|
||||
- Set up observability for orchestrated workflows (tracing every step, cost tracking)
|
||||
- Operationalize AI across cross-functional systems spanning engineering, product, and domain teams
|
||||
|
||||
### Required skills
|
||||
|
||||
| Technical | Soft |
|
||||
|-----------|------|
|
||||
| Orchestration platforms (n8n, LangChain, LlamaIndex) | Process analysis |
|
||||
| API integration (REST, GraphQL, webhooks) | Cross-functional collaboration |
|
||||
| Python or JavaScript (workflow scripting) | Systems thinking |
|
||||
| Data transformation and mapping | Business process intuition |
|
||||
| Observability and tracing (LangSmith, Langfuse) | |
|
||||
|
||||
### Distinction from AI Agent Engineer
|
||||
|
||||
| AI Agent Engineer | AI Orchestration Engineer |
|
||||
|-------------------|--------------------------|
|
||||
| Builds autonomous reasoning systems | Builds integration workflows connecting AI to existing systems |
|
||||
| Focus: planning, memory, multi-step reasoning | Focus: connectivity, reliability, process automation |
|
||||
| Core challenge: non-determinism | Core challenge: integration complexity |
|
||||
| Primarily product-facing | Primarily internal/enterprise-facing |
|
||||
|
||||
### Where this role appears in job postings
|
||||
|
||||
Title varies significantly: "AI-First Orchestration Engineer" (Vista Equity Partners), "Staff AI Engineer (Orchestration)" (Heidi Health), "Sr. Software Engineer (AI Orchestration Zone)" (Zapier), "AI Engineer, AI Orchestration" (Adobe). The function is consistent even when the title isn't.
|
||||
|
||||
### Entry paths
|
||||
|
||||
Integration engineer, backend engineer with workflow automation experience, DevOps engineer adding AI tooling, business process automation specialist who's moved into code.
|
||||
|
||||
---
|
||||
|
||||
## 17. Career Decision Matrix
|
||||
|
||||
Which role fits your current background and goals?
|
||||
|
||||
|
|
@ -486,6 +620,9 @@ Which role fits your current background and goals?
|
|||
| Non-technical who works with AI daily | Prompt Engineer → Context Engineer | 6-18 months |
|
||||
| PM who wants to stay PM but be more relevant | AI Product Manager | 3-6 months upskill |
|
||||
| Engineer obsessed with reliability and architecture | Harness Engineer (emerging) | Pioneers' territory |
|
||||
| DevOps/platform engineer who wants to work with models | MLOps Engineer | 3-6 months upskill |
|
||||
| Engineer with public presence and community instincts | AI Developer Advocate | 6-12 months |
|
||||
| Integration or automation engineer adding AI | AI Orchestration Engineer | 3-6 months |
|
||||
|
||||
### The fastest path to AI employment in 2025-2026
|
||||
|
||||
|
|
@ -498,7 +635,7 @@ Note: 76% of candidates claiming AI expertise lack production-level deployment e
|
|||
|
||||
---
|
||||
|
||||
## 15. Salary Benchmarks (2025-2026)
|
||||
## 18. Salary Benchmarks (2025-2026)
|
||||
|
||||
> **Indicative only — large variance applies.** These figures are US market base salaries (2025-2026). Europe runs 30-50% lower, other markets 40-60% lower. Total compensation (equity, bonus, RSUs) can significantly exceed base, especially at startups and FAANG. Experience level, location within a country, company stage, and negotiation all create wide variance. Use these as orientation, not negotiation anchors.
|
||||
|
||||
|
|
@ -516,12 +653,15 @@ Note: 76% of candidates claiming AI expertise lack production-level deployment e
|
|||
| AI Product Manager | $130K-$170K | $170K-$230K | $230K-$350K | FAANG premium significant |
|
||||
| AI Safety/Eval Engineer | $140K-$180K | $180K-$250K | $250K-$400K | Lab compensation highest |
|
||||
| ML Engineer | $100K-$140K | $140K-$200K | $200K-$280K | Lower demand outside labs |
|
||||
| MLOps Engineer | $110K-$150K | $150K-$200K | $200K-$270K | High demand in enterprises deploying at scale |
|
||||
| AI Developer Advocate | $120K-$160K | $160K-$220K | $220K-$300K | Active hiring at AI platforms |
|
||||
| AI Orchestration Engineer | $100K-$140K | $140K-$190K | $190K-$260K | Emerging — title varies across companies |
|
||||
|
||||
> **Sources**: FinalRoundAI (2025), Alcor AI Salary Report (2025), RiseWorks AI Talent Report (2025), job postings analysis.
|
||||
|
||||
---
|
||||
|
||||
## 16. What's Not a Role (Yet)
|
||||
## 19. What's Not a Role (Yet)
|
||||
|
||||
Some terms you'll hear that describe practices or methodologies, not job titles:
|
||||
|
||||
|
|
@ -529,13 +669,11 @@ Some terms you'll hear that describe practices or methodologies, not job titles:
|
|||
|
||||
**AI-native engineer** — Describes a quality expected of all engineers increasingly, not a specialized role. It means: you use AI tools fluently in your daily workflow. It's the bar, not the title.
|
||||
|
||||
**Orchestration engineer** — Sometimes used for agent systems, overlaps significantly with AI Agent Engineer. Not yet a distinct category.
|
||||
|
||||
These terms are worth knowing (you'll encounter them in job descriptions and articles) but don't represent distinct career paths — yet.
|
||||
|
||||
---
|
||||
|
||||
## 17. Job Listings
|
||||
## 20. Job Listings
|
||||
|
||||
> **Coming soon** — Curated listings for AI roles at companies building seriously with Claude Code and agentic AI.
|
||||
|
||||
|
|
|
|||
Loading…
Add table
Add a link
Reference in a new issue