Skip to content
Replicating Anthropic's Self-Service Analytics
On Trove · from Matan Marudi

Replicating Anthropic's Self-Service Analytics

A Practitioner's Implementation Guide

Step-by-step guide to replicating Anthropic's internal self-service analytics system — the one that automated 95% of business analytics quer.

Read a preview

You can read the contents and the first 2,000 characters.
Free accounts get the whole book.

  • By Matan Marudi
  • 12 chapters
  • Updated Nov 2, 2026
  • 78 readers

What’s inside

  • The System Anthropic Built
  • Building the Data Foundation
  • Building the Skills
  • The Agentic Loop
  • The Evaluation System
  • Going to Production

What is Replicating Anthropic's Self-Service Analytics?

Step-by-step guide to replicating Anthropic's internal self-service analytics system — the one that automated 95% of business analytics queries at ~95% accuracy. Covers the exact four-layer architecture (data foundations, sources of truth, skills, validation), how to write Knowledge Skills and Unbook Skills, semantic layer integration (dbt/Cube/Looker), adversarial review sub-agents, offline eval framework with correction harvesting, and CI/colocation enforcement. All specifics sourced from the Anthropic engineering blog post (Chen Chang, Clement Peng, Justin Leder, Johanne Jiao, Josh Cherry, June 2026). Includes copy-paste templates for every component.

Read more

markdown · Listed Jun 4, 2026

From the first page

A working blueprint for building an LLM-driven analytics system that automates 95% of business questions at 95% accuracy. All specifics traced to the Anthropic engineering blog post by Chen Chang, Clement Peng, Justin Leder, Johanne Jiao, and Josh Cherry (June 2026). Templates included are complete and copy-paste ready.

This book is written for Claude Code so it can guide a data engineer or analytics engineer through replicating Anthropic's stack. Each chapter is actionable. The target reader is technically fluent and impatient: the prose is dense and the examples are real.

Anthropic built an internal self-service analytics system that resolves natural-language questions to correct SQL against the company's warehouse with approximately 95% accuracy. About 95% of business analytics queries that previously routed to human analysts now flow through this system without intervention. In some domains, accuracy peaks near 99%.

The 21% baseline is the number to internalize: without the skills, sources of truth, and validation harness described in this book, a fluent LLM with warehouse access answers only about one-in-five analytics questions correctly. Most of those wrong answers look right. That gap — 21% to 95% — is engineering work, not a model upgrade.

The 95% number is aggregated over a production query stream. It includes both queries the system answered correctly on first attempt and queries where the adversarial reviewer caught and corrected a mistake before the answer was returned. It excludes queries where the system explicitly refused (red flag detection, ungated domains, missing authentication).

Preview · 2,000 of 171,187 characters

The Complete Guide to Building Skills for Claude
Web Application Security for AI Agents
UI/UX Design Principles for AI Agents
Building Your Agent Team: A Practitioner's Guide to Multi-Agent AI Systems

Contents

  1. 01Replicating Anthropic's Self-Service Analytics: A Practitioner's Implementation Guidep.1
  2. 02The System Anthropic Builtp.2
    1. Chapter 1: What They Built and Why It Worksp.2
    2. Chapter 2: The Three Failure Modesp.2
    3. Chapter 3: The Four-Layer Architecturep.2
  3. 03Building the Data Foundationp.3
    1. Chapter 4: Canonical Datasets — Your Single Sources of Truthp.3
    2. Chapter 5: The Semantic Layerp.3
    3. Chapter 6: The Query Corpus — What to Skip and What to Keepp.3
  4. 04Building the Skillsp.4
    1. Chapter 7: What Skills Arep.4
    2. Chapter 8: Writing Knowledge Skillsp.4
    3. Chapter 9: Writing Unbook Skillsp.4
    4. Chapter 10: The Skill Taxonomy — Organizing for Scalep.4
  5. 05The Agentic Loopp.5
    1. Chapter 11: System Prompt Architecturep.5
    2. Chapter 12: Adversarial Review Sub-Agentsp.5
    3. Chapter 13: Provenance Footersp.5
    4. Chapter 14: Entity Disambiguationp.5
  6. 06The Evaluation Systemp.6
    1. Chapter 15: Offline Evals — The Foundationp.6
    2. Chapter 16: Eval Versioningp.6
    3. Chapter 17: Correction Harvestingp.6
    4. Chapter 18: The CI/CD Enforcement Systemp.6
  7. 07Going to Productionp.7
    1. Chapter 19: The 3-Week Launch Planp.7
    2. Chapter 20: Monitoring and Governancep.7
    3. Chapter 21: Scaling Across Domainsp.7
    4. Chapter 22: What NOT to Dop.7
  8. 08Appendix A: Copy-Paste Templatesp.8
    1. A.1 Knowledge Skill — Index Filep.8
    2. A.2 Knowledge Skill — Domain Reference Filep.8
    3. A.3 Unbook Skill — Main Filep.8
    4. A.4 Analysis Pattern — Retention Curvep.8
    5. A.5 Analysis Pattern — Rate Decompositionp.8
    6. A.6 Analysis Pattern — Funnel Analysisp.8
    7. A.7 System Prompt — Full Templatep.8
    8. A.8 Adversarial Reviewer — Prompt Templatep.8
    9. +6 more sections
See all 12 parts
  1. 09Appendix B: Accuracy Diagnostic Checklistp.9
    1. Step 1: Identify the failure categoryp.9
    2. Step 2: If concept-entity ambiguity dominatesp.9
    3. Step 3: If staleness dominatesp.9
    4. Step 4: If retrieval failure dominatesp.9
    5. Step 5: If "other" dominatesp.9
    6. Step 6: Re-run evalsp.9
  2. 10Appendix C: Tool Stack Optionsp.10
    1. Semantic layerp.10
    2. Warehousep.10
    3. Skill storagep.10
    4. Eval trackingp.10
    5. Correction harvestingp.10
  3. 11Appendix D: Glossaryp.11
  4. 12Extended Practitioner Sectionsp.12
    1. The No-Semantic-Layer Onboarding Pathp.12
    2. Cross-Grain Derived Metricsp.12
    3. Pattern: Trailing-Period Deviation Flaggingp.12
    4. Query Cost Controlsp.12
    5. Cost Estimate Reconciliationp.12
    6. Event Naming Conventionp.12
    7. Chapter 11a: The Meta-Tool Pattern — One Exec Surface Instead of Forty Toolsp.12
    8. Chapter 14a: Choosing a Chart Type for Rate-by-Category Metricsp.12
    9. +3 more sections