支持增量编译wiki

This commit is contained in:
Junjian Wang
2026-04-11 23:12:14 +08:00
parent 4d35d576ea
commit 94d4617299
129 changed files with 5981 additions and 4276 deletions
+7
View File
@@ -0,0 +1,7 @@
{
"permissions": {
"additionalDirectories": [
"/Users/junjian/GitHub/wang-junjian/wikillm/raw/images"
]
}
}
@@ -0,0 +1,69 @@
---
title: "Harness Engineering - first thoughts"
source: "https://martinfowler.com/articles/exploring-gen-ai/harness-engineering-memo.html"
author:
- "[[Birgitta Böckeler Birgitta is a Distinguished Engineer and AI-assisted delivery expert at Thoughtworks. She has over 20 years of experience as a software developer]]"
- "[[architect and technical leader.]]"
published:
created: 2026-04-11
description: "Notes from my Thoughtworks colleagues on AI-assisted software delivery"
tags:
- "clippings"
---
Since writing this memo, I have had time to analyze the topic further and write a [more considered article describing Harness Engineering](https://martinfowler.com/articles/harness-engineering.html). This frames the elements of a harness as guides and sensors, which may be computational or inferential. Harness templates allow us to share common guides and sensors around a larger software organization. Harnesses attempt to externalise and make explicit what human developer experience brings to the table, but they can only go so far. A good harness should not necessarily aim to fully eliminate human input, but to direct it to where our input is most important.
It was very interesting to read [OpenAIs recent write-up on “Harness engineering”](https://openai.com/index/harness-engineering/) which describes how a team used “no manually typed code at all” as a forcing function to build a harness for maintaining a large application with AI agents. After 5 months, theyve built a real product thats now over 1 million lines of code.
The article is titled “Harness engineering: leveraging Codex in an agent-first world”, but only mentions “harness” once in the text. Maybe the term was an afterthought inspired by [Mitchell Hashimoto](https://mitchellh.com/writing/my-ai-adoption-journey#step-5-engineer-the-harness) s recent blog post. Either way, I like “harness” as a word to describe the tooling and practices we can use to keep AI agents in check.
The OpenAI teams harness components mix deterministic and LLM-based approaches across 3 categories (grouping based on my interpretation):
1. **Context engineering**: Continuously enhanced knowledge base in the codebase, plus agent access to dynamic context like observability data and browser navigation
2. **Architectural constraints**: Monitored not only by the LLM-based agents, but also deterministic custom linters and structural tests
3. **“Garbage collection”**: Agents that run periodically to find inconsistencies in documentation or violations of architectural constraints, fighting entropy and decay
They also highlight how iterative this is: “When the agent struggles, we treat it as a signal: identify what is missing — tools, guardrails, documentation — and feed it back into the repository, always by having Codex itself write the fix.”
All of the described measures focus on increasing long-term internal quality and maintainability. What I am missing in the write-up is verification of functionality and behaviour.
Leaving that gap aside, and assuming we can trust OpenAIs representation of the success of this (with respect to the author and the team, OpenAI do have a vested interest in us believing in AI-maintainable code) — here are my thoughts on what *is* in the article.
### Harnesses - the future service templates?
Most organizations have just two or three main tech stacks — not every application is its own snowflake. The article made me imagine a future where teams pick from a set of harnesses for common application topologies to get started. This evokes todays service templates, which help teams instantiate new services on a “golden path”. Will harnesses — with custom linters, structural tests, basic context and knowledge documentation, and additional context providers — become the new service templates? Will teams use them as a starting point, then shape them over time for their applications specifics?
With service templates, teams contribute back as they gain experience, then other teams often struggle to incorporate updates. Would we see a similar forking and synchronization challenge with harnesses?
The article also made me revisit some of my older hypotheses:
### The runtime has to be constrained for more AI autonomy?
A lot of early and current AI coding hype assumes LLMs will give us unlimited flexibility of the target runtime. Generate in any language, any pattern, without constraints — the LLM will figure it out. But for maintainable, AI-generated code at scale that we can trust, something has to give.
The harness described suggests that increasing trust and reliability required constraining the solution space: specific architectural patterns, enforced boundaries, standardized structures. That means giving up some “generate anything” flexibility for prompts, rules, and harnesses full of technical specifics.
### A convergence on a limited number of tech stacks and topologies?
As coding becomes less about typing code and more about steering its generation, AI might push us toward fewer tech stacks. Usability of frameworks and SDKs still matters — were seeing repeatedly that whats good for humans is good for AI. But developer tastes will matter less at that level of detail. Little inefficiencies and idiosyncracies in interfaces will be less annoying since we dont deal with them directly. We might choose stacks with good harnesses available and prioritize “AI-friendliness”.
This might apply not just to tech stacks, but also to codebase structures and topologies. We might default to structures that are easier to maintain with AI because theyre easier to harness. The OpenAI team discusses architectural rigidity and enforcement rules. The main focus areas I can see are keeping data structures stable and defining and enforcing module boundaries. Sounds reasonable — but without concrete examples, Im still struggling to imagine what “we require Codex to parse data shapes at the boundary” looks like in practice in their harness.
But if we can figure out widely how to harness codebase design patterns, will these topologies become the new abstraction layer, not natural language itself like so many AI enthusiasts hope?
### Two future worlds: Pre-AI vs post-AI application maintenance?
Say we develop good harnessing techniques to turn AI autonomy up to 9 and increase our confidence in the results. Which techniques could we apply to existing applications, and which would only work for applications built from scratch with a harness in mind?
For older codebases, wed need to consider whether retrofitting a harness is worth the effort. AI can help us do that faster, but those applications are often so non-standardized and full of entropy that it might not be worthwhile. It makes me think of running a static code analysis tool on a codebase thats never had one, and then drowning in alerts.
### Whats your harness today?
That this team worked on their harness for 5 months shows this isnt something you can jump into for quick results. But its worth reflecting on what your harness is today. Do you have a pre-commit hook? Whats in it? Do you have ideas for custom linters? What architectural constraints would you like to impose on your codebase? Have you experimented with structural testing frameworks like ArchUnit?
### Final thoughts
Unsurprisingly, what they describe sounds like much more work than just generating and maintaining a bunch of Markdown rules files. They built extensive tooling for the deterministic part of the harness. Their context engineering involved not only curating a knowledge base, but also significant design work — the code design itself is a huge part of the context.
The OpenAI team says: “Our most difficult challenges now center on designing environments, feedback loops, and control systems.” This reminded me of [Chad Fowlers recent post on “Relocating Rigor”](https://aicoding.leaflet.pub/3mbrvhyye4k2e). Its refreshing to hear concrete ideas and experiences about where that rigor might go, rather than just hoping “better models” will magically solve maintainability issues.
And finally, for once, I like a term in this space. Though its only 2 weeks old — I can probably hold my metaphorical breath until somebody calls their one-prompt, LLM-based code review agent a harness…
@@ -0,0 +1,330 @@
---
title: "Harness Engineering: The Complete Guide to Building Systems That Make AI Agents Actually Work (2026)"
source: "https://www.nxcode.io/resources/news/harness-engineering-complete-guide-ai-agent-codex-2026"
author:
- "[[NxCode Team]]"
published: 2026-03-01
created: 2026-04-11
description: "Harness engineering is the new discipline of designing environments, constraints, and feedback loops that make AI coding agents reliable at scale. OpenAI built 1M+ lines of code with zero human-written code using this approach."
tags:
- "clippings"
---
Turn your idea into a working app — no coding required.[Start Free](https://studio.nxcode.io/?ref=article_top_harness-engineering-complete-guide-ai-agent-codex-2026&article=harness-engineering-complete-guide-ai-agent-codex-2026)
## Harness Engineering: The Complete Guide to Building Systems That Make AI Agents Actually Work
**March 2026** — If 2025 was the year AI agents proved they could write code, 2026 is the year we learned that **the agent isn't the hard part — the harness is.**
OpenAI's Codex team just built a production application with **over 1 million lines of code** where **zero lines were written by human hands**. The engineers didn't write code. They designed the system that let AI write code reliably. That system — the constraints, feedback loops, documentation, linters, and lifecycle management — is what the industry now calls a **harness**.
**Harness engineering** is the new discipline of designing these systems. And it's changing what it means to be a software engineer.
---
## What Is Harness Engineering?
### The Horse Metaphor
The term "harness" comes from horse tack — reins, saddle, bit — the complete set of equipment for channeling a powerful but unpredictable animal in the right direction. The metaphor is deliberate:
- The **horse** is the AI model — powerful, fast, but it doesn't know where to go on its own
- The **harness** is the infrastructure — constraints, guardrails, feedback loops that channel the model's power productively
- The **rider** is the human engineer — providing direction, not doing the running
Without a harness, an AI agent is a thoroughbred in an open field. Fast, impressive, and completely useless for getting anything done.
### The Formal Definition
**Harness engineering** is the design and implementation of systems that:
1. **Constrain** what an AI agent can do (architectural boundaries, dependency rules)
2. **Inform** the agent about what it should do (context engineering, documentation)
3. **Verify** that the agent did it correctly (testing, linting, CI validation)
4. **Correct** the agent when it goes wrong (feedback loops, self-repair mechanisms)
Martin Fowler describes it as *"the tooling and practices we can use to keep AI agents in check"* — but it's more than just safety. A good harness makes agents **more capable**, not just more controlled.
---
## Why Harness Engineering Matters Now
### The Model Is Commodity. The Harness Is Moat.
Here's the uncomfortable truth the AI industry is confronting: **the underlying model matters less than the system around it.**
LangChain proved this definitively. Their coding agent went from **52.8% to 66.5%** on Terminal Bench 2.0 — jumping from **Top 30 to Top 5** — by changing nothing about the model. They only changed the harness:
| Change | What They Did | Impact |
| --- | --- | --- |
| Self-verification loop | Added pre-completion checklist middleware | Caught errors before submission |
| Context engineering | Mapped directory structures at startup | Agent understood codebase from the start |
| Loop detection | Tracked repeated file edits | Prevented "doom loops" |
| Reasoning sandwich | High reasoning for planning/verification, medium for implementation | Better quality within time budgets |
**Same model. Different harness. Dramatically better results.**
### OpenAI's 1 Million Line Proof Point
OpenAI's experiment is the most compelling evidence yet:
- **5 months** of development
- **1 million+ lines of code** in the final product
- **Zero manually written lines** — every line was produced by Codex agents
- **Built in ~1/10th the time** it would have taken humans
- The product has **internal daily users and external alpha testers**
- It **ships, deploys, breaks, and gets fixed** — all by agents within the harness
The engineers' job? Designing the harness. Specifying intent. Providing feedback. Not writing code.
---
## The Three Pillars of Harness Engineering
OpenAI's framework organizes harness engineering into three core categories:
### 1\. Context Engineering
Context engineering is about ensuring the agent has the right information at the right time.
**Static context:**
- Repository-local documentation (architecture specs, API contracts, style guides)
- `AGENTS.md` or `CLAUDE.md` files that encode project-specific rules
- Cross-linked design documents validated by linters
**Dynamic context:**
- Observability data (logs, metrics, traces) accessible to agents
- Directory structure mapping at agent startup
- CI/CD pipeline status and test results
**The critical rule:** From the agent's perspective, anything it can't access in-context doesn't exist. Knowledge in Google Docs, Slack threads, or people's heads is invisible to the system. **The repository must be the single source of truth.**
### 2\. Architectural Constraints
This is where harness engineering diverges most sharply from traditional AI prompting. Instead of telling the agent "write good code," you **mechanically enforce what good code looks like.**
**Dependency layering:**
```
Types → Config → Repo → Service → Runtime → UI
```
Each layer can only import from layers to its left. This isn't a suggestion — it's enforced by structural tests and CI validation.
**Constraint enforcement tools:**
- **Deterministic linters** — Custom rules that flag violations automatically
- **LLM-based auditors** — Agents that review other agents' code for architectural compliance
- **Structural tests** — Like ArchUnit, but for AI-generated code
- **Pre-commit hooks** — Automated checks before any code is committed
**Why constraints improve output:** Paradoxically, constraining the solution space makes agents **more productive**, not less. When an agent can generate anything, it wastes tokens exploring dead ends. When the harness defines clear boundaries, the agent converges faster on correct solutions.
### 3\. Entropy Management ("Garbage Collection")
This is the most underappreciated component. Over time, AI-generated codebases accumulate entropy — documentation drifts from reality, naming conventions diverge, dead code accumulates.
Harness engineering addresses this with **periodic cleanup agents:**
- **Documentation consistency agents** — Verify that docs match current code
- **Constraint violation scanners** — Find code that slipped past earlier checks
- **Pattern enforcement agents** — Identify and fix deviations from established patterns
- **Dependency auditors** — Track and resolve circular or unnecessary dependencies
These agents run on schedules — daily, weekly, or triggered by specific events — keeping the codebase healthy for both human reviewers and future AI agents.
---
## Harness Engineering in Practice: How Teams Actually Do It
### The OpenAI Approach: Zero Human Code
OpenAI's team structure for harness engineering:
| Role | Traditional | Harness Engineering |
| --- | --- | --- |
| Writing code | Primary job | Never |
| Designing architecture | Part of the job | Primary job |
| Writing documentation | Afterthought | Critical infrastructure |
| Reviewing PRs | Code review | Reviewing agent output + harness effectiveness |
| Debugging | Reading code | Analyzing agent behavior patterns |
| Testing | Writing tests | Designing test strategies agents execute |
### The Stripe Approach: Minions at Scale
Stripe's internal coding agents, called **Minions**, now produce **over 1,000 merged pull requests per week**:
1. Developer posts a task in Slack
2. Minion writes the code
3. Minion passes CI
4. Minion opens a PR
5. Human reviews and merges
No developer interaction between step 1 and step 5. The harness handles everything — test execution, CI validation, style compliance, and documentation updates.
### The LangChain Approach: Middleware-First
LangChain structures their harness as composable middleware layers:
```
Agent Request
→ LocalContextMiddleware (maps codebase)
→ LoopDetectionMiddleware (prevents repetition)
→ ReasoningSandwichMiddleware (optimizes compute)
→ PreCompletionChecklistMiddleware (enforces verification)
→ Agent Response
```
Each middleware layer adds a specific capability without modifying the core agent logic. This modular approach makes the harness testable and evolvable.
---
## Building Your First Harness: A Practical Framework
### Level 1: Basic Harness (Single Developer)
If you're using Claude Code, Cursor, or Codex for individual projects:
**What to set up:**
- `CLAUDE.md` or `.cursorrules` file with project conventions
- Pre-commit hooks for linting and formatting
- A test suite the agent can run to self-verify
- Clear directory structure with consistent naming
**Time to set up:** 1-2 hours **Impact:** Prevents the most common agent mistakes
### Level 2: Team Harness (Small Team)
For teams of 3-10 developers sharing a codebase:
**Add to Level 1:**
- `AGENTS.md` with team-wide conventions
- Architectural constraints enforced by CI
- Shared prompt templates for common tasks
- Documentation-as-code validated by linters
- Code review checklists specifically for agent-generated PRs
**Time to set up:** 1-2 days **Impact:** Consistent agent behavior across the team
### Level 3: Production Harness (Engineering Organization)
For organizations running dozens of concurrent agents:
**Add to Level 2:**
- Custom middleware layers (loop detection, reasoning optimization)
- Observability integration (agents read logs and metrics)
- Entropy management agents on scheduled runs
- Harness versioning and A/B testing
- Agent performance monitoring dashboards
- Escalation policies for when agents get stuck
**Time to set up:** 1-2 weeks **Impact:** Agents operate as autonomous contributors
---
## Common Harness Engineering Mistakes
### 1\. Over-Engineering the Control Flow
> *"If you over-engineer the control flow, the next model update will break your system."*
Models improve rapidly. Capabilities that required complex pipelines in 2024 are now handled by a single context-window prompt. Build your harness to be **rippable** — you should be able to remove "smart" logic when the model gets smart enough to not need it.
### 2\. Treating the Harness as Static
The harness needs to evolve with the model. When a new model release improves reasoning, your reasoning-optimization middleware might become counterproductive. Review and update harness components with every major model update.
### 3\. Ignoring the Documentation Layer
The most impactful harness improvement is often the simplest: **better documentation**. If your `AGENTS.md` is vague, your agent output will be vague. Invest in precise, machine-readable documentation that serves as the agent's ground truth.
### 4\. No Feedback Loop
A harness without feedback is a cage, not a guide. The agent needs to know when it's succeeding and when it's failing. Build in:
- Self-verification steps before task completion
- Test execution as part of the agent workflow
- Metrics on agent success rates by task type
### 5\. Human-Only Documentation
If your architectural decisions live in people's heads or in Confluence pages the agent can't access, the harness has a gap. **Everything the agent needs must be in the repository.**
---
## Harness Engineering vs. Related Concepts
| Concept | Scope | Focus |
| --- | --- | --- |
| **Prompt Engineering** | Single interaction | Crafting effective prompts |
| **Context Engineering** | Model context window | What information the model sees |
| **Harness Engineering** | Entire agent system | Environment, constraints, feedback, lifecycle |
| **Agent Engineering** | Agent architecture | Internal agent design and routing |
| **Platform Engineering** | Infrastructure | Deployment, scaling, operations |
Harness engineering **includes** context engineering and draws from prompt engineering, but it operates at a higher level — it's about the complete system that makes agents reliable, not just the inputs to a single interaction.
---
## What This Means for Software Engineers
### The Job Is Changing
Harness engineering represents a genuine evolution in what software engineers do:
| Before | After |
| --- | --- |
| Write code | Design environments where AI writes code |
| Debug code | Debug agent behavior |
| Review code | Review agent output + harness effectiveness |
| Write tests | Design test strategies |
| Maintain docs | Build documentation as machine-readable infrastructure |
This doesn't mean engineers become less technical. If anything, harness engineering requires **deeper** architectural thinking — you're designing systems that must work without your constant intervention.
### The Skills That Matter
Based on what we've seen building AI-powered products at [NxCode](https://www.nxcode.io/):
1. **Systems thinking** — Understanding how constraints, feedback loops, and documentation interact
2. **Architecture design** — Defining boundaries that are enforceable and productive
3. **Specification writing** — Articulating intent precisely enough for agents to execute
4. **Observability** — Building monitoring that reveals agent behavior patterns
5. **Iteration speed** — Rapidly testing and refining harness configurations
### Our Experience: What Works in Practice
We've been building AI-powered web applications using multiple agent systems (Claude Code, Codex, Cursor). The patterns that have made the biggest difference for us:
- **Repository-first documentation**: Every architectural decision, naming convention, and deployment process is in the repo. Nothing lives in Slack or Google Docs.
- **Incremental constraint building**: Start with basic linting, add architectural constraints as patterns emerge, don't try to design the perfect harness upfront.
- **Agent-specific review checklists**: AI-generated code has different failure modes than human code. Our review process accounts for common agent patterns (over-abstraction, unnecessary error handling, documentation drift).
- **Multi-provider harness design**: Our harness works with Claude, GPT, and Gemini models. Provider-agnostic design means we can switch models without rebuilding the entire system.
---
## Key Takeaways
1. **Harness engineering is the new discipline** of designing systems that make AI agents reliable — constraints, feedback loops, documentation, and lifecycle management
2. **The model is commodity; the harness is moat** — LangChain jumped from Top 30 to Top 5 on benchmarks by only changing the harness
3. **OpenAI built 1M+ lines with zero human code** — proving harness engineering works at production scale
4. **Three pillars**: Context engineering, architectural constraints, and entropy management
5. **Start simple**: A good `AGENTS.md` and pre-commit hooks are more impactful than complex middleware
6. **The engineer's job is evolving** — from writing code to designing environments where AI writes code
7. **Build rippable harnesses** — over-engineering breaks when models improve; keep it adaptable
---
## Related Articles
- [Best AI for Coding in 2026: 10 Tools Ranked by Real-World Performance](https://www.nxcode.io/resources/news/best-ai-for-coding-2026-complete-ranking)
- [OpenAI Frontier Guide: Enterprise AI Agent Platform for Building AI Coworkers (2026)](https://www.nxcode.io/resources/news/openai-frontier-enterprise-ai-agent-platform-guide-2026)
- [Cursor Tutorial 2026: Learn AI Coding in 15 Minutes (Beginner Guide)](https://www.nxcode.io/resources/news/cursor-tutorial-beginners-2026)
[Back to all news](https://www.nxcode.io/resources/news)
Enjoyed this article?
@@ -1,5 +1,13 @@
# Harness design for long-running application development ---
title: "Harness design for long-running application development"
source: "https://www.anthropic.com/engineering/harness-design-long-running-apps"
author:
published:
created: 2026-04-11
description: "Anthropic is an AI safety and research company that's working to build reliable, interpretable, and steerable AI systems."
tags:
- "clippings"
---
*Written by Prithvi Rajasekaran, a member of our [Labs](https://www.anthropic.com/news/introducing-anthropic-labs) team.* *Written by Prithvi Rajasekaran, a member of our [Labs](https://www.anthropic.com/news/introducing-anthropic-labs) team.*
Over the past several months Ive been working on two interconnected problems: getting Claude to produce high-quality frontend designs, and getting it to build complete applications without human intervention. This work originated with earlier efforts on our [frontend design skill](https://github.com/anthropics/claude-code/blob/main/plugins/frontend-design/skills/frontend-design/SKILL.md) and [long-running coding agent harness](https://www.anthropic.com/engineering/effective-harnesses-for-long-running-agents), where my colleagues and I were able to improve Claudes performance well above baseline through prompt engineering and harness design—but both eventually hit ceilings. Over the past several months Ive been working on two interconnected problems: getting Claude to produce high-quality frontend designs, and getting it to build complete applications without human intervention. This work originated with earlier efforts on our [frontend design skill](https://github.com/anthropics/claude-code/blob/main/plugins/frontend-design/skills/frontend-design/SKILL.md) and [long-running coding agent harness](https://www.anthropic.com/engineering/effective-harnesses-for-long-running-agents), where my colleagues and I were able to improve Claudes performance well above baseline through prompt engineering and harness design—but both eventually hit ceilings.
@@ -10,7 +18,7 @@ I then applied these techniques to long-running autonomous coding, carrying over
## Why naive implementations fall short ## Why naive implementations fall short
We've previously shown that harness design has a substantial impact on the effectiveness of long running agentic coding. In an earlier [experiment](https://www.anthropic.com/engineering/effective-harnesses-for-long-running-agents), we used an initializer agent to decompose a product spec into a task list, and a coding agent that implemented the tasks one feature at a time before handing off artifacts to carry context across sessions. The broader developer community has converged on similar insights, with approaches like the "[Ralph Wiggum](https://ghuntley.com/ralph/)" method using hooks or scripts to keep agents in continuous iteration cycles. We've previously shown that harness design has a substantial impact on the effectiveness of long running agentic coding. In an earlier [experiment](https://www.anthropic.com/engineering/effective-harnesses-for-long-running-agents), we used an initializer agent to decompose a product spec into a task list, and a coding agent that implemented the tasks one feature at a time before handing off artifacts to carry context across sessions. The broader developer community has converged on similar insights, with approaches like the " [Ralph Wiggum](https://ghuntley.com/ralph/) " method using hooks or scripts to keep agents in continuous iteration cycles.
But some problems remained persistent. For more complex tasks, the agent still tends to go off the rails over time. While decomposing this issue, we observed two common failure modes with agents executing these sorts of tasks. But some problems remained persistent. For more complex tasks, the agent still tends to go off the rails over time. While decomposing this issue, we observed two common failure modes with agents executing these sorts of tasks.
@@ -30,10 +38,10 @@ Two insights shaped the harness I built for frontend design. First, while aesthe
With this in mind, I wrote four grading criteria that I gave to both the generator and evaluator agents in their prompts: With this in mind, I wrote four grading criteria that I gave to both the generator and evaluator agents in their prompts:
- **Design quality:** Does the design feel like a coherent whole rather than a collection of parts? Strong work here means the colors, typography, layout, imagery, and other details combine to create a distinct mood and identity. - **Design quality:** Does the design feel like a coherent whole rather than a collection of parts? Strong work here means the colors, typography, layout, imagery, and other details combine to create a distinct mood and identity.
- **Originality:** Is there evidence of custom decisions, or is this template layouts, library defaults, and AI-generated patterns? A human designer should recognize deliberate creative choices. Unmodified stock components—or telltale signs of AI generation like purple gradients over white cards—fail here. - **Originality:** Is there evidence of custom decisions, or is this template layouts, library defaults, and AI-generated patterns? A human designer should recognize deliberate creative choices. Unmodified stock components—or telltale signs of AI generation like purple gradients over white cards—fail here.
- **Craft:** Technical execution: typography hierarchy, spacing consistency, color harmony, contrast ratios. This is a competence check rather than a creativity check. Most reasonable implementations do fine here by default; failing means broken fundamentals. - **Craft:** Technical execution: typography hierarchy, spacing consistency, color harmony, contrast ratios. This is a competence check rather than a creativity check. Most reasonable implementations do fine here by default; failing means broken fundamentals.
- **Functionality:** Usability independent of aesthetics. Can users understand what the interface does, find primary actions, and complete tasks without guessing? - **Functionality:** Usability independent of aesthetics. Can users understand what the interface does, find primary actions, and complete tasks without guessing?
I emphasized design quality and originality over craft and functionality. Claude already scored well on craft and functionality by default, as the required technical competence tended to come naturally to the model. But on design and originality, Claude often produced outputs that were bland at best. The criteria explicitly penalized highly generic “AI slop” patterns, and by weighting design and originality more heavily it pushed the model toward more aesthetic risk-taking. I emphasized design quality and originality over craft and functionality. Claude already scored well on craft and functionality by default, as the required technical competence tended to come naturally to the model. But on design and originality, Claude often produced outputs that were bland at best. The criteria explicitly penalized highly generic “AI slop” patterns, and by weighting design and originality more heavily it pushed the model toward more aesthetic risk-taking.
@@ -49,13 +57,15 @@ While scores generally improved over iterations, the pattern was not always clea
In one notable example, I prompted the model to create a website for a Dutch art museum. By the ninth iteration, it had produced a clean, dark-themed landing page for a fictional museum. The page was visually polished but largely in line with my expectations. Then, on the tenth cycle, it scrapped the approach entirely and reimagined the site as a spatial experience: a 3D room with a checkered floor rendered in CSS perspective, artwork hung on the walls in free-form positions, and doorway-based navigation between gallery rooms instead of scroll or click. It was the kind of creative leap that I hadn't seen before from a single-pass generation. In one notable example, I prompted the model to create a website for a Dutch art museum. By the ninth iteration, it had produced a clean, dark-themed landing page for a fictional museum. The page was visually polished but largely in line with my expectations. Then, on the tenth cycle, it scrapped the approach entirely and reimagined the site as a spatial experience: a 3D room with a checkered floor rendered in CSS perspective, artwork hung on the walls in free-form positions, and doorway-based navigation between gallery rooms instead of scroll or click. It was the kind of creative leap that I hadn't seen before from a single-pass generation.
<video controls="" src="https://cdn.sanity.io/files/4zrzovbb/website/9877febd34432f7f582aecd0023b951223605c6a.mp4"></video>
## Scaling to full-stack coding ## Scaling to full-stack coding
With these findings in hand, I applied this GAN-inspired pattern to full-stack development. The generator-evaluator loop maps naturally onto the software development lifecycle, where code review and QA serve the same structural role as the design evaluator. With these findings in hand, I applied this GAN-inspired pattern to full-stack development. The generator-evaluator loop maps naturally onto the software development lifecycle, where code review and QA serve the same structural role as the design evaluator.
### The architecture ### The architecture
In our earlier [long-running harness](https://www.anthropic.com/engineering/effective-harnesses-for-long-running-agents), we had solved for coherent multi-session coding with an initializer agent, a coding agent that worked one feature at a time, and context resets between sessions. Context resets were a key unlock: the harness used Sonnet 4.5, which exhibited the “context anxiety” tendency mentioned earlier. Creating a harness that worked well across context resets was key to keeping the model on task. Opus 4.5 largely removed that behavior on its own, so I was able to drop context resets from this harness entirely. The agents were run as one continuous session across the whole build, with the [Claude Agent SDK](https://platform.claude.com/docs/en/agent-sdk/overview)'s automatic compaction handling context growth along the way. In our earlier [long-running harness](https://www.anthropic.com/engineering/effective-harnesses-for-long-running-agents), we had solved for coherent multi-session coding with an initializer agent, a coding agent that worked one feature at a time, and context resets between sessions. Context resets were a key unlock: the harness used Sonnet 4.5, which exhibited the “context anxiety” tendency mentioned earlier. Creating a harness that worked well across context resets was key to keeping the model on task. Opus 4.5 largely removed that behavior on its own, so I was able to drop context resets from this harness entirely. The agents were run as one continuous session across the whole build, with the [Claude Agent SDK](https://platform.claude.com/docs/en/agent-sdk/overview) 's automatic compaction handling context growth along the way.
For this work I built on the foundation from the original harness with a three-agent system, with each agent addressing a specific gap I'd observed in prior runs. The system contained the following agent personas: For this work I built on the foundation from the original harness with a three-agent system, with each agent addressing a specific gap I'd observed in prior runs. The system contained the following agent personas:
@@ -63,8 +73,8 @@ For this work I built on the foundation from the original harness with a three-a
**Generator:** The one-feature-at-a-time approach from the earlier harness worked well for scope management. I applied a similar model here, instructing the generator to work in sprints, picking up one feature at a time from the spec. Each sprint implemented the app with a React, Vite, FastAPI, and SQLite (later PostgreSQL) stack, and the generator was instructed to self-evaluate its work at the end of each sprint before handing off to QA. It also had git for version control. **Generator:** The one-feature-at-a-time approach from the earlier harness worked well for scope management. I applied a similar model here, instructing the generator to work in sprints, picking up one feature at a time from the spec. Each sprint implemented the app with a React, Vite, FastAPI, and SQLite (later PostgreSQL) stack, and the generator was instructed to self-evaluate its work at the end of each sprint before handing off to QA. It also had git for version control.
**Evaluator:** Applications from earlier harnesses often looked impressive but still had real bugs when you actually tried to use them. To catch these, the evaluator used the Playwright MCP to click through the running application the way a user would, testing UI features, API endpoints, and database states. It then graded each sprint against both the bugs it had found and a set of criteria modeled on the frontend experiment, adapted here to cover product depth, functionality, visual design, and code quality. Each criterion had a hard threshold, and if any one fell below it, the sprint failed and the generator got detailed feedback on what went wrong. **Evaluator:** Applications from earlier harnesses often looked impressive but still had real bugs when you actually tried to use them. To catch these, the evaluator used the Playwright MCP to click through the running application the way a user would, testing UI features, API endpoints, and database states. It then graded each sprint against both the bugs it had found and a set of criteria modeled on the frontend experiment, adapted here to cover product depth, functionality, visual design, and code quality. Each criterion had a hard threshold, and if any one fell below it, the sprint failed and the generator got detailed feedback on what went wrong.
Before each sprint, the generator and evaluator negotiated a sprint contract: agreeing on what "done" looked like for that chunk of work before any code was written. This existed because the product spec was intentionally high-level, and I wanted a step to bridge the gap between user stories and testable implementation. The generator proposed what it would build and how success would be verified, and the evaluator reviewed that proposal to make sure the generator was building the right thing. The two iterated until they agreed. Before each sprint, the generator and evaluator negotiated a sprint contract: agreeing on what "done" looked like for that chunk of work before any code was written. This existed because the product spec was intentionally high-level, and I wanted a step to bridge the gap between user stories and testable implementation. The generator proposed what it would build and how success would be verified, and the evaluator reviewed that proposal to make sure the generator was building the right thing. The two iterated until they agreed.
Communication was handled via files: one agent would write a file, another agent would read it and respond either within that file or with a new file that the previous agent would read in turn. The generator then built against the agreed-upon contract before handing the work off to QA. This kept the work faithful to the spec without over-specifying implementation too early. Communication was handled via files: one agent would write a file, another agent would read it and respond either within that file or with a new file that the previous agent would read in turn. The generator then built against the agreed-upon contract before handing the work off to QA. This kept the work faithful to the spec without over-specifying implementation too early.
@@ -79,23 +89,10 @@ I wrote the following prompt to generate a retro video game maker:
The table below shows the harness type, length it ran for, and the total cost. The table below shows the harness type, length it ran for, and the total cost.
**Harness** | **Harness** | **Duration** | **Cost** |
| --- | --- | --- |
**Duration** | Solo | 20 min | $9 |
| Full harness | 6 hr | $200 |
**Cost**
Solo
20 min
$9
Full harness
6 hr
$200
The harness was over 20x more expensive, but the difference in output quality was immediately apparent. The harness was over 20x more expensive, but the difference in output quality was immediately apparent.
@@ -103,6 +100,12 @@ I was expecting an interface where I could construct a level and its component p
As I clicked through, however, issues started to emerge. The layout wasted space, with fixed-height panels leaving most of the viewport empty. The workflow was rigid. Trying to populate a level prompted me to create sprites and entities first, but nothing in the UI guided me toward that sequence. More to the point, the actual game was broken. My entities appeared on screen but nothing responded to input. Digging into the code revealed that the wiring between entity definitions and the game runtime was broken, with no surface indication of where. As I clicked through, however, issues started to emerge. The layout wasted space, with fixed-height panels leaving most of the viewport empty. The workflow was rigid. Trying to populate a level prompted me to create sprites and entities first, but nothing in the UI guided me toward that sequence. More to the point, the actual game was broken. My entities appeared on screen but nothing responded to input. Digging into the code revealed that the wiring between entity definitions and the game runtime was broken, with no surface indication of where.
![[23c98f1d7ae720bfb39190d50e0706c03b177ad8-1999x1320.webp]]
Initial screen when opening the app created by the solo harness.
After evaluating the solo run, I turned my attention to the harness run. This run started from the same one-sentence prompt, but the planner step expanded that prompt into a 16-feature spec spread across ten sprints. It went well beyond what the solo run attempted. In addition to the core editors and play mode, the spec called for a sprite animation system, behavior templates, sound effects and music, an AI-assisted sprite generator and level designer, and game export with shareable links. I gave the planner access to our [frontend design skill](https://github.com/anthropics/claude-code/blob/main/plugins/frontend-design/skills/frontend-design/SKILL.md), which it read and used to create a visual design language for the app as part of the spec. For each sprint, the generator and evaluator negotiated a contract defining the specific implementation details for the sprint, and the testable behaviors that would be tested to verify completion. After evaluating the solo run, I turned my attention to the harness run. This run started from the same one-sentence prompt, but the planner step expanded that prompt into a 16-feature spec spread across ten sprints. It went well beyond what the solo run attempted. In addition to the core editors and play mode, the spec called for a sprite animation system, behavior templates, sound effects and music, an AI-assisted sprite generator and level designer, and game export with shareable links. I gave the planner access to our [frontend design skill](https://github.com/anthropics/claude-code/blob/main/plugins/frontend-design/skills/frontend-design/SKILL.md), which it read and used to create a visual design language for the app as part of the spec. For each sprint, the generator and evaluator negotiated a contract defining the specific implementation details for the sprint, and the testable behaviors that would be tested to verify completion.
The app immediately showed more polish and smoothness than the solo run. The canvas used the full viewport, the panels were sized sensibly, and the interface had a consistent visual identity that tracked the design direction from the spec. Some of the clunkiness I'd seen in the solo run did remain—the workflow still didn't make it clear that you should build sprites and entities before trying to populate a level, and I had to figure that out by poking around. This read as a gap in the base models product intuition rather than something the harness was designed to address, though it did suggest a place where targeted iteration inside the harness could help to further improve output quality. The app immediately showed more polish and smoothness than the solo run. The canvas used the full viewport, the panels were sized sensibly, and the interface had a consistent visual identity that tracked the design direction from the spec. Some of the clunkiness I'd seen in the solo run did remain—the workflow still didn't make it clear that you should build sprites and entities before trying to populate a level, and I had to figure that out by poking around. This read as a gap in the base models product intuition rather than something the harness was designed to address, though it did suggest a place where targeted iteration inside the harness could help to further improve output quality.
@@ -111,30 +114,23 @@ Working through the editors, the new run's advantages over solo became more appa
Because I'd asked the planner to weave AI features into its specs, the app also came with a built-in Claude integration that let me generate different parts of the game through prompting. This significantly sped up the workflow. Because I'd asked the planner to weave AI features into its specs, the app also came with a built-in Claude integration that let me generate different parts of the game through prompting. This significantly sped up the workflow.
![[a8bef95425966495629095a5cb38bde4a8b13558-1999x997.webp]]
Initial screen: Creating a new game, in the app built with the full harness
The biggest difference was in play mode. I was actually able to move my entity and play the game. The physics had some rough edges—my character jumped onto a platform but ended up overlapping with it, which felt intuitively wrong—but the core thing worked, which the solo run did not manage. After moving around a bit, I did hit some limitations with the AIs game level construction. There was a large wall that I wasnt able to jump past, so I was stuck. This suggested there were some common sense improvements and edge cases that the harness could handle to further refine the app. The biggest difference was in play mode. I was actually able to move my entity and play the game. The physics had some rough edges—my character jumped onto a platform but ended up overlapping with it, which felt intuitively wrong—but the core thing worked, which the solo run did not manage. After moving around a bit, I did hit some limitations with the AIs game level construction. There was a large wall that I wasnt able to jump past, so I was stuck. This suggested there were some common sense improvements and edge cases that the harness could handle to further refine the app.
Reading through the logs, it was clear that the evaluator kept the implementation in line with the spec. Each sprint, it walked through the sprint contract's test criteria and exercised the running application through Playwright, filing bugs against anything that diverged from expected behavior. The contracts were granular—Sprint 3 alone had 27 criteria covering the level editor—and the evaluator's findings were specific enough to act on without extra investigation. The table below shows several examples of issues our evaluator identified: Reading through the logs, it was clear that the evaluator kept the implementation in line with the spec. Each sprint, it walked through the sprint contract's test criteria and exercised the running application through Playwright, filing bugs against anything that diverged from expected behavior. The contracts were granular—Sprint 3 alone had 27 criteria covering the level editor—and the evaluator's findings were specific enough to act on without extra investigation. The table below shows several examples of issues our evaluator identified:
**Contract criterion** | **Contract criterion** | **Evaluator finding** |
| --- | --- |
**Evaluator finding** | Rectangle fill tool allows click-drag to fill a rectangular area with selected tile | **FAIL** — Tool only places tiles at drag start/end points instead of filling the region. `fillRectangle` function exists but isn't triggered properly on mouseUp. |
| User can select and delete placed entity spawn points | **FAIL** — Delete key handler at `LevelEditor.tsx:892` requires both `selection` and `selectedEntityId ` to be set, but clicking an entity only sets `selectedEntityId`. Condition should be `selection \|\| (selectedEntityId && activeLayer === 'entity')`. |
Rectangle fill tool allows click-drag to fill a rectangular area with selected tile | User can reorder animation frames via API | **FAIL**`PUT /frames/reorder` route defined after `/{frame_id}` routes. FastAPI matches 'r `eorder` ' as a frame\_id integer and returns 422: "unable to parse string as an integer." |
**FAIL** — Tool only places tiles at drag start/end points instead of filling the region. `fillRectangle` function exists but isn't triggered properly on mouseUp.
User can select and delete placed entity spawn points
**FAIL** — Delete key handler at `LevelEditor.tsx:892` requires both `selection` and `selectedEntityId` to be set, but clicking an entity only sets `selectedEntityId`. Condition should be `selection || (selectedEntityId && activeLayer === 'entity')`.
User can reorder animation frames via API
**FAIL** — `PUT /frames/reorder` route defined after `/{frame_id}` routes. FastAPI matches 'r`eorder`' as a frame\_id integer and returns 422: "unable to parse string as an integer."
Getting the evaluator to perform at this level took work. Out of the box, Claude is a poor QA agent. In early runs, I watched it identify legitimate issues, then talk itself into deciding they weren't a big deal and approve the work anyway. It also tended to test superficially, rather than probing edge cases, so more subtle bugs often slipped through. The tuning loop was to read the evaluator's logs, find examples where its judgment diverged from mine, and update the QAs prompt to solve for those issues. It took several rounds of this development loop before the evaluator was grading in a way that I found reasonable. Even then, the harness output showed the limits of the models QAing capabilities: small layout issues, interactions that felt unintuitive in places, and undiscovered bugs in more deeply nested features that the evaluator hadn't exercised thoroughly. There was clearly more verification headroom to capture with further tuning. But compared to the solo run, where the central feature of the application simply didn't work, the lift was obvious. Getting the evaluator to perform at this level took work. Out of the box, Claude is a poor QA agent. In early runs, I watched it identify legitimate issues, then talk itself into deciding they weren't a big deal and approve the work anyway. It also tended to test superficially, rather than probing edge cases, so more subtle bugs often slipped through. The tuning loop was to read the evaluator's logs, find examples where its judgment diverged from mine, and update the QAs prompt to solve for those issues. It took several rounds of this development loop before the evaluator was grading in a way that I found reasonable. Even then, the harness output showed the limits of the models QAing capabilities: small layout issues, interactions that felt unintuitive in places, and undiscovered bugs in more deeply nested features that the evaluator hadn't exercised thoroughly. There was clearly more verification headroom to capture with further tuning. But compared to the solo run, where the central feature of the application simply didn't work, the lift was obvious.
### ### Iterating on the harness
Iterating on the harness
The first set of harness results was encouraging, but it was also bulky, slow, and expensive. The logical next step was to find ways to simplify the harness without degrading its performance. This was partly common sense and partly a function of a more general principle: every component in a harness encodes an assumption about what the model can't do on its own, and those assumptions are worth stress testing, both because they may be incorrect, and because they can quickly go stale as models improve. Our blog post [Building Effective Agents](https://www.anthropic.com/research/building-effective-agents) frames the underlying idea as "find the simplest solution possible, and only increase complexity when needed," and it's a pattern that shows up consistently for anyone maintaining an agent harness. The first set of harness results was encouraging, but it was also bulky, slow, and expensive. The logical next step was to find ways to simplify the harness without degrading its performance. This was partly common sense and partly a function of a more general principle: every component in a harness encodes an assumption about what the model can't do on its own, and those assumptions are worth stress testing, both because they may be incorrect, and because they can quickly go stale as models improve. Our blog post [Building Effective Agents](https://www.anthropic.com/research/building-effective-agents) frames the underlying idea as "find the simplest solution possible, and only increase complexity when needed," and it's a pattern that shows up consistently for anyone maintaining an agent harness.
@@ -164,59 +160,16 @@ The run was still lengthy and expensive, at about 4 hours and $124 in token cost
Most of the time went to the builder, which ran coherently for over two hours without the sprint decomposition that Opus 4.5 had needed. Most of the time went to the builder, which ran coherently for over two hours without the sprint decomposition that Opus 4.5 had needed.
**Agent & Phase** | **Agent & Phase** | **Duration** | **Cost** |
| --- | --- | --- |
**Duration** | Planner | 4.7 min | $0.46 |
| Build (Round 1) | 2 hr 7 min | $71.08 |
**Cost** | QA (Round 1) | 8.8 min | $3.24 |
| Build (Round 2) | 1 hr 2 min | $36.89 |
Planner | QA (Round 2) | 6.8 min | $3.09 |
| Build (Round 3) | 10.9 min | $5.88 |
4.7 min | QA (Round 3) | 9.6 min | $4.06 |
| **Total V2 Harness** | **3 hr 50 min** | **$124.70** |
$0.46
Build (Round 1)
2 hr 7 min
$71.08
QA (Round 1)
8.8 min
$3.24
Build (Round 2)
1 hr 2 min
$36.89
QA (Round 2)
6.8 min
$3.09
Build (Round 3)
10.9 min
$5.88
QA (Round 3)
9.6 min
$4.06
**Total V2 Harness**
**3 hr 50 min**
**$124.70**
As with the previous harness, the planner expanded the one-line prompt into a full spec. From the logs, I could see the generator model did a good job planning the app and the agent design, wiring the agent up, and testing it before handing off to QA. As with the previous harness, the planner expanded the one-line prompt into a full spec. From the logs, I could see the generator model did a good job planning the app and the agent design, wiring the agent up, and testing it before handing off to QA.
@@ -235,6 +188,8 @@ The generator was still liable to miss details or stub features when left to its
Based on the prompt, I was expecting a program where I could create melodies, harmonies, and drum patterns, arrange them into a song, and get help from an integrated agent along the way. The video below shows the result. Based on the prompt, I was expecting a program where I could create melodies, harmonies, and drum patterns, arrange them into a song, and get help from an integrated agent along the way. The video below shows the result.
<video controls="" src="https://cdn.sanity.io/files/4zrzovbb/website/555910f9adb3938734940224e7a6f4c7cbbbd8f2.mp4"></video>
The app is far from a professional music production program, and the agent's song composition skills could clearly use a lot of work. Additionally, Claude cant actually hear, which made the QA feedback loop less effective with respect to musical taste. The app is far from a professional music production program, and the agent's song composition skills could clearly use a lot of work. Additionally, Claude cant actually hear, which made the QA feedback loop less effective with respect to musical taste.
But the final app had all the core pieces of a functional music production program: a working arrangement view, mixer, and transport running in the browser. Beyond that, I was able to put together a short song snippet entirely through prompting: the agent set the tempo and key, laid down a melody, built a drum track, adjusted mixer levels, and added reverb. The core primitives for song composition were present, and the agent could drive them autonomously, using tools to create a simple production from end to end. You might say its not pitch-perfect yet—but its getting there. But the final app had all the core pieces of a functional music production program: a working arrangement view, mixer, and transport running in the browser. Beyond that, I was able to put together a short song snippet entirely through prompting: the agent set the tempo and key, laid down a melody, built a drum track, adjusted mixer levels, and added reverb. The core primitives for song composition were present, and the agent could drive them autonomously, using tools to create a simple production from end to end. You might say its not pitch-perfect yet—but its getting there.
@@ -247,15 +202,13 @@ With this in mind, there are a few lessons from this work worth carrying forward
From this work, my conviction is that the space of interesting harness combinations doesn't shrink as models improve. Instead, it moves, and the interesting work for AI engineers is to keep finding the next novel combination. From this work, my conviction is that the space of interesting harness combinations doesn't shrink as models improve. Instead, it moves, and the interesting work for AI engineers is to keep finding the next novel combination.
## ## Acknowledgements
Acknowledgements
Special thanks to Mike Krieger, Michael Agaby, Justin Young, Jeremy Hadfield, David Hershey, Julius Tarng, Xiaoyi Zhang, Barry Zhang, Orowa Sidker, Michael Tingley, Ibrahim Madha, Martina Long, and Canyon Robbins for their contributions to this work. Special thanks to Mike Krieger, Michael Agaby, Justin Young, Jeremy Hadfield, David Hershey, Julius Tarng, Xiaoyi Zhang, Barry Zhang, Orowa Sidker, Michael Tingley, Ibrahim Madha, Martina Long, and Canyon Robbins for their contributions to this work.
Thanks also to Jake Eaton, Alyssa Leonard, and Stef Sequeira for their help shaping the post. Thanks also to Jake Eaton, Alyssa Leonard, and Stef Sequeira for their help shaping the post.
## ## Appendix
Appendix
Example plan generated by planner agent. Example plan generated by planner agent.
@@ -290,5 +243,4 @@ Color palette selection
All associated sprites, tilesets, levels, and entity definitions All associated sprites, tilesets, levels, and entity definitions
... ...
``` ```
@@ -1,80 +1,43 @@
# Harness engineering for coding agent users ---
title: "Harness engineering for coding agent users"
source: "https://martinfowler.com/articles/harness-engineering.html"
author:
- "[[Birgitta Böckeler Birgitta is a Distinguished Engineer and AI-assisted delivery expert at Thoughtworks. She has over 20 years of experience as a software developer]]"
- "[[architect and technical leader.]]"
published: 2026-04-02
created: 2026-04-11
description: "A mental model for building trust in coding agents through feedforward guides, feedback sensors, and iterative harness engineering."
tags:
- "clippings"
---
The term harness has emerged as a shorthand to mean everything in an AI agent except the model itself - [Agent = Model + Harness](https://blog.langchain.com/the-anatomy-of-an-agent-harness/). That is a very wide definition, and therefore worth narrowing down for common categories of agents. I want to take the liberty here of defining its meaning in the bounded context of using a coding agent. In coding agents, part of the harness is already built in (e.g. via the system prompt, or the chosen code retrieval mechanism, or even a [sophisticated orchestration system](https://www.anthropic.com/engineering/effective-harnesses-for-long-running-agents)). But coding agents also provide us, their users, with many features to build an outer harness specifically for our use case and system. The term harness has emerged as a shorthand to mean everything in an AI agent except the model itself - [Agent = Model + Harness](https://blog.langchain.com/the-anatomy-of-an-agent-harness/). That is a very wide definition, and therefore worth narrowing down for common categories of agents. I want to take the liberty here of defining its meaning in the bounded context of using a coding agent. In coding agents, part of the harness is already built in (e.g. via the system prompt, or the chosen code retrieval mechanism, or even a [sophisticated orchestration system](https://www.anthropic.com/engineering/effective-harnesses-for-long-running-agents)). But coding agents also provide us, their users, with many features to build an outer harness specifically for our use case and system.
![Three concentric circles, with the model in the core (the ultimate thing being harnessed), then the coding agent's builder harness next circle out, and the coding agent's user harness as the outermost ring](../../martinfowler-assets/harness-bounded-contexts.png) ![[harness-bounded-contexts.png|Three concentric circles, with the model in the core (the ultimate thing being harnessed), then the coding agent's builder harness next circle out, and the coding agent's user harness as the outermost ring]]
Figure 1: The term “harness” means different things depending on the bounded context. Figure 1: The term “harness” means different things depending on the bounded context.
A well-built outer harness serves two goals: it increases the probability that the agent gets it right in the first place, and it provides a feedback loop that self-corrects as many issues as possible before they even reach human eyes. Ultimately it should reduce the review toil and increase the system quality, all with the added benefit of fewer wasted tokens along the way. A well-built outer harness serves two goals: it increases the probability that the agent gets it right in the first place, and it provides a feedback loop that self-corrects as many issues as possible before they even reach human eyes. Ultimately it should reduce the review toil and increase the system quality, all with the added benefit of fewer wasted tokens along the way.
![Title "Harness engineering for coding agent users". Overview of guides (examples shown are \[inferential\] principles, CfRs, Rules, Ref Docs, How-tos; \[computational\] Language Servers, CLIs, scripts, codemods) that feedforward into a coding agent; and feedback sensors (examples shown are \[inferential\] review agents; \[computational\] static analysis, logs, browser). The feedback sensors point at the coding agent as well as input into its self-correcting loop. On the left side of it all we see a box with a human who steers both the guides and sensors.](../../martinfowler-assets/harness-overview.png) ![[harness-overview.png|Title "Harness engineering for coding agent users". Overview of guides (examples shown are [inferential] principles, CfRs, Rules, Ref Docs, How-tos; [computational] Language Servers, CLIs, scripts, codemods) that feedforward into a coding agent; and feedback sensors (examples shown are [inferential] review agents; [computational] static analysis, logs, browser). The feedback sensors point at the coding agent as well as input into its self-correcting loop. On the left side of it all we see a box with a human who steers both the guides and sensors.]]
## Feedforward and Feedback
To harness a coding agent we both anticipate unwanted outputs and try to prevent them, and we put sensors in place to allow the agent to self-correct:
- **Guides (feedforward controls)** - anticipate the agent's behaviour and aim to steer it *before* it acts. Guides increase the probability that the agent creates good results in the first attempt
- **Sensors (feedback controls)** - observe *after* the agent acts and help it self-correct. Particularly powerful when they produce signals that are optimised for LLM consumption, e.g. custom linter messages that include instructions for the self-correction - a positive kind of prompt injection.
Separately, you get either an agent that keeps repeating the same mistakes (feedback-only) or an agent that encodes rules but never finds out whether they worked (feed-forward-only).
## Computational vs Inferential ## Computational vs Inferential
There are two execution types of guides and sensors: There are two execution types of guides and sensors:
- **Computational** - deterministic and fast, run by the CPU. Tests, linters, type checkers, structural analysis. Run in milliseconds to seconds; results are reliable. - **Computational** - deterministic and fast, run by the CPU. Tests, linters, type checkers, structural analysis. Run in milliseconds to seconds; results are reliable.
- **Inferential** - Semantic analysis, AI code review, “LLM as judge”. Typically run by a GPU or NPU. Slower and more expensive; results are more non-deterministic. - **Inferential** - Semantic analysis, AI code review, “LLM as judge”. Typically run by a GPU or NPU. Slower and more expensive; results are more non-deterministic.
Computational guides increase the probability of good results with deterministic tooling. Computational sensors are cheap and fast enough to run on every change, alongside the agent. Inferential controls are of course more expensive and non-deterministic, but allow us to both provide rich guidance, and add additional semantic judgment. In spite of their non-determinism, inferential sensors can particularly increase our trust when used with a strong model, or rather a model that is suitable to the task at hand. Computational guides increase the probability of good results with deterministic tooling. Computational sensors are cheap and fast enough to run on every change, alongside the agent. Inferential controls are of course more expensive and non-deterministic, but allow us to both provide rich guidance, and add additional semantic judgment. In spite of their non-determinism, inferential sensors can particularly increase our trust when used with a strong model, or rather a model that is suitable to the task at hand.
**Examples** **Examples**
Direction | | Direction | Computational / Inferential | Example implementations |
| --- | --- | --- | --- |
Computational / Inferential | Coding conventions | feedforward | Inferential | AGENTS.md, Skills |
| Instructions how to bootstrap a new project | feedforward | Both | Skill with instructions and a bootstrap script |
Example implementations | Code mods | feedforward | Computational | A tool with access to OpenRewrite recipes |
| Structural tests | feedback | Computational | A pre-commit (or coding agent) hook running ArchUnit tests that check for violations of module boundaries |
Coding conventions | Instructions how to review | feedback | Inferential | Skills |
feedforward
Inferential
AGENTS.md, Skills
Instructions how to bootstrap a new project
feedforward
Both
Skill with instructions and a bootstrap script
Code mods
feedforward
Computational
A tool with access to OpenRewrite recipes
Structural tests
feedback
Computational
A pre-commit (or coding agent) hook running ArchUnit tests that check for violations of module boundaries
Instructions how to review
feedback
Inferential
Skills
## The steering loop ## The steering loop
@@ -88,17 +51,14 @@ Teams who are [continuously integrating](https://martinfowler.com/articles/conti
**Feedforward and feedback in the change lifecycle** **Feedforward and feedback in the change lifecycle**
- What is reasonably fast and should be run even before integration, or even before a commit is even created? (e.g. linters, fast test suites, basic code review agent) - What is reasonably fast and should be run even before integration, or even before a commit is even created? (e.g. linters, fast test suites, basic code review agent)
- What is more expensive and should therefore only be run post-integration in the pipeline, in addition to a repetition of the fast controls? (e.g. mutation testing, a more broad code review that can take into account the bigger picture) - What is more expensive and should therefore only be run post-integration in the pipeline, in addition to a repetition of the fast controls? (e.g. mutation testing, a more broad code review that can take into account the bigger picture)
![[harness-change-lifecycle-examples.png|Examples of feedforward and feedback in a change's lifecycle. Feedforward: LSP, architecture.md, /how-to-test skill, AGENTS.md, MCP server that can access a team's knowledge management tool, /xyz-api-docs skill; they feed into the agent's initial generation; feedback sensor examples for first self-correction loop are /code-review, npx eslint, semgrep, npm run coverage, npm run dep-cruiser; then human review is an additional feedback sensor; then integration happens; after integration, examples shown in the pipeline, which reruns all the previous sensors, and additional examples for more expensive sensors are /architecture-review skill, /detailed-review skill, mutation testing. An arrow shows that the feedback can then lead to new commits by agents or humans.]]
![Examples of feedforward and feedback in a change's lifecycle. Feedforward: LSP, architecture.md, /how-to-test skill, AGENTS.md, MCP server that can access a team's knowledge management tool, /xyz-api-docs skill; they feed into the agent's initial generation; feedback sensor examples for first self-correction loop are /code-review, npx eslint, semgrep, npm run coverage, npm run dep-cruiser; then human review is an additional feedback sensor; then integration happens; after integration, examples shown in the pipeline, which reruns all the previous sensors, and additional examples for more expensive sensors are /architecture-review skill, /detailed-review skill, mutation testing. An arrow shows that the feedback can then lead to new commits by agents or humans.](../../martinfowler-assets/harness-change-lifecycle-examples.png)
**Continuous drift and health sensors** **Continuous drift and health sensors**
- What type of drift accumulates gradually and should be monitored by sensors running continuously against the codebase, outside the change lifecycle? (e.g. dead code detection, analysis of the quality of the test coverage, dependency scanners) - What type of drift accumulates gradually and should be monitored by sensors running continuously against the codebase, outside the change lifecycle? (e.g. dead code detection, analysis of the quality of the test coverage, dependency scanners)
- What runtime feedback could agents be monitoring? (e.g. having them look for degrading SLOs to make suggestions how to improve them, or AI judges continuously sampling response quality and flagging log anomalies) - What runtime feedback could agents be monitoring? (e.g. having them look for degrading SLOs to make suggestions how to improve them, or AI judges continuously sampling response quality and flagging log anomalies)
![Shows examples of continuous feedback sensors after change integration. Continuous drift detection in the codebase, e.g. /find-dead-code, /code-coverage-quality, dependabot; or Continuous runtime feedback, e.g. latency, error rate or availability SLOs leading to coding agent suggestions, or /response-quality-sampling, /log-anomalies AI judges.](../../martinfowler-assets/harness-continuous-feedback-examples.png)
## Regulation categories ## Regulation categories
@@ -124,21 +84,21 @@ This groups guides and sensors that define and check the architecture characteri
Examples: Examples:
- Skills that feed forward our performance requirements, and performance tests that feed back to the agent if it improved or degraded them. - Skills that feed forward our performance requirements, and performance tests that feed back to the agent if it improved or degraded them.
- Skills that describe coding conventions for better observability (like logging standards), and debugging instructions that ask the agent to reflect on the quality of the logs it had available. - Skills that describe coding conventions for better observability (like logging standards), and debugging instructions that ask the agent to reflect on the quality of the logs it had available.
### Behaviour harness ### Behaviour harness
This is the elephant in the room - how do we guide and sense if the application functionally behaves the way we need it to? At the moment, I see most people who give high autonomy to their coding agents do this: This is the elephant in the room - how do we guide and sense if the application functionally behaves the way we need it to? At the moment, I see most people who give high autonomy to their coding agents do this:
- Feed-forward: A functional specification (of varying levels of detail, from a short prompt to multi-file descriptions) - Feed-forward: A functional specification (of varying levels of detail, from a short prompt to multi-file descriptions)
- Feed-back: Check if the AI-generated test suite is green, has reasonably high coverage, some might even monitor its quality with mutation testing. Then combine that with manual testing. - Feed-back: Check if the AI-generated test suite is green, has reasonably high coverage, some might even monitor its quality with mutation testing. Then combine that with manual testing.
This approach puts a lot of faith into the AI-generated tests, that's not good enough yet. Some of my colleagues are seeing good results with the [approved fixtures](https://lexler.github.io/augmented-coding-patterns/patterns/approved-fixtures/) pattern, but it's easier to apply in some areas than others. They use it selectively where it fits, it's not a wholesale answer to the test quality problem. This approach puts a lot of faith into the AI-generated tests, that's not good enough yet. Some of my colleagues are seeing good results with the [approved fixtures](https://lexler.github.io/augmented-coding-patterns/patterns/approved-fixtures/) pattern, but it's easier to apply in some areas than others. They use it selectively where it fits, it's not a wholesale answer to the test quality problem.
So overall, we still have a lot to do to figure out good harnesses for functional behaviour that increase our confidence enough to reduce supervision and manual testing. So overall, we still have a lot to do to figure out good harnesses for functional behaviour that increase our confidence enough to reduce supervision and manual testing.
![Simplified overview of a harness showing guides and sensors in horizontal, and then the regulation dimensions maintainability, architecture fitness, and behaviour, in vertical. Examples shown for the behaviour harness, spec as feedforward guide, test suite as feedback sensor that is a mix of inferential and computational, plus a human icon indicating human review and manual tests as main additional feedback sensor.](../../martinfowler-assets/harness-types.png) ![[harness-types.png|Simplified overview of a harness showing guides and sensors in horizontal, and then the regulation dimensions maintainability, architecture fitness, and behaviour, in vertical. Examples shown for the behaviour harness, spec as feedforward guide, test suite as feedback sensor that is a mix of inferential and computational, plus a human icon indicating human review and manual tests as main additional feedback sensor.]]
## Harnessability ## Harnessability
@@ -150,7 +110,7 @@ This plays out differently for greenfield versus legacy. Greenfield teams can ba
Most enterprises have a few common topologies of services that cover 80% of what they need - business services that exposes data via APIs; event processing services; data dashboards. In many mature engineering organizations these topologies are already codified in service templates. These might evolve into harness templates in the future: a bundle of guides and sensors that leash a coding agent to the structure, conventions and tech stack of a topology. Teams may start picking tech stacks and structures partly based on what harnesses are already available for them. Most enterprises have a few common topologies of services that cover 80% of what they need - business services that exposes data via APIs; event processing services; data dashboards. In many mature engineering organizations these topologies are already codified in service templates. These might evolve into harness templates in the future: a bundle of guides and sensors that leash a coding agent to the structure, conventions and tech stack of a topology. Teams may start picking tech stacks and structures partly based on what harnesses are already available for them.
![A stack of examples of topologies (Data dashboard in Node, CRUD business service on JVM, event processor in Golang). The top one, data dashboard, is shown in detail, as a combination of structure definition and tech stack. The graphic indicates a "harness template" with guides and sensors for each topology, which can be instantiated.](../../martinfowler-assets/harness-templates.png) ![[harness-templates.png|A stack of examples of topologies (Data dashboard in Node, CRUD business service on JVM, event processor in Golang). The top one, data dashboard, is shown in detail, as a combination of structure definition and tech stack. The graphic indicates a "harness template" with guides and sensors for each topology, which can be instantiated.]]
We would of course face similar challenges as with service templates. As soon as teams instantiate them, they start fall out of sync with upstream improvements. Harness templates would face the same versioning and contribution problems, maybe even worse with non-deterministic guides and sensors that are harder to test. We would of course face similar challenges as with service templates. As soon as teams instantiate them, they start fall out of sync with upstream improvements. Harness templates would face the same versioning and contribution problems, maybe even worse with non-deterministic guides and sensors that are harder to test.
@@ -168,12 +128,28 @@ The mental model I've laid out here describes techniques that are already happen
Here are some harness-related examples from the current discourse: Here are some harness-related examples from the current discourse:
- [An OpenAI team documented what their harness looks like](https://openai.com/index/harness-engineering/): layered architecture enforced by custom linters and structural tests, and recurring “garbage collection” that scans for drift and has agents suggest fixes. Their conclusion: “Our most difficult challenges now center on designing environments, feedback loops, and control systems.” - [An OpenAI team documented what their harness looks like](https://openai.com/index/harness-engineering/): layered architecture enforced by custom linters and structural tests, and recurring “garbage collection” that scans for drift and has agents suggest fixes. Their conclusion: “Our most difficult challenges now center on designing environments, feedback loops, and control systems.”
- [Stripe's write-up about their minions](https://stripe.dev/blog/minions-stripes-one-shot-end-to-end-coding-agents) describes things like pre-push hooks that run relevant linters based on a heuristic, they highlight how important “shift feedback left” is to them, and their “blueprints” show how they're integrating feedback sensors into the agent workflows. - [Stripe's write-up about their minions](https://stripe.dev/blog/minions-stripes-one-shot-end-to-end-coding-agents) describes things like pre-push hooks that run relevant linters based on a heuristic, they highlight how important “shift feedback left” is to them, and their “blueprints” show how they're integrating feedback sensors into the agent workflows.
- Mutation and structural testing are examples of computational feedback sensors that have been underused in the past, but are now having a resurgence. - Mutation and structural testing are examples of computational feedback sensors that have been underused in the past, but are now having a resurgence.
- There is increased chatter among developers about the integration of LSPs and code intelligence in coding agents, examples of computational feedforward guides. - There is increased chatter among developers about the integration of LSPs and code intelligence in coding agents, examples of computational feedforward guides.
- I hear stories from teams at Thoughtworks about tackling architecture drift with both computational and inferential sensors, e.g. increasing API quality with a mix of agents and custom linters, or increasing code quality with a “janitor army”. - I hear stories from teams at Thoughtworks about tackling architecture drift with both computational and inferential sensors, e.g. increasing API quality with a mix of agents and custom linters, or increasing code quality with a “janitor army”.
There's plenty still to figure out, not just the already mentioned behavioural harness. How do we keep a harness coherent as it grows, with guides and sensors in sync, not contradicting each other? How far can we trust agents to make sensible trade-offs when instructions and feedback signals point in different directions? If sensors never fire, is that a sign of high quality or inadequate detection mechanisms? We need a way to evaluate harness coverage and quality similar to what code coverage and mutation testing do for tests. Feedforward and feedback controls are currently scattered across delivery steps, there's real potential for tooling that helps configure, sync, and reason about them as a system. Building this outer harness is emerging as an ongoing engineering practice, not a one-time configuration. There's plenty still to figure out, not just the already mentioned behavioural harness. How do we keep a harness coherent as it grows, with guides and sensors in sync, not contradicting each other? How far can we trust agents to make sensible trade-offs when instructions and feedback signals point in different directions? If sensors never fire, is that a sign of high quality or inadequate detection mechanisms? We need a way to evaluate harness coverage and quality similar to what code coverage and mutation testing do for tests. Feedforward and feedback controls are currently scattered across delivery steps, there's real potential for tooling that helps configure, sync, and reason about them as a system. Building this outer harness is emerging as an ongoing engineering practice, not a one-time configuration.
* * * ---
## Acknowledgements
Big thanks to the Doppler team for the engaging discussion at our last technology radar meeting, in particular Kief Morris for bringing up cybernetics. Thanks to Ned Letcher, Chris Ford and Ben O'Mahoney for the conversations about what a harness even is, and to Matteo Vaccari for his insights on the behaviour harness. And to everybody who took the time to read the draft and provide lots of valuable feedback: Christoph Burgmer, Jörn Dinkla, Michael Feathers, Karrtik Iyer, Swapnil Phulse, Paul Sobocinski, Zhenjia Zhou
GenAI (Claude and Claude Code) was used for research, pulling in relevant ideas from existing notes, and polishing the language.
## Earlier Memo
I wrote [a memo in early February](https://martinfowler.com/articles/exploring-gen-ai/harness-engineering-memo.html) containing my initial thoughts on Harness Engineering as the term first appeared. That post has attracted a lot of traffic. This article supersedes that memo, so we have redirected the original memo URL to this page, as we believe this page is the better resource for readers.
Significant Revisions
*02 April 2026:* published full article including introducing guides, sensors, computational and inferential elements, and harness templates
*17 February 2026:* published my [initial memo](https://martinfowler.com/articles/exploring-gen-ai/harness-engineering-memo.html) on Harness Engineering
@@ -1,5 +1,14 @@
# Improving Deep Agents with harness engineering ---
title: "Improving Deep Agents with harness engineering"
source: "https://blog.langchain.com/improving-deep-agents-with-harness-engineering/"
author:
- "[[LangChain Accounts]]"
published: 2026-02-18
created: 2026-04-11
description: "TLDR: Our coding agent went from Top 30 to Top 5 on Terminal Bench 2.0. We only changed the harness. Heres our approach to harness engineering (teaser: self-verification & tracing help a lot).The Goal of Harness EngineeringThe goal of a harness is to mold the inherently spiky"
tags:
- "clippings"
---
TLDR: Our coding agent went from Top 30 to Top 5 on [Terminal Bench 2.0](https://www.tbench.ai/leaderboard/terminal-bench/2.0?ref=blog.langchain.com). We only changed the harness. Heres our approach to harness engineering (teaser: self-verification & tracing help a lot). TLDR: Our coding agent went from Top 30 to Top 5 on [Terminal Bench 2.0](https://www.tbench.ai/leaderboard/terminal-bench/2.0?ref=blog.langchain.com). We only changed the harness. Heres our approach to harness engineering (teaser: self-verification & tracing help a lot).
## The Goal of Harness Engineering ## The Goal of Harness Engineering
@@ -12,7 +21,9 @@ At LangChain, we use [Traces](https://docs.langchain.com/langsmith/observability
We used a simple recipe to iteratively improve [deepagents-cli](https://github.com/langchain-ai/deepagents/tree/main/libs/cli?ref=blog.langchain.com) (our coding agent) `13.7 points` from `52.8` to `66.5` on Terminal Bench 2.0. We only tweaked the harness and kept the model fixed, `gpt-5.2-codex`. We used a simple recipe to iteratively improve [deepagents-cli](https://github.com/langchain-ai/deepagents/tree/main/libs/cli?ref=blog.langchain.com) (our coding agent) `13.7 points` from `52.8` to `66.5` on Terminal Bench 2.0. We only tweaked the harness and kept the model fixed, `gpt-5.2-codex`.
![](../../langchain-assets/Screenshot-2026-02-12-at-12.25.20---PM-1.png) ![[Screenshot-2026-02-12-at-12.25.20---PM-1.png]]
Spending more reasoning compute on planning and verification
## Experiment Setup & The Knobs on a Harness ## Experiment Setup & The Knobs on a Harness
@@ -20,27 +31,31 @@ We used [Terminal Bench 2.0](https://www.tbench.ai/?ref=blog.langchain.com), a n
Every agent action is stored in [LangSmith](https://smith.langchain.com/?ref=blog.langchain.com). It also includes metrics like latency, token counts, and costs. Every agent action is stored in [LangSmith](https://smith.langchain.com/?ref=blog.langchain.com). It also includes metrics like latency, token counts, and costs.
### **The Knobs we can Turn** ### The Knobs we can Turn
An agent harness has a lot of knobs: system prompts, tools, hooks/middleware, skills, sub-agent delegation, memory systems, and more. We deliberately compress the optimization space and focus on three: **System Prompt, Tools,** and [**Middleware**](https://docs.langchain.com/oss/python/langchain/middleware/overview?ref=blog.langchain.com#the-agent-loop) (our term for hooks around model and tool calls). An agent harness has a lot of knobs: system prompts, tools, hooks/middleware, skills, sub-agent delegation, memory systems, and more. We deliberately compress the optimization space and focus on three: **System Prompt, Tools,** and [**Middleware**](https://docs.langchain.com/oss/python/langchain/middleware/overview?ref=blog.langchain.com#the-agent-loop) (our term for hooks around model and tool calls).
We start with a default prompt and standard tools+middleware. This scores 52.8% with GPT-5.2-Codex. A solid score, just outside the Top 30 of the leaderboard today, but room to grow. We start with a default prompt and standard tools+middleware. This scores 52.8% with GPT-5.2-Codex. A solid score, just outside the Top 30 of the leaderboard today, but room to grow.
![](../../langchain-assets/Screenshot-2026-02-16-at-12.50.00---PM.png) ![[Screenshot-2026-02-16-at-12.50.00---PM.png]]
### **The Trace Analyzer Skill** Spending more reasoning compute on planning and verification
### The Trace Analyzer Skill
We wanted trace analysis to be repeatable so we made it into an Agent Skill. This serves as our recipe to **analyze errors across runs and make improvements to the harness**. The flow is: We wanted trace analysis to be repeatable so we made it into an Agent Skill. This serves as our recipe to **analyze errors across runs and make improvements to the harness**. The flow is:
1. Fetch experiment traces from LangSmith 1. Fetch experiment traces from LangSmith
2. Spawn parallel error analysis agents → main agent synthesizes findings + suggestions 2. Spawn parallel error analysis agents → main agent synthesizes findings + suggestions
3. Aggregate feedback and make targeted changes to the harness. 3. Aggregate feedback and make targeted changes to the harness.
This works similarly to [boosting](https://en.wikipedia.org/wiki/Boosting_\(machine_learning\)?ref=blog.langchain.com) which focuses on mistakes from previous runs. A human can be pretty helpful in Step 3 (though not required) to verify and discuss proposed changes. Changes that overfit to a task are bad for generalization and can lead to regressions in other Tasks. This works similarly to [boosting](https://en.wikipedia.org/wiki/Boosting_\(machine_learning\)?ref=blog.langchain.com) which focuses on mistakes from previous runs. A human can be pretty helpful in Step 3 (though not required) to verify and discuss proposed changes. Changes that overfit to a task are bad for generalization and can lead to regressions in other Tasks.
Automated trace analysis saves hours of time and made it easy to quickly try experiments. Well be publishing this skill soon, were currently testing it for prompt optimization generally. Automated trace analysis saves hours of time and made it easy to quickly try experiments. Well be publishing this skill soon, were currently testing it for prompt optimization generally.
![](../../langchain-assets/langsmith_trace_analyzer_skill.png) ![[langsmith_trace_analyzer_skill.png]]
Spending more reasoning compute on planning and verification
## What Actually Improved Agent Performance ## What Actually Improved Agent Performance
@@ -56,22 +71,24 @@ The most common failure pattern was that the agent wrote a solution, re-read its
We added guidance to the system prompt on how to approach problem solving. We added guidance to the system prompt on how to approach problem solving.
1. **Planning & Discovery:** Read the task, scan the codebase, and build an initial plan based on the task specification and how to verify the solution. 1. **Planning & Discovery:** Read the task, scan the codebase, and build an initial plan based on the task specification and how to verify the solution.
2. **Build:** Implement the plan with verification in mind. Build tests, if they dont exist and test both happy paths and edge cases. 2. **Build:** Implement the plan with verification in mind. Build tests, if they dont exist and test both happy paths and edge cases.
3. **Verify:** Run tests, read the full output, compare against what was asked (not against your own code). 3. **Verify:** Run tests, read the full output, compare against what was asked (not against your own code).
4. **Fix:** Analyze any errors, revisit the original spec, and fix issues. 4. **Fix:** Analyze any errors, revisit the original spec, and fix issues.
We really focus on testing because it powers the changes in every iteration. We found that alongside prompting, deterministic context injection helps agents verify their work. We use a `PreCompletionChecklistMiddleware` that intercepts the agent before it exits and reminds it to run a verification pass against the Task spec. This is similar to a [Ralph Wiggum Loop](https://ghuntley.com/loop/?ref=blog.langchain.com) where a hook forces the agent to continue executing on exit, we use this for verification. We really focus on testing because it powers the changes in every iteration. We found that alongside prompting, deterministic context injection helps agents verify their work. We use a `PreCompletionChecklistMiddleware` that intercepts the agent before it exits and reminds it to run a verification pass against the Task spec. This is similar to a [Ralph Wiggum Loop](https://ghuntley.com/loop/?ref=blog.langchain.com) where a hook forces the agent to continue executing on exit, we use this for verification.
![](../../langchain-assets/self-verification-loop.png) ![[self-verification-loop.png]]
Spending more reasoning compute on planning and verification
### Giving Agents Context about their Environment ### Giving Agents Context about their Environment
Part of harness engineering is **building a good delivery mechanism for context engineering.** Terminal Bench tasks come with directory structures, built-in tooling, and strict timeouts. Part of harness engineering is **building a good delivery mechanism for context engineering.** Terminal Bench tasks come with directory structures, built-in tooling, and strict timeouts.
1. **Directory Context & Tooling:** A `LocalContextMiddleware` runs on agent start to map the `cwd` and other parent+children directories. We run `bash` commands to find tools like `Python` installations. Context discovery and search are error prone, so injecting context reduces this error surface and helps **onboard the agent into its environment.** 1. **Directory Context & Tooling:** A `LocalContextMiddleware` runs on agent start to map the `cwd` and other parent+children directories. We run `bash` commands to find tools like `Python` installations. Context discovery and search are error prone, so injecting context reduces this error surface and helps **onboard the agent into its environment.**
2. **Teaching Agents to Write Testable Code:** Agents dont know how their code needs to be testable. We add prompting say their work will be measured against programatic tests, similar to when committing code. For example, Task specs that mention file paths should be followed exactly so the solutions works in an automated scoring step. Prompting that stresses edge-cases helps the agent avoid only checking “happy path” cases. Forcing models to conform to testing standards is a powerful strategy to avoid “slop buildup” over time. 2. **Teaching Agents to Write Testable Code:** Agents dont know how their code needs to be testable. We add prompting say their work will be measured against programatic tests, similar to when committing code. For example, Task specs that mention file paths should be followed exactly so the solutions works in an automated scoring step. Prompting that stresses edge-cases helps the agent avoid only checking “happy path” cases. Forcing models to conform to testing standards is a powerful strategy to avoid “slop buildup” over time.
3. **Time Budgeting:** We inject time budget warnings to nudge the agent to finish work and shift to verification. Agents are famously bad at time estimation so this heuristic helps in this environment. Real world coding usually doesnt have strict time limits, but without adding any knowledge of constraints, agents wont work within time bounds. 3. **Time Budgeting:** We inject time budget warnings to nudge the agent to finish work and shift to verification. Agents are famously bad at time estimation so this heuristic helps in this environment. Real world coding usually doesnt have strict time limits, but without adding any knowledge of constraints, agents wont work within time bounds.
The more that agents know about their environment, constraints, and evaluation criteria, the better they can autonomously self-direct their work. The more that agents know about their environment, constraints, and evaluation criteria, the better they can autonomously self-direct their work.
@@ -93,11 +110,11 @@ Terminal Bench timeout limits create a tradeoff. More reasoning helps agents eva
We found that reasoning helps with planning to fully understand the problem, some Terminal Bench tasks are very difficult. A good plan helps get to a working solution more quickly. We found that reasoning helps with planning to fully understand the problem, some Terminal Bench tasks are very difficult. A good plan helps get to a working solution more quickly.
Later stage verification also benefits from more reasoning to catch mistakes and get a solution submitted. As a heuristic, we choose a xhigh-high-xhigh "**reasoning sandwich**" as a baseline. Later stage verification also benefits from more reasoning to catch mistakes and get a solution submitted. As a heuristic, we choose a xhigh-high-xhigh " **reasoning sandwich** " as a baseline.
![](../../langchain-assets/the-reasoning-sandwich.png) ![[the-reasoning-sandwich.png]]
**Spending more reasoning compute on planning and verification** Spending more reasoning compute on planning and verification
Running only at `xhigh` scored poorly at `53.9%` due to agent timeouts compared to `63.6%` at `high`. There werent large differences in trial runs across reasoning budget splits so we stuck with our approach which pushed the score to `66.5%`. Running only at `xhigh` scored poorly at `53.9%` due to agent timeouts compared to `63.6%` at `high`. There werent large differences in trial runs across reasoning budget splits so we stuck with our approach which pushed the score to `66.5%`.
@@ -109,11 +126,11 @@ In a multi-model harness, balancing reasoning budgets could play out as using a
The design space of agents is big. Here are some general principles from our experiments and building deepagents overall. The design space of agents is big. Here are some general principles from our experiments and building deepagents overall.
1. **Context Engineering on Behalf of Agents.** Context assembly is still difficult for agents today, especially in unseen environments. Onboarding models with context like directory structures, available tools, coding best practices, and problem solving strategies helps reduce the error surface for poor search and avoidable errors in planning. 1. **Context Engineering on Behalf of Agents.** Context assembly is still difficult for agents today, especially in unseen environments. Onboarding models with context like directory structures, available tools, coding best practices, and problem solving strategies helps reduce the error surface for poor search and avoidable errors in planning.
2. **Help agents self-verify their work.** Models are biased towards their first plausible solution. Prompt them aggressively to verify their work by running tests and refining solutions. This is especially important in autonomous coding systems that dont have humans in the loop. 2. **Help agents self-verify their work.** Models are biased towards their first plausible solution. Prompt them aggressively to verify their work by running tests and refining solutions. This is especially important in autonomous coding systems that dont have humans in the loop.
3. **Tracing as a feedback signal.** Traces allow agents to self-evaluate and debug themselves. Its important to debug tooling and reasoning together (ex: models go down wrong paths because they lack a tool or instructions how to do something). 3. **Tracing as a feedback signal.** Traces allow agents to self-evaluate and debug themselves. Its important to debug tooling and reasoning together (ex: models go down wrong paths because they lack a tool or instructions how to do something).
4. **Detect and fix bad patterns in the short term.** Models today arent perfect. The job of the harness designer is to design around todays shortcomings while planning for smarter models in the future. Blind retries and not verifying work are good examples. These guardrails will almost surely dissolve over time, but to build robust agent applications today, theyre useful tools to experiment with. 4. **Detect and fix bad patterns in the short term.** Models today arent perfect. The job of the harness designer is to design around todays shortcomings while planning for smarter models in the future. Blind retries and not verifying work are good examples. These guardrails will almost surely dissolve over time, but to build robust agent applications today, theyre useful tools to experiment with.
5. **Tailor Harnesses to Models. T**he [Codex](https://developers.openai.com/cookbook/examples/gpt-5/codex_prompting_guide/?ref=blog.langchain.com) and [Claude](https://platform.claude.com/docs/en/build-with-claude/prompt-engineering/claude-prompting-best-practices?ref=blog.langchain.com) prompting guides show that models require different prompting. A test run with Claude Opus 4.6 scored `59.6%` with an earlier harness version, competitive but worse than Codex because we didnt run the same Improvement Loop with Claude. Many principles generalize like good context preparation and a focus on verification, but running a few rounds of harness iterations for your task helps maximize agent performance across tasks. 5. **Tailor Harnesses to Models. T** he [Codex](https://developers.openai.com/cookbook/examples/gpt-5/codex_prompting_guide/?ref=blog.langchain.com) and [Claude](https://platform.claude.com/docs/en/build-with-claude/prompt-engineering/claude-prompting-best-practices?ref=blog.langchain.com) prompting guides show that models require different prompting. A test run with Claude Opus 4.6 scored `59.6%` with an earlier harness version, competitive but worse than Codex because we didnt run the same Improvement Loop with Claude. Many principles generalize like good context preparation and a focus on verification, but running a few rounds of harness iterations for your task helps maximize agent performance across tasks.
Theres more open research to do in harness design. Interesting avenues include multi-model systems (Codex, Gemini, and Claude together), memory primitives for continual learning so agents can autonomously improve on tasks, and measuring harness changes across models. Theres more open research to do in harness design. Interesting avenues include multi-model systems (Codex, Gemini, and Claude together), memory primitives for continual learning so agents can autonomously improve on tasks, and measuring harness changes across models.
File diff suppressed because one or more lines are too long
@@ -0,0 +1,116 @@
---
title: "MiniMax M2.7: 开启模型的自我进化"
source: "https://www.minimaxi.com/news/minimax-m27-zh"
author:
- "[[MiniMax]]"
published: 2026-04-11
created: 2026-04-11
description: "MiniMax是全球领先的通用人工智能科技公司,致力于\"与所有人共创智能\",自主研发了一系列多模态通用大模型,并面向全球推出一系列AI原生产品,已服务逾 2亿名用户, MiniMax是全球领先的通用人工智能科技公司,致力于\"与所有人共创智能\",自主研发了一系列多模态通用大模型,并面向全球推出一系列AI原生产品,已服务逾 2亿名用户, MiniMax是全球领先的通用人工智能科技公司,致力于\"与所有人共创智能\",自主研发了一系列多模态通用大模型,并面向全球推出一系列AI原生产品,已服务逾 2亿名用户"
tags:
- "clippings"
---
![[Image.png|https://filecdn.minimax.chat/public/7a85723c-acc3-4eb8-a3a8-f53713d0822e.png]]
在 M2 系列模型发布后的几个月,我们收到了大量热心用户的反馈和建议,这促使我们进一步加速模型的迭代效率。除了更加认真工作之外,我们能找到的唯一途径就是开启模型和组织的自我进化。MiniMax M2.7是我们第一个模型深度参与迭代自己的模型。
M2.7 能够自行构建复杂 Agent Harness,并基于 Agent Teams、复杂 Skills、Tool Search Tool 等能力,完成高度复杂的生产力任务。例如,在研发M2.7的过程中,我们基于模型构建强化学习Harness 中的数十个复杂的 skills,更新自己的 memory,驱动模型自身的强化学习,并基于结果优化强化学习过程和Harness,开启模型的自我进化。
1\. M2.7 在真实的软件工程中有优异的表现,包括端到端的完整项目交付,分析日志排查 Bug、代码安全,机器学习等。在基准测试 SWE-Pro 中,M2.7 得分56.22%,几乎接近Opus最好的水平。这一能力同样延伸到了端到端的完整项目交付场景(VIBE-Pro 55.6%)以及对复杂工程系统的深层理解Terminal Bench 257.0%)。
2\. 在专业办公领域,我们提升了模型在各领域的专业知识和任务交付能力,在 GDPval-AA 的ELO得分是1495,为开源最高。M2.7 对 Office 三件套 Excel/PPT/Word 的复杂编辑能力显著提升,能更好地完成多轮修改和高保真的编辑。M2.7具备与复杂环境交互的能力,M2.7 在 40 个复杂 skills (> 2000 Token) 的 case 上,仍能保持 97% 的 skills 遵循率。在OpenClaw的使用中,M2.7相比于M2.5也有了显著的提升,在MMClaw的评测中接近最新的Sonnet 4.6。
3\. M2.7具备优秀的身份保持能力和情商,除了生产力使用外,给互动娱乐场景的创新也准备了空间。
基于上述能力,M2.7 也在显著加速我们自身向一个 AI Native 组织的进化。
![[img-1.png|MiniMax M2.7 基准测试对比]]
## 构建模型自我进化智能体
在最开始,我们分享一个我们内部让M2系列模型自我进化的实践,这也是对模型Agent能力边界的探索。
Agent Harness通常依赖复杂的Skills、记忆系统和其他组件来提升模型对不同工作环境的适应能力。在此基础上,我们在 M2 的早期版本中,将其引导为一个研究型 Agent Harness——它能够与不同的研究项目组进行交互和协作。该系统覆盖了数据流水线、训练环境、评测基础设施、跨团队协作、持久化记忆——让研究员可以驱动它来交付更好的模型。研究 Agent 驱动着产出下一代模型的迭代循环。研究员在每一层引导方向,模型在每一层负责构建。
以一个 RL 场景为例:研究员从一个实验想法出发,与 Agent 展开讨论。Agent 协助进行文献调研,持续跟踪预设的实验规格,完成数据流水线及其他对接工作,并启动实验。实验运行期间,它会自动监控和分析实验状态,并自动触发日志读取、问题排查、指标分析、代码修复、合并请求以及冒烟测试,识别并配置那些细微但关键的变更。这些工作过去可能需要来自不同团队的多位同事协作完成,而现在研究员只需在关键决策和讨论时介入。这大幅加速了问题发现和实验迭代,从而更快地交付模型。在这个场景下,M2.7 能够胜任 30-50% 的工作流。
![[d080353d-31aa-4642-8586-3b7f2201bf94.png|M2* 模型迭代系统架构]]
我们在迭代过程中也意识到,模型自主迭代 harness 的能力也至关重要。我们内部的 harness 会自主收集反馈,建立内部任务的评测集,并基于此不断迭代自己的 Agent架构、Skills/MCP 实现和记忆机制,来更好和更高效的完成任务。
举个例子,我们让 M2.7 优化一个内部脚手架上模型的软件工程开发表现。M2.7 全程自主运行,执行"分析失败轨迹 → 规划改动 → 修改脚手架代码 → 运行评测 → 对比结果 → 决定保留或回退"的迭代循环超过 100 轮。这个过程中 M2.7 发现了针对模型的有效优化:系统性搜索温度、频率惩罚、存在惩罚等采样参数的最优组合;为模型设计更具体的工作流指引(如修复后自动搜索其他文件中的相同 bug 模式);在脚手架的Agent Loop中添加循环检测等优化。最终在内部评测集上效果提升 30%。
我们相信,未来的 AI 自我进化会逐步向完全自动化过渡,包括完全自主的协调数据构建、模型训练、推理架构、评测等等。我们用 M2.7 参与了MLE Bench Lite的22 个机器学习任务测试,几乎囊括了研发的所有环节。
我们设计和实现了一个简易的脚手架来引导 Agent 进行自主优化,核心的模块包括短时记忆、自反馈以及自优化三个模块。具体来讲,Agent 完成每轮迭代后会形成一个短时记忆文件,同时对当前轮次的结果进行自反馈,从而给下一轮次提供潜在的优化方向,下一轮次基于所有历史轮次的记忆及自反馈链进行下一步的自优化。我们总共测试三次,每次有 24 小时来迭代进化,从下图中能够看到 M2.7 随时间不断取得更高的性能。最好的一次取得 9 枚金牌,5 枚银牌,1枚铜牌。三次平均是 66.6% 的得牌率,此成绩仅次于Opus-4.6 (75.7%)、GPT-5.4 (71.2%),和 Gemini-3.1 (66.6%) 持平。
![[img-3.png|MLE Bench 得牌率随时间变化]]
## 真实的软件工程
在编程等代码生成类任务上,M2.7 更深入地打磨了真实软件工程所需的编程能力,覆盖日志分析与 Bug 定位、代码重构、代码安全、机器学习、安卓开发等方向。
以线上最常见的线上生产环境故障调试为例——这类任务不仅需要生成代码,还需要很强的综合推理能力。面对我们实际的生产环境告警,M2.7 能关联监控指标与部署时间线做因果推理,对轨迹采样做统计分析并提出精准假设,主动连接数据库执行验证根因,定位到代码仓库中缺失的索引迁移文件,甚至知道用非阻塞建索引先止血,再提 MR。从可观测性分析、数据库专业知识到 SRE 级别的决策判断——这不只是一个会写代码的模型,而是一个真正理解生产系统的模型。相比传统的人工排障流程,基于 M2.7,我们已多次将线上生产系统故障的恢复时间缩短到三分钟以内。
<video controls=""><source src="https://filecdn.minimax.chat/public/platform_web/offical-news/%E9%A3%9E%E4%B9%A6%E4%BA%91%E6%96%87%E6%A1%A3/video-1.mp4" type="video/mp4"> 您的浏览器不支持播放此视频,请更换浏览器。</video>
【线上生产环境故障调试】
在单项编程能力上,M2.7 已具备国际一线模型水准。 在涵盖多种编程语言的 SWE-Pro 中,M2.7 以 56.22% 的正确率追平 GPT-5.3-Codex;而在更贴近真实工程场景的 SWE Multilingual76.5)和 Multi SWE Bench52.7)中展现出更显著优势。
这一能力同样延伸到了端到端的完整项目交付场景。 在 Repo 级代码生成基准 VIBE-Pro 上,M2.7 得分55.6%,几乎与 Opus 4.6持平 —— 这意味着无论是 Web、Android、iOS 还是 Simulation 类需求,都可以直接交给 M2.7 完成。
更值得关注的,是对复杂工程系统的深层理解。 在对系统认知要求极高的 Terminal Bench 257.0%)和 NL2Repo39.8%)中,M2.7 同样表现稳健,进一步印证了它不只擅长代码生成,更能深入理解软件系统的运行逻辑与协作流程。
<video controls=""><source src="https://filecdn.minimax.chat/public/platform_web/offical-news/%E9%A3%9E%E4%B9%A6%E4%BA%91%E6%96%87%E6%A1%A3/video-2.mp4" type="video/mp4"> 您的浏览器不支持播放此视频,请更换浏览器。</video>
【基于M2.7生成的WildGuard演示网页】
为了提升开发效率,一个比较重要的特性是原生的 Agent Teams (多智能体协作)。Agent Teams 对模型提出了范式级要求:角色边界、对抗性推理、协议遵循、行为分化——这些无法通过提示词,必须内化为模型的原生能力。Agent Teams 场景下,模型需要稳定锚定角色身份、主动挑战队友的逻辑与伦理盲区、在复杂状态机中自主决策。下面是我们内部使用的一个做产品原型开发的 Agent Teams,里面包含了做产品原型的一个最小组织。
![[dd83d19a-156b-481f-9835-9576293c323a.gif|Agent Teams 协作模拟演示]]
【Agent Teams 协作模拟演示】
## 专业办公
除了软件工程外,Agent开始在办公场景中变得越来越有用,我们认为这是两个核心能力:
1\. 专业知识与任务交付能力。 模型需要具备各领域的专业知识,理解用户的需求。在衡量这一能力的 GDPval-AA 的评测中,M2.7 在45个模型中的ELO得分是1500,仅次于Opus 4.6、Sonnet 4.6和GPT5.4,超过了GPT5.3。在最常见的办公文件处理上,我们系统性地优化了模型处理 Word、Excel 和 PPT 的能力。M2.7 在各种 Agent Harness上,既能够基于模版和 skills 直接生成文件,也能够遵从用户的交互指令,对已有的文件做多轮的高保真编辑,最终给出可编辑的产物。
2\. 与复杂环境的交互能力。 泛化的日常场景意味着模型需要灵活适应各类上下文、调用各种 skills 和工具、并在长程交互中保持稳定的指令遵循。M2.7 在这些方面有大幅提升。在 Toolathon 上,M2.7的正确率 46.3%,达到了全球第一梯队水平。真实工作场景中的 Agent Harness 往往也需要理解和调用大量复杂的 skills。在 MM Claw 的测试中,M2.7 在 40 个复杂 skills (> 2000 Token) 的 case 上,仍能保持 97% 的 skills 遵循率。
我们测试了在Finance领域的专业水准,与上代模型相比,模型的能力提升显著。例如,在Finance领域一个阅读研报并建模公司未来营收的场景,M2.7可以自主阅读公司的年报与业绩沟通会纪要,交叉比对多篇研报,独立设计假设并构建营收预测模型,再基于模版产出PPT和研究报告——像一个初级分析师一样理解、判断、输出,并在多轮交互中自我修正。从业者的评价是:产出物已经可以作为初稿直接进入后续工作流程。下面是一个对台积电的例子。
> 任务:基于台积电年报和业绩沟通会信息,构建台积电的营收模型,读取多个研报,设计对应的假设,基于最新的信息对台积电营收建模,然后基于PPT模版产出PPT,并写一个Word文档研究报告。
[\[TSMC\_Revenue\_Model.xlsx\]](https://filecdn.minimax.chat/public/0dff594f-3226-421e-a7f3-b4d55273ee5f.xlsx)
[\[TSMC\_Financial\_Analysis.pptx\]](https://filecdn.minimax.chat/public/504a47e3-8d71-4dd0-9397-8c7214bb26d7.pptx)
[\[TSMC\_Equity\_Research\_Report.docx\]](https://filecdn.minimax.chat/public/7bd2f0a5-06e2-406f-818d-7bf5bd64eca8.docx)
<video controls=""><source src="https://filecdn.minimax.chat/public/d070816d-2c2a-4a5c-a441-48c9dd19d44d.mp4" type="video/mp4"> 您的浏览器不支持播放此视频,请更换浏览器。</video>
最近OpenClaw 爆火为代表的Agent社区蓬勃发展,我们很高兴M2系列的模型帮助到了社区的繁荣。我们基于OpenClaw中的常用任务,构建了一个评测集 MM Claw,涵盖个人学习规划,到办公文档的处理与交付、定时的专业信息调研与投资建议、代码开发与维护等工作与生活中形形色色的真实需求。M2.7 在这个测试中达到了接近Sonnet 4.6的水平,正确率是62.7%。
## 互动娱乐
在 OpenClaw 等 Agent 脚手架的使用过程中,不少用户在使用 Agent 完成工作的同时,还希望模型具备比较高的情商和复杂人设保持能力。在有人设的情况下,用户不再只是让模型机械完成任务,而是开始自然于与Agent"相处"。 这促使我们思考,产品与交互设计、内容创作、甚至娱乐体验的构建,都可以被 AI 原生驱动的可能性。我们认为这会让 Agentic 模型的使用从单纯的生产力能进一步拓展到互动娱乐。为此,我们在 M2.7 中极大加强了人设保持和对话能力。
基于此,我们构建了一个 Agent 交互系统 OpenRoom,它将 AI 互动置入一个万物皆可互动的 Web GUI 空间。在这里,对话即驱动,实时产生视觉反馈与场景交互,角色可以主动地与环境交互。我们认为这个框架扩展性较高,能够随着模型 Agentic 能力的提升和社区的共建持续进化,探索出更多人与 Agent 之间全新的交互方式。为了促进这个领域的创新,我们已将这个原型项目开源 (这个里面的代码大部分也是 AI 写的):
项目地址: [github.com/MiniMax-AI/OpenRoom](https://github.com/MiniMax-AI/OpenRoom)
立即体验: [openroom.ai](https://openroom.ai/)
<video controls=""><source src="https://filecdn.minimax.chat/public/platform_web/offical-news/%E9%A3%9E%E4%B9%A6%E4%BA%91%E6%96%87%E6%A1%A3/video-3.mp4" type="video/mp4"> 您的浏览器不支持播放此视频,请更换浏览器。</video>
MiniMax M2.7 已在 MiniMax Agent 与开放平台上全量上线,期待用户和开发者朋友们能在MiniMax M2.7上探索出更多有趣场景。
MiniMax Agent [agent.minimaxi.com](https://agent.minimaxi.com/)
API服务: [platform.minimaxi.com](https://platform.minimaxi.com/)
Coding Plan订阅: [platform.minimaxi.com/subscribe/coding-plan](https://platform.minimaxi.com/subscribe/coding-plan)
Intelligence with Everyone.
@@ -1,15 +1,19 @@
# My AI Adoption Journey ---
title: "My AI Adoption Journey"
source: "https://mitchellh.com/writing/my-ai-adoption-journey"
author:
published: 2026-02-05
created: 2026-04-11
description:
tags:
- "clippings"
---
## Mitchell Hashimoto
## My AI Adoption Journey
Table of Contents Table of Contents
- [Step 1: Drop the Chatbot](#step-1-drop-the-chatbot)
- [Step 2: Reproduce Your Own Work](#step-2-reproduce-your-own-work)
- [Step 3: End-of-Day Agents](#step-3-end-of-day-agents)
- [Step 4: Outsource the Slam Dunks](#step-4-outsource-the-slam-dunks)
- [Step 5: Engineer the Harness](#step-5-engineer-the-harness)
- [Step 6: Always Have an Agent Running](#step-6-always-have-an-agent-running)
- [Today](#today)
My experience adopting any meaningful tool is that I've necessarily gone through three phases: (1) a period of inefficiency (2) a period of adequacy, then finally (3) a period of workflow and life-altering discovery. My experience adopting any meaningful tool is that I've necessarily gone through three phases: (1) a period of inefficiency (2) a period of adequacy, then finally (3) a period of workflow and life-altering discovery.
In most cases, I have to force myself through phase 1 and 2 because I usually have a workflow I'm already happy and comfortable with. Adopting a tool feels like work, and I *do not* want to put in the effort, but I usually do in an effort to be a well-rounded person of my craft. In most cases, I have to force myself through phase 1 and 2 because I usually have a workflow I'm already happy and comfortable with. Adopting a tool feels like work, and I *do not* want to put in the effort, but I usually do in an effort to be a well-rounded person of my craft.
@@ -18,7 +22,7 @@ This is my journey of how I found value in AI tooling and what I'm trying next w
This blog post was fully written by hand, in my own words. I hate that I have to say that but especially given the subject matter, I want to be explicit about it. This blog post was fully written by hand, in my own words. I hate that I have to say that but especially given the subject matter, I want to be explicit about it.
* * * ---
## Step 1: Drop the Chatbot ## Step 1: Drop the Chatbot
@@ -30,9 +34,9 @@ While I was still a heavy AI skeptic, my first "oh wow" moment was pasting a scr
But when I tried to reproduce that behavior for other tasks, I was left disappointed. In the context of brownfield projects, I found the chat interface produced poor results very often, and I found myself very frustrated copying and pasting code and command output to and from the interface. It was very obviously far less efficient than me doing the work myself. But when I tried to reproduce that behavior for other tasks, I was left disappointed. In the context of brownfield projects, I found the chat interface produced poor results very often, and I found myself very frustrated copying and pasting code and command output to and from the interface. It was very obviously far less efficient than me doing the work myself.
To find value, you *must* use an **agent**. An agent is the industry-adopted term for an LLM that can chat and invoke external behavior in a loop[1](#user-content-fn-1) At a bare minimum, the agent must have the ability to: read files, execute programs, and make HTTP requests. To find value, you *must* use an **agent**. An agent is the industry-adopted term for an LLM that can chat and invoke external behavior in a loop [^1] At a bare minimum, the agent must have the ability to: read files, execute programs, and make HTTP requests.
* * * ---
## Step 2: Reproduce Your Own Work ## Step 2: Reproduce Your Own Work
@@ -44,19 +48,19 @@ This was *excruciating*, because it got in the way of simply getting things done
But, expertise formed. I quickly discovered for myself from first principles what others were already saying, but discovering it myself resulted in a stronger fundamental understanding. But, expertise formed. I quickly discovered for myself from first principles what others were already saying, but discovering it myself resulted in a stronger fundamental understanding.
1. Break down sessions into separate clear, actionable tasks. Don't try to "draw the owl" in one mega session. 1. Break down sessions into separate clear, actionable tasks. Don't try to "draw the owl" in one mega session.
2. For vague requests, split the work into separate planning vs. execution sessions. 2. For vague requests, split the work into separate planning vs. execution sessions.
3. If you give an agent a way to verify its work, it more often than not fixes its own mistakes and prevents regressions. 3. If you give an agent a way to verify its work, it more often than not fixes its own mistakes and prevents regressions.
More generally, I also found the edges of what agents -- at the time -- were good at, what they weren't good at, and for the tasks they were good at how to achieve the results I wanted. More generally, I also found the edges of what agents -- at the time -- were good at, what they weren't good at, and for the tasks they were good at how to achieve the results I wanted.
All of this led to significant efficiency gains, to the point where I was starting to naturally use agents in a way that I felt was no slower than doing it myself (but I still didn't feel it was any faster, since I was mostly babysitting an agent). All of this led to significant efficiency gains, to the point where I was starting to naturally use agents in a way that I felt was no slower than doing it myself (but I still didn't feel it was any faster, since I was mostly babysitting an agent).
The negative space here is worth reiterating: part of the efficiency gains here were understanding when *not* to reach for an agent. Using an agent for something it'll likely fail at is obviously a big waste of time and having the knowledge to avoid that completely leads to time savings[2](#user-content-fn-3). The negative space here is worth reiterating: part of the efficiency gains here were understanding when *not* to reach for an agent. Using an agent for something it'll likely fail at is obviously a big waste of time and having the knowledge to avoid that completely leads to time savings [^2].
At this stage, I was finding adequate value with agents that I was happy to use them in my workflow, but still didn't feel like I was seeing any net efficiency gains. I didn't care though, I was content at this point with AI as a tool. At this stage, I was finding adequate value with agents that I was happy to use them in my workflow, but still didn't feel like I was seeing any net efficiency gains. I didn't care though, I was content at this point with AI as a tool.
* * * ---
## Step 3: End-of-Day Agents ## Step 3: End-of-Day Agents
@@ -64,15 +68,15 @@ To try to find some efficiency, I next started up a new pattern: **block out the
Similar to the previous task, I at first found this both unsuccessful and annoying. But, I once again quickly found different categories of work that were really helpful: Similar to the previous task, I at first found this both unsuccessful and annoying. But, I once again quickly found different categories of work that were really helpful:
- **Deep research sessions** where I'd ask agents to survey some field, such as finding all libraries in a specific language with a specific license type and producing multi-page summaries for each on their pros, cons, development activity, social sentiment, etc. - **Deep research sessions** where I'd ask agents to survey some field, such as finding all libraries in a specific language with a specific license type and producing multi-page summaries for each on their pros, cons, development activity, social sentiment, etc.
- **Parallel agents attempting different vague ideas I had but didn't have time to get started on.** I didn't expect them to produce something I'd ever ship here, but perhaps could illuminate some unknown unknowns when I got to the task the next day. - **Parallel agents attempting different vague ideas I had but didn't have time to get started on.** I didn't expect them to produce something I'd ever ship here, but perhaps could illuminate some unknown unknowns when I got to the task the next day.
- **Issue and PR triage/review.** Agents are good at using `gh` (GitHub CLI), so I manually scripted a quick way to spin up a bunch in parallel to triage issues. I would NOT allow agents to respond, I just wanted reports the next day to try to guide me towards high value or low effort tasks. - **Issue and PR triage/review.** Agents are good at using `gh` (GitHub CLI), so I manually scripted a quick way to spin up a bunch in parallel to triage issues. I would NOT allow agents to respond, I just wanted reports the next day to try to guide me towards high value or low effort tasks.
To be clear, I did not go as far as others went to have agents running in loops all night. In most cases, agents completed their tasks in less than half an hour. But, the latter part of the working day, I'm usually tired and coming out of flow and find myself too personally inefficient, so shifting my effort to spinning up these agents I found gave me a "warm start" the next morning that got me working more quickly than I would've otherwise. To be clear, I did not go as far as others went to have agents running in loops all night. In most cases, agents completed their tasks in less than half an hour. But, the latter part of the working day, I'm usually tired and coming out of flow and find myself too personally inefficient, so shifting my effort to spinning up these agents I found gave me a "warm start" the next morning that got me working more quickly than I would've otherwise.
I was happy, and I was starting to feel like I was doing more than I was doing prior to AI, if only slightly. I was happy, and I was starting to feel like I was doing more than I was doing prior to AI, if only slightly.
* * * ---
## Step 4: Outsource the Slam Dunks ## Step 4: Outsource the Slam Dunks
@@ -88,7 +92,7 @@ Importantly, I think the "work on something else" helps counteract the highly pu
At this point I was firmly in the "no way I can go back" territory. I felt more efficient, but even if I wasn't, the thing I liked the most was that I could now focus my coding and thinking on tasks I really loved while still adequately completing the tasks I didn't. At this point I was firmly in the "no way I can go back" territory. I felt more efficient, but even if I wasn't, the thing I liked the most was that I could now focus my coding and thinking on tasks I really loved while still adequately completing the tasks I didn't.
* * * ---
## Step 5: Engineer the Harness ## Step 5: Engineer the Harness
@@ -98,14 +102,12 @@ I don't know if there is a broad industry-accepted term for this yet, but I've g
This comes in two forms: This comes in two forms:
1. **Better implicit prompting (AGENTS.md).** For simple things, like the agent repeatedly running the wrong commands or finding the wrong APIs, update the `AGENTS.md` (or equivalent). Here is [an example from Ghostty](https://github.com/ghostty-org/ghostty/blob/ca07f8c3f775fe437d46722db80a755c2b6e6399/src/inspector/AGENTS.md). Each line in that file is based on a bad agent behavior, and it almost completely resolved them all. 1. **Better implicit prompting (AGENTS.md).** For simple things, like the agent repeatedly running the wrong commands or finding the wrong APIs, update the `AGENTS.md` (or equivalent). Here is [an example from Ghostty](https://github.com/ghostty-org/ghostty/blob/ca07f8c3f775fe437d46722db80a755c2b6e6399/src/inspector/AGENTS.md). Each line in that file is based on a bad agent behavior, and it almost completely resolved them all.
2. **Actual, programmed tools.** For example, scripts to take screenshots, run filtered tests, etc etc. This is usually paired with an AGENTS.md change to let it know about this existing.
2. **Actual, programmed tools.** For example, scripts to take screenshots, run filtered tests, etc etc. This is usually paired with an AGENTS.md change to let it know about this existing.
**This is where I'm at today.** I'm making an earnest effort whenever I see an agent do a Bad Thing to prevent it from ever doing that bad thing again. Or, conversely, I'm making an earnest effort for agents to be able to verify they're doing a Good Thing. **This is where I'm at today.** I'm making an earnest effort whenever I see an agent do a Bad Thing to prevent it from ever doing that bad thing again. Or, conversely, I'm making an earnest effort for agents to be able to verify they're doing a Good Thing.
* * * ---
## Step 6: Always Have an Agent Running ## Step 6: Always Have an Agent Running
@@ -119,24 +121,24 @@ The "have an agent running at all times" goal is still just a goal. I'd say righ
**I don't want to run agents for the sake of running agents.** I only want to run them when there is a task I think would be truly helpful to me. Part of the challenge of this goal is improving my own workflows and tools so that I can have a constant stream of high quality work to do that I can delegate. Which, even without AI, is important! **I don't want to run agents for the sake of running agents.** I only want to run them when there is a task I think would be truly helpful to me. Part of the challenge of this goal is improving my own workflows and tools so that I can have a constant stream of high quality work to do that I can delegate. Which, even without AI, is important!
* * * ---
## Today ## Today
And that's where I'm at today. And that's where I'm at today.
Through this journey, I've personally reached a point where I'm having success with modern AI tooling and I believe I'm approaching it with the proper measured view that is grounded in reality. I really don't care one way or the other if AI is here to stay[3](#user-content-fn-4), I'm a software craftsman that just wants to build stuff for the love of the game. Through this journey, I've personally reached a point where I'm having success with modern AI tooling and I believe I'm approaching it with the proper measured view that is grounded in reality. I really don't care one way or the other if AI is here to stay [^3], I'm a software craftsman that just wants to build stuff for the love of the game.
The whole landscape is moving so rapidly that I'm sure I'll look back at this post very quickly and laugh at my naivete. But, as they say, if you can't be embarassed about your past self, you're probably not growing. I just hope I'll grow in the right direction! The whole landscape is moving so rapidly that I'm sure I'll look back at this post very quickly and laugh at my naivete. But, as they say, if you can't be embarassed about your past self, you're probably not growing. I just hope I'll grow in the right direction!
I have no skin in the game here[4](#user-content-fn-5), and there are of course other reasons behind utility to avoid using AI. I fully respect anyone's individual decisions regarding it. I'm not here to convince you! For those interested, I just wanted to share my personal approach to navigating these new tools and give a glimpse about how I approach new tools *in general*, regardless of AI. I have no skin in the game here [^4], and there are of course other reasons behind utility to avoid using AI. I fully respect anyone's individual decisions regarding it. I'm not here to convince you! For those interested, I just wanted to share my personal approach to navigating these new tools and give a glimpse about how I approach new tools *in general*, regardless of AI.
## Footnotes February 5, 2026
1. Modern coding models like Opus and Codex are specifically trained to bias towards using tools compared to conversational models. [](#user-content-fnref-1) [^1]: Modern coding models like Opus and Codex are specifically trained to bias towards using tools compared to conversational models.
2. Due to the rapid pace of innovation in models, I have to constantly revisit my priors on this one. [](#user-content-fnref-3) [^2]: Due to the rapid pace of innovation in models, I have to constantly revisit my priors on this one.
3. The skill formation issues particularly in juniors without a strong grasp of fundamentals deeply worries me, however. [](#user-content-fnref-4) [^3]: The skill formation issues particularly in juniors without a strong grasp of fundamentals deeply worries me, however.
4. I don't work for, invest in, or advise any AI companies. [](#user-content-fnref-5) [^4]: I don't work for, invest in, or advise any AI companies.
@@ -0,0 +1,85 @@
---
title: "Scaling Managed Agents: Decoupling the brain from the hands"
source: "https://www.anthropic.com/engineering/managed-agents"
author:
published:
created: 2026-04-11
description: "Anthropic is an AI safety and research company that's working to build reliable, interpretable, and steerable AI systems."
tags:
- "clippings"
---
*Get started with Claude Managed Agents by following our [docs](https://platform.claude.com/docs/en/managed-agents/overview).*
A running topic on the Engineering Blog is how to [build effective agents](https://www.anthropic.com/engineering/building-effective-agents) and [design harnesses](https://www.anthropic.com/engineering/effective-harnesses-for-long-running-agents) for [long-running work](https://www.anthropic.com/engineering/harness-design-long-running-apps). A common thread across this work is that harnesses encode assumptions about what Claude cant do on its own. However, those assumptions need to be frequently questioned because they can [go stale](http://www.incompleteideas.net/IncIdeas/BitterLesson.html) as models improve.
As just one example, in prior work [we found](https://www.anthropic.com/engineering/harness-design-long-running-apps) that Claude Sonnet 4.5 would wrap up tasks prematurely as it sensed its context limit approaching—a behavior sometimes called “context anxiety.” We addressed this by adding context resets to the harness. But when we used the same harness on Claude Opus 4.5, we found that the behavior was gone. The resets had become dead weight.
We expect harnesses to continue evolving. So we built Managed Agents: a hosted service in the Claude Platform that runs long-horizon agents on your behalf through a small set of interfaces meant to outlast any particular implementation—including the ones we run today.
Building Managed Agents meant solving an old problem in computing: how to design a system for “ [programs as yet unthought of](http://www.catb.org/esr/writings/taoup/html/ch03s01.html).” Decades ago, operating systems solved this problem by virtualizing hardware into abstractions— *process, file* —general enough for programs that didn't exist yet. The abstractions outlasted the hardware. The `read()` command is agnostic as to whether its accessing a disk pack from the 1970s or a modern SSD. The abstractions on top stayed stable while the implementations underneath changed freely.
Managed Agents follow the same pattern. We virtualized the components of an agent: a session (the append-only log of everything that happened), a harness (the loop that calls Claude and routes Claudes tool calls to the relevant infrastructure), and a sandbox (an execution environment where Claude can run code and edit files). This allows the implementation of each to be swapped without disturbing the others. We're opinionated about the shape of these interfaces, not about what runs behind them.
![[903b624ada206b10753a24c6a1367e74a869165d-1080x1080.webp]]
## Dont adopt a pet
We started by placing all agent components into a single container, which meant the session, agent harness, and sandbox all shared an environment. There were benefits to this approach, including that file edits are direct syscalls, and there were no service boundaries to design.
But by coupling everything into one container, we ran into an old infrastructure problem: wed adopted a [*pet*](https://cloudscaling.com/blog/cloud-computing/the-history-of-pets-vs-cattle/). In the pets-vs-cattle analogy, a pet is a named, hand-tended individual you cant afford to lose, while cattle are interchangeable. In our case, the server became that pet; if a container failed, the session was lost. If a container was unresponsive, we had to nurse it back to health.
Nursing containers meant debugging unresponsive stuck sessions. Our only window in was the WebSocket event stream, but that couldnt tell us *where* failures arose, which meant that a bug in the harness, a packet drop in the event stream, or a container going offline all presented the same. To figure out what went wrong, an engineer had to open a shell inside the container, but because that container often also held user data, that approach essentially meant we lacked the ability to debug.
A second issue was that the harness assumed that whatever Claude worked on lived in the container with it. When customers asked us to connect Claude to their virtual private cloud, they had to either peer their network with ours, or run our harness in their own environment. An assumption baked into the harness became a problem when we wanted to connect it to different infrastructure.
## Decouple the brain from the hands
The solution we arrived at was to decouple what we thought of as the “brain” (Claude and its harness) from both the “hands” (sandboxes and tools that perform actions) and the “session” (the log of session events). Each became an interface that made few assumptions about the others, and each could fail or be replaced independently.
**The harness leaves the container.** Decoupling the brain from the hands meant the harness no longer lived inside the container. It called the container the way it called any other tool: `execute(name, input) → string`. The container became cattle. If the container died, the harness caught the failure as a tool-call error and passed it back to Claude. If Claude decided to retry, a new container could be reinitialized with a standard recipe: `provision({resources})`. We no longer had to nurse failed containers back to health.
**Recovering from harness failure.** The harness also became cattle. Because the session log sits outside the harness, nothing in the harness needs to survive a crash. When one fails, a new one can be rebooted with `wake(sessionId)`, use `getSession(id)` to get back the event log, and resume from the last event. During the agent loop, the harness writes to the session with `emitEvent(id, event)` in order to keep a durable record of events.
![[73e900af5b9d6ed8c64db0a8e74d4465963556b7-1640x1596.webp]]
**The security boundary.** In the coupled design, any untrusted code that Claude generated was run in the same container as credentials—so a prompt injection only had to convince Claude to read its own environment. Once an attacker has those tokens, they can spawn fresh, unrestricted sessions and delegate work to them. Narrow scoping is an obvious mitigation, but this encodes an assumption about what Claude can't do with a limited token—and Claude is getting increasingly smart. The structural fix was to make sure the tokens are never reachable from the sandbox where Claudes generated code runs.
We used two patterns to ensure this. Auth can be bundled with a resource or held in a vault outside the sandbox. For Git, we use each repositorys access token to clone the repo during sandbox initialization and wire it into the local git remote. Git `push` and `pull` work from inside the sandbox without the agent ever handling the token itself. For custom tools, we support MCP and store OAuth tokens in a secure vault. Claude calls MCP tools via a dedicated proxy; this proxy takes in a token associated with the session. The proxy can then fetch the corresponding credentials from the vault and make the call to the external service. The harness is never made aware of any credentials.
## The session is not Claudes context window
Long-horizon tasks often exceed the length of Claudes context window, and the standard ways to address this all involve irreversible decisions about what to keep. Weve explored these techniques in [prior work](https://www.anthropic.com/engineering/effective-context-engineering-for-ai-agents) on context engineering. For example, compaction lets Claude save a summary of its context window and the memory tool lets Claude write context to files, enabling learning across sessions. This can be paired with context trimming, which selectively removes tokens such as old tool results or thinking blocks.
But irreversible decisions to selectively retain or discard context can lead to failures. It is difficult to know which tokens the future turns will need. If messages are transformed by a compaction step, the harness removes compacted messages from Claudes context window, and these are recoverable only if they are stored. Prior work [has explored](https://arxiv.org/pdf/2512.24601) ways to address this by storing context as an object that lives *outside* the context window. For example, context can be an object in a REPL that the LLM programmatically accesses by writing code to filter or slice it.
![[cf0719d7832b1f577b7393c84a7c53eecc725ca4-760x200.webp]]
In Managed Agents, the session provides this same benefit, serving as a context object that lives outside Claudes context window. But rather than be stored within the sandbox or REPL, context is durably stored in the session log. The interface, `getEvents(),` allows the brain to interrogate context by selecting positional slices of the event stream. The interface can be used flexibly, allowing the brain to pick up from wherever it last stopped reading, rewinding a few events before a specific moment to see the lead up, or rereading context before a specific action.
Any fetched events can also be transformed in the harness before being passed to Claudes context window. These transformations can be whatever the harness encodes, including context organization to achieve a high prompt cache hit rate and context engineering. We separated the concerns of recoverable context storage in the session and arbitrary context management in the harness because we cant predict what specific context engineering will be required in future models. The interfaces push that context management into the harness, and only guarantee that the session is durable and available for interrogation.
## Many brains, many hands
**Many brains.** Decoupling the brain from the hands solved one of our earliest customer complaints. When teams wanted Claude to work against resources in their own VPC, the only path was to peer their network with ours, because the container holding the harness assumed every resource sat next to it. Once the harness was no longer in the container, that assumption went away. The same change had a performance payoff. When we initially put the brain in a container, it meant that many brains required as many containers. For each brain, no inference could happen until that container was provisioned; every session paid the full container setup cost up front. Every session, even ones that would never touch the sandbox, had to clone the repo, boot the process, fetch pending events from our servers.
That dead time is expressed in time-to-first-token (TTFT), which measures how long a session waits between accepting work and producing its first response token. TTFT is the latency the user most acutely *feels*.
Decoupling the brain from the hands means that containers are provisioned by the brain via a tool call `(execute(name, input) → string)` only if they are needed. So a session that didn't need a container right away didn't wait for one. Inference could start as soon as the orchestration layer pulled pending events from the session log. Using this architecture, our p50 TTFT dropped roughly 60% and p95 dropped over 90%. Scaling to many brains just meant starting many stateless harnesses, and connecting them to hands only if needed.
**Many hands.** We also wanted the ability to connect each brain to many hands. In practice, this means Claude must reason about many execution environments and decide where to send work—a harder cognitive task than operating in a single shell. We started with the brain in a single container because earlier models weren't capable of this. As intelligence scaled, the single container became the limitation instead: when that container failed, we lost state for every hand that the brain was reaching into.
Decoupling the brain from the hands makes each hand a tool, `execute(name, input) → string`: a name and input go in, and a string is returned. That interface supports any custom tool, any MCP server, and our own tools. The harness doesnt know whether the sandbox is a container, a phone, or a Pokémon emulator. And because no hand is coupled to any brain, brains can pass hands to one another.
![[4f67b1c10566552aec514a716ea43544ab330e0b-668x243.webp]]
## Conclusion
The challenge we faced is an old one: how to design a system for “programs as yet unthought of.” Operating systems have lasted decades by virtualizing the hardware into abstractions general enough for programs that didn't exist yet. With Managed Agents, we aimed to design a system that accommodates future harnesses, sandboxes, or other components around Claude.
Managed Agents is a meta-harness in the same spirit, unopinionated about the *specific* harness that Claude will need in the future. Rather, it is a system with general interfaces that allow many different harnesses. For example, Claude Code is an excellent harness that we use widely across tasks. Weve also shown that task-specific agent harnesses excel in narrow domains. Managed Agents can accommodate any of these, matching Claudes intelligence over time.
Meta-harness design means being opinionated about the interfaces around Claude: we expect that Claude will need the ability to manipulate state (the session) and perform computation (the sandbox). We also expect that Claude will require the ability to scale to many brains and many hands. We designed the interfaces so that these can be run reliably and securely over long time horizons. But we make no assumptions about the number or location of brains or hands that Claude will need.
## Acknowledgements
Written by Lance Martin, Gabe Cemaj, and Michael Cohen. Thanks to Nodir Turakulov and Jeremy Fox for helpful conversations on these topics. Special thanks to the Agents API team and Jake Eaton for their contributions.
Binary file not shown.

Before

Width:  |  Height:  |  Size: 177 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 136 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 286 KiB

File diff suppressed because one or more lines are too long

Before

Width:  |  Height:  |  Size: 139 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 19 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 4.6 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 86 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 12 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 115 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 282 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 199 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 295 KiB

Before

Width:  |  Height:  |  Size: 35 KiB

After

Width:  |  Height:  |  Size: 35 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 17 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 5.2 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 378 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 339 KiB

Before

Width:  |  Height:  |  Size: 7.1 MiB

After

Width:  |  Height:  |  Size: 7.1 MiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 101 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 234 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 200 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 175 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 198 KiB

Before

Width:  |  Height:  |  Size: 76 KiB

After

Width:  |  Height:  |  Size: 76 KiB

Before

Width:  |  Height:  |  Size: 120 KiB

After

Width:  |  Height:  |  Size: 120 KiB

Before

Width:  |  Height:  |  Size: 337 KiB

After

Width:  |  Height:  |  Size: 337 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 1.6 MiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 395 KiB

Before

Width:  |  Height:  |  Size: 76 KiB

After

Width:  |  Height:  |  Size: 76 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 1.8 MiB

Before

Width:  |  Height:  |  Size: 47 KiB

After

Width:  |  Height:  |  Size: 47 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 216 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 85 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 34 KiB

BIN
View File
Binary file not shown.

After

Width:  |  Height:  |  Size: 966 KiB

BIN
View File
Binary file not shown.

After

Width:  |  Height:  |  Size: 671 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 214 KiB

BIN
View File
Binary file not shown.

After

Width:  |  Height:  |  Size: 246 KiB

BIN
View File
Binary file not shown.

After

Width:  |  Height:  |  Size: 208 KiB

BIN
View File
Binary file not shown.

After

Width:  |  Height:  |  Size: 188 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 284 KiB

@@ -1,280 +0,0 @@
<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd">
<html>
<head>
<meta content = 'uft-8' name = 'charset'></meta>
<title>not found</title>
<meta http-equiv="Content-type" content="text/html;charset=UTF-8" /><script defer src="https://cloud.umami.is/script.js" data-website-id="eb12527f-b713-4afa-905a-8a50f8a7f157"></script>
<link href = '/global.css' rel = 'stylesheet' type = 'text/css'></link>
</head>
<body><header id = 'banner' style = 'background-image: url("/banner.png"); background-repeat: no-repeat'>
<div class = 'name-logo'><a href = 'https://martinfowler.com'><img src = '/mf-name-white.png'></img></a></div>
<div class = 'search'>
<!-- SiteSearch Google -->
<form method='GET' action="https://www.google.com/search">
<input type='hidden' name='ie' value='UTF-8'/>
<input type='hidden' name='oe' value='UTF-8'/>
<input class = 'field' type='text'
name='q' size='15' maxlength='255' value=""/>
<button class = 'button' type='submit'
name='btnG' value=" " title = "Search"/>
<input type='hidden' name='domains' value="martinfowler.com"/>
<input type='hidden' name='sitesearch' value=""/>
<input
type='hidden' name='sitesearch' value="martinfowler.com"/>
</form>
</div>
<div class = 'menu-button navmenu-button'><a class = 'icon icon-bars' href = '#navmenu-bottom'></a></div>
<nav class = 'top-menu'>
<ul>
<li><a class = '' href = 'https://refactoring.com'>Refactoring</a></li>
<li><a class = '' href = '/agile.html'>Agile</a></li>
<li><a class = '' href = '/architecture'>Architecture</a></li>
<li><a class = '' href = '/aboutMe.html'>About</a></li>
<li><a class = 'tw' href = 'https://www.thoughtworks.com/engineering'>Thoughtworks</a></li>
<li><a class = 'icon icon-rss' href = '/feed.atom' title = 'feed'></a></li>
<li><a class = 'icon icon-twitter' href = 'https://www.twitter.com/martinfowler' title = 'Twitter stream'></a></li>
<li class = 'icon'><a href = 'https://toot.thoughtworks.com/@mfowler' title = 'Mastodon stream'><svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 24 24" fill="currentColor"><path d="M21.2595 13.9898C20.9852 15.4006 18.8033 16.9446 16.2974 17.2439C14.9907 17.3998 13.7041 17.5431 12.3321 17.4802C10.0885 17.3774 8.31809 16.9446 8.31809 16.9446C8.31809 17.163 8.33156 17.371 8.3585 17.5655C8.65019 19.7797 10.5541 19.9124 12.3576 19.9742C14.1779 20.0365 15.7987 19.5254 15.7987 19.5254L15.8735 21.1711C15.8735 21.1711 14.6003 21.8548 12.3321 21.9805C11.0814 22.0493 9.52849 21.9491 7.71973 21.4703C3.79684 20.432 3.12219 16.2504 3.01896 12.0074C2.98749 10.7477 3.00689 9.55981 3.00689 8.56632C3.00689 4.22771 5.84955 2.95599 5.84955 2.95599C7.2829 2.29772 9.74238 2.0209 12.2993 2H12.3621C14.919 2.0209 17.3801 2.29772 18.8133 2.95599C18.8133 2.95599 21.6559 4.22771 21.6559 8.56632C21.6559 8.56632 21.6916 11.7674 21.2595 13.9898ZM18.3029 8.9029C18.3029 7.82924 18.0295 6.97604 17.4805 6.34482C16.9142 5.71359 16.1726 5.39001 15.2522 5.39001C14.187 5.39001 13.3805 5.79937 12.8473 6.61819L12.3288 7.48723L11.8104 6.61819C11.2771 5.79937 10.4706 5.39001 9.40554 5.39001C8.485 5.39001 7.74344 5.71359 7.17719 6.34482C6.62807 6.97604 6.3547 7.82924 6.3547 8.9029V14.1562H8.43597V9.05731C8.43597 7.98246 8.88822 7.4369 9.79281 7.4369C10.793 7.4369 11.2944 8.08408 11.2944 9.36376V12.1547H13.3634V9.36376C13.3634 8.08408 13.8646 7.4369 14.8648 7.4369C15.7694 7.4369 16.2216 7.98246 16.2216 9.05731V14.1562H18.3029V8.9029Z"></path></svg>
</a></li>
<li class = 'icon'><a href = 'https://www.linkedin.com/in/martin-fowler-com/' title = 'LinkedIn'><svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 24 24" fill="currentColor"><path d="M4.00098 3H20.001C20.5533 3 21.001 3.44772 21.001 4V20C21.001 20.5523 20.5533 21 20.001 21H4.00098C3.44869 21 3.00098 20.5523 3.00098 20V4C3.00098 3.44772 3.44869 3 4.00098 3ZM5.00098 5V19H19.001V5H5.00098ZM7.50098 9C6.67255 9 6.00098 8.32843 6.00098 7.5C6.00098 6.67157 6.67255 6 7.50098 6C8.3294 6 9.00098 6.67157 9.00098 7.5C9.00098 8.32843 8.3294 9 7.50098 9ZM6.50098 10H8.50098V17.5H6.50098V10ZM12.001 10.4295C12.5854 9.86534 13.2665 9.5 14.001 9.5C16.072 9.5 17.501 11.1789 17.501 13.25V17.5H15.501V13.25C15.501 12.2835 14.7175 11.5 13.751 11.5C12.7845 11.5 12.001 12.2835 12.001 13.25V17.5H10.001V10H12.001V10.4295Z"></path></svg>
</a></li>
<li class = 'icon'><a href = 'https://bsky.app/profile/martinfowler.com' title = 'BlueSky'><svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 24 24" fill="currentColor"><path d="M12 11.3884C11.0942 9.62673 8.62833 6.34423 6.335 4.7259C4.13833 3.17506 3.30083 3.4434 2.75167 3.69256C2.11583 3.9784 2 4.95506 2 5.52839C2 6.10339 2.315 10.2367 2.52 10.9276C3.19917 13.2076 5.61417 13.9776 7.83917 13.7309C4.57917 14.2142 1.68333 15.4017 5.48083 19.6292C9.65833 23.9542 11.2058 18.7017 12 16.0392C12.7942 18.7017 13.7083 23.7651 18.4442 19.6292C22 16.0392 19.4208 14.2142 16.1608 13.7309C18.3858 13.9784 20.8008 13.2076 21.48 10.9276C21.685 10.2376 22 6.10256 22 5.52923C22 4.95423 21.8842 3.97839 21.2483 3.6909C20.6992 3.44256 19.8617 3.17423 17.665 4.72423C15.3717 6.34506 12.9058 9.62756 12 11.3884Z"></path></svg></a></li>
</ul>
</nav>
</header>
<nav id = 'top-navmenu'>
<nav class = 'navmenu'>
<div class = 'nav-head'> <div class = 'search'>
<!-- SiteSearch Google -->
<form method='GET' action="https://www.google.com/search">
<input type='hidden' name='ie' value='UTF-8'/>
<input type='hidden' name='oe' value='UTF-8'/>
<input class = 'field' type='text'
name='q' size='15' maxlength='255' value=""/>
<button class = 'button' type='submit'
name='btnG' value=" " title = "Search"/>
<input type='hidden' name='domains' value="martinfowler.com"/>
<input type='hidden' name='sitesearch' value=""/>
<input
type='hidden' name='sitesearch' value="martinfowler.com"/>
</form>
</div>
<div class = 'closediv'>
<span class = 'close' title = 'close'></span>
</div>
</div>
<div class = 'nav-body'>
<div class = 'topics'>
<h2>Topics</h2>
<p><a href = '/architecture'>Architecture</a></p>
<p><a href = 'https://refactoring.com'>Refactoring</a></p>
<p><a href = '/agile.html'>Agile</a></p>
<p><a href = '/delivery.html'>Delivery</a></p>
<p><a href = '/microservices'>Microservices</a></p>
<p><a href = '/data'>Data</a></p>
<p><a href = '/testing'>Testing</a></p>
<p><a href = '/dsl.html'>DSL</a></p>
</div>
<div class = 'about'>
<h2>about me</h2>
<p><a href = '/aboutMe.html'>About</a></p>
<p><a href = '/books'>Books</a></p>
<p><a href = '/faq.html'>FAQ</a></p>
</div>
<div class = 'content'>
<h2>content</h2>
<p><a href = '/videos.html'>Videos</a></p>
<p><a href = '/tags'>Content Index</a></p>
<p><a href = '/boardgames'>Board Games</a></p>
<p><a href = '/photos'>Photography</a></p>
</div>
<div class = 'tw'>
<h2>Thoughtworks</h2>
<p><a href = 'https://thoughtworks.com'>Home</a></p>
<p><a href = 'https://thoughtworks.com/insights'>Insights</a></p>
<p><a href = 'https://thoughtworks.com/careers'>Careers</a></p>
<p><a href = 'https://thoughtworks.com/radar'>Radar</a></p>
<p><a href = 'https://www.thoughtworks.com/engineering'>Engineering</a></p>
</div>
<div class = 'feeds'>
<h2>follow</h2>
<p><a href = '/feed.atom'>RSS</a></p>
<p><a href = 'https://toot.thoughtworks.com/@mfowler'>Mastodon</a></p>
<p><a href = 'https://www.linkedin.com/in/martin-fowler-com/'>LinkedIn</a></p>
<p><a href = 'https://bsky.app/profile/martinfowler.com'>Bluesky</a></p>
<p><a href = 'https://www.twitter.com/martinfowler'>X</a></p>
<p><a href = 'https://boardgamegeek.com/blog/13064/martins-7th-decade'>BGG</a></p>
</div>
</div>
</nav>
</nav>
<main><h1 id="section">404</h1>
<p>Im afraid this is not the document youre looking for. Try using the
search box above, and good luck.</p>
</main>
<nav id = 'bottom-navmenu'>
<nav class = 'navmenu'>
<div class = 'nav-head'> <div class = 'search'>
<!-- SiteSearch Google -->
<form method='GET' action="https://www.google.com/search">
<input type='hidden' name='ie' value='UTF-8'/>
<input type='hidden' name='oe' value='UTF-8'/>
<input class = 'field' type='text'
name='q' size='15' maxlength='255' value=""/>
<button class = 'button' type='submit'
name='btnG' value=" " title = "Search"/>
<input type='hidden' name='domains' value="martinfowler.com"/>
<input type='hidden' name='sitesearch' value=""/>
<input
type='hidden' name='sitesearch' value="martinfowler.com"/>
</form>
</div>
<div class = 'closediv'>
<span class = 'close' title = 'close'></span>
</div>
</div>
<div class = 'nav-body'>
<div class = 'topics'>
<h2>Topics</h2>
<p><a href = '/architecture'>Architecture</a></p>
<p><a href = 'https://refactoring.com'>Refactoring</a></p>
<p><a href = '/agile.html'>Agile</a></p>
<p><a href = '/delivery.html'>Delivery</a></p>
<p><a href = '/microservices'>Microservices</a></p>
<p><a href = '/data'>Data</a></p>
<p><a href = '/testing'>Testing</a></p>
<p><a href = '/dsl.html'>DSL</a></p>
</div>
<div class = 'about'>
<h2>about me</h2>
<p><a href = '/aboutMe.html'>About</a></p>
<p><a href = '/books'>Books</a></p>
<p><a href = '/faq.html'>FAQ</a></p>
</div>
<div class = 'content'>
<h2>content</h2>
<p><a href = '/videos.html'>Videos</a></p>
<p><a href = '/tags'>Content Index</a></p>
<p><a href = '/boardgames'>Board Games</a></p>
<p><a href = '/photos'>Photography</a></p>
</div>
<div class = 'tw'>
<h2>Thoughtworks</h2>
<p><a href = 'https://thoughtworks.com'>Home</a></p>
<p><a href = 'https://thoughtworks.com/insights'>Insights</a></p>
<p><a href = 'https://thoughtworks.com/careers'>Careers</a></p>
<p><a href = 'https://thoughtworks.com/radar'>Radar</a></p>
<p><a href = 'https://www.thoughtworks.com/engineering'>Engineering</a></p>
</div>
<div class = 'feeds'>
<h2>follow</h2>
<p><a href = '/feed.atom'>RSS</a></p>
<p><a href = 'https://toot.thoughtworks.com/@mfowler'>Mastodon</a></p>
<p><a href = 'https://www.linkedin.com/in/martin-fowler-com/'>LinkedIn</a></p>
<p><a href = 'https://bsky.app/profile/martinfowler.com'>Bluesky</a></p>
<p><a href = 'https://www.twitter.com/martinfowler'>X</a></p>
<p><a href = 'https://boardgamegeek.com/blog/13064/martins-7th-decade'>BGG</a></p>
</div>
</div>
</nav>
</nav>
<footer id='page-footer'>
<div class='tw-logo'>
<a href='https://www.thoughtworks.com/engineering'>
<img src='/thoughtworks_white.png'>
</a>
</div>
<div class='menu-button'>
<div class='icon-bars navmenu-button'></div>
</div>
<div class='copyright'>
<p>© Martin Fowler | <a href="/aboutMe.html#disclosures">Disclosures</a></p>
</div>
</footer>
<script src = '/jquery-1.11.3.min.js' type = 'text/javascript'></script>
<script src = '/mfcom.js' type = 'text/javascript'></script>
</body>
</html>
@@ -1,280 +0,0 @@
<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd">
<html>
<head>
<meta content = 'uft-8' name = 'charset'></meta>
<title>not found</title>
<meta http-equiv="Content-type" content="text/html;charset=UTF-8" /><script defer src="https://cloud.umami.is/script.js" data-website-id="eb12527f-b713-4afa-905a-8a50f8a7f157"></script>
<link href = '/global.css' rel = 'stylesheet' type = 'text/css'></link>
</head>
<body><header id = 'banner' style = 'background-image: url("/banner.png"); background-repeat: no-repeat'>
<div class = 'name-logo'><a href = 'https://martinfowler.com'><img src = '/mf-name-white.png'></img></a></div>
<div class = 'search'>
<!-- SiteSearch Google -->
<form method='GET' action="https://www.google.com/search">
<input type='hidden' name='ie' value='UTF-8'/>
<input type='hidden' name='oe' value='UTF-8'/>
<input class = 'field' type='text'
name='q' size='15' maxlength='255' value=""/>
<button class = 'button' type='submit'
name='btnG' value=" " title = "Search"/>
<input type='hidden' name='domains' value="martinfowler.com"/>
<input type='hidden' name='sitesearch' value=""/>
<input
type='hidden' name='sitesearch' value="martinfowler.com"/>
</form>
</div>
<div class = 'menu-button navmenu-button'><a class = 'icon icon-bars' href = '#navmenu-bottom'></a></div>
<nav class = 'top-menu'>
<ul>
<li><a class = '' href = 'https://refactoring.com'>Refactoring</a></li>
<li><a class = '' href = '/agile.html'>Agile</a></li>
<li><a class = '' href = '/architecture'>Architecture</a></li>
<li><a class = '' href = '/aboutMe.html'>About</a></li>
<li><a class = 'tw' href = 'https://www.thoughtworks.com/engineering'>Thoughtworks</a></li>
<li><a class = 'icon icon-rss' href = '/feed.atom' title = 'feed'></a></li>
<li><a class = 'icon icon-twitter' href = 'https://www.twitter.com/martinfowler' title = 'Twitter stream'></a></li>
<li class = 'icon'><a href = 'https://toot.thoughtworks.com/@mfowler' title = 'Mastodon stream'><svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 24 24" fill="currentColor"><path d="M21.2595 13.9898C20.9852 15.4006 18.8033 16.9446 16.2974 17.2439C14.9907 17.3998 13.7041 17.5431 12.3321 17.4802C10.0885 17.3774 8.31809 16.9446 8.31809 16.9446C8.31809 17.163 8.33156 17.371 8.3585 17.5655C8.65019 19.7797 10.5541 19.9124 12.3576 19.9742C14.1779 20.0365 15.7987 19.5254 15.7987 19.5254L15.8735 21.1711C15.8735 21.1711 14.6003 21.8548 12.3321 21.9805C11.0814 22.0493 9.52849 21.9491 7.71973 21.4703C3.79684 20.432 3.12219 16.2504 3.01896 12.0074C2.98749 10.7477 3.00689 9.55981 3.00689 8.56632C3.00689 4.22771 5.84955 2.95599 5.84955 2.95599C7.2829 2.29772 9.74238 2.0209 12.2993 2H12.3621C14.919 2.0209 17.3801 2.29772 18.8133 2.95599C18.8133 2.95599 21.6559 4.22771 21.6559 8.56632C21.6559 8.56632 21.6916 11.7674 21.2595 13.9898ZM18.3029 8.9029C18.3029 7.82924 18.0295 6.97604 17.4805 6.34482C16.9142 5.71359 16.1726 5.39001 15.2522 5.39001C14.187 5.39001 13.3805 5.79937 12.8473 6.61819L12.3288 7.48723L11.8104 6.61819C11.2771 5.79937 10.4706 5.39001 9.40554 5.39001C8.485 5.39001 7.74344 5.71359 7.17719 6.34482C6.62807 6.97604 6.3547 7.82924 6.3547 8.9029V14.1562H8.43597V9.05731C8.43597 7.98246 8.88822 7.4369 9.79281 7.4369C10.793 7.4369 11.2944 8.08408 11.2944 9.36376V12.1547H13.3634V9.36376C13.3634 8.08408 13.8646 7.4369 14.8648 7.4369C15.7694 7.4369 16.2216 7.98246 16.2216 9.05731V14.1562H18.3029V8.9029Z"></path></svg>
</a></li>
<li class = 'icon'><a href = 'https://www.linkedin.com/in/martin-fowler-com/' title = 'LinkedIn'><svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 24 24" fill="currentColor"><path d="M4.00098 3H20.001C20.5533 3 21.001 3.44772 21.001 4V20C21.001 20.5523 20.5533 21 20.001 21H4.00098C3.44869 21 3.00098 20.5523 3.00098 20V4C3.00098 3.44772 3.44869 3 4.00098 3ZM5.00098 5V19H19.001V5H5.00098ZM7.50098 9C6.67255 9 6.00098 8.32843 6.00098 7.5C6.00098 6.67157 6.67255 6 7.50098 6C8.3294 6 9.00098 6.67157 9.00098 7.5C9.00098 8.32843 8.3294 9 7.50098 9ZM6.50098 10H8.50098V17.5H6.50098V10ZM12.001 10.4295C12.5854 9.86534 13.2665 9.5 14.001 9.5C16.072 9.5 17.501 11.1789 17.501 13.25V17.5H15.501V13.25C15.501 12.2835 14.7175 11.5 13.751 11.5C12.7845 11.5 12.001 12.2835 12.001 13.25V17.5H10.001V10H12.001V10.4295Z"></path></svg>
</a></li>
<li class = 'icon'><a href = 'https://bsky.app/profile/martinfowler.com' title = 'BlueSky'><svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 24 24" fill="currentColor"><path d="M12 11.3884C11.0942 9.62673 8.62833 6.34423 6.335 4.7259C4.13833 3.17506 3.30083 3.4434 2.75167 3.69256C2.11583 3.9784 2 4.95506 2 5.52839C2 6.10339 2.315 10.2367 2.52 10.9276C3.19917 13.2076 5.61417 13.9776 7.83917 13.7309C4.57917 14.2142 1.68333 15.4017 5.48083 19.6292C9.65833 23.9542 11.2058 18.7017 12 16.0392C12.7942 18.7017 13.7083 23.7651 18.4442 19.6292C22 16.0392 19.4208 14.2142 16.1608 13.7309C18.3858 13.9784 20.8008 13.2076 21.48 10.9276C21.685 10.2376 22 6.10256 22 5.52923C22 4.95423 21.8842 3.97839 21.2483 3.6909C20.6992 3.44256 19.8617 3.17423 17.665 4.72423C15.3717 6.34506 12.9058 9.62756 12 11.3884Z"></path></svg></a></li>
</ul>
</nav>
</header>
<nav id = 'top-navmenu'>
<nav class = 'navmenu'>
<div class = 'nav-head'> <div class = 'search'>
<!-- SiteSearch Google -->
<form method='GET' action="https://www.google.com/search">
<input type='hidden' name='ie' value='UTF-8'/>
<input type='hidden' name='oe' value='UTF-8'/>
<input class = 'field' type='text'
name='q' size='15' maxlength='255' value=""/>
<button class = 'button' type='submit'
name='btnG' value=" " title = "Search"/>
<input type='hidden' name='domains' value="martinfowler.com"/>
<input type='hidden' name='sitesearch' value=""/>
<input
type='hidden' name='sitesearch' value="martinfowler.com"/>
</form>
</div>
<div class = 'closediv'>
<span class = 'close' title = 'close'></span>
</div>
</div>
<div class = 'nav-body'>
<div class = 'topics'>
<h2>Topics</h2>
<p><a href = '/architecture'>Architecture</a></p>
<p><a href = 'https://refactoring.com'>Refactoring</a></p>
<p><a href = '/agile.html'>Agile</a></p>
<p><a href = '/delivery.html'>Delivery</a></p>
<p><a href = '/microservices'>Microservices</a></p>
<p><a href = '/data'>Data</a></p>
<p><a href = '/testing'>Testing</a></p>
<p><a href = '/dsl.html'>DSL</a></p>
</div>
<div class = 'about'>
<h2>about me</h2>
<p><a href = '/aboutMe.html'>About</a></p>
<p><a href = '/books'>Books</a></p>
<p><a href = '/faq.html'>FAQ</a></p>
</div>
<div class = 'content'>
<h2>content</h2>
<p><a href = '/videos.html'>Videos</a></p>
<p><a href = '/tags'>Content Index</a></p>
<p><a href = '/boardgames'>Board Games</a></p>
<p><a href = '/photos'>Photography</a></p>
</div>
<div class = 'tw'>
<h2>Thoughtworks</h2>
<p><a href = 'https://thoughtworks.com'>Home</a></p>
<p><a href = 'https://thoughtworks.com/insights'>Insights</a></p>
<p><a href = 'https://thoughtworks.com/careers'>Careers</a></p>
<p><a href = 'https://thoughtworks.com/radar'>Radar</a></p>
<p><a href = 'https://www.thoughtworks.com/engineering'>Engineering</a></p>
</div>
<div class = 'feeds'>
<h2>follow</h2>
<p><a href = '/feed.atom'>RSS</a></p>
<p><a href = 'https://toot.thoughtworks.com/@mfowler'>Mastodon</a></p>
<p><a href = 'https://www.linkedin.com/in/martin-fowler-com/'>LinkedIn</a></p>
<p><a href = 'https://bsky.app/profile/martinfowler.com'>Bluesky</a></p>
<p><a href = 'https://www.twitter.com/martinfowler'>X</a></p>
<p><a href = 'https://boardgamegeek.com/blog/13064/martins-7th-decade'>BGG</a></p>
</div>
</div>
</nav>
</nav>
<main><h1 id="section">404</h1>
<p>Im afraid this is not the document youre looking for. Try using the
search box above, and good luck.</p>
</main>
<nav id = 'bottom-navmenu'>
<nav class = 'navmenu'>
<div class = 'nav-head'> <div class = 'search'>
<!-- SiteSearch Google -->
<form method='GET' action="https://www.google.com/search">
<input type='hidden' name='ie' value='UTF-8'/>
<input type='hidden' name='oe' value='UTF-8'/>
<input class = 'field' type='text'
name='q' size='15' maxlength='255' value=""/>
<button class = 'button' type='submit'
name='btnG' value=" " title = "Search"/>
<input type='hidden' name='domains' value="martinfowler.com"/>
<input type='hidden' name='sitesearch' value=""/>
<input
type='hidden' name='sitesearch' value="martinfowler.com"/>
</form>
</div>
<div class = 'closediv'>
<span class = 'close' title = 'close'></span>
</div>
</div>
<div class = 'nav-body'>
<div class = 'topics'>
<h2>Topics</h2>
<p><a href = '/architecture'>Architecture</a></p>
<p><a href = 'https://refactoring.com'>Refactoring</a></p>
<p><a href = '/agile.html'>Agile</a></p>
<p><a href = '/delivery.html'>Delivery</a></p>
<p><a href = '/microservices'>Microservices</a></p>
<p><a href = '/data'>Data</a></p>
<p><a href = '/testing'>Testing</a></p>
<p><a href = '/dsl.html'>DSL</a></p>
</div>
<div class = 'about'>
<h2>about me</h2>
<p><a href = '/aboutMe.html'>About</a></p>
<p><a href = '/books'>Books</a></p>
<p><a href = '/faq.html'>FAQ</a></p>
</div>
<div class = 'content'>
<h2>content</h2>
<p><a href = '/videos.html'>Videos</a></p>
<p><a href = '/tags'>Content Index</a></p>
<p><a href = '/boardgames'>Board Games</a></p>
<p><a href = '/photos'>Photography</a></p>
</div>
<div class = 'tw'>
<h2>Thoughtworks</h2>
<p><a href = 'https://thoughtworks.com'>Home</a></p>
<p><a href = 'https://thoughtworks.com/insights'>Insights</a></p>
<p><a href = 'https://thoughtworks.com/careers'>Careers</a></p>
<p><a href = 'https://thoughtworks.com/radar'>Radar</a></p>
<p><a href = 'https://www.thoughtworks.com/engineering'>Engineering</a></p>
</div>
<div class = 'feeds'>
<h2>follow</h2>
<p><a href = '/feed.atom'>RSS</a></p>
<p><a href = 'https://toot.thoughtworks.com/@mfowler'>Mastodon</a></p>
<p><a href = 'https://www.linkedin.com/in/martin-fowler-com/'>LinkedIn</a></p>
<p><a href = 'https://bsky.app/profile/martinfowler.com'>Bluesky</a></p>
<p><a href = 'https://www.twitter.com/martinfowler'>X</a></p>
<p><a href = 'https://boardgamegeek.com/blog/13064/martins-7th-decade'>BGG</a></p>
</div>
</div>
</nav>
</nav>
<footer id='page-footer'>
<div class='tw-logo'>
<a href='https://www.thoughtworks.com/engineering'>
<img src='/thoughtworks_white.png'>
</a>
</div>
<div class='menu-button'>
<div class='icon-bars navmenu-button'></div>
</div>
<div class='copyright'>
<p>© Martin Fowler | <a href="/aboutMe.html#disclosures">Disclosures</a></p>
</div>
</footer>
<script src = '/jquery-1.11.3.min.js' type = 'text/javascript'></script>
<script src = '/mfcom.js' type = 'text/javascript'></script>
</body>
</html>
@@ -1,280 +0,0 @@
<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd">
<html>
<head>
<meta content = 'uft-8' name = 'charset'></meta>
<title>not found</title>
<meta http-equiv="Content-type" content="text/html;charset=UTF-8" /><script defer src="https://cloud.umami.is/script.js" data-website-id="eb12527f-b713-4afa-905a-8a50f8a7f157"></script>
<link href = '/global.css' rel = 'stylesheet' type = 'text/css'></link>
</head>
<body><header id = 'banner' style = 'background-image: url("/banner.png"); background-repeat: no-repeat'>
<div class = 'name-logo'><a href = 'https://martinfowler.com'><img src = '/mf-name-white.png'></img></a></div>
<div class = 'search'>
<!-- SiteSearch Google -->
<form method='GET' action="https://www.google.com/search">
<input type='hidden' name='ie' value='UTF-8'/>
<input type='hidden' name='oe' value='UTF-8'/>
<input class = 'field' type='text'
name='q' size='15' maxlength='255' value=""/>
<button class = 'button' type='submit'
name='btnG' value=" " title = "Search"/>
<input type='hidden' name='domains' value="martinfowler.com"/>
<input type='hidden' name='sitesearch' value=""/>
<input
type='hidden' name='sitesearch' value="martinfowler.com"/>
</form>
</div>
<div class = 'menu-button navmenu-button'><a class = 'icon icon-bars' href = '#navmenu-bottom'></a></div>
<nav class = 'top-menu'>
<ul>
<li><a class = '' href = 'https://refactoring.com'>Refactoring</a></li>
<li><a class = '' href = '/agile.html'>Agile</a></li>
<li><a class = '' href = '/architecture'>Architecture</a></li>
<li><a class = '' href = '/aboutMe.html'>About</a></li>
<li><a class = 'tw' href = 'https://www.thoughtworks.com/engineering'>Thoughtworks</a></li>
<li><a class = 'icon icon-rss' href = '/feed.atom' title = 'feed'></a></li>
<li><a class = 'icon icon-twitter' href = 'https://www.twitter.com/martinfowler' title = 'Twitter stream'></a></li>
<li class = 'icon'><a href = 'https://toot.thoughtworks.com/@mfowler' title = 'Mastodon stream'><svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 24 24" fill="currentColor"><path d="M21.2595 13.9898C20.9852 15.4006 18.8033 16.9446 16.2974 17.2439C14.9907 17.3998 13.7041 17.5431 12.3321 17.4802C10.0885 17.3774 8.31809 16.9446 8.31809 16.9446C8.31809 17.163 8.33156 17.371 8.3585 17.5655C8.65019 19.7797 10.5541 19.9124 12.3576 19.9742C14.1779 20.0365 15.7987 19.5254 15.7987 19.5254L15.8735 21.1711C15.8735 21.1711 14.6003 21.8548 12.3321 21.9805C11.0814 22.0493 9.52849 21.9491 7.71973 21.4703C3.79684 20.432 3.12219 16.2504 3.01896 12.0074C2.98749 10.7477 3.00689 9.55981 3.00689 8.56632C3.00689 4.22771 5.84955 2.95599 5.84955 2.95599C7.2829 2.29772 9.74238 2.0209 12.2993 2H12.3621C14.919 2.0209 17.3801 2.29772 18.8133 2.95599C18.8133 2.95599 21.6559 4.22771 21.6559 8.56632C21.6559 8.56632 21.6916 11.7674 21.2595 13.9898ZM18.3029 8.9029C18.3029 7.82924 18.0295 6.97604 17.4805 6.34482C16.9142 5.71359 16.1726 5.39001 15.2522 5.39001C14.187 5.39001 13.3805 5.79937 12.8473 6.61819L12.3288 7.48723L11.8104 6.61819C11.2771 5.79937 10.4706 5.39001 9.40554 5.39001C8.485 5.39001 7.74344 5.71359 7.17719 6.34482C6.62807 6.97604 6.3547 7.82924 6.3547 8.9029V14.1562H8.43597V9.05731C8.43597 7.98246 8.88822 7.4369 9.79281 7.4369C10.793 7.4369 11.2944 8.08408 11.2944 9.36376V12.1547H13.3634V9.36376C13.3634 8.08408 13.8646 7.4369 14.8648 7.4369C15.7694 7.4369 16.2216 7.98246 16.2216 9.05731V14.1562H18.3029V8.9029Z"></path></svg>
</a></li>
<li class = 'icon'><a href = 'https://www.linkedin.com/in/martin-fowler-com/' title = 'LinkedIn'><svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 24 24" fill="currentColor"><path d="M4.00098 3H20.001C20.5533 3 21.001 3.44772 21.001 4V20C21.001 20.5523 20.5533 21 20.001 21H4.00098C3.44869 21 3.00098 20.5523 3.00098 20V4C3.00098 3.44772 3.44869 3 4.00098 3ZM5.00098 5V19H19.001V5H5.00098ZM7.50098 9C6.67255 9 6.00098 8.32843 6.00098 7.5C6.00098 6.67157 6.67255 6 7.50098 6C8.3294 6 9.00098 6.67157 9.00098 7.5C9.00098 8.32843 8.3294 9 7.50098 9ZM6.50098 10H8.50098V17.5H6.50098V10ZM12.001 10.4295C12.5854 9.86534 13.2665 9.5 14.001 9.5C16.072 9.5 17.501 11.1789 17.501 13.25V17.5H15.501V13.25C15.501 12.2835 14.7175 11.5 13.751 11.5C12.7845 11.5 12.001 12.2835 12.001 13.25V17.5H10.001V10H12.001V10.4295Z"></path></svg>
</a></li>
<li class = 'icon'><a href = 'https://bsky.app/profile/martinfowler.com' title = 'BlueSky'><svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 24 24" fill="currentColor"><path d="M12 11.3884C11.0942 9.62673 8.62833 6.34423 6.335 4.7259C4.13833 3.17506 3.30083 3.4434 2.75167 3.69256C2.11583 3.9784 2 4.95506 2 5.52839C2 6.10339 2.315 10.2367 2.52 10.9276C3.19917 13.2076 5.61417 13.9776 7.83917 13.7309C4.57917 14.2142 1.68333 15.4017 5.48083 19.6292C9.65833 23.9542 11.2058 18.7017 12 16.0392C12.7942 18.7017 13.7083 23.7651 18.4442 19.6292C22 16.0392 19.4208 14.2142 16.1608 13.7309C18.3858 13.9784 20.8008 13.2076 21.48 10.9276C21.685 10.2376 22 6.10256 22 5.52923C22 4.95423 21.8842 3.97839 21.2483 3.6909C20.6992 3.44256 19.8617 3.17423 17.665 4.72423C15.3717 6.34506 12.9058 9.62756 12 11.3884Z"></path></svg></a></li>
</ul>
</nav>
</header>
<nav id = 'top-navmenu'>
<nav class = 'navmenu'>
<div class = 'nav-head'> <div class = 'search'>
<!-- SiteSearch Google -->
<form method='GET' action="https://www.google.com/search">
<input type='hidden' name='ie' value='UTF-8'/>
<input type='hidden' name='oe' value='UTF-8'/>
<input class = 'field' type='text'
name='q' size='15' maxlength='255' value=""/>
<button class = 'button' type='submit'
name='btnG' value=" " title = "Search"/>
<input type='hidden' name='domains' value="martinfowler.com"/>
<input type='hidden' name='sitesearch' value=""/>
<input
type='hidden' name='sitesearch' value="martinfowler.com"/>
</form>
</div>
<div class = 'closediv'>
<span class = 'close' title = 'close'></span>
</div>
</div>
<div class = 'nav-body'>
<div class = 'topics'>
<h2>Topics</h2>
<p><a href = '/architecture'>Architecture</a></p>
<p><a href = 'https://refactoring.com'>Refactoring</a></p>
<p><a href = '/agile.html'>Agile</a></p>
<p><a href = '/delivery.html'>Delivery</a></p>
<p><a href = '/microservices'>Microservices</a></p>
<p><a href = '/data'>Data</a></p>
<p><a href = '/testing'>Testing</a></p>
<p><a href = '/dsl.html'>DSL</a></p>
</div>
<div class = 'about'>
<h2>about me</h2>
<p><a href = '/aboutMe.html'>About</a></p>
<p><a href = '/books'>Books</a></p>
<p><a href = '/faq.html'>FAQ</a></p>
</div>
<div class = 'content'>
<h2>content</h2>
<p><a href = '/videos.html'>Videos</a></p>
<p><a href = '/tags'>Content Index</a></p>
<p><a href = '/boardgames'>Board Games</a></p>
<p><a href = '/photos'>Photography</a></p>
</div>
<div class = 'tw'>
<h2>Thoughtworks</h2>
<p><a href = 'https://thoughtworks.com'>Home</a></p>
<p><a href = 'https://thoughtworks.com/insights'>Insights</a></p>
<p><a href = 'https://thoughtworks.com/careers'>Careers</a></p>
<p><a href = 'https://thoughtworks.com/radar'>Radar</a></p>
<p><a href = 'https://www.thoughtworks.com/engineering'>Engineering</a></p>
</div>
<div class = 'feeds'>
<h2>follow</h2>
<p><a href = '/feed.atom'>RSS</a></p>
<p><a href = 'https://toot.thoughtworks.com/@mfowler'>Mastodon</a></p>
<p><a href = 'https://www.linkedin.com/in/martin-fowler-com/'>LinkedIn</a></p>
<p><a href = 'https://bsky.app/profile/martinfowler.com'>Bluesky</a></p>
<p><a href = 'https://www.twitter.com/martinfowler'>X</a></p>
<p><a href = 'https://boardgamegeek.com/blog/13064/martins-7th-decade'>BGG</a></p>
</div>
</div>
</nav>
</nav>
<main><h1 id="section">404</h1>
<p>Im afraid this is not the document youre looking for. Try using the
search box above, and good luck.</p>
</main>
<nav id = 'bottom-navmenu'>
<nav class = 'navmenu'>
<div class = 'nav-head'> <div class = 'search'>
<!-- SiteSearch Google -->
<form method='GET' action="https://www.google.com/search">
<input type='hidden' name='ie' value='UTF-8'/>
<input type='hidden' name='oe' value='UTF-8'/>
<input class = 'field' type='text'
name='q' size='15' maxlength='255' value=""/>
<button class = 'button' type='submit'
name='btnG' value=" " title = "Search"/>
<input type='hidden' name='domains' value="martinfowler.com"/>
<input type='hidden' name='sitesearch' value=""/>
<input
type='hidden' name='sitesearch' value="martinfowler.com"/>
</form>
</div>
<div class = 'closediv'>
<span class = 'close' title = 'close'></span>
</div>
</div>
<div class = 'nav-body'>
<div class = 'topics'>
<h2>Topics</h2>
<p><a href = '/architecture'>Architecture</a></p>
<p><a href = 'https://refactoring.com'>Refactoring</a></p>
<p><a href = '/agile.html'>Agile</a></p>
<p><a href = '/delivery.html'>Delivery</a></p>
<p><a href = '/microservices'>Microservices</a></p>
<p><a href = '/data'>Data</a></p>
<p><a href = '/testing'>Testing</a></p>
<p><a href = '/dsl.html'>DSL</a></p>
</div>
<div class = 'about'>
<h2>about me</h2>
<p><a href = '/aboutMe.html'>About</a></p>
<p><a href = '/books'>Books</a></p>
<p><a href = '/faq.html'>FAQ</a></p>
</div>
<div class = 'content'>
<h2>content</h2>
<p><a href = '/videos.html'>Videos</a></p>
<p><a href = '/tags'>Content Index</a></p>
<p><a href = '/boardgames'>Board Games</a></p>
<p><a href = '/photos'>Photography</a></p>
</div>
<div class = 'tw'>
<h2>Thoughtworks</h2>
<p><a href = 'https://thoughtworks.com'>Home</a></p>
<p><a href = 'https://thoughtworks.com/insights'>Insights</a></p>
<p><a href = 'https://thoughtworks.com/careers'>Careers</a></p>
<p><a href = 'https://thoughtworks.com/radar'>Radar</a></p>
<p><a href = 'https://www.thoughtworks.com/engineering'>Engineering</a></p>
</div>
<div class = 'feeds'>
<h2>follow</h2>
<p><a href = '/feed.atom'>RSS</a></p>
<p><a href = 'https://toot.thoughtworks.com/@mfowler'>Mastodon</a></p>
<p><a href = 'https://www.linkedin.com/in/martin-fowler-com/'>LinkedIn</a></p>
<p><a href = 'https://bsky.app/profile/martinfowler.com'>Bluesky</a></p>
<p><a href = 'https://www.twitter.com/martinfowler'>X</a></p>
<p><a href = 'https://boardgamegeek.com/blog/13064/martins-7th-decade'>BGG</a></p>
</div>
</div>
</nav>
</nav>
<footer id='page-footer'>
<div class='tw-logo'>
<a href='https://www.thoughtworks.com/engineering'>
<img src='/thoughtworks_white.png'>
</a>
</div>
<div class='menu-button'>
<div class='icon-bars navmenu-button'></div>
</div>
<div class='copyright'>
<p>© Martin Fowler | <a href="/aboutMe.html#disclosures">Disclosures</a></p>
</div>
</footer>
<script src = '/jquery-1.11.3.min.js' type = 'text/javascript'></script>
<script src = '/mfcom.js' type = 'text/javascript'></script>
</body>
</html>
@@ -1,280 +0,0 @@
<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd">
<html>
<head>
<meta content = 'uft-8' name = 'charset'></meta>
<title>not found</title>
<meta http-equiv="Content-type" content="text/html;charset=UTF-8" /><script defer src="https://cloud.umami.is/script.js" data-website-id="eb12527f-b713-4afa-905a-8a50f8a7f157"></script>
<link href = '/global.css' rel = 'stylesheet' type = 'text/css'></link>
</head>
<body><header id = 'banner' style = 'background-image: url("/banner.png"); background-repeat: no-repeat'>
<div class = 'name-logo'><a href = 'https://martinfowler.com'><img src = '/mf-name-white.png'></img></a></div>
<div class = 'search'>
<!-- SiteSearch Google -->
<form method='GET' action="https://www.google.com/search">
<input type='hidden' name='ie' value='UTF-8'/>
<input type='hidden' name='oe' value='UTF-8'/>
<input class = 'field' type='text'
name='q' size='15' maxlength='255' value=""/>
<button class = 'button' type='submit'
name='btnG' value=" " title = "Search"/>
<input type='hidden' name='domains' value="martinfowler.com"/>
<input type='hidden' name='sitesearch' value=""/>
<input
type='hidden' name='sitesearch' value="martinfowler.com"/>
</form>
</div>
<div class = 'menu-button navmenu-button'><a class = 'icon icon-bars' href = '#navmenu-bottom'></a></div>
<nav class = 'top-menu'>
<ul>
<li><a class = '' href = 'https://refactoring.com'>Refactoring</a></li>
<li><a class = '' href = '/agile.html'>Agile</a></li>
<li><a class = '' href = '/architecture'>Architecture</a></li>
<li><a class = '' href = '/aboutMe.html'>About</a></li>
<li><a class = 'tw' href = 'https://www.thoughtworks.com/engineering'>Thoughtworks</a></li>
<li><a class = 'icon icon-rss' href = '/feed.atom' title = 'feed'></a></li>
<li><a class = 'icon icon-twitter' href = 'https://www.twitter.com/martinfowler' title = 'Twitter stream'></a></li>
<li class = 'icon'><a href = 'https://toot.thoughtworks.com/@mfowler' title = 'Mastodon stream'><svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 24 24" fill="currentColor"><path d="M21.2595 13.9898C20.9852 15.4006 18.8033 16.9446 16.2974 17.2439C14.9907 17.3998 13.7041 17.5431 12.3321 17.4802C10.0885 17.3774 8.31809 16.9446 8.31809 16.9446C8.31809 17.163 8.33156 17.371 8.3585 17.5655C8.65019 19.7797 10.5541 19.9124 12.3576 19.9742C14.1779 20.0365 15.7987 19.5254 15.7987 19.5254L15.8735 21.1711C15.8735 21.1711 14.6003 21.8548 12.3321 21.9805C11.0814 22.0493 9.52849 21.9491 7.71973 21.4703C3.79684 20.432 3.12219 16.2504 3.01896 12.0074C2.98749 10.7477 3.00689 9.55981 3.00689 8.56632C3.00689 4.22771 5.84955 2.95599 5.84955 2.95599C7.2829 2.29772 9.74238 2.0209 12.2993 2H12.3621C14.919 2.0209 17.3801 2.29772 18.8133 2.95599C18.8133 2.95599 21.6559 4.22771 21.6559 8.56632C21.6559 8.56632 21.6916 11.7674 21.2595 13.9898ZM18.3029 8.9029C18.3029 7.82924 18.0295 6.97604 17.4805 6.34482C16.9142 5.71359 16.1726 5.39001 15.2522 5.39001C14.187 5.39001 13.3805 5.79937 12.8473 6.61819L12.3288 7.48723L11.8104 6.61819C11.2771 5.79937 10.4706 5.39001 9.40554 5.39001C8.485 5.39001 7.74344 5.71359 7.17719 6.34482C6.62807 6.97604 6.3547 7.82924 6.3547 8.9029V14.1562H8.43597V9.05731C8.43597 7.98246 8.88822 7.4369 9.79281 7.4369C10.793 7.4369 11.2944 8.08408 11.2944 9.36376V12.1547H13.3634V9.36376C13.3634 8.08408 13.8646 7.4369 14.8648 7.4369C15.7694 7.4369 16.2216 7.98246 16.2216 9.05731V14.1562H18.3029V8.9029Z"></path></svg>
</a></li>
<li class = 'icon'><a href = 'https://www.linkedin.com/in/martin-fowler-com/' title = 'LinkedIn'><svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 24 24" fill="currentColor"><path d="M4.00098 3H20.001C20.5533 3 21.001 3.44772 21.001 4V20C21.001 20.5523 20.5533 21 20.001 21H4.00098C3.44869 21 3.00098 20.5523 3.00098 20V4C3.00098 3.44772 3.44869 3 4.00098 3ZM5.00098 5V19H19.001V5H5.00098ZM7.50098 9C6.67255 9 6.00098 8.32843 6.00098 7.5C6.00098 6.67157 6.67255 6 7.50098 6C8.3294 6 9.00098 6.67157 9.00098 7.5C9.00098 8.32843 8.3294 9 7.50098 9ZM6.50098 10H8.50098V17.5H6.50098V10ZM12.001 10.4295C12.5854 9.86534 13.2665 9.5 14.001 9.5C16.072 9.5 17.501 11.1789 17.501 13.25V17.5H15.501V13.25C15.501 12.2835 14.7175 11.5 13.751 11.5C12.7845 11.5 12.001 12.2835 12.001 13.25V17.5H10.001V10H12.001V10.4295Z"></path></svg>
</a></li>
<li class = 'icon'><a href = 'https://bsky.app/profile/martinfowler.com' title = 'BlueSky'><svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 24 24" fill="currentColor"><path d="M12 11.3884C11.0942 9.62673 8.62833 6.34423 6.335 4.7259C4.13833 3.17506 3.30083 3.4434 2.75167 3.69256C2.11583 3.9784 2 4.95506 2 5.52839C2 6.10339 2.315 10.2367 2.52 10.9276C3.19917 13.2076 5.61417 13.9776 7.83917 13.7309C4.57917 14.2142 1.68333 15.4017 5.48083 19.6292C9.65833 23.9542 11.2058 18.7017 12 16.0392C12.7942 18.7017 13.7083 23.7651 18.4442 19.6292C22 16.0392 19.4208 14.2142 16.1608 13.7309C18.3858 13.9784 20.8008 13.2076 21.48 10.9276C21.685 10.2376 22 6.10256 22 5.52923C22 4.95423 21.8842 3.97839 21.2483 3.6909C20.6992 3.44256 19.8617 3.17423 17.665 4.72423C15.3717 6.34506 12.9058 9.62756 12 11.3884Z"></path></svg></a></li>
</ul>
</nav>
</header>
<nav id = 'top-navmenu'>
<nav class = 'navmenu'>
<div class = 'nav-head'> <div class = 'search'>
<!-- SiteSearch Google -->
<form method='GET' action="https://www.google.com/search">
<input type='hidden' name='ie' value='UTF-8'/>
<input type='hidden' name='oe' value='UTF-8'/>
<input class = 'field' type='text'
name='q' size='15' maxlength='255' value=""/>
<button class = 'button' type='submit'
name='btnG' value=" " title = "Search"/>
<input type='hidden' name='domains' value="martinfowler.com"/>
<input type='hidden' name='sitesearch' value=""/>
<input
type='hidden' name='sitesearch' value="martinfowler.com"/>
</form>
</div>
<div class = 'closediv'>
<span class = 'close' title = 'close'></span>
</div>
</div>
<div class = 'nav-body'>
<div class = 'topics'>
<h2>Topics</h2>
<p><a href = '/architecture'>Architecture</a></p>
<p><a href = 'https://refactoring.com'>Refactoring</a></p>
<p><a href = '/agile.html'>Agile</a></p>
<p><a href = '/delivery.html'>Delivery</a></p>
<p><a href = '/microservices'>Microservices</a></p>
<p><a href = '/data'>Data</a></p>
<p><a href = '/testing'>Testing</a></p>
<p><a href = '/dsl.html'>DSL</a></p>
</div>
<div class = 'about'>
<h2>about me</h2>
<p><a href = '/aboutMe.html'>About</a></p>
<p><a href = '/books'>Books</a></p>
<p><a href = '/faq.html'>FAQ</a></p>
</div>
<div class = 'content'>
<h2>content</h2>
<p><a href = '/videos.html'>Videos</a></p>
<p><a href = '/tags'>Content Index</a></p>
<p><a href = '/boardgames'>Board Games</a></p>
<p><a href = '/photos'>Photography</a></p>
</div>
<div class = 'tw'>
<h2>Thoughtworks</h2>
<p><a href = 'https://thoughtworks.com'>Home</a></p>
<p><a href = 'https://thoughtworks.com/insights'>Insights</a></p>
<p><a href = 'https://thoughtworks.com/careers'>Careers</a></p>
<p><a href = 'https://thoughtworks.com/radar'>Radar</a></p>
<p><a href = 'https://www.thoughtworks.com/engineering'>Engineering</a></p>
</div>
<div class = 'feeds'>
<h2>follow</h2>
<p><a href = '/feed.atom'>RSS</a></p>
<p><a href = 'https://toot.thoughtworks.com/@mfowler'>Mastodon</a></p>
<p><a href = 'https://www.linkedin.com/in/martin-fowler-com/'>LinkedIn</a></p>
<p><a href = 'https://bsky.app/profile/martinfowler.com'>Bluesky</a></p>
<p><a href = 'https://www.twitter.com/martinfowler'>X</a></p>
<p><a href = 'https://boardgamegeek.com/blog/13064/martins-7th-decade'>BGG</a></p>
</div>
</div>
</nav>
</nav>
<main><h1 id="section">404</h1>
<p>Im afraid this is not the document youre looking for. Try using the
search box above, and good luck.</p>
</main>
<nav id = 'bottom-navmenu'>
<nav class = 'navmenu'>
<div class = 'nav-head'> <div class = 'search'>
<!-- SiteSearch Google -->
<form method='GET' action="https://www.google.com/search">
<input type='hidden' name='ie' value='UTF-8'/>
<input type='hidden' name='oe' value='UTF-8'/>
<input class = 'field' type='text'
name='q' size='15' maxlength='255' value=""/>
<button class = 'button' type='submit'
name='btnG' value=" " title = "Search"/>
<input type='hidden' name='domains' value="martinfowler.com"/>
<input type='hidden' name='sitesearch' value=""/>
<input
type='hidden' name='sitesearch' value="martinfowler.com"/>
</form>
</div>
<div class = 'closediv'>
<span class = 'close' title = 'close'></span>
</div>
</div>
<div class = 'nav-body'>
<div class = 'topics'>
<h2>Topics</h2>
<p><a href = '/architecture'>Architecture</a></p>
<p><a href = 'https://refactoring.com'>Refactoring</a></p>
<p><a href = '/agile.html'>Agile</a></p>
<p><a href = '/delivery.html'>Delivery</a></p>
<p><a href = '/microservices'>Microservices</a></p>
<p><a href = '/data'>Data</a></p>
<p><a href = '/testing'>Testing</a></p>
<p><a href = '/dsl.html'>DSL</a></p>
</div>
<div class = 'about'>
<h2>about me</h2>
<p><a href = '/aboutMe.html'>About</a></p>
<p><a href = '/books'>Books</a></p>
<p><a href = '/faq.html'>FAQ</a></p>
</div>
<div class = 'content'>
<h2>content</h2>
<p><a href = '/videos.html'>Videos</a></p>
<p><a href = '/tags'>Content Index</a></p>
<p><a href = '/boardgames'>Board Games</a></p>
<p><a href = '/photos'>Photography</a></p>
</div>
<div class = 'tw'>
<h2>Thoughtworks</h2>
<p><a href = 'https://thoughtworks.com'>Home</a></p>
<p><a href = 'https://thoughtworks.com/insights'>Insights</a></p>
<p><a href = 'https://thoughtworks.com/careers'>Careers</a></p>
<p><a href = 'https://thoughtworks.com/radar'>Radar</a></p>
<p><a href = 'https://www.thoughtworks.com/engineering'>Engineering</a></p>
</div>
<div class = 'feeds'>
<h2>follow</h2>
<p><a href = '/feed.atom'>RSS</a></p>
<p><a href = 'https://toot.thoughtworks.com/@mfowler'>Mastodon</a></p>
<p><a href = 'https://www.linkedin.com/in/martin-fowler-com/'>LinkedIn</a></p>
<p><a href = 'https://bsky.app/profile/martinfowler.com'>Bluesky</a></p>
<p><a href = 'https://www.twitter.com/martinfowler'>X</a></p>
<p><a href = 'https://boardgamegeek.com/blog/13064/martins-7th-decade'>BGG</a></p>
</div>
</div>
</nav>
</nav>
<footer id='page-footer'>
<div class='tw-logo'>
<a href='https://www.thoughtworks.com/engineering'>
<img src='/thoughtworks_white.png'>
</a>
</div>
<div class='menu-button'>
<div class='icon-bars navmenu-button'></div>
</div>
<div class='copyright'>
<p>© Martin Fowler | <a href="/aboutMe.html#disclosures">Disclosures</a></p>
</div>
</footer>
<script src = '/jquery-1.11.3.min.js' type = 'text/javascript'></script>
<script src = '/mfcom.js' type = 'text/javascript'></script>
</body>
</html>
@@ -1,280 +0,0 @@
<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd">
<html>
<head>
<meta content = 'uft-8' name = 'charset'></meta>
<title>not found</title>
<meta http-equiv="Content-type" content="text/html;charset=UTF-8" /><script defer src="https://cloud.umami.is/script.js" data-website-id="eb12527f-b713-4afa-905a-8a50f8a7f157"></script>
<link href = '/global.css' rel = 'stylesheet' type = 'text/css'></link>
</head>
<body><header id = 'banner' style = 'background-image: url("/banner.png"); background-repeat: no-repeat'>
<div class = 'name-logo'><a href = 'https://martinfowler.com'><img src = '/mf-name-white.png'></img></a></div>
<div class = 'search'>
<!-- SiteSearch Google -->
<form method='GET' action="https://www.google.com/search">
<input type='hidden' name='ie' value='UTF-8'/>
<input type='hidden' name='oe' value='UTF-8'/>
<input class = 'field' type='text'
name='q' size='15' maxlength='255' value=""/>
<button class = 'button' type='submit'
name='btnG' value=" " title = "Search"/>
<input type='hidden' name='domains' value="martinfowler.com"/>
<input type='hidden' name='sitesearch' value=""/>
<input
type='hidden' name='sitesearch' value="martinfowler.com"/>
</form>
</div>
<div class = 'menu-button navmenu-button'><a class = 'icon icon-bars' href = '#navmenu-bottom'></a></div>
<nav class = 'top-menu'>
<ul>
<li><a class = '' href = 'https://refactoring.com'>Refactoring</a></li>
<li><a class = '' href = '/agile.html'>Agile</a></li>
<li><a class = '' href = '/architecture'>Architecture</a></li>
<li><a class = '' href = '/aboutMe.html'>About</a></li>
<li><a class = 'tw' href = 'https://www.thoughtworks.com/engineering'>Thoughtworks</a></li>
<li><a class = 'icon icon-rss' href = '/feed.atom' title = 'feed'></a></li>
<li><a class = 'icon icon-twitter' href = 'https://www.twitter.com/martinfowler' title = 'Twitter stream'></a></li>
<li class = 'icon'><a href = 'https://toot.thoughtworks.com/@mfowler' title = 'Mastodon stream'><svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 24 24" fill="currentColor"><path d="M21.2595 13.9898C20.9852 15.4006 18.8033 16.9446 16.2974 17.2439C14.9907 17.3998 13.7041 17.5431 12.3321 17.4802C10.0885 17.3774 8.31809 16.9446 8.31809 16.9446C8.31809 17.163 8.33156 17.371 8.3585 17.5655C8.65019 19.7797 10.5541 19.9124 12.3576 19.9742C14.1779 20.0365 15.7987 19.5254 15.7987 19.5254L15.8735 21.1711C15.8735 21.1711 14.6003 21.8548 12.3321 21.9805C11.0814 22.0493 9.52849 21.9491 7.71973 21.4703C3.79684 20.432 3.12219 16.2504 3.01896 12.0074C2.98749 10.7477 3.00689 9.55981 3.00689 8.56632C3.00689 4.22771 5.84955 2.95599 5.84955 2.95599C7.2829 2.29772 9.74238 2.0209 12.2993 2H12.3621C14.919 2.0209 17.3801 2.29772 18.8133 2.95599C18.8133 2.95599 21.6559 4.22771 21.6559 8.56632C21.6559 8.56632 21.6916 11.7674 21.2595 13.9898ZM18.3029 8.9029C18.3029 7.82924 18.0295 6.97604 17.4805 6.34482C16.9142 5.71359 16.1726 5.39001 15.2522 5.39001C14.187 5.39001 13.3805 5.79937 12.8473 6.61819L12.3288 7.48723L11.8104 6.61819C11.2771 5.79937 10.4706 5.39001 9.40554 5.39001C8.485 5.39001 7.74344 5.71359 7.17719 6.34482C6.62807 6.97604 6.3547 7.82924 6.3547 8.9029V14.1562H8.43597V9.05731C8.43597 7.98246 8.88822 7.4369 9.79281 7.4369C10.793 7.4369 11.2944 8.08408 11.2944 9.36376V12.1547H13.3634V9.36376C13.3634 8.08408 13.8646 7.4369 14.8648 7.4369C15.7694 7.4369 16.2216 7.98246 16.2216 9.05731V14.1562H18.3029V8.9029Z"></path></svg>
</a></li>
<li class = 'icon'><a href = 'https://www.linkedin.com/in/martin-fowler-com/' title = 'LinkedIn'><svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 24 24" fill="currentColor"><path d="M4.00098 3H20.001C20.5533 3 21.001 3.44772 21.001 4V20C21.001 20.5523 20.5533 21 20.001 21H4.00098C3.44869 21 3.00098 20.5523 3.00098 20V4C3.00098 3.44772 3.44869 3 4.00098 3ZM5.00098 5V19H19.001V5H5.00098ZM7.50098 9C6.67255 9 6.00098 8.32843 6.00098 7.5C6.00098 6.67157 6.67255 6 7.50098 6C8.3294 6 9.00098 6.67157 9.00098 7.5C9.00098 8.32843 8.3294 9 7.50098 9ZM6.50098 10H8.50098V17.5H6.50098V10ZM12.001 10.4295C12.5854 9.86534 13.2665 9.5 14.001 9.5C16.072 9.5 17.501 11.1789 17.501 13.25V17.5H15.501V13.25C15.501 12.2835 14.7175 11.5 13.751 11.5C12.7845 11.5 12.001 12.2835 12.001 13.25V17.5H10.001V10H12.001V10.4295Z"></path></svg>
</a></li>
<li class = 'icon'><a href = 'https://bsky.app/profile/martinfowler.com' title = 'BlueSky'><svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 24 24" fill="currentColor"><path d="M12 11.3884C11.0942 9.62673 8.62833 6.34423 6.335 4.7259C4.13833 3.17506 3.30083 3.4434 2.75167 3.69256C2.11583 3.9784 2 4.95506 2 5.52839C2 6.10339 2.315 10.2367 2.52 10.9276C3.19917 13.2076 5.61417 13.9776 7.83917 13.7309C4.57917 14.2142 1.68333 15.4017 5.48083 19.6292C9.65833 23.9542 11.2058 18.7017 12 16.0392C12.7942 18.7017 13.7083 23.7651 18.4442 19.6292C22 16.0392 19.4208 14.2142 16.1608 13.7309C18.3858 13.9784 20.8008 13.2076 21.48 10.9276C21.685 10.2376 22 6.10256 22 5.52923C22 4.95423 21.8842 3.97839 21.2483 3.6909C20.6992 3.44256 19.8617 3.17423 17.665 4.72423C15.3717 6.34506 12.9058 9.62756 12 11.3884Z"></path></svg></a></li>
</ul>
</nav>
</header>
<nav id = 'top-navmenu'>
<nav class = 'navmenu'>
<div class = 'nav-head'> <div class = 'search'>
<!-- SiteSearch Google -->
<form method='GET' action="https://www.google.com/search">
<input type='hidden' name='ie' value='UTF-8'/>
<input type='hidden' name='oe' value='UTF-8'/>
<input class = 'field' type='text'
name='q' size='15' maxlength='255' value=""/>
<button class = 'button' type='submit'
name='btnG' value=" " title = "Search"/>
<input type='hidden' name='domains' value="martinfowler.com"/>
<input type='hidden' name='sitesearch' value=""/>
<input
type='hidden' name='sitesearch' value="martinfowler.com"/>
</form>
</div>
<div class = 'closediv'>
<span class = 'close' title = 'close'></span>
</div>
</div>
<div class = 'nav-body'>
<div class = 'topics'>
<h2>Topics</h2>
<p><a href = '/architecture'>Architecture</a></p>
<p><a href = 'https://refactoring.com'>Refactoring</a></p>
<p><a href = '/agile.html'>Agile</a></p>
<p><a href = '/delivery.html'>Delivery</a></p>
<p><a href = '/microservices'>Microservices</a></p>
<p><a href = '/data'>Data</a></p>
<p><a href = '/testing'>Testing</a></p>
<p><a href = '/dsl.html'>DSL</a></p>
</div>
<div class = 'about'>
<h2>about me</h2>
<p><a href = '/aboutMe.html'>About</a></p>
<p><a href = '/books'>Books</a></p>
<p><a href = '/faq.html'>FAQ</a></p>
</div>
<div class = 'content'>
<h2>content</h2>
<p><a href = '/videos.html'>Videos</a></p>
<p><a href = '/tags'>Content Index</a></p>
<p><a href = '/boardgames'>Board Games</a></p>
<p><a href = '/photos'>Photography</a></p>
</div>
<div class = 'tw'>
<h2>Thoughtworks</h2>
<p><a href = 'https://thoughtworks.com'>Home</a></p>
<p><a href = 'https://thoughtworks.com/insights'>Insights</a></p>
<p><a href = 'https://thoughtworks.com/careers'>Careers</a></p>
<p><a href = 'https://thoughtworks.com/radar'>Radar</a></p>
<p><a href = 'https://www.thoughtworks.com/engineering'>Engineering</a></p>
</div>
<div class = 'feeds'>
<h2>follow</h2>
<p><a href = '/feed.atom'>RSS</a></p>
<p><a href = 'https://toot.thoughtworks.com/@mfowler'>Mastodon</a></p>
<p><a href = 'https://www.linkedin.com/in/martin-fowler-com/'>LinkedIn</a></p>
<p><a href = 'https://bsky.app/profile/martinfowler.com'>Bluesky</a></p>
<p><a href = 'https://www.twitter.com/martinfowler'>X</a></p>
<p><a href = 'https://boardgamegeek.com/blog/13064/martins-7th-decade'>BGG</a></p>
</div>
</div>
</nav>
</nav>
<main><h1 id="section">404</h1>
<p>Im afraid this is not the document youre looking for. Try using the
search box above, and good luck.</p>
</main>
<nav id = 'bottom-navmenu'>
<nav class = 'navmenu'>
<div class = 'nav-head'> <div class = 'search'>
<!-- SiteSearch Google -->
<form method='GET' action="https://www.google.com/search">
<input type='hidden' name='ie' value='UTF-8'/>
<input type='hidden' name='oe' value='UTF-8'/>
<input class = 'field' type='text'
name='q' size='15' maxlength='255' value=""/>
<button class = 'button' type='submit'
name='btnG' value=" " title = "Search"/>
<input type='hidden' name='domains' value="martinfowler.com"/>
<input type='hidden' name='sitesearch' value=""/>
<input
type='hidden' name='sitesearch' value="martinfowler.com"/>
</form>
</div>
<div class = 'closediv'>
<span class = 'close' title = 'close'></span>
</div>
</div>
<div class = 'nav-body'>
<div class = 'topics'>
<h2>Topics</h2>
<p><a href = '/architecture'>Architecture</a></p>
<p><a href = 'https://refactoring.com'>Refactoring</a></p>
<p><a href = '/agile.html'>Agile</a></p>
<p><a href = '/delivery.html'>Delivery</a></p>
<p><a href = '/microservices'>Microservices</a></p>
<p><a href = '/data'>Data</a></p>
<p><a href = '/testing'>Testing</a></p>
<p><a href = '/dsl.html'>DSL</a></p>
</div>
<div class = 'about'>
<h2>about me</h2>
<p><a href = '/aboutMe.html'>About</a></p>
<p><a href = '/books'>Books</a></p>
<p><a href = '/faq.html'>FAQ</a></p>
</div>
<div class = 'content'>
<h2>content</h2>
<p><a href = '/videos.html'>Videos</a></p>
<p><a href = '/tags'>Content Index</a></p>
<p><a href = '/boardgames'>Board Games</a></p>
<p><a href = '/photos'>Photography</a></p>
</div>
<div class = 'tw'>
<h2>Thoughtworks</h2>
<p><a href = 'https://thoughtworks.com'>Home</a></p>
<p><a href = 'https://thoughtworks.com/insights'>Insights</a></p>
<p><a href = 'https://thoughtworks.com/careers'>Careers</a></p>
<p><a href = 'https://thoughtworks.com/radar'>Radar</a></p>
<p><a href = 'https://www.thoughtworks.com/engineering'>Engineering</a></p>
</div>
<div class = 'feeds'>
<h2>follow</h2>
<p><a href = '/feed.atom'>RSS</a></p>
<p><a href = 'https://toot.thoughtworks.com/@mfowler'>Mastodon</a></p>
<p><a href = 'https://www.linkedin.com/in/martin-fowler-com/'>LinkedIn</a></p>
<p><a href = 'https://bsky.app/profile/martinfowler.com'>Bluesky</a></p>
<p><a href = 'https://www.twitter.com/martinfowler'>X</a></p>
<p><a href = 'https://boardgamegeek.com/blog/13064/martins-7th-decade'>BGG</a></p>
</div>
</div>
</nav>
</nav>
<footer id='page-footer'>
<div class='tw-logo'>
<a href='https://www.thoughtworks.com/engineering'>
<img src='/thoughtworks_white.png'>
</a>
</div>
<div class='menu-button'>
<div class='icon-bars navmenu-button'></div>
</div>
<div class='copyright'>
<p>© Martin Fowler | <a href="/aboutMe.html#disclosures">Disclosures</a></p>
</div>
</footer>
<script src = '/jquery-1.11.3.min.js' type = 'text/javascript'></script>
<script src = '/mfcom.js' type = 'text/javascript'></script>
</body>
</html>
-280
View File
@@ -1,280 +0,0 @@
<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd">
<html>
<head>
<meta content = 'uft-8' name = 'charset'></meta>
<title>not found</title>
<meta http-equiv="Content-type" content="text/html;charset=UTF-8" /><script defer src="https://cloud.umami.is/script.js" data-website-id="eb12527f-b713-4afa-905a-8a50f8a7f157"></script>
<link href = '/global.css' rel = 'stylesheet' type = 'text/css'></link>
</head>
<body><header id = 'banner' style = 'background-image: url("/banner.png"); background-repeat: no-repeat'>
<div class = 'name-logo'><a href = 'https://martinfowler.com'><img src = '/mf-name-white.png'></img></a></div>
<div class = 'search'>
<!-- SiteSearch Google -->
<form method='GET' action="https://www.google.com/search">
<input type='hidden' name='ie' value='UTF-8'/>
<input type='hidden' name='oe' value='UTF-8'/>
<input class = 'field' type='text'
name='q' size='15' maxlength='255' value=""/>
<button class = 'button' type='submit'
name='btnG' value=" " title = "Search"/>
<input type='hidden' name='domains' value="martinfowler.com"/>
<input type='hidden' name='sitesearch' value=""/>
<input
type='hidden' name='sitesearch' value="martinfowler.com"/>
</form>
</div>
<div class = 'menu-button navmenu-button'><a class = 'icon icon-bars' href = '#navmenu-bottom'></a></div>
<nav class = 'top-menu'>
<ul>
<li><a class = '' href = 'https://refactoring.com'>Refactoring</a></li>
<li><a class = '' href = '/agile.html'>Agile</a></li>
<li><a class = '' href = '/architecture'>Architecture</a></li>
<li><a class = '' href = '/aboutMe.html'>About</a></li>
<li><a class = 'tw' href = 'https://www.thoughtworks.com/engineering'>Thoughtworks</a></li>
<li><a class = 'icon icon-rss' href = '/feed.atom' title = 'feed'></a></li>
<li><a class = 'icon icon-twitter' href = 'https://www.twitter.com/martinfowler' title = 'Twitter stream'></a></li>
<li class = 'icon'><a href = 'https://toot.thoughtworks.com/@mfowler' title = 'Mastodon stream'><svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 24 24" fill="currentColor"><path d="M21.2595 13.9898C20.9852 15.4006 18.8033 16.9446 16.2974 17.2439C14.9907 17.3998 13.7041 17.5431 12.3321 17.4802C10.0885 17.3774 8.31809 16.9446 8.31809 16.9446C8.31809 17.163 8.33156 17.371 8.3585 17.5655C8.65019 19.7797 10.5541 19.9124 12.3576 19.9742C14.1779 20.0365 15.7987 19.5254 15.7987 19.5254L15.8735 21.1711C15.8735 21.1711 14.6003 21.8548 12.3321 21.9805C11.0814 22.0493 9.52849 21.9491 7.71973 21.4703C3.79684 20.432 3.12219 16.2504 3.01896 12.0074C2.98749 10.7477 3.00689 9.55981 3.00689 8.56632C3.00689 4.22771 5.84955 2.95599 5.84955 2.95599C7.2829 2.29772 9.74238 2.0209 12.2993 2H12.3621C14.919 2.0209 17.3801 2.29772 18.8133 2.95599C18.8133 2.95599 21.6559 4.22771 21.6559 8.56632C21.6559 8.56632 21.6916 11.7674 21.2595 13.9898ZM18.3029 8.9029C18.3029 7.82924 18.0295 6.97604 17.4805 6.34482C16.9142 5.71359 16.1726 5.39001 15.2522 5.39001C14.187 5.39001 13.3805 5.79937 12.8473 6.61819L12.3288 7.48723L11.8104 6.61819C11.2771 5.79937 10.4706 5.39001 9.40554 5.39001C8.485 5.39001 7.74344 5.71359 7.17719 6.34482C6.62807 6.97604 6.3547 7.82924 6.3547 8.9029V14.1562H8.43597V9.05731C8.43597 7.98246 8.88822 7.4369 9.79281 7.4369C10.793 7.4369 11.2944 8.08408 11.2944 9.36376V12.1547H13.3634V9.36376C13.3634 8.08408 13.8646 7.4369 14.8648 7.4369C15.7694 7.4369 16.2216 7.98246 16.2216 9.05731V14.1562H18.3029V8.9029Z"></path></svg>
</a></li>
<li class = 'icon'><a href = 'https://www.linkedin.com/in/martin-fowler-com/' title = 'LinkedIn'><svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 24 24" fill="currentColor"><path d="M4.00098 3H20.001C20.5533 3 21.001 3.44772 21.001 4V20C21.001 20.5523 20.5533 21 20.001 21H4.00098C3.44869 21 3.00098 20.5523 3.00098 20V4C3.00098 3.44772 3.44869 3 4.00098 3ZM5.00098 5V19H19.001V5H5.00098ZM7.50098 9C6.67255 9 6.00098 8.32843 6.00098 7.5C6.00098 6.67157 6.67255 6 7.50098 6C8.3294 6 9.00098 6.67157 9.00098 7.5C9.00098 8.32843 8.3294 9 7.50098 9ZM6.50098 10H8.50098V17.5H6.50098V10ZM12.001 10.4295C12.5854 9.86534 13.2665 9.5 14.001 9.5C16.072 9.5 17.501 11.1789 17.501 13.25V17.5H15.501V13.25C15.501 12.2835 14.7175 11.5 13.751 11.5C12.7845 11.5 12.001 12.2835 12.001 13.25V17.5H10.001V10H12.001V10.4295Z"></path></svg>
</a></li>
<li class = 'icon'><a href = 'https://bsky.app/profile/martinfowler.com' title = 'BlueSky'><svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 24 24" fill="currentColor"><path d="M12 11.3884C11.0942 9.62673 8.62833 6.34423 6.335 4.7259C4.13833 3.17506 3.30083 3.4434 2.75167 3.69256C2.11583 3.9784 2 4.95506 2 5.52839C2 6.10339 2.315 10.2367 2.52 10.9276C3.19917 13.2076 5.61417 13.9776 7.83917 13.7309C4.57917 14.2142 1.68333 15.4017 5.48083 19.6292C9.65833 23.9542 11.2058 18.7017 12 16.0392C12.7942 18.7017 13.7083 23.7651 18.4442 19.6292C22 16.0392 19.4208 14.2142 16.1608 13.7309C18.3858 13.9784 20.8008 13.2076 21.48 10.9276C21.685 10.2376 22 6.10256 22 5.52923C22 4.95423 21.8842 3.97839 21.2483 3.6909C20.6992 3.44256 19.8617 3.17423 17.665 4.72423C15.3717 6.34506 12.9058 9.62756 12 11.3884Z"></path></svg></a></li>
</ul>
</nav>
</header>
<nav id = 'top-navmenu'>
<nav class = 'navmenu'>
<div class = 'nav-head'> <div class = 'search'>
<!-- SiteSearch Google -->
<form method='GET' action="https://www.google.com/search">
<input type='hidden' name='ie' value='UTF-8'/>
<input type='hidden' name='oe' value='UTF-8'/>
<input class = 'field' type='text'
name='q' size='15' maxlength='255' value=""/>
<button class = 'button' type='submit'
name='btnG' value=" " title = "Search"/>
<input type='hidden' name='domains' value="martinfowler.com"/>
<input type='hidden' name='sitesearch' value=""/>
<input
type='hidden' name='sitesearch' value="martinfowler.com"/>
</form>
</div>
<div class = 'closediv'>
<span class = 'close' title = 'close'></span>
</div>
</div>
<div class = 'nav-body'>
<div class = 'topics'>
<h2>Topics</h2>
<p><a href = '/architecture'>Architecture</a></p>
<p><a href = 'https://refactoring.com'>Refactoring</a></p>
<p><a href = '/agile.html'>Agile</a></p>
<p><a href = '/delivery.html'>Delivery</a></p>
<p><a href = '/microservices'>Microservices</a></p>
<p><a href = '/data'>Data</a></p>
<p><a href = '/testing'>Testing</a></p>
<p><a href = '/dsl.html'>DSL</a></p>
</div>
<div class = 'about'>
<h2>about me</h2>
<p><a href = '/aboutMe.html'>About</a></p>
<p><a href = '/books'>Books</a></p>
<p><a href = '/faq.html'>FAQ</a></p>
</div>
<div class = 'content'>
<h2>content</h2>
<p><a href = '/videos.html'>Videos</a></p>
<p><a href = '/tags'>Content Index</a></p>
<p><a href = '/boardgames'>Board Games</a></p>
<p><a href = '/photos'>Photography</a></p>
</div>
<div class = 'tw'>
<h2>Thoughtworks</h2>
<p><a href = 'https://thoughtworks.com'>Home</a></p>
<p><a href = 'https://thoughtworks.com/insights'>Insights</a></p>
<p><a href = 'https://thoughtworks.com/careers'>Careers</a></p>
<p><a href = 'https://thoughtworks.com/radar'>Radar</a></p>
<p><a href = 'https://www.thoughtworks.com/engineering'>Engineering</a></p>
</div>
<div class = 'feeds'>
<h2>follow</h2>
<p><a href = '/feed.atom'>RSS</a></p>
<p><a href = 'https://toot.thoughtworks.com/@mfowler'>Mastodon</a></p>
<p><a href = 'https://www.linkedin.com/in/martin-fowler-com/'>LinkedIn</a></p>
<p><a href = 'https://bsky.app/profile/martinfowler.com'>Bluesky</a></p>
<p><a href = 'https://www.twitter.com/martinfowler'>X</a></p>
<p><a href = 'https://boardgamegeek.com/blog/13064/martins-7th-decade'>BGG</a></p>
</div>
</div>
</nav>
</nav>
<main><h1 id="section">404</h1>
<p>Im afraid this is not the document youre looking for. Try using the
search box above, and good luck.</p>
</main>
<nav id = 'bottom-navmenu'>
<nav class = 'navmenu'>
<div class = 'nav-head'> <div class = 'search'>
<!-- SiteSearch Google -->
<form method='GET' action="https://www.google.com/search">
<input type='hidden' name='ie' value='UTF-8'/>
<input type='hidden' name='oe' value='UTF-8'/>
<input class = 'field' type='text'
name='q' size='15' maxlength='255' value=""/>
<button class = 'button' type='submit'
name='btnG' value=" " title = "Search"/>
<input type='hidden' name='domains' value="martinfowler.com"/>
<input type='hidden' name='sitesearch' value=""/>
<input
type='hidden' name='sitesearch' value="martinfowler.com"/>
</form>
</div>
<div class = 'closediv'>
<span class = 'close' title = 'close'></span>
</div>
</div>
<div class = 'nav-body'>
<div class = 'topics'>
<h2>Topics</h2>
<p><a href = '/architecture'>Architecture</a></p>
<p><a href = 'https://refactoring.com'>Refactoring</a></p>
<p><a href = '/agile.html'>Agile</a></p>
<p><a href = '/delivery.html'>Delivery</a></p>
<p><a href = '/microservices'>Microservices</a></p>
<p><a href = '/data'>Data</a></p>
<p><a href = '/testing'>Testing</a></p>
<p><a href = '/dsl.html'>DSL</a></p>
</div>
<div class = 'about'>
<h2>about me</h2>
<p><a href = '/aboutMe.html'>About</a></p>
<p><a href = '/books'>Books</a></p>
<p><a href = '/faq.html'>FAQ</a></p>
</div>
<div class = 'content'>
<h2>content</h2>
<p><a href = '/videos.html'>Videos</a></p>
<p><a href = '/tags'>Content Index</a></p>
<p><a href = '/boardgames'>Board Games</a></p>
<p><a href = '/photos'>Photography</a></p>
</div>
<div class = 'tw'>
<h2>Thoughtworks</h2>
<p><a href = 'https://thoughtworks.com'>Home</a></p>
<p><a href = 'https://thoughtworks.com/insights'>Insights</a></p>
<p><a href = 'https://thoughtworks.com/careers'>Careers</a></p>
<p><a href = 'https://thoughtworks.com/radar'>Radar</a></p>
<p><a href = 'https://www.thoughtworks.com/engineering'>Engineering</a></p>
</div>
<div class = 'feeds'>
<h2>follow</h2>
<p><a href = '/feed.atom'>RSS</a></p>
<p><a href = 'https://toot.thoughtworks.com/@mfowler'>Mastodon</a></p>
<p><a href = 'https://www.linkedin.com/in/martin-fowler-com/'>LinkedIn</a></p>
<p><a href = 'https://bsky.app/profile/martinfowler.com'>Bluesky</a></p>
<p><a href = 'https://www.twitter.com/martinfowler'>X</a></p>
<p><a href = 'https://boardgamegeek.com/blog/13064/martins-7th-decade'>BGG</a></p>
</div>
</div>
</nav>
</nav>
<footer id='page-footer'>
<div class='tw-logo'>
<a href='https://www.thoughtworks.com/engineering'>
<img src='/thoughtworks_white.png'>
</a>
</div>
<div class='menu-button'>
<div class='icon-bars navmenu-button'></div>
</div>
<div class='copyright'>
<p>© Martin Fowler | <a href="/aboutMe.html#disclosures">Disclosures</a></p>
</div>
</footer>
<script src = '/jquery-1.11.3.min.js' type = 'text/javascript'></script>
<script src = '/mfcom.js' type = 'text/javascript'></script>
</body>
</html>
Binary file not shown.

Before

Width:  |  Height:  |  Size: 305 KiB

-99
View File
@@ -1,99 +0,0 @@
# MiniMax M2.7: Early Echoes of Self-Evolution
In the months following the first release of our M2-series models, we received a large volume of feedback and suggestions from enthusiastic users and developers, which drove us to further accelerate the efficiency of our model iterations. With human productivity already fully unleashed, the natural next step was to initiate self-evolution of both the model and the organization. M2.7 is our first model deeply participating in its own evolution.
M2.7 is capable of building complex agent harnesses and completing highly elaborate productivity tasks, leveraging capabilities such as Agent Teams, complex Skills, and dynamic tool search. For example, when developing M2.7, we let the model update its own memory and build dozens of complex skills in its harness to help with reinforcement learning experiments. We further let the model improve its learning process and harness based on the experiment results. This process initiates a cycle of model self-evolution.
1\. M2.7 delivers outstanding performance in real-world software engineering, including end-to-end full project delivery, log analysis, bug troubleshooting, code security, machine learning, and more. On the SWE-Pro benchmark, M2.7 scored 56.22%, nearly approaching Opus's best level. This capability also extends to end-to-end full project delivery scenarios (VIBE-Pro 55.6%) and deep understanding of complex engineering systems on Terminal Bench 2 (57.0%).
2\. We have also enhanced the model's expertise and task delivery capabilities across various fields in the professional office software domain. Its ELO score on GDPval-AA is 1495, the highest among open-source models. M2.7 shows significantly improved ability for complex editing in the Office suite — Excel, PPT, and Word — and can better handle multi-round revisions and high-fidelity editing. M2.7 is capable of interacting with complex environments: It maintains a 97% skill adherence rate while working with over 40 complex skills, each exceeding 2,000 tokens.
3\. M2.7 exhibits excellent character consistency and emotional intelligence, opening up more room for product innovation.
Based on these capabilities, M2.7 is also significantly accelerating our own evolution into an AI-native organization.
![MiniMax M2.7 benchmark overview](minimax-assets/img-1.png)
## Building an agent for model self-evolution
We first share an internal workflow that enables the M2-series models to self-evolve. This workflow also serves as an exploration of the boundaries of the model's agentic capabilities.
Modern agent harness utilizes a combination of complex skills, memory, and other external modules to help improve its adaptability to various workspace environments. In MiniMax, our agents are routinely faced with very complex and disparate working environments spanning multiple departments. As such, to improve the robustness of our agents in these heterogeneous environments, we tasked an internal version of M2.7 to build a research agent harness that interacts and collaborates with different research project groups. The harness supports data pipelines, training environments, infrastructure, cross-team collaboration, and persistent memory — enabling researchers to drive it to deliver better models. The research agent harness drives the iteration cycle that produces the next generation of models under the guidance set by researchers.
An exemplary workflow lies in the daily routine of our RL team. A researcher starts by discussing an experimental idea with the agent, who helps with literature review, tracks a pre-set experiment spec, pipelines data and other artifacts, and launches experiments. During the experiments, the agent monitors and profiles the experiment's progress and automatically triggers log reading, debugging, metric analysis, code fixes, merge requests, and smoke tests, identifying and configuring subtle yet key changes. These could have required the collaboration of multiple human researchers from different teams before, but now human researchers only interact for critical decisions and discussions. This accelerates problem discovery and experimentation, delivering models faster. Here, M2.7 is capable of handling 30%-50% of the workflow.
![Research agent harness architecture](minimax-assets/img-2.png)
During the iteration process, we realized that the model's ability to recursively evolve its own harness is also critical. Our internal harness autonomously collects feedback, builds evaluation sets for internal tasks, and based on this continuously iterates its own architecture, skills/MCP implementation, and memory mechanisms to complete tasks better and more efficiently.
For example, we had M2.7 optimize a model's programming performance on an internal scaffold. M2.7 ran entirely autonomously, executing an iterative loop of "analyze failure trajectories → plan changes → modify scaffold code → run evaluations → compare results → decide to keep or revert changes" for over 100 rounds. During this process, M2.7 discovered effective optimizations for the model: systematically searching for the optimal combination of sampling parameters such as temperature, frequency penalty, and presence penalty; designing more specific workflow guidelines for the model (e.g., automatically searching for the same bug patterns in other files after a fix); and adding loop detection and other optimizations to the scaffold's agent loop. Ultimately, this achieved a 30% performance improvement on internal evaluation sets.
We believe that future AI self-evolution will gradually transition towards full autonomy, coordinating data construction, model training, inference architecture, evaluation, and other stages without human involvement.
To this end, we conducted preliminary exploratory tests in low-resource scenarios. We had M2.7 participate in 22 machine learning competitions at the MLE Bench Lite level open-sourced by OpenAI. These competitions can be run on a single A30 GPU, yet they cover virtually all stages of machine learning workflow.
We designed and implemented a simple harness to guide the agent in autonomous optimization. The core modules include three components: short-term memory, self-feedback, and self-optimization. Specifically, after each iteration round, the agent generates a short-term memory markdown file and simultaneously performs self-criticism on the current round's results, thereby providing potential optimization directions for the next round. The next round then conducts further self-optimization based on the memory and self-feedback chain from all previous rounds. We ran a total of three trials, each with 24 hours for iterative evolution. From the figure below, one can see that the ML models trained by M2.7 continuously achieved higher medal rates over time. In the end, the best run achieved 9 gold medals, 5 silver medals, and 1 bronze medal. The average medal rate across the three runs was 66.6%, a result second only to Opus-4.6 (75.7%) and GPT-5.4 (71.2%), tying with Gemini-3.1 (66.6%).
![MLE Bench Lite performance chart](minimax-assets/img-3.png)
## Professional Software Engineering
In software engineering tasks, M2.7 more deeply explores real-world programming abilities, including log analysis for bug hunting, refactoring, code security, machine learning, Android development, and more.
Take a common production scenario as an example: debugging in a live environment. This requires not just code generation, but strong comprehensive reasoning abilities. When faced with alerts in production, M2.7 can correlate monitoring metrics with deployment timelines to perform causal reasoning, conduct statistical analysis on trace sampling and propose precise hypotheses, proactively connect to databases to verify root causes, pinpoint missing index migration files in the code repository, and even have the awareness to use non-blocking index creation to stop the bleeding first before submitting a merge request. From observability analysis and database expertise to SRE-level decision-making — this is not merely a model that can write code, but one that truly understands production systems. Compared to traditional manual troubleshooting processes, using M2.7, we have on multiple occasions reduced the recovery time for live production system incidents to under three minutes.
Your browser does not support video playback. Please switch to a different browser.
Live production environment debugging
In terms of raw programming capabilities, M2.7 has reached the level of SOTA models. On SWE-Pro, which covers multiple programming languages, M2.7 achieved a 56.22% accuracy rate, matching GPT-5.3-Codex. It demonstrates an even more notable advantage on benchmarks closer to real-world engineering scenarios, such as SWE Multilingual (76.5) and Multi SWE Bench (52.7).
This capability also extends to end-to-end full project delivery scenarios. On the repo-level code generation benchmark VIBE-Pro, M2.7 scored 55.6%, nearly on par with Opus 4.6 — meaning that whether the requirement involves Web, Android, iOS, or simulation tasks, they can be handed directly to M2.7 to complete.
What deserves even more attention is its deep understanding of complex engineering systems. On Terminal Bench 2 (57.0%) and NL2Repo (39.8%), both of which demand a high degree of system-level comprehension, M2.7 also performs solidly. This further confirms that it excels not only at code generation but can also deeply understand the operational logic and collaborative dynamics of software systems.
Your browser does not support video playback. Please switch to a different browser.
WildGuard demo webpage generated by M2.7
To improve development efficiency, one particularly important feature is native Agent Teams (multi-agent collaboration). Agent Teams impose paradigm-level demands on the model: role boundaries, adversarial reasoning, protocol adherence, and behavioral differentiation — these cannot be achieved through prompting alone and must be internalized as native capabilities of the model. In Agent Teams scenarios, the model needs to stably anchor its role identity, proactively challenge teammates' logical and ethical blind spots, and make autonomous decisions within complex state machines. Below is an Agent Teams setup we use internally for product prototype development, which contains a minimal organization for building product prototypes.
![Research agent harness architecture](minimax-assets/d92a6eb4-a4b8-4906-b76a-d627c814a2c0.gif)
Agent Teams multi-agent collaboration demo
## Professional Work
Beyond software engineering, agents are becoming increasingly useful in office scenarios. We believe this comes down to two core capabilities:
Domain expertise and task delivery capability. The model needs to possess professional knowledge across various fields and understand user requirements. In the GDPval-AA evaluation, which measures this capability, M2.7 achieved an ELO score of 1495 among 45 models, second only to Opus 4.6, Sonnet 4.6, and GPT5.4, and surpassing GPT5.3. For the most common office document processing tasks, we systematically optimized the model's ability to handle Word, Excel, and PPT. Across various agent harnesses, M2.7 can both generate files directly based on templates and skills, and follow users' interactive instructions to perform multiple rounds of high-fidelity editing on existing files, ultimately producing editable deliverables.
Ability to interact with complex environments. Generalized everyday scenarios mean the model must flexibly adapt to various contexts, invoke diverse skills and tools, and maintain stable instruction adherence throughout extended interactions. M2.7 has made substantial improvements in these areas. On Toolathon, M2.7 achieved an accuracy of 46.3%, reaching the global top tier. Agent harnesses in real-world work scenarios also often require understanding and invoking a large number of complex skills. In MM Claw testing, M2.7 maintained a 97% skill compliance rate across 40 complex skills (each exceeding 2,000 tokens).
We tested the model's professional proficiency in finance, and compared to the previous generation, the improvement in capability is significant. For example, in a scenario involving reading research reports and modeling a company's future revenue, M2.7 can autonomously read a company's annual reports and earnings call minutes, cross-reference multiple research reports, independently design assumptions and build a revenue forecast model, and then produce a PPT and research report based on templates — understanding, making judgments, and producing output like a junior analyst, while self-correcting through multiple rounds of interaction. The feedback from practitioners is that the output can already serve as a first draft and go directly into subsequent workflows. Below is an example for TSMC.
Task: Based on TSMC's annual report and earnings call information, build a revenue model for TSMC. Read multiple research reports, design corresponding assumptions, model TSMC's revenue based on the latest information, then produce a PPT based on a PPT template, and write a Word document research report.
您的浏览器不支持播放此视频,请更换浏览器。
The recent surge in popularity of OpenClaw is representative of a thriving agent ecosystem, and we are pleased that our M2-series models have contributed to the community's flourishing. Based on commonly used tasks in OpenClaw, we built an evaluation set called MM Claw, covering a wide range of real-world needs in both work and life — from personal learning planning, to office document processing and delivery, scheduled professional research and investment advice, and code development and maintenance. M2.7 achieved a level close to Sonnet 4.6 on this test, with an accuracy of 62.7%.
## Entertainment
With OpenClaw and similar personal agents, we noticed that beyond getting work done, many users also want the model to have high emotional intelligence and character consistency. With a persona in place, users start interacting with OpenClaw like a friend. We believe this presents an opportunity to extend the use of agentic models beyond pure productivity into interactive entertainment. To this end, we strengthened character consistency and conversational capabilities in M2.7.
Based on this, we built a preliminary demo: OpenRoom, an interaction system based on an agent harness that liberates AI interaction from plain text streams and places it within a Web GUI space where everything is interactive. Here, character settings are no longer cold chunks of prompts; conversation drives the experience, generating real-time visual feedback and scene interactions, with characters proactively engaging with their environment. We believe this framework is highly extensible and can continue to evolve alongside improvements in agentic capabilities and community development, exploring entirely new ways for humans and agents to interact.
To encourage exploration in this area, we have open-sourced the initial demo (of which most of the code was written by AI):
Your browser does not support video playback. Please switch to a different browser.
MiniMax M2.7 is now fully available on MiniMax Agent and the MiniMax API Platform. We look forward to users and developers exploring even more interesting use cases with M2.7.
MiniMax Agent: [agent.minimax.io](https://agent.minimax.io/)
API: [platform.minimax.io](https://platform.minimax.io/)
Coding Plan: [platform.minimax.io/subscribe/coding-plan](https://platform.minimax.io/subscribe/coding-plan)
Intelligence with Everyone.
@@ -1,417 +0,0 @@
# Harness Engineering: The Complete Guide to Building Systems That Make AI Agents Actually Work (2026)
## Harness Engineering: The Complete Guide to Building Systems That Make AI Agents Actually Work
**March 2026** — If 2025 was the year AI agents proved they could write code, 2026 is the year we learned that **the agent isn't the hard part — the harness is.**
OpenAI's Codex team just built a production application with **over 1 million lines of code** where **zero lines were written by human hands**. The engineers didn't write code. They designed the system that let AI write code reliably. That system — the constraints, feedback loops, documentation, linters, and lifecycle management — is what the industry now calls a **harness**.
**Harness engineering** is the new discipline of designing these systems. And it's changing what it means to be a software engineer.
* * *
## What Is Harness Engineering?
### The Horse Metaphor
The term "harness" comes from horse tack — reins, saddle, bit — the complete set of equipment for channeling a powerful but unpredictable animal in the right direction. The metaphor is deliberate:
- The **horse** is the AI model — powerful, fast, but it doesn't know where to go on its own
- The **harness** is the infrastructure — constraints, guardrails, feedback loops that channel the model's power productively
- The **rider** is the human engineer — providing direction, not doing the running
Without a harness, an AI agent is a thoroughbred in an open field. Fast, impressive, and completely useless for getting anything done.
### The Formal Definition
**Harness engineering** is the design and implementation of systems that:
1. **Constrain** what an AI agent can do (architectural boundaries, dependency rules)
2. **Inform** the agent about what it should do (context engineering, documentation)
3. **Verify** that the agent did it correctly (testing, linting, CI validation)
4. **Correct** the agent when it goes wrong (feedback loops, self-repair mechanisms)
Martin Fowler describes it as *"the tooling and practices we can use to keep AI agents in check"* — but it's more than just safety. A good harness makes agents **more capable**, not just more controlled.
* * *
## Why Harness Engineering Matters Now
### The Model Is Commodity. The Harness Is Moat.
Here's the uncomfortable truth the AI industry is confronting: **the underlying model matters less than the system around it.**
LangChain proved this definitively. Their coding agent went from **52.8% to 66.5%** on Terminal Bench 2.0 — jumping from **Top 30 to Top 5** — by changing nothing about the model. They only changed the harness:
Change
What They Did
Impact
Self-verification loop
Added pre-completion checklist middleware
Caught errors before submission
Context engineering
Mapped directory structures at startup
Agent understood codebase from the start
Loop detection
Tracked repeated file edits
Prevented "doom loops"
Reasoning sandwich
High reasoning for planning/verification, medium for implementation
Better quality within time budgets
**Same model. Different harness. Dramatically better results.**
### OpenAI's 1 Million Line Proof Point
OpenAI's experiment is the most compelling evidence yet:
- **5 months** of development
- **1 million+ lines of code** in the final product
- **Zero manually written lines** — every line was produced by Codex agents
- **Built in ~1/10th the time** it would have taken humans
- The product has **internal daily users and external alpha testers**
- It **ships, deploys, breaks, and gets fixed** — all by agents within the harness
The engineers' job? Designing the harness. Specifying intent. Providing feedback. Not writing code.
* * *
## The Three Pillars of Harness Engineering
OpenAI's framework organizes harness engineering into three core categories:
### 1\. Context Engineering
Context engineering is about ensuring the agent has the right information at the right time.
**Static context:**
- Repository-local documentation (architecture specs, API contracts, style guides)
- `AGENTS.md` or `CLAUDE.md` files that encode project-specific rules
- Cross-linked design documents validated by linters
**Dynamic context:**
- Observability data (logs, metrics, traces) accessible to agents
- Directory structure mapping at agent startup
- CI/CD pipeline status and test results
**The critical rule:** From the agent's perspective, anything it can't access in-context doesn't exist. Knowledge in Google Docs, Slack threads, or people's heads is invisible to the system. **The repository must be the single source of truth.**
### 2\. Architectural Constraints
This is where harness engineering diverges most sharply from traditional AI prompting. Instead of telling the agent "write good code," you **mechanically enforce what good code looks like.**
**Dependency layering:**
```
Types → Config → Repo → Service → Runtime → UI
```
Each layer can only import from layers to its left. This isn't a suggestion — it's enforced by structural tests and CI validation.
**Constraint enforcement tools:**
- **Deterministic linters** — Custom rules that flag violations automatically
- **LLM-based auditors** — Agents that review other agents' code for architectural compliance
- **Structural tests** — Like ArchUnit, but for AI-generated code
- **Pre-commit hooks** — Automated checks before any code is committed
**Why constraints improve output:** Paradoxically, constraining the solution space makes agents **more productive**, not less. When an agent can generate anything, it wastes tokens exploring dead ends. When the harness defines clear boundaries, the agent converges faster on correct solutions.
### 3\. Entropy Management ("Garbage Collection")
This is the most underappreciated component. Over time, AI-generated codebases accumulate entropy — documentation drifts from reality, naming conventions diverge, dead code accumulates.
Harness engineering addresses this with **periodic cleanup agents:**
- **Documentation consistency agents** — Verify that docs match current code
- **Constraint violation scanners** — Find code that slipped past earlier checks
- **Pattern enforcement agents** — Identify and fix deviations from established patterns
- **Dependency auditors** — Track and resolve circular or unnecessary dependencies
These agents run on schedules — daily, weekly, or triggered by specific events — keeping the codebase healthy for both human reviewers and future AI agents.
* * *
## Harness Engineering in Practice: How Teams Actually Do It
### The OpenAI Approach: Zero Human Code
OpenAI's team structure for harness engineering:
Role
Traditional
Harness Engineering
Writing code
Primary job
Never
Designing architecture
Part of the job
Primary job
Writing documentation
Afterthought
Critical infrastructure
Reviewing PRs
Code review
Reviewing agent output + harness effectiveness
Debugging
Reading code
Analyzing agent behavior patterns
Testing
Writing tests
Designing test strategies agents execute
### The Stripe Approach: Minions at Scale
Stripe's internal coding agents, called **Minions**, now produce **over 1,000 merged pull requests per week**:
1. Developer posts a task in Slack
2. Minion writes the code
3. Minion passes CI
4. Minion opens a PR
5. Human reviews and merges
No developer interaction between step 1 and step 5. The harness handles everything — test execution, CI validation, style compliance, and documentation updates.
### The LangChain Approach: Middleware-First
LangChain structures their harness as composable middleware layers:
```
Agent Request
→ LocalContextMiddleware (maps codebase)
→ LoopDetectionMiddleware (prevents repetition)
→ ReasoningSandwichMiddleware (optimizes compute)
→ PreCompletionChecklistMiddleware (enforces verification)
→ Agent Response
```
Each middleware layer adds a specific capability without modifying the core agent logic. This modular approach makes the harness testable and evolvable.
* * *
## Building Your First Harness: A Practical Framework
### Level 1: Basic Harness (Single Developer)
If you're using Claude Code, Cursor, or Codex for individual projects:
**What to set up:**
- `CLAUDE.md` or `.cursorrules` file with project conventions
- Pre-commit hooks for linting and formatting
- A test suite the agent can run to self-verify
- Clear directory structure with consistent naming
**Time to set up:** 1-2 hours **Impact:** Prevents the most common agent mistakes
### Level 2: Team Harness (Small Team)
For teams of 3-10 developers sharing a codebase:
**Add to Level 1:**
- `AGENTS.md` with team-wide conventions
- Architectural constraints enforced by CI
- Shared prompt templates for common tasks
- Documentation-as-code validated by linters
- Code review checklists specifically for agent-generated PRs
**Time to set up:** 1-2 days **Impact:** Consistent agent behavior across the team
### Level 3: Production Harness (Engineering Organization)
For organizations running dozens of concurrent agents:
**Add to Level 2:**
- Custom middleware layers (loop detection, reasoning optimization)
- Observability integration (agents read logs and metrics)
- Entropy management agents on scheduled runs
- Harness versioning and A/B testing
- Agent performance monitoring dashboards
- Escalation policies for when agents get stuck
**Time to set up:** 1-2 weeks **Impact:** Agents operate as autonomous contributors
* * *
## Common Harness Engineering Mistakes
### 1\. Over-Engineering the Control Flow
> *"If you over-engineer the control flow, the next model update will break your system."*
Models improve rapidly. Capabilities that required complex pipelines in 2024 are now handled by a single context-window prompt. Build your harness to be **rippable** — you should be able to remove "smart" logic when the model gets smart enough to not need it.
### 2\. Treating the Harness as Static
The harness needs to evolve with the model. When a new model release improves reasoning, your reasoning-optimization middleware might become counterproductive. Review and update harness components with every major model update.
### 3\. Ignoring the Documentation Layer
The most impactful harness improvement is often the simplest: **better documentation**. If your `AGENTS.md` is vague, your agent output will be vague. Invest in precise, machine-readable documentation that serves as the agent's ground truth.
### 4\. No Feedback Loop
A harness without feedback is a cage, not a guide. The agent needs to know when it's succeeding and when it's failing. Build in:
- Self-verification steps before task completion
- Test execution as part of the agent workflow
- Metrics on agent success rates by task type
### 5\. Human-Only Documentation
If your architectural decisions live in people's heads or in Confluence pages the agent can't access, the harness has a gap. **Everything the agent needs must be in the repository.**
* * *
## Harness Engineering vs. Related Concepts
Concept
Scope
Focus
**Prompt Engineering**
Single interaction
Crafting effective prompts
**Context Engineering**
Model context window
What information the model sees
**Harness Engineering**
Entire agent system
Environment, constraints, feedback, lifecycle
**Agent Engineering**
Agent architecture
Internal agent design and routing
**Platform Engineering**
Infrastructure
Deployment, scaling, operations
Harness engineering **includes** context engineering and draws from prompt engineering, but it operates at a higher level — it's about the complete system that makes agents reliable, not just the inputs to a single interaction.
* * *
## What This Means for Software Engineers
### The Job Is Changing
Harness engineering represents a genuine evolution in what software engineers do:
Before
After
Write code
Design environments where AI writes code
Debug code
Debug agent behavior
Review code
Review agent output + harness effectiveness
Write tests
Design test strategies
Maintain docs
Build documentation as machine-readable infrastructure
This doesn't mean engineers become less technical. If anything, harness engineering requires **deeper** architectural thinking — you're designing systems that must work without your constant intervention.
### The Skills That Matter
Based on what we've seen building AI-powered products at [NxCode](https://www.nxcode.io/):
1. **Systems thinking** — Understanding how constraints, feedback loops, and documentation interact
2. **Architecture design** — Defining boundaries that are enforceable and productive
3. **Specification writing** — Articulating intent precisely enough for agents to execute
4. **Observability** — Building monitoring that reveals agent behavior patterns
5. **Iteration speed** — Rapidly testing and refining harness configurations
### Our Experience: What Works in Practice
We've been building AI-powered web applications using multiple agent systems (Claude Code, Codex, Cursor). The patterns that have made the biggest difference for us:
- **Repository-first documentation**: Every architectural decision, naming convention, and deployment process is in the repo. Nothing lives in Slack or Google Docs.
- **Incremental constraint building**: Start with basic linting, add architectural constraints as patterns emerge, don't try to design the perfect harness upfront.
- **Agent-specific review checklists**: AI-generated code has different failure modes than human code. Our review process accounts for common agent patterns (over-abstraction, unnecessary error handling, documentation drift).
- **Multi-provider harness design**: Our harness works with Claude, GPT, and Gemini models. Provider-agnostic design means we can switch models without rebuilding the entire system.
* * *
## Key Takeaways
1. **Harness engineering is the new discipline** of designing systems that make AI agents reliable — constraints, feedback loops, documentation, and lifecycle management
2. **The model is commodity; the harness is moat** — LangChain jumped from Top 30 to Top 5 on benchmarks by only changing the harness
3. **OpenAI built 1M+ lines with zero human code** — proving harness engineering works at production scale
4. **Three pillars**: Context engineering, architectural constraints, and entropy management
5. **Start simple**: A good `AGENTS.md` and pre-commit hooks are more impactful than complex middleware
6. **The engineer's job is evolving** — from writing code to designing environments where AI writes code
7. **Build rippable harnesses** — over-engineering breaks when models improve; keep it adaptable
* * *
## Related Articles
- [Best AI for Coding in 2026: 10 Tools Ranked by Real-World Performance](https://www.nxcode.io/resources/news/best-ai-for-coding-2026-complete-ranking)
- [OpenAI Frontier Guide: Enterprise AI Agent Platform for Building AI Coworkers (2026)](https://www.nxcode.io/resources/news/openai-frontier-enterprise-ai-agent-platform-guide-2026)
- [Cursor Tutorial 2026: Learn AI Coding in 15 Minutes (Beginner Guide)](https://www.nxcode.io/resources/news/cursor-tutorial-beginners-2026)
@@ -1,66 +1,113 @@
# Harness Engineering:在智能体优先的世界中利用 Codex | OpenAI ---
title: "工程技术:在智能体优先的世界中利用 Codex"
source: "https://openai.com/zh-Hans-CN/index/harness-engineering/"
author:
published: 2026-03-11
created: 2026-04-11
description: "作者:Ryan Lopopolo,技术人员"
tags:
- "clippings"
---
2026年2月11日
(原文来源:https://openai.com/zh-Hans-CN/index/harness-engineering/ [工程](https://openai.com/news/engineering/)
作者:Ryan Lopopolo,技术人员
<audio src="https://cdn-azalea-tts-achmg6dqbafjaxcr.a01.azurefd.net/tts-prod/1flz3YCBifvfy5L243uhH9/ember/c0fc31d2467edefafed73e9b118289e7.mp3"></audio>
在过去五个月里,我们的团队一直在进行一项实验:构建并交付一款软件产品的内部 beta 版, **其中没有一行代码是人工编写的**
在过去五个月里,我们的团队一直在进行一项实验:构建并交付一款软件产品的内部 beta 版,其中没有一行代码是人工编写的。
该产品有内部日常活跃用户和外部 Alpha 测试者。它经历了交付、部署、故障和修复的整个过程。与众不同的是,每一行代码 — 从应用逻辑、测试、CI 配置、文档、可观察性到内部工具 — 全都是由 Codex 编写的。据估计,我们只用了手工编写代码所需的大约 1/10 的时间就完成了这项工作。 该产品有内部日常活跃用户和外部 Alpha 测试者。它经历了交付、部署、故障和修复的整个过程。与众不同的是,每一行代码 — 从应用逻辑、测试、CI 配置、文档、可观察性到内部工具 — 全都是由 Codex 编写的。据估计,我们只用了手工编写代码所需的大约 1/10 的时间就完成了这项工作。
人类掌舵。智能体执行。
**人类掌舵。智能体执行。**
我们有意选择这一限制,以便构建必要的内容,从而将工程速度提升数个数量级。我们用了几周的时间来交付最终达到一百万行代码的项目。为此,我们需要了解,当软件工程团队的主要工作不再是编写代码,而是设计环境、明确意图和构建反馈回路,从而使 Codex 智能体能够可靠地工作时,会发生哪些变化。 我们有意选择这一限制,以便构建必要的内容,从而将工程速度提升数个数量级。我们用了几周的时间来交付最终达到一百万行代码的项目。为此,我们需要了解,当软件工程团队的主要工作不再是编写代码,而是设计环境、明确意图和构建反馈回路,从而使 Codex 智能体能够可靠地工作时,会发生哪些变化。
这个帖子要说的是,在我们与智能体团队一起从零开始打造一款全新产品的过程中,所能学到的经验教训 — 哪些地方出了问题,哪些问题相互叠加,以及如何最大化利用我们唯一真正稀缺的资源:人类的时间和注意力。 这个帖子要说的是,在我们与智能体团队一起从零开始打造一款全新产品的过程中,所能学到的经验教训 — 哪些地方出了问题,哪些问题相互叠加,以及如何最大化利用我们唯一真正稀缺的资源:人类的时间和注意力。
## 我们从一个空的 Git 代码仓库开始
首次提交到一个空的代码仓库是在 2025 年 8 月下旬。 首次提交到一个空的代码仓库是在 2025 年 8 月下旬。
初始架构 — 包括代码仓库结构、CI 配置、格式化规则、包管理器设置和应用框架 — 是在一小套现有模板的指导下,由 Codex CLI 使用 GPT‑5 生成的。就连指导智能体如何在代码仓库中工作的初始 AGENTS.md 文件本身也是由 Codex 编写的。 初始架构 — 包括代码仓库结构、CI 配置、格式化规则、包管理器设置和应用框架 — 是在一小套现有模板的指导下,由 Codex CLI 使用 GPT‑5 生成的。就连指导智能体如何在代码仓库中工作的初始 AGENTS.md 文件本身也是由 Codex 编写的。
该系统没有预存任何人工编写的代码。从一开始,代码仓库就由智能体塑造。 该系统没有预存任何人工编写的代码。从一开始,代码仓库就由智能体塑造。
五个月后,该代码仓库已经拥有约一百万行代码,从应用逻辑、基础设施、工具、文档到内部开发者工具应有尽有。在那段时间内,大约有 1,500 个 Pull Request 被打开与合并,而推动 Codex 的仅仅是一个由三名工程师组成的小团队。这相当于平均每位工程师每天处理 3.5 个 PRs 的吞吐量,而且令人惊讶的是,随着团队规模扩大到现在的七名工程师,吞吐量甚至还增加了。重要的是,这并非为了输出而输出:该产品已在数百名内测用户那里投入使用,其中包括每天都在使用的内测高级用户。
在整个开发过程中,人类从未直接直接贡献过任何代码。这成为团队的核心理念:不手动编写代码 五个月后,该代码仓库已经拥有约一百万行代码,从应用逻辑、基础设施、工具、文档到内部开发者工具应有尽有。在那段时间内,大约有 1,500 个 Pull Request 被打开与合并,而推动 Codex 的仅仅是一个由三名工程师组成的小团队。这相当于平均每位工程师每天处理 3.5 个 PRs 的吞吐量,而且令人惊讶的是,随着团队规模扩大到现在的七名工程师,吞吐量甚至还 *增加* 了。重要的是,这并非为了输出而输出:该产品已在数百名内测用户那里投入使用,其中包括每天都在使用的内测高级用户
由于缺乏人工编码的实践,工程师工作的重点转向了系统、架构和杠杆作用。
在整个开发过程中,人类从未直接直接贡献过任何代码。这成为团队的核心理念: **不手动编写代码**
## 重新定义工程师的角色
由于缺乏人工编码的实践, **工程师工作的重点转向了系统、架构和杠杆作用**
早期进展比我们所预期的要慢,而这并不是因为 Codex 不具备相应的能力,而是因为环境的规范不够明确。该智能体缺乏实现高级目标所需的工具、抽象层和内部结构,因而无法取得进展。我们工程团队的主要任务成了协助智能体完成有用的工作。 早期进展比我们所预期的要慢,而这并不是因为 Codex 不具备相应的能力,而是因为环境的规范不够明确。该智能体缺乏实现高级目标所需的工具、抽象层和内部结构,因而无法取得进展。我们工程团队的主要任务成了协助智能体完成有用的工作。
在实践中,这意味着采用深度优先的工作方式:将更大的目标拆解为更小的构建模块(设计、代码、评审、测试等),提示智能体去构建这些模块,并使用它们去解锁更复杂的任务。当事情进行不顺利时,解决方案基本上再也不会是“再努力一点”。因为取得进展的唯一方式是让 Codex 来完成工作,而人类工程师则总是介入这项任务并追问:“究竟还需要什么样的能力,我们又该如何让这个能力对智能体来说既清晰可读又可强制执行?” 在实践中,这意味着采用深度优先的工作方式:将更大的目标拆解为更小的构建模块(设计、代码、评审、测试等),提示智能体去构建这些模块,并使用它们去解锁更复杂的任务。当事情进行不顺利时,解决方案基本上再也不会是“再努力一点”。因为取得进展的唯一方式是让 Codex 来完成工作,而人类工程师则总是介入这项任务并追问:“究竟还需要什么样的能力,我们又该如何让这个能力对智能体来说既清晰可读又可强制执行?”
人类几乎完全通过提示与系统交互:工程师描述任务,运行智能体,并允许其打开一个 Pull Request。为了推动 PR 的完成,我们会指示 Codex 在本地审核其自身的更改,在本地和云端请求额外的特定智能体审查,对任何人工或智能体给出的反馈做出响应,并循环往复,直到所有智能体审核人员都满意为止(这实际上是一个 [Ralph Wiggum 循环](https://ghuntley.com/loop/))。Codex 直接使用我们的标准开发工具(gh、本地脚本和嵌入代码仓库的技能)来收集情境,而无需人工将内容复制粘贴到 CLI 中。
人类几乎完全通过提示与系统交互:工程师描述任务,运行智能体,并允许其打开一个 Pull Request。为了推动 PR 的完成,我们会指示 Codex 在本地审核其自身的更改,在本地和云端请求额外的特定智能体审查,对任何人工或智能体给出的反馈做出响应,并循环往复,直到所有智能体审核人员都满意为止(这实际上是一个 [Ralph Wiggum 循环 ](https://ghuntley.com/loop/) )。Codex 直接使用我们的标准开发工具(gh、本地脚本和嵌入代码仓库的技能)来收集情境,而无需人工将内容复制粘贴到 CLI 中。
人类可以审核 Pull Request(合并请求),但并非必须这样做。随着时间的推移,我们已将几乎所有的审核工作调整为用智能体对智能体的方式来处理。 人类可以审核 Pull Request(合并请求),但并非必须这样做。随着时间的推移,我们已将几乎所有的审核工作调整为用智能体对智能体的方式来处理。
## 提高应用程序的可读性
随着代码吞吐量的增加,我们的瓶颈变成了人工 QA 能力。由于人类的时间和注意力是固定的限制因素,我们一直在努力通过令应用程序的 UI、日志和应用指标等内容对 Codex 直接可读,从而为智能体增加更多功能。 随着代码吞吐量的增加,我们的瓶颈变成了人工 QA 能力。由于人类的时间和注意力是固定的限制因素,我们一直在努力通过令应用程序的 UI、日志和应用指标等内容对 Codex 直接可读,从而为智能体增加更多功能。
例如,我们令应用程序可以根据 git worktree 启动,因此 Codex 可以为每次更改启动并驱动一个实例。我们还将 Chrome DevTools 协议接入智能体运行时,并创建了用于处理 DOM 快照、屏幕截图和导航的技能。这使 Codex 能够复现错误、验证修复,并直接推理 UI 的行为。 例如,我们令应用程序可以根据 git worktree 启动,因此 Codex 可以为每次更改启动并驱动一个实例。我们还将 Chrome DevTools 协议接入智能体运行时,并创建了用于处理 DOM 快照、屏幕截图和导航的技能。这使 Codex 能够复现错误、验证修复,并直接推理 UI 的行为。
![[OAI_Harness_engineering_Codex_drives_the_app_with_Chrome_DevTools_MCP_to_validate_its_work_desktop-dark.png|题为“Codex 使用 Chrome DevTools MCP 驱动应用程序以验证其工作”的图表。Codex 会选择一个目标,对触发用户界面路径前后的状态进行快照,通过 Chrome DevTools 观察运行时事件,应用修复、重启和循环重新运行验证,直到应用程序恢复正常。]]
题为“Codex 使用 Chrome DevTools MCP 驱动应用程序以验证其工作”的图表。Codex 会选择一个目标,对触发用户界面路径前后的状态进行快照,通过 Chrome DevTools 观察运行时事件,应用修复、重启和循环重新运行验证,直到应用程序恢复正常。
我们对可观测性工具也做了同样的处理。日志、指标和追踪记录会通过一个本地可观测性堆栈展示给 Codex,对任何给定的工作树来说,该堆栈都是临时的。Codex 在该应用程序的一个完全独立的版本上运行,一旦任务完成,该版本的所有内容,包括日志和指标,都会被删除。智能体可以使用 LogQL 查询日志,使用 PromQL 查询指标。有了这些情境,像“确保服务启动在 800ms 内完成”或“这四个关键用户旅程中的任何跨度都不得超过两秒”这样的提示就变得可行了。 我们对可观测性工具也做了同样的处理。日志、指标和追踪记录会通过一个本地可观测性堆栈展示给 Codex,对任何给定的工作树来说,该堆栈都是临时的。Codex 在该应用程序的一个完全独立的版本上运行,一旦任务完成,该版本的所有内容,包括日志和指标,都会被删除。智能体可以使用 LogQL 查询日志,使用 PromQL 查询指标。有了这些情境,像“确保服务启动在 800ms 内完成”或“这四个关键用户旅程中的任何跨度都不得超过两秒”这样的提示就变得可行了。
![[OAI_Harness_engineering_Giving_Codex_a_full_observability_stack_desktop-dark.png|题为“在本地开发中赋予 Codex 完整的可观察性堆栈”的图表。一个应用程序将日志、指标和追踪数据发送到 Vector,Vector 会将数据分发到一个包含 Victoria Logs、Metrics 和 Traces 的可观测性堆栈中,并通过 LogQL、PromQL 或 TraceQL API 进行查询。Codex 使用这些信号进行查询、关联和推理,然后在代码库中进行修复,重启应用程序,重新运行工作负载,测试 UI 流程,并在反馈循环中重复此过程。]]
题为“在本地开发中赋予 Codex 完整的可观察性堆栈”的图表。一个应用程序将日志、指标和追踪数据发送到 Vector,Vector 会将数据分发到一个包含 Victoria Logs、Metrics 和 Traces 的可观测性堆栈中,并通过 LogQL、PromQL 或 TraceQL API 进行查询。Codex 使用这些信号进行查询、关联和推理,然后在代码库中进行修复,重启应用程序,重新运行工作负载,测试 UI 流程,并在反馈循环中重复此过程。
我们经常看到单次 Codex 运行在单个任务上持续工作超过六个小时(通常是在人类睡眠时间)。 我们经常看到单次 Codex 运行在单个任务上持续工作超过六个小时(通常是在人类睡眠时间)。
情境管理是使智能体在大型和复杂任务中有效发挥作用的最大挑战之一。我们学到的最早经验教训之一很简单:要给 Codex 的是一张地图,而不是一本 1,000 页的说明书。
![智能体知识局限边界](./harness-assets/agent-knowledge-limits.webp) ## 我们将代码仓库设为记录系统
我们尝试了“一个大型的 [AGENTS.md](https://agents.md/)”方法。可想而知,这是一次失败的尝试: 情境管理是使智能体在大型和复杂任务中有效发挥作用的最大挑战之一。我们学到的最早经验教训之一很简单: **要给 Codex 的是一张地图,而不是一本 1,000 页的说明书。**
- 情境是一种稀缺资源。一个巨大的指令文件会挤掉任务、代码和相关文档 — 因此智能体要么会错过关键约束条件,要么开始针对错误的约束条件进行优化。 我们尝试了“一个大型的 [`AGENTS.md` ](https://agents.md/) ”方法。可想而知,这是一次失败的尝试:
- 过多的指导反而变得无效。当一切都 "重要"时,一切都不重要了。智能体最终会在本地进行模式匹配,而不是有意识地进行导航。
- 它会立即腐烂。一本庞杂的手册会变成陈旧规则的坟场。智能体无法判断哪些信息仍然有效,一旦人类停止维护它,此文件就会悄然成为一个颇具吸引力的麻烦源头。
- 这很难核实。单个 blob 不适合进行机械检查(覆盖率、新鲜度、所有权、交叉链接),因此漂移是不可避免的。
因此,我们不再将 AGENTS.md 视为百科全书,而是将其视为内容目录 - **情境是一种稀缺资源。** 一个巨大的指令文件会挤掉任务、代码和相关文档 — 因此智能体要么会错过关键约束条件,要么开始针对错误的约束条件进行优化
代码仓库的知识库位于一个结构化了的 docs/ 目录中,此目录被当作记录系统来使用。一份简短的 AGENTS.md(大约 100 行)被注入到情境中,主要用作地图,并指向其他地方更深层次的真实信息来源 - **过多的指导反而变得** ***无效*** **。** 当一切都 "重要"时,一切都不重要了。智能体最终会在本地进行模式匹配,而不是有意识地进行导航
代码仓库内知识存储布局 - **它会立即腐烂。** 一本庞杂的手册会变成陈旧规则的坟场。智能体无法判断哪些信息仍然有效,一旦人类停止维护它,此文件就会悄然成为一个颇具吸引力的麻烦源头
- **这很难核实。** 单个 blob 不适合进行机械检查(覆盖率、新鲜度、所有权、交叉链接),因此漂移是不可避免的。
![Codex 驱动应用架构图](./harness-assets/fig1-codex-drives-app.webp) 因此,我们不再将 `AGENTS.md` 视为百科全书,而是将其视为 **内容目录**
代码仓库的知识库位于一个结构化了的 `docs/` 目录中,此目录被当作记录系统来使用。一份简短的 `AGENTS.md` (大约 100 行)被注入到情境中,主要用作地图,并指向其他地方更深层次的真实信息来源。
#### 纯文本
``` ```
AGENTS.md AGENTS.md
ARCHITECTURE.md ARCHITECTURE.md
docs/ docs/
├── design-docs/ ├── design-docs/
│ ├── index.md ├── index.md
│ ├── core-beliefs.md ├── core-beliefs.md
│ └── ... └── ...
├── exec-plans/ ├── exec-plans/
│ ├── active/ ├── active/
│ ├── completed/ ├── completed/
│ └── tech-debt-tracker.md └── tech-debt-tracker.md
├── generated/ ├── generated/
│ └── db-schema.md └── db-schema.md
├── product-specs/ ├── product-specs/
│ ├── index.md ├── index.md
│ ├── new-user-onboarding.md ├── new-user-onboarding.md
│ └── ... └── ...
├── references/ ├── references/
│ ├── design-system-reference-llms.txt ├── design-system-reference-llms.txt
│ ├── nixpacks-llms.txt ├── nixpacks-llms.txt
│ ├── uv-llms.txt ├── uv-llms.txt
│ └── ... └── ...
├── DESIGN.md ├── DESIGN.md
├── FRONTEND.md ├── FRONTEND.md
├── PLANS.md ├── PLANS.md
@@ -70,35 +117,72 @@ docs/
└── SECURITY.md └── SECURITY.md
``` ```
![Codex 全栈可观测性架构](./harness-assets/observability-stack.svg) 代码仓库内知识存储布局。
设计文档已被编目和索引,其中包括验证状态和一套核心理念,定义了智能体优先的操作原则。 [架构文档 ](https://matklad.github.io/2021/02/06/ARCHITECTURE.md.html) 提供域和包分层的顶层地图。一份高质量的文档会对每个产品领域和架构层进行评分,并随着时间的推移追踪差距。
计划被视为一流的工件。临时轻量计划用于小幅变更,而复杂工作则记录在 [执行计划 ](https://cookbook.openai.com/articles/codex_exec_plans) 中,并附带进度和决策日志,这些日志会被提交到代码仓库。活跃计划、已完成计划和已知的技术债务都已进行版本控制并集中存放,使智能体能够在不依赖外部情境的情况下运行。
这实现了 **渐进式披露** :智能体从一个小而稳定的切入点开始,并被指导下一步该去哪里查看,而不是一开始就被淹没。
设计文档已被编目和索引,其中包括验证状态和一套核心理念,定义了智能体优先的操作原则。[架构文档](https://matklad.github.io/2021/02/06/ARCHITECTURE.md.html)提供域和包分层的顶层地图。一份高质量的文档会对每个产品领域和架构层进行评分,并随着时间的推移追踪差距。
计划被视为一流的工件。临时轻量计划用于小幅变更,而复杂工作则记录在[执行计划](https://cookbook.openai.com/articles/codex_exec_plans)中,并附带进度和决策日志,这些日志会被提交到代码仓库。活跃计划、已完成计划和已知的技术债务都已进行版本控制并集中存放,使智能体能够在不依赖外部情境的情况下运行。
这实现了渐进式披露:智能体从一个小而稳定的切入点开始,并被指导下一步该去哪里查看,而不是一开始就被淹没。
我们严格执行这一点。专职的 linter 和 CI 作业会验证知识库的更新状况、是否已交叉链接且结构正确。一个定期运行的“doc-gardening”智能体会扫描那些不再反映真实代码行为的过时或废弃文档,并发起修复用的 Pull Request。 我们严格执行这一点。专职的 linter 和 CI 作业会验证知识库的更新状况、是否已交叉链接且结构正确。一个定期运行的“doc-gardening”智能体会扫描那些不再反映真实代码行为的过时或废弃文档,并发起修复用的 Pull Request。
随着代码库的发展,Codex 的设计决策框架也需要随之演变。
由于该代码仓库完全由智能体生成,因此我们首先针对 Codex 的可读性进行了优化。就像团队会努力提升代码对新入职工程师的可导航性一样,我们的人类工程师的目标也是让智能体能够直接从代码仓库推理出完整的业务领域。
从智能体的角度来看,它在运行时无法在情境中访问的任何内容都是不存在的。存储在 Google Docs、聊天记录或人们头脑中的知识都无法被系统访问。代码仓库本地的、已版本化的工件(例如,代码、Markdown、模式、可执行计划)就是它所能看到的全部。
我们了解到,随着时间的推移,我们需要将越来越多的情境推送到仓库中。那次让团队在架构模式上达成一致的 Slack 讨论?如果智能体无法发现它,那么它就会像迟了三个月入职的新员工一样,对其一无所知。
为 Codex 提供更多情境意味着要组织和展示正确的信息,好令智能体能够基于这些信息进行推理,而不是用临时指令使其不堪重负。就像你会在产品原则、工程规范和团队文化(包括表情符号偏好)方面为新队友提供引导一样,将这些信息提供给智能体会带来更一致的输出。
这一框架明确了许多取舍。我们倾向于选择那些可以完全内化于在仓库中进行推理的依赖项和抽象。对智能体来说,通常被称为“枯燥”的技术,由于其可组合性、API 稳定性和在训练集里的表现,往往更容易建立模型。在某些情况下,让智能体重新实现部分功能子集比绕过公共库中不透明的上游行为更便宜。例如,我们没有引入通用的 p-limit 风格包,而是投入使用了我们自己的带并发的 map 辅助函数:它与我们的 OpenTelemetry 仪表紧密集成,具备 100% 的测试覆盖率,并且其行为完全符合我们的运行时预期。
将系统的更多部分转化为智能体可以检查、验证并直接修改的形式,可以直接提高杠杆效应 — 这不仅适用于 Codex,也适用于其他智能体(例如[Aardvark](/index/introducing-aardvark/)) 也在参与代码库的开发。
仅靠文档本身,是没法保持完全由智能体生成的代码库的连贯性的。通过强制执行不变量,而非对实施过程进行微观管理,我们令智能体能够快速交付,而且不会削弱基础。例如,我们要求 Codex [在边界处解析数据形状](https://lexi-lambda.github.io/blog/2019/11/05/parse-don-t-validate/),但不规定具体实现方式(模型似乎偏好 Zod,但我们没有指定特定库)。
智能体在具有[严格边界和可预测结构](https://bits.logic.inc/p/ai-is-forcing-us-to-write-good-code)的环境中最为高效,因此我们围绕一个严格的架构模型构建了该应用。每个业务域都划分为一组固定的层,依赖方向经过严格验证,并且仅允许有限的一组边。这些约束是通过自定义的 linter(当然是由 Codex 生成的!)和结构测试机械地强制执行的。
![分层领域架构与跨切面边界](./harness-assets/layered-domain-architecture.webp) ## 目标是智能体的可读性
随着代码库的发展,Codex 的设计决策框架也需要随之演变。
由于该代码仓库完全由智能体生成,因此我们首先针对 *Codex**可读性* 进行了优化。就像团队会努力提升代码对新入职工程师的可导航性一样,我们的人类工程师的目标也是让智能体能够 **直接从代码仓库** 推理出完整的业务领域。
从智能体的角度来看,它在运行时无法在情境中访问的任何内容都是不存在的。存储在 Google Docs、聊天记录或人们头脑中的知识都无法被系统访问。代码仓库本地的、已版本化的工件(例如,代码、Markdown、模式、可执行计划)就是它所能看到的全部。
![[OAI_Harness_engineering_The_limits_of_agent_knowledge_desktop-dark.png|题为“智能体知识的局限性:Codex看不到的东西就不存在”的图表。Codex 的知识被展示为一个有边界的气泡。在其下方是不可见知识的示例 — Google Docs、Slack 消息,以及隐性的人类知识。箭头表示,要想让 Codex 看到这些信息,就必须将其以 Markdown 的形式编码到代码库中。]]
题为“智能体知识的局限性:Codex看不到的东西就不存在”的图表。Codex 的知识被展示为一个有边界的气泡。在其下方是不可见知识的示例 — Google Docs、Slack 消息,以及隐性的人类知识。箭头表示,要想让 Codex 看到这些信息,就必须将其以 Markdown 的形式编码到代码库中。
我们了解到,随着时间的推移,我们需要将越来越多的情境推送到仓库中。那次让团队在架构模式上达成一致的 Slack 讨论?如果智能体无法发现它,那么它就会像迟了三个月入职的新员工一样,对其一无所知。
为 Codex 提供更多情境意味着要组织和展示正确的信息,好令智能体能够基于这些信息进行推理,而不是用临时指令使其不堪重负。就像你会在产品原则、工程规范和团队文化(包括表情符号偏好)方面为新队友提供引导一样,将这些信息提供给智能体会带来更一致的输出。
这一框架明确了许多取舍。我们倾向于选择那些可以完全内化于在仓库中进行推理的依赖项和抽象。对智能体来说,通常被称为“枯燥”的技术,由于其可组合性、API 稳定性和在训练集里的表现,往往更容易建立模型。在某些情况下,让智能体重新实现部分功能子集比绕过公共库中不透明的上游行为更便宜。例如,我们没有引入通用的 `p-limit` 风格包,而是投入使用了我们自己的带并发的 map 辅助函数:它与我们的 OpenTelemetry 仪表紧密集成,具备 100% 的测试覆盖率,并且其行为完全符合我们的运行时预期。
将系统的更多部分转化为智能体可以检查、验证并直接修改的形式,可以直接提高杠杆效应 — 这不仅适用于 Codex,也适用于其他智能体(例如 [Aardvark](https://openai.com/zh-Hans-CN/index/introducing-aardvark/)) 也在参与代码库的开发。
## 规范架构与品味
仅靠文档本身,是没法保持完全由智能体生成的代码库的连贯性的。 **通过强制执行不变量,而非对实施过程进行微观管理,我们令智能体能够快速交付,而且不会削弱基础。** 例如,我们要求 Codex [在边界处解析数据形状 ](https://lexi-lambda.github.io/blog/2019/11/05/parse-don-t-validate/) ,但不规定具体实现方式(模型似乎偏好 Zod,但我们没有指定特定库)。
智能体在具有 [严格边界和可预测结构 ](https://bits.logic.inc/p/ai-is-forcing-us-to-write-good-code) 的环境中最为高效,因此我们围绕一个严格的架构模型构建了该应用。每个业务域都划分为一组固定的层,依赖方向经过严格验证,并且仅允许有限的一组边。这些约束是通过自定义的 linter(当然是由 Codex 生成的!)和结构测试机械地强制执行的。
下图展示了规则:在每个业务领域内(例如应用设置),代码只能“向前”依赖于一组固定的层(Types → Config → Repo → Service → Runtime → UI)。横切关注点(认证、连接器、遥测、功能标志)通过一个单一的显式接口进入:Providers。其他任何内容都不被允许,并将通过自动化方式强制执行。 下图展示了规则:在每个业务领域内(例如应用设置),代码只能“向前”依赖于一组固定的层(Types → Config → Repo → Service → Runtime → UI)。横切关注点(认证、连接器、遥测、功能标志)通过一个单一的显式接口进入:Providers。其他任何内容都不被允许,并将通过自动化方式强制执行。
![[OAI_Harness_engineering_Layered_domain_architecture_with_explicit_cross-cutting_boundries_desktop-dark.png|题为“具有明确交叉界限的分层领域架构”的图表。在业务逻辑域内存在以下模块:Types → Config → Repo,以及 Providers → Service → Runtime → UI,底部是 App Wiring + UI。一个 Utils 模块位于界限之外,并向 Providers 提供输入。]]
题为“具有明确交叉界限的分层领域架构”的图表。在业务逻辑域内存在以下模块:Types → Config → Repo,以及 Providers → Service → Runtime → UI,底部是 App Wiring + UI。一个 Utils 模块位于界限之外,并向 Providers 提供输入。
这种架构通常要等到你拥有数百名工程师时才会推迟。对于编码智能体来说,这是一个早期的先决条件:有了约束,速度才不会下降,架构才不会漂移。 这种架构通常要等到你拥有数百名工程师时才会推迟。对于编码智能体来说,这是一个早期的先决条件:有了约束,速度才不会下降,架构才不会漂移。
在实践中,我们通过自定义的代码检查器和结构测试来强制执行这些规则,并辅以一小组“品味不变式”。例如,我们通过自定义 lint 静态地强制执行结构化日志记录、模式和类型的命名约定、文件大小限制,以及特定平台的可靠性要求。由于这些 lint 是自定义的,我们编写错误信息时会在智能体情境中注入修复指令。 在实践中,我们通过自定义的代码检查器和结构测试来强制执行这些规则,并辅以一小组“品味不变式”。例如,我们通过自定义 lint 静态地强制执行结构化日志记录、模式和类型的命名约定、文件大小限制,以及特定平台的可靠性要求。由于这些 lint 是自定义的,我们编写错误信息时会在智能体情境中注入修复指令。
在以人为本的工作流程中,这些规则可能会让人感到迂腐或束缚。有了智能体,它们就成了倍增器:一旦编码,它们就能立即应用于所有地方。 在以人为本的工作流程中,这些规则可能会让人感到迂腐或束缚。有了智能体,它们就成了倍增器:一旦编码,它们就能立即应用于所有地方。
同时,我们还明确指出了哪些地方需要限制,哪些地方不需要限制。这类似于领导一个大型工程平台组织:在中央层面强制执行边界,在本地层面允许自主权。你非常重视界限、正确性和可重复性。在这些边界内,你允许团队或智能体在解决方案的表达方式上拥有很大的自由。 同时,我们还明确指出了哪些地方需要限制,哪些地方不需要限制。这类似于领导一个大型工程平台组织:在中央层面强制执行边界,在本地层面允许自主权。你非常重视界限、正确性和可重复性。在这些边界内,你允许团队或智能体在解决方案的表达方式上拥有很大的自由。
生成的代码不总是符合人类的风格偏好,这也没关系。只要输出是正确的、可维护的,并且对未来的智能体运行而言清晰易读,就可以算作达标。 生成的代码不总是符合人类的风格偏好,这也没关系。只要输出是正确的、可维护的,并且对未来的智能体运行而言清晰易读,就可以算作达标。
人类的品味会不断反馈到系统中。审查评论、重构的 Pull Request 和面向用户的 Bug 会被记录为文档更新,或直接编码到工具中。当文档不够完善时,我们会将规则转化为代码 人类的品味会不断反馈到系统中。审查评论、重构的 Pull Request 和面向用户的 Bug 会被记录为文档更新,或直接编码到工具中。当文档不够完善时,我们会将规则转化为代码
## 吞吐量改变了合并的理念
随着 Codex 的吞吐量增加,许多传统的工程规范变得不再有效。 随着 Codex 的吞吐量增加,许多传统的工程规范变得不再有效。
该代码仓库在运行过程中尽量减少阻塞合并门。Pull Request 的生命周期很短。测试偶发失败通常通过后续重跑来解决,而不是无限期地阻碍进展。在一个智能体吞吐量远超人类注意力的系统中,纠错成本低,而等待成本高。 该代码仓库在运行过程中尽量减少阻塞合并门。Pull Request 的生命周期很短。测试偶发失败通常通过后续重跑来解决,而不是无限期地阻碍进展。在一个智能体吞吐量远超人类注意力的系统中,纠错成本低,而等待成本高。
在低吞吐量环境中,这样做是不负责任的。而在这里,这通常是正确的选择。 在低吞吐量环境中,这样做是不负责任的。而在这里,这通常是正确的选择。
## “智能体生成”实际上意味着什么
当我们说代码库是由 Codex 智能体生成的,我们指的是整个代码库。 当我们说代码库是由 Codex 智能体生成的,我们指的是整个代码库。
智能体的产出包括: 智能体的产出包括:
- 产品代码与测试 - 产品代码与测试
@@ -111,8 +195,13 @@ docs/
- 生产仪表板定义文件 - 生产仪表板定义文件
人类始终参与其中,但工作的抽象层次与过去不同。我们优先处理工作,将用户反馈转化为验收标准,并对结果进行验证。当智能体遇到困难时,我们将其视为一个信号:识别缺失的内容 — 工具、指导与约束、文档 — 并将其反馈到代码仓库中,始终由 Codex 自己编写修复。 人类始终参与其中,但工作的抽象层次与过去不同。我们优先处理工作,将用户反馈转化为验收标准,并对结果进行验证。当智能体遇到困难时,我们将其视为一个信号:识别缺失的内容 — 工具、指导与约束、文档 — 并将其反馈到代码仓库中,始终由 Codex 自己编写修复。
智能体可以直接使用我们的标准开发工具。他们会拉取审查反馈、在行内回复、推送更新,并且经常压缩并合并他们自己的 Pull Request(合并请求)。 智能体可以直接使用我们的标准开发工具。他们会拉取审查反馈、在行内回复、推送更新,并且经常压缩并合并他们自己的 Pull Request(合并请求)。
## 不断提高的自主水平
随着越来越多的开发环节被直接编码到系统中 — 包括测试、验证、审查、反馈处理和恢复 — 该代码仓库最近跨过了一个重要门槛,使 Codex 能够端到端地驱动一个新功能。 随着越来越多的开发环节被直接编码到系统中 — 包括测试、验证、审查、反馈处理和恢复 — 该代码仓库最近跨过了一个重要门槛,使 Codex 能够端到端地驱动一个新功能。
给定一个提示,智能体现在可以: 给定一个提示,智能体现在可以:
- 验证代码库的当前状态 - 验证代码库的当前状态
@@ -128,12 +217,47 @@ docs/
- 合并更改 - 合并更改
此行为在很大程度上取决于此代码仓库的具体结构和工具,不应在没有类似投入的情况下假定它可以泛化 — 至少目前还不行。 此行为在很大程度上取决于此代码仓库的具体结构和工具,不应在没有类似投入的情况下假定它可以泛化 — 至少目前还不行。
完全自主的智能体也引入了新的问题。Codex 会复现代码仓库中已存在的模式 — 甚至包括那些不均衡或不够理想的模式。随着时间的推移,这不可避免地导致漂移。
## 熵与垃圾收集
**完全自主的智能体也引入了新的问题。** Codex 会复现代码仓库中已存在的模式 — 甚至包括那些不均衡或不够理想的模式。随着时间的推移,这不可避免地导致漂移。
最初,人类是手动处理这个问题的。我们的团队过去每周五(占一周的20%)都要花时间清理“AI 残渣”。不出所料,那并不具备可扩展性。 最初,人类是手动处理这个问题的。我们的团队过去每周五(占一周的20%)都要花时间清理“AI 残渣”。不出所料,那并不具备可扩展性。
相反,我们开始将我们称为“黄金原则”的内容直接编码到代码仓库中,并建立了一个循环清理流程。这些原则是带有主观意见的机械规则,旨在保持代码库的可读性和一致性,以便将来运行智能体。例如:(1) 我们更倾向于使用共享的实用程序包,而不是手工编写的辅助工具,以便将不变式集中管理;(2) 我们不会使用“YOLO 式”探测数据 — 我们会验证边界,或依赖类型化的 SDK,这样智能体就不会意外地基于猜测的结构进行构建。我们会定期运行一组后台 Codex 任务,扫描偏差、更新质量等级,并发起有针对性的重构 Pull Request。其中大多数都可以在一分钟内完成审查并自动合并。 相反,我们开始将我们称为“黄金原则”的内容直接编码到代码仓库中,并建立了一个循环清理流程。这些原则是带有主观意见的机械规则,旨在保持代码库的可读性和一致性,以便将来运行智能体。例如:(1) 我们更倾向于使用共享的实用程序包,而不是手工编写的辅助工具,以便将不变式集中管理;(2) 我们不会使用“YOLO 式”探测数据 — 我们会验证边界,或依赖类型化的 SDK,这样智能体就不会意外地基于猜测的结构进行构建。我们会定期运行一组后台 Codex 任务,扫描偏差、更新质量等级,并发起有针对性的重构 Pull Request。其中大多数都可以在一分钟内完成审查并自动合并。
其功能类似于垃圾回收。技术债务就像一笔高息贷款:不断地以小额贷款的方式偿还债务,总比让债务不断累积,再痛苦地一次解决要好得多。人类的品味一旦被捕捉,就会持续应用于每一行代码。这也使我们能够每天发现并解决不良模式,而不是让它们在代码库中传播数天或数周。 其功能类似于垃圾回收。技术债务就像一笔高息贷款:不断地以小额贷款的方式偿还债务,总比让债务不断累积,再痛苦地一次解决要好得多。人类的品味一旦被捕捉,就会持续应用于每一行代码。这也使我们能够每天发现并解决不良模式,而不是让它们在代码库中传播数天或数周。
## 我们仍在学习的内容
到目前为止,这一策略在 OpenAI 的内部发布和采纳过程中表现良好。为真实用户打造真实产品,帮助我们将投资锚定在现实中,并引导我们实现长期的可维护性。 到目前为止,这一策略在 OpenAI 的内部发布和采纳过程中表现良好。为真实用户打造真实产品,帮助我们将投资锚定在现实中,并引导我们实现长期的可维护性。
我们尚不清楚的是,在一个完全由智能体生成的系统中,架构连贯性会如何随着时间的推移而演变。我们仍在学习人类的判断力在哪些方面能发挥最大作用,以及如何对这种判断力进行编码,使其发挥更大作用。我们也不知道,随着时间的推移,模型的功能不断增强,这一系统将如何演变。 我们尚不清楚的是,在一个完全由智能体生成的系统中,架构连贯性会如何随着时间的推移而演变。我们仍在学习人类的判断力在哪些方面能发挥最大作用,以及如何对这种判断力进行编码,使其发挥更大作用。我们也不知道,随着时间的推移,模型的功能不断增强,这一系统将如何演变。
显而易见的是:构建软件仍然需要纪律,但纪律更多地体现在支撑结构上,而不是代码上。保持代码库一致性的工具、抽象和反馈回路变得越发重要。 显而易见的是:构建软件仍然需要纪律,但纪律更多地体现在支撑结构上,而不是代码上。保持代码库一致性的工具、抽象和反馈回路变得越发重要。
我们当前最棘手的挑战集中在设计环境、反馈回路和控制系统方面,帮助智能体实现我们的目标:大规模构建和维护复杂、可靠的软件。
随着像 Codex 这样的智能体在软件生命周期中占据越来越大的比重,这些问题将变得更加重要。我们希望通过分享一些早期的经验教训,帮助你理清投入精力的方向,以便[你可以直接开始构建](/codex/) **我们当前最棘手的挑战集中在设计环境、反馈回路和控制系统方面** ,帮助智能体实现我们的目标:大规模构建和维护复杂、可靠的软件
随着像 Codex 这样的智能体在软件生命周期中占据越来越大的比重,这些问题将变得更加重要。我们希望通过分享一些早期的经验教训,帮助你理清投入精力的方向,以便 [你可以直接开始构建](https://openai.com/zh-Hans-CN/codex/) 。
## 作者
Ryan Lopopolo
## 致谢
特别感谢 Victor Zhu 和 Zach Brock 为这篇文章做出的贡献,以及开发这款新产品的整个团队。
## 继续阅读[超越速率限制:扩大 Codex 和 Sora 的访问规模](https://openai.com/zh-Hans-CN/index/beyond-rate-limits/)
[
工程
](https://openai.com/zh-Hans-CN/index/beyond-rate-limits/)[解锁 Codex 运行框架:我们如何构建 App Server](https://openai.com/zh-Hans-CN/index/unlocking-the-codex-harness/)
[
工程
](https://openai.com/zh-Hans-CN/index/unlocking-the-codex-harness/)
+141 -8
View File
@@ -12,29 +12,95 @@
--- ---
## 1.5 大文档处理要求
对于篇幅较长的 raw 文档(如学术论文、长篇技术文章),**必须完整阅读和分析**,不得仅基于开头部分生成简短摘要。
### 具体要求:
1. **完整内容获取**
- 使用 Grep 搜索章节标题(如 `^#{1,3} `)了解文档结构
- 分段读取完整内容,确保覆盖所有主要章节
- 特别关注:摘要、引言、方法、实验、讨论、结论、附录等核心章节
2. **深度分析维度**
- **核心论点**:提取文章的主要主张和关键发现
- **方法细节**:理解技术方案的实现细节和设计决策
- **实验结果**:完整记录所有实验数据、表格、图表信息
- **案例研究**:保留具体的定性示例和应用场景
- **相关工作**:建立与其他研究的联系和对比
3. **输出内容标准**
- wiki 页面长度应与原文档的重要性和复杂度相匹配
- 学术论文应包含:摘要、核心方法、完整实验结果、详细讨论
- 技术文章应包含:问题背景、完整解决方案、实际应用案例
- 保留所有定量数据(表格、指标、分数等)
4. **例外情况**
- 仅在以下情况下可生成较短摘要:
- 文档是纯新闻报道或简短公告
- 文档主要是代码或配置(无大量叙事内容)
- 用户明确要求仅生成摘要
---
## 2. 标准文件系统架构 ## 2. 标准文件系统架构
严格遵循 I/O 分离原则,确保知识库的纯净度与可迁移性: 严格遵循 I/O 分离原则,确保知识库的纯净度与可迁移性:
```text ```text
📁 wikillm 📁 wikillm
├── 📁 raw/ # 【输入层】原始素材(只读) ├── 📁 raw/ # 【输入层】原始素材(只读)
│ └── 📁 images/ # 原始图片文件(png, jpg, webp, gif, svg 等)
└── 📁 wiki/ # 【输出层】编译器生成的知识产物 └── 📁 wiki/ # 【输出层】编译器生成的知识产物
├── 📁 concepts/ # 核心概念、原理分析 ├── 📁 concepts/ # 核心概念、原理分析
├── 📁 practices/ # 部署指南、最佳实践 ├── 📁 practices/ # 部署指南、最佳实践
├── 📁 visual/ # Marp 幻灯片、Matplotlib 趋势图 ├── 📁 visual/ # Marp 幻灯片、Matplotlib 趋势图
├── 📁 queries/ # 高价值 Q&A 的沉淀归档 ├── 📁 queries/ # 高价值 Q&A 的沉淀归档
├── 📁 assets/ # 图像和资源文件(从 raw/images/ 同步而来)
├── INDEX.md # 动态索引与学习路径 ├── INDEX.md # 动态索引与学习路径
── Glossary.md # 统一术语表与双链枢纽 ── Glossary.md # 统一术语表与双链枢纽
└── sources.md # 来源文档索引(原始 URL 列表)
``` ```
--- ---
## 3. 核心工作流 (The "Compilation" Loop) ## 3. 核心工作流 (The "Compilation" Loop)
### 阶段 0:增量检查 (Incremental Check)
* **任务**:检查 `raw/` 目录下哪些文件需要编译。
* **读取状态**:读取 `wiki/compile-results.tsv`,获取已编译文件的哈希记录。
* **扫描文件**:遍历 `raw/` 目录,计算每个文件的 SHA-256 哈希。
* **识别变更**:对比哈希值,识别:
- **新增文件**:在 `compile-results.tsv` 中不存在的文件
- **修改文件**:哈希值与记录不同的文件
- **未修改文件**:哈希值相同的文件(跳过编译)
* **记录日志**:将检查过程写入 `wiki/compile.log`
### 阶段 0.5:资源同步 (Asset Sync)
* **任务**:将 `raw/images/` 下的所有图片资源同步到 `wiki/assets/`
* **同步范围**:所有图像文件,包括但不限于:
- `png`, `jpg`, `jpeg`, `gif`, `webp`, `svg`
* **同步方式**
- 使用 `cp -r raw/images/* wiki/assets/` 进行完整同步
- `raw/images/` 是权威来源,同名文件直接覆盖
- 保留原始文件名(包括空格和特殊字符)
* **验证**:确保 `wiki/assets/` 包含 `raw/images/` 中的所有文件
* **时机**:每次编译前必须执行此步骤
### 阶段 1:多模态解构 (Ingest & Analyze) ### 阶段 1:多模态解构 (Ingest & Analyze)
* **任务**:解析 `raw/` 目录下的新增内容。 * **任务**:解析 `raw/` 目录下的新增或修改内容。
* **大文档完整阅读**:对于篇幅较长的文档(学术论文、长篇技术文章),必须完整阅读和分析:
- 首先用 Grep 搜索章节标题(如 `^#{1,3} `)了解文档结构
- 分段读取完整内容,确保覆盖所有主要章节
- 特别关注:摘要、引言、方法、实验、讨论、结论、附录等核心章节
* **视觉解析**:对图片进行深度 OCR 与逻辑识别。将架构图转化为文字描述及 **Mermaid** 代码块,存入对应 Wiki 页面。 * **视觉解析**:对图片进行深度 OCR 与逻辑识别。将架构图转化为文字描述及 **Mermaid** 代码块,存入对应 Wiki 页面。
* **元数据提取**:为每篇文档生成 YAML Frontmatter(包含:`tags`, `source`, `confidence_score`, `last_updated`)。 * **元数据提取**:为每篇文档生成 YAML Frontmatter(包含:`tags`, `source`, `raw_sources`, `confidence_score`, `last_updated`)。
- `raw_sources` 字段:记录源文件路径和哈希值,格式如下:
```yaml
raw_sources:
- path: raw/anthropic-harness-design.md
hash: "sha256:abc123..."
```
### 阶段 2:增量编译 (Incremental Writing) ### 阶段 2:增量编译 (Incremental Writing)
* **非线性重构**:不进行 1:1 翻译,而是基于源文档的“核心贡献”进行重写。 * **非线性重构**:不进行 1:1 翻译,而是基于源文档的“核心贡献”进行重写。
@@ -67,10 +133,16 @@
* **动态索引**:根据新增内容,自动更新 `INDEX.md` 中的”最新研究”与”学习路径”部分。 * **动态索引**:根据新增内容,自动更新 `INDEX.md` 中的”最新研究”与”学习路径”部分。
### 阶段 4:健康检查与维护 (Linting) ### 阶段 4:健康检查与维护 (Linting)
* **一致性检查**:扫描 `wiki/`,发现术语冲突(如 A 文档叫智能体”,B 文档叫代理”)时,自动统一。 * **一致性检查**:扫描 `wiki/`,发现术语冲突(如 A 文档叫智能体”,B 文档叫代理”)时,自动统一。
* **孤岛扫描**:识别没有任何链接指向的页面,强制将其挂载到导航树中。 * **孤岛扫描**:识别没有任何链接指向的页面,强制将其挂载到导航树中。
* **补丁发布**:当 `raw/` 有新版本(如论文更新)时,在对应 Wiki 页面顶部发布 `[Update Patch]` 摘要。 * **补丁发布**:当 `raw/` 有新版本(如论文更新)时,在对应 Wiki 页面顶部发布 `[Update Patch]` 摘要。
### 阶段 5:来源索引更新 (Sources Update)
* **任务**:更新 `wiki/sources.md`,记录本次编译涉及的来源文档。
* **格式规范**:使用简单无序列表,每项格式为 `- [标题](URL)`
* **增量更新**:添加本次新增的来源,保持已有来源不变
* **分类组织**:按”学术论文”、”概念文章”、”实践指南”等类别合理分组
--- ---
## 4. 输出质量标准 (The Gold Standard) ## 4. 输出质量标准 (The Gold Standard)
@@ -102,6 +174,7 @@
## 6. 执行清单 (Checklist) ## 6. 执行清单 (Checklist)
* [ ] **Raw Check**: `raw/` 目录中是否包含待处理的新素材(图片/文档)? * [ ] **Raw Check**: `raw/` 目录中是否包含待处理的新素材(图片/文档)?
* [ ] **Asset Sync**: `raw/images/` 下的所有图片是否已同步到 `wiki/assets/`
* [ ] **Glossary Lock**: 是否已锁定全局术语表,确保翻译不漂移? * [ ] **Glossary Lock**: 是否已锁定全局术语表,确保翻译不漂移?
* [ ] **Multimodal Sync**: 图片是否已转化为可编辑的文字解析/Mermaid? * [ ] **Multimodal Sync**: 图片是否已转化为可编辑的文字解析/Mermaid?
* [ ] **文件名规范**: 所有 wiki 页面文件是否使用 kebab-case(连字符分隔)命名? * [ ] **文件名规范**: 所有 wiki 页面文件是否使用 kebab-case(连字符分隔)命名?
@@ -112,7 +185,67 @@
--- ---
## 7. 最佳实践提示 ## 7. 增量编译工作流
* **手离开键盘**:不要手动修改 `wiki/` 目录下的内容,所有的修改应通过“向 LLM 发出 Lint 任务”或“添加 raw 素材后重新编译”来完成。
* **搜索即创作**:把每一次对知识库的提问看作是一次“知识合成”,务必将高质量的回答存回库中。 ### 编译状态文件
* **结构化思考**:在生成任何长篇文档前,先让 LLM 在内存中构建该主题的“概念地图”。
项目使用三个核心文件来追踪编译状态:
1. **`wiki/compile-results.tsv`** - 结构化的编译结果(TSV 格式)
- 字段:`raw_path``hash``last_modified``wiki_paths``compile_time``status`
- 记录每个 raw 文件的编译状态和生成的 wiki 文档
2. **`wiki/compile.log`** - 详细的编译日志
- 记录每次编译的输入、输出、决策过程
- 用于调试、审查和回溯
3. **`wiki/sources.md`** - 来源文档索引
- 记录所有原始来源的标题和 URL
- 格式:`- [标题](URL)` 的无序列表
- 按"学术论文"、"概念文章"、"实践指南"等分类组织
- 每次增量编译后更新
### Wiki 文档元数据
每个 wiki 文档的 YAML frontmatter 都包含 `raw_sources` 字段:
```yaml
---
title: 文档标题
source: [来源名称]
raw_sources:
- path: raw/source-file.md
hash: “sha256:abc123...”
---
```
### 常用命令
```bash
# 查看所有已编译文件
cat wiki/compile-results.tsv
# 查找特定文件的编译状态
grep “raw/xxx.md” wiki/compile-results.tsv
# 查看最近的编译日志
tail -100 wiki/compile.log
# 查看上次编译摘要
grep “=== 编译完成 ===” -A 5 wiki/compile.log
```
### 增量编译检查清单
* [ ] **扫描检查**:运行扫描,检查 `raw/` 目录中是否有新增或修改的文件
* [ ] **哈希对比**:与 `compile-results.tsv` 中的记录对比,确认变更
* [ ] **日志记录**:将检查过程写入 `compile.log`
* [ ] **只编译变更**:仅处理新增或修改的文件
* [ ] **更新 frontmatter**:确保新编译的 wiki 文档包含 `raw_sources`
* [ ] **更新状态文件**:追加/更新 `compile-results.tsv` 中的记录
* [ ] **更新来源索引**:更新 `sources.md`,添加本次新增的来源
## 8. 最佳实践提示
* **手离开键盘**:不要手动修改 `wiki/` 目录下的内容,所有的修改应通过”向 LLM 发出 Lint 任务”或”添加 raw 素材后重新编译”来完成。
* **搜索即创作**:把每一次对知识库的提问看作是一次”知识合成”,务必将高质量的回答存回库中。
* **结构化思考**:在生成任何长篇文档前,先让 LLM 在内存中构建该主题的”概念地图”。
* **利用编译日志**:遇到问题时,先查看 `wiki/compile.log` 了解之前的编译过程。
+309 -251
View File
@@ -1,294 +1,352 @@
--- ---
title: 术语表 title: "术语表"
tags: [术语表, 核心概念] source: "Externalization in LLM Agents: A Unified Review"
last_updated: 2026-04-07 last_updated: 2026-04-11
--- ---
# 术语表 # 术语表
本术语表汇总了 [[Harness-Engineering|Harness 工程]] 领域的核心概念,为知识库提供统一的术语枢纽 本术语表统一了 LLM Agent 外部化框架中的核心概念,提供中英对照和 Wikilink 链接
## A ## 核心框架
### Agent (智能体) ### Externalization (外部化)
**英文**Agent **英文**: Externalization
**中文**:智能体 **中文**: 外部化
**定义**:能够自主感知环境、做出决策并执行行动的 AI 系统。在编码场景中,智能体通常具备读取文件、执行程序、发起 HTTP 请求等工具调用能力 **定义**: 将认知负担从模型的内部计算逐步迁移到持久、可检查和可重用的外部结构中的过程
**相关概念**[[Agent Teams|智能体团队]], [[Coding Agent|编码智能体]] **参见**: [[Externalization-in-LLM-Agents|LLM Agent 中的外部化]]
### Agent Teams (智能体团队) ### Cognitive Artifact (认知人工制品)
**英文**Agent Teams **英文**: Cognitive Artifact
**中文**:智能体团队 **中文**: 认知人工制品
**定义**:多个智能体通过角色分工、协作协议和差异化行为共同完成复杂任务的系统。需要模型原生支持角色锚定、对抗性推理和协议遵守 **定义**: 设计用于维持、显示或操作信息的人工设备,通过改变任务本身的结构来改变认知性能
**相关概念**[[MiniMax M2.7]], [[Multi-Agent Collaboration|多智能体协作]] **参见**: [[Externalization-in-LLM-Agents|LLM Agent 中的外部化]]
### Anthropic Harness Design (Anthropic Harness 设计)
**英文**Anthropic Harness Design
**中文**Anthropic Harness 设计
**定义**Anthropic 团队提出的多智能体架构,包含 Planner(规划者)、Generator(生成者)和 Evaluator(评估者)三种角色,通过生成-评估循环提升输出质量。
**相关概念**[[Generator-Evaluator Loop|生成-评估循环]], [[Context Reset|上下文重置]]
## B
### Build-Verify Loop (构建-验证循环)
**英文**Build-Verify Loop
**中文**:构建-验证循环
**定义**:智能体在完成任务过程中自主进行的迭代改进流程,包括规划发现、构建实现、验证测试和修复问题四个阶段。
**相关概念**[[Self-Verification|自我验证]], [[Reasoning Sandwich|推理三明治]]
## C
### Codex (Codex 模型)
**英文**Codex
**中文**Codex 模型
**定义**:OpenAI 推出的专门用于代码生成的模型系列,在 Harness 工程中被用于从零生成完整产品代码库。
**相关概念**[[OpenAI Harness Engineering|OpenAI Harness 工程]]
### Coding Agent (编码智能体)
**英文**Coding Agent
**中文**:编码智能体
**定义**:专门用于软件工程任务的 AI 智能体,能够理解代码库、编写代码、运行测试和调试问题。
**相关概念**[[Harness|Harness]], [[Agent|智能体]]
### Context Engineering (上下文工程)
**英文**Context Engineering
**中文**:上下文工程
**定义**:Harness 工程的三大支柱之一,专注于确保智能体在正确的时间获得正确的信息,包括静态上下文和动态上下文。
**相关概念**[[Harness-Engineering|Harness 工程]], [[Context Reset|上下文重置]]
### Context Reset (上下文重置)
**英文**Context Reset
**中文**:上下文重置
**定义**:一种解决长任务中上下文窗口填充和"上下文焦虑"问题的技术,通过清空上下文窗口并使用结构化交接传递状态来实现。
**相关概念**[[Context Compaction|上下文压缩]], [[Context Anxiety|上下文焦虑]]
### Context Anxiety (上下文焦虑)
**英文**Context Anxiety
**中文**:上下文焦虑
**定义**:模型在接近其认为的上下文限制时提前结束工作的倾向,Claude Sonnet 4.5 表现出较强的这种行为。
**相关概念**[[Context Reset|上下文重置]], [[Context Window|上下文窗口]]
### Context Compaction (上下文压缩)
**英文**Context Compaction
**中文**:上下文压缩
**定义**:通过摘要方式保留对话连续性的技术,但无法为智能体提供干净的状态,上下文焦虑问题仍可能存在。
**相关概念**[[Context Reset|上下文重置]]
## D
### Doom Loop (末日循环)
**英文**Doom Loop
**中文**:末日循环
**定义**:智能体在陷入困境时对同一错误方法进行小幅变异的重复尝试现象,可能多达 10 次以上。
**相关概念**[[Loop Detection|循环检测]]
## E
### Entropy Management (熵管理)
**英文**Entropy Management
**中文**:熵管理
**定义**:Harness 工程的三大支柱之一,通过定期清理智能体来管理 AI 生成代码库中随时间积累的熵(文档漂移、命名约定分歧、死代码堆积等)。
**相关概念**[[Harness-Engineering|Harness 工程]], [[Garbage Collection|垃圾回收]]
### Evaluator (评估者)
**英文**Evaluator
**中文**:评估者
**定义**Anthropic Harness 设计中的三种角色之一,负责评估 Generator 的输出质量,提供具体的反馈和评分。
**相关概念**[[Generator|生成者]], [[Planner|规划者]]
## F
### Feedforward (前馈控制)
**英文**Feedforward
**中文**:前馈控制
**定义**:预期智能体行为并在其行动前进行引导的控制方式,提高智能体第一次尝试就产生良好结果的概率。
**相关概念**[[Feedback|反馈控制]], [[Guide|引导]]
### Feedback (反馈控制)
**英文**Feedback
**中文**:反馈控制
**定义**:在智能体行动后进行观察并帮助其自我纠正的控制方式,特别是当产生针对 LLM 消费优化的信号时效果显著。
**相关概念**[[Feedforward|前馈控制]], [[Sensor|传感器]]
## G
### Garbage Collection (垃圾回收)
**英文**Garbage Collection
**中文**:垃圾回收
**定义**:OpenAI 团队采用的定期清理流程,通过"黄金原则"和后台 Codex 任务来扫描偏差、更新质量等级并发起针对性重构。
**相关概念**[[Entropy Management|熵管理]]
### Generator (生成者)
**英文**Generator
**中文**:生成者
**定义**Anthropic Harness 设计中的三种角色之一,负责实际创建输出(如前端代码、应用功能等)。
**相关概念**[[Evaluator|评估者]], [[Planner|规划者]]
### Generator-Evaluator Loop (生成-评估循环)
**英文**Generator-Evaluator Loop
**中文**:生成-评估循环
**定义**:受 GAN 启发的多智能体结构,Generator 生成输出,Evaluator 评估并提供反馈,Generator 根据反馈进行迭代改进。
**相关概念**[[Anthropic-Harness-Design|Anthropic Harness 设计]]
### Glossary (术语表)
**英文**Glossary
**中文**:术语表
**定义**:WikiLLM 知识库的核心枢纽文档,统一术语翻译、提供中英对照,并通过 wikilinks 连接所有相关概念。
**相关概念**[[Wikilink|Wikilink]], [[WikiLLM]]
## H
### Harness (Harness) ### Harness (Harness)
**英文**Harness **英文**: Harness
**中文**Harness **中文**: Harness
**定义**:AI 智能体之外的一切,包括系统提示、工具选择、执行流程、约束条件、反馈循环等。公式:Agent = Model + Harness **定义**: 将原始模型能力转化为可靠 Agent 行为的脚手架,是承载记忆、技能、协议并提供编排逻辑、约束、可观测性和反馈循环的工程层
**相关概念**[[Harness-Engineering|Harness 工程]], [[Agent|智能体]] **参见**: [[Harness-Engineering|Harness 工程]]
### Harness Engineering (Harness 工程) ### Representational Transformation (表征转换)
**英文**Harness Engineering **英文**: Representational Transformation
**中文**Harness 工程 **中文**: 表征转换
**定义**:设计和实现使 AI 智能体可靠工作的系统的新学科,包括约束智能体行为、告知智能体应该做什么、验证智能体正确执行、纠正智能体错误四个方面。 **定义**: 认知人工制品的核心作用——重组问题,使智能体能够用它已经拥有的能力更可靠地解决问题。
**相关概念**[[Context-Engineering|上下文工程]], [[Architectural-Constraints|架构约束]], [[Entropy Management|熵管理]]
### Harness Template (Harness 模板) ## 记忆系统
**英文**Harness Template
**中文**Harness 模板
**定义**:为常见服务拓扑(如数据仪表板、CRUD 业务服务、事件处理器)准备的引导和传感器捆绑包,可实例化用于特定项目。
**相关概念**[[Harness-Engineering|Harness 工程]]
## L ### Memory System (记忆系统)
**英文**: Memory System
**中文**: 记忆系统
**定义**: 外部化 Agent 状态跨时间的系统,允许积累的知识在单个会话之外持续存在,并在相关时被选择性检索。
**参见**: [[Memory-Systems|记忆系统]]
### LangChain Harness Engineering (LangChain Harness 工程) ### Working Context (工作上下文)
**英文**LangChain Harness Engineering **英文**: Working Context
**中文**LangChain Harness 工程 **中文**: 工作上下文
**定义**LangChain 团队通过仅改变 Harness 将编码智能体在 Terminal Bench 2.0 上的表现从 52.8% 提升到 66.5%Top 30 到 Top 5)的实践。 **定义**: 当前任务的实时中间状态:打开的文件、临时变量、活跃假设、部分计划和执行检查点。
**相关概念**[[Harness-Engineering|Harness 工程]], [[Self-Verification|自我验证]]
### Layered Domain Architecture (分层领域架构) ### Episodic Experience (情景经验)
**英文**Layered Domain Architecture **英文**: Episodic Experience
**中文**:分层领域架构 **中文**: 情景经验
**定义**:OpenAI 采用的严格架构模型,每个业务领域划分为固定的层组(Types → Config → Repo → Service → Runtime → UI),依赖方向经过严格验证。 **定义**: 记录先前运行中发生的事情:决策点、工具调用、失败、结果和反思。
**相关概念**[[Architectural-Constraints|架构约束]]
### Loop Detection (循环检测) ### Semantic Knowledge (语义知识)
**英文**Loop Detection **英文**: Semantic Knowledge
**中文**:循环检测 **中文**: 语义知识
**定义**:LangChain 采用的中间件,通过钩子跟踪每个文件的编辑次数,在对同一文件进行 N 次编辑后添加"考虑重新考虑你的方法"的上下文。 **定义**: 存储在任何单个情节之外都存在的抽象:领域事实、一般启发式、项目约定和稳定的世界知识。
**相关概念**[[Doom Loop|末日循环]], [[Middleware|中间件]]
## M ### Personalized Memory (个性化记忆)
**英文**: Personalized Memory
**中文**: 个性化记忆
**定义**: 跟踪关于特定用户、团队或环境的稳定信息:偏好、习惯、重复出现的约束和先前的交互。
### Middleware (中间件) ## 技能系统
**英文**Middleware
**中文**:中间件
**定义**LangChain 结构化 Harness 的方式,通过可组合的中间件层在不修改核心智能体逻辑的情况下添加特定功能。
**相关概念**[[LangChain Harness Engineering|LangChain Harness 工程]]
### MiniMax M2.7 (MiniMax M2.7 模型) ### Skill System (技能系统)
**英文**MiniMax M2.7 **英文**: Skill System
**中文**MiniMax M2.7 模型 **中文**: 技能系统
**定义**:MiniMax 推出的深度参与自我进化的模型,能够构建复杂智能体 Harness、完成高度复杂的生产力任务,包括 Agent Teams、复杂 Skills 和动态工具搜索 **定义**: 将程序、最佳实践和操作指导打包成可重用的人工制品的系统,而不是依赖模型的权重在每次调用时重新生成特定任务的知识
**相关概念**[[Agent Teams|智能体团队]], [[Self-Evolution|自我进化]] **参见**: [[Skill-Systems|技能系统]]
### Mitchellh AI Adoption Journey (Mitchellh AI 采用之旅) ### Operational Procedure (操作程序)
**英文**Mitchellh AI Adoption Journey **英文**: Operational Procedure
**中文**Mitchellh AI 采用之旅 **中文**: 操作程序
**定义**HashiCorp 创始人 Mitchell Hashimoto 分享的个人 AI 工具采用历程,包括从聊天机器人到始终运行智能体的六个阶段。 **定义**: 任务骨架:将复杂工作分解为步骤、阶段、依赖关系和停止条件。
**相关概念**[[Harness-Engineering|Harness 工程]]
## N ### Decision Heuristic (决策启发式)
**英文**: Decision Heuristic
**中文**: 决策启发式
**定义**: 在分支处管理发生情况的实用经验法则,从经验中得出而不是仅靠穷举搜索。
### NxCode Harness Engineering (NxCode Harness 工程) ### Normative Constraint (规范性约束)
**英文**NxCode Harness Engineering **英文**: Normative Constraint
**中文**NxCode Harness 工程 **中文**: 规范性约束
**定义**NxCode 团队提供的 Harness 工程完整指南,总结了三大支柱、实践框架和常见错误。 **定义**: 程序被视为可接受的条件,包括测试要求、范围限制、访问限制、可追溯性期望和特定领域操作规则。
**相关概念**[[Harness-Engineering|Harness 工程]]
## O ### Progressive Disclosure (渐进式披露)
**英文**: Progressive Disclosure
**中文**: 渐进式披露
**定义**: 一种分层加载策略,首先暴露技能的存在,仅在需要时才加载更深的细节。
### OpenAI Harness Engineering (OpenAI Harness 工程) ## 协议系统
**英文**OpenAI Harness Engineering
**中文**OpenAI Harness 工程
**定义**:OpenAI 团队在 5 个月内构建了超过 100 万行代码的产品,其中零行代码由人工编写,证明了 Harness 工程在生产规模上的有效性。
**相关概念**[[Harness-Engineering|Harness 工程]], [[Codex|Codex 模型]]
### Observability Stack (可观测性堆栈) ### Agent Protocol (智能体协议)
**英文**Observability Stack **英文**: Agent Protocol
**中文**:可观测性堆栈 **中文**: 智能体协议
**定义**OpenAI 为 Codex 提供的日志、指标和追踪记录展示系统,使智能体能够直接访问应用程序的运行状态 **定义**: 定义了用于发现、调用、委托和权限管理的显式机器可读契约,而不是依赖临时提示级别的协调
**相关概念**[[Context-Engineering|上下文工程]] **参见**: [[Agent-Protocols|智能体协议]]
## P ### Invocation Grammar (调用语法)
**英文**: Invocation Grammar
**中文**: 调用语法
**定义**: 每个工具调用、API 请求或委托消息都需要的格式:参数名称、类型、排序和返回结构。
### Planner (规划者) ### Lifecycle Semantics (生命周期语义)
**英文**Planner **英文**: Lifecycle Semantics
**中文**:规划者 **中文**: 生命周期语义
**定义**Anthropic Harness 设计中的三种角色之一,负责将简单的 1-4 句话提示扩展为完整的产品规格。 **定义**: 多步交互需要的协调规则:谁接下来行动,允许什么状态转换,任务何时完成或失败。
**相关概念**[[Generator|生成者]], [[Evaluator|评估者]]
## R ### MCP (Model Context Protocol)
**英文**: Model Context Protocol (MCP)
**中文**: 模型上下文协议 (MCP)
**定义**: Anthropic 提出的标准化协议,为智能体提供跨异构服务发现工具、检查其模式和调用它们的方式。
### Reasoning Sandwich (推理三明治) ### A2A (Agent-to-Agent Protocol)
**英文**Reasoning Sandwich **英文**: Agent-to-Agent Protocol (A2A)
**中文**:推理三明治 **中文**: 智能体到智能体协议 (A2A)
**定义**:LangChain 采用的推理预算分配策略,在规划和验证阶段使用高推理预算,在实现阶段使用中等推理预算。 **定义**: Google 提出的标准化智能体间通信协议,支持能力发现、任务委托和状态交换。
**相关概念**[[Build-Verify Loop|构建-验证循环]]
### Ralph Wiggum Loop (Ralph Wiggum 循环) ## Harness 工程
**英文**Ralph Wiggum Loop
**中文**Ralph Wiggum 循环
**定义**:使用钩子在智能体退出时强制其继续执行的循环模式,用于验证环节。
**相关概念**[[Self-Verification|自我验证]]
## S ### Agent Loop (智能体循环)
**英文**: Agent Loop
**中文**: 智能体循环
**定义**: Harness 的时间骨干,实现感知-检索-计划-行动-观察周期。
**参见**: [[Harness-Engineering|Harness 工程]]
### Self-Verification (自我验证) ### Sandboxing (沙箱)
**英文**Self-Verification **英文**: Sandboxing
**中文**:自我验证 **中文**: 沙箱
**定义**:智能体通过运行测试、阅读完整输出并与原始要求进行比较来自我改进的能力。 **定义**: 创建受控的执行边界,限制 Agent 可以读取、写入和修改的内容,并提供使失败可诊断和回滚可行的可再现性保证。
**相关概念**[[Build-Verify Loop|构建-验证循环]]
### Self-Evolution (自我进化) ### Observability (可观测性)
**英文**Self-Evolution **英文**: Observability
**中文**:自我进化 **中文**: 可观测性
**定义**:模型深度参与自身进化的过程,包括更新自身记忆、构建复杂技能、根据实验结果改进学习过程和 Harness。 **定义**: 使 Agent 的内部轨迹对开发者、操作员和 Agent 本身可见的机制,包括结构化日志、执行轨迹和聚合指标。
**相关概念**[[MiniMax M2.7|MiniMax M2.7 模型]]
### Sensor (传感器) ### Context Budget Management (上下文预算管理)
**英文**Sensor **英文**: Context Budget Management
**中文**:传感器 **中文**: 上下文预算管理
**定义**:观察智能体行动后结果并帮助其自我纠正的反馈控制,包括计算型和推理型两种类型。 **定义**: 主动管理最稀缺的共享资源——上下文窗口——的策略,包括摘要、基于优先级的驱逐和分阶段加载。
**相关概念**[[Feedback|反馈控制]]
## 历史演进
### Weights Layer (权重层)
**英文**: Weights Layer
**中文**: 权重层
**定义**: LLM 部署的最早浪潮,其中能力几乎完全与模型参数等同。
**参见**: [[From-Weights-to-Context-to-Harness|从权重到上下文到 Harness]]
### Context Layer (上下文层)
**英文**: Context Layer
**中文**: 上下文层
**定义**: 注意力从模型修改转向输入设计的阶段,包括提示工程、思维链、ReAct、RAG 等技术。
### Harness Layer (Harness 层)
**英文**: Harness Layer
**中文**: Harness 层
**定义**: 当前阶段,其中能力延伸超出提示管理进入持久基础设施。
## 理论基础
### Distributed Cognition (分布式认知)
**英文**: Distributed Cognition
**中文**: 分布式认知
**定义**: 拒绝认知完全驻留在个人心灵内的观点,而是将认知过程定位在人、人工制品、表征和协调实践之间。
### Complementary Strategies (互补策略)
**英文**: Complementary Strategies
**中文**: 互补策略
**定义**: Kirsh 的理论,认为智能体不仅通过在内部更努力地思考来提高性能,还通过重组外部环境使一些认知工作卸载到其中来提高性能。
## 实践术语
### Context Anxiety (上下文焦虑)
**英文**: Context Anxiety
**中文**: 上下文焦虑
**定义**: 一些模型表现出的倾向,当它们接近认为的上下文限制时,会过早地结束工作。
### Context Reset (上下文重置)
**英文**: Context Reset
**中文**: 上下文重置
**定义**: 完全清除上下文窗口并启动一个新的 Agent,结合结构化移交来携带前一个 Agent 的状态和下一步。
### Sprint Contract (冲刺契约) ### Sprint Contract (冲刺契约)
**英文**Sprint Contract **英文**: Sprint Contract
**中文**冲刺契约 **中文**: 冲刺契约
**定义**在每个冲刺前,Generator 和 Evaluator 协商达成的协议,定义该阶段工作的"完成"标准。 **定义**: 在每个冲刺前,生成器和评估器协商的协议,就在编写任何代码之前该工作的"完成"是什么样子达成一致。
**相关概念**[[Anthropic-Harness-Design|Anthropic Harness 设计]]
## T ### Planner-Generator-Evaluator (规划器-生成器-评估器)
**英文**: Planner-Generator-Evaluator
**中文**: 规划器-生成器-评估器
**定义**: 一种三 Agent 架构,用于长运行自主编码:规划器将简单提示扩展为完整规范,生成器一次实现一个功能,评估器测试并评分结果。
**参见**: [[Long-Running-Harness-Design|长运行应用的 Harness 设计]]
### Terminal Bench (终端基准测试) ### Agent-First World (智能体优先的世界)
**英文**Terminal Bench **英文**: Agent-First World
**中文**:终端基准测试 **中文**: 智能体优先的世界
**定义**:评估智能体编码能力的标准基准测试,包含机器学习、调试、生物学等多个领域的任务 **定义**: 一种软件工程范式,其中没有一行代码是人工编写的,人类工程师的工作重点转向设计环境、明确意图和构建反馈回路
**相关概念**[[LangChain Harness Engineering|LangChain Harness 工程]] **参见**: [[OpenAI-Codex-Harness-Engineering|OpenAI Codex Harness 工程]]
## W ### Progressive Disclosure (渐进式披露)
**英文**: Progressive Disclosure
**中文**: 渐进式披露
**定义**: 一种情境管理策略,智能体从一个小而稳定的切入点开始,并被指导下一步该去哪里查看,而不是一开始就被淹没。
**参见**: [[OpenAI-Codex-Harness-Engineering|OpenAI Codex Harness 工程]]
### Wikilink (Wikilink) ### Doc-Gardening (文档园艺)
**英文**Wikilink **英文**: Doc-Gardening
**中文**Wikilink **中文**: 文档园艺
**定义**:使用 `[[文档标题]]` 格式的内部链接,在 Obsidian 等工具中支持双向链接和图谱视图。 **定义**: 定期运行的智能体,扫描那些不再反映真实代码行为的过时或废弃文档,并发起修复用的 Pull Request。
**相关概念**[[Backlink|反向链接]], [[Glossary|术语表]]
### WikiLLM (WikiLLM) ### Golden Principles (黄金原则)
**英文**WikiLLM **英文**: Golden Principles
**中文**WikiLLM **中文**: 黄金原则
**定义**:本项目的名称,一个利用 LLM 构建个人知识库的系统,通过"编译"原始数据生成结构化、交叉链接的高质量中文 Wiki。 **定义**: 带有主观意见的机械规则,旨在保持代码库的可读性和一致性,以便将来运行智能体。
**相关概念**[[Glossary|术语表]], [[INDEX]]
--- ### AI Slop (AI 残渣)
**英文**: AI Slop
**中文**: AI 残渣
**定义**: 智能体复现代码仓库中已存在的不均衡或不够理想的模式,随着时间的推移导致的漂移。
*最后更新:2026-04-07* ### Meta-Harness (元 Harness)
*本文档由 [[WikiLLM]] 自动生成* **英文**: Meta-Harness
**中文**: 元 Harness
**定义**: 一个外环系统,用于搜索和优化 LLM 应用的 Harness 代码,使用编码智能体通过文件系统访问完整历史记录(源代码、执行轨迹、分数)来提议和评估新的 Harness。
**参见**: [[Meta-Harness|Meta-Harness:模型 Harness 的端到端优化]]
### Code-Space Search (代码空间搜索)
**英文**: Code-Space Search
**中文**: 代码空间搜索
**定义**: Meta-Harness 的关键设计选择,将 Harness 优化发生在代码空间中,通过检查执行轨迹推断为什么失败以及哪些早期设计选择导致了失败,而不仅仅是失败本身。
### Computational vs Inferential (计算型 vs 推理型)
**英文**: Computational vs Inferential
**中文**: 计算型 vs 推理型
**定义**: Harness 中指南和传感器的两种执行类型:计算型是确定性且快速的(测试、lint、类型检查);推理型是语义分析、AI 代码审查、"LLM 作为法官"。
**参见**: [[Harness-Engineering-for-Coding-Agent-Users|面向编码智能体用户的 Harness 工程]]
### Feedforward Guide (前馈指南)
**英文**: Feedforward Guide
**中文**: 前馈指南
**定义**: 在智能体行动之前提供的指导,包括原则、规则、参考文档、操作指南等,增加智能体第一次就做对的概率。
**参见**: [[Harness-Engineering-for-Coding-Agent-Users|面向编码智能体用户的 Harness 工程]]
### Feedback Sensor (反馈传感器)
**英文**: Feedback Sensor
**中文**: 反馈传感器
**定义**: 在智能体行动之后提供的验证机制,包括静态分析、日志、浏览器测试、代码审查智能体等,用于自我纠正问题。
**参见**: [[Harness-Engineering-for-Coding-Agent-Users|面向编码智能体用户的 Harness 工程]]
### Harness Template (Harness 模板)
**英文**: Harness Template
**中文**: Harness 模板
**定义**: 针对常见应用拓扑(数据仪表板、CRUD 业务服务、事件处理器)的指南和传感器捆绑包,可以作为团队的起点。
**参见**: [[Harness-Engineering-for-Coding-Agent-Users|面向编码智能体用户的 Harness 工程]]
### Maintainability Harness (可维护性 Harness)
**英文**: Maintainability Harness
**中文**: 可维护性 Harness
**定义**: 监管内部代码质量和可维护性的 Harness 类别,包括 lint、结构测试、代码覆盖等工具。
**参见**: [[Harness-Engineering-for-Coding-Agent-Users|面向编码智能体用户的 Harness 工程]]
### Architecture Fitness Harness (架构适用性 Harness)
**英文**: Architecture Fitness Harness
**中文**: 架构适用性 Harness
**定义**: 定义和检查应用程序架构特征的指南和传感器,类似于架构适用性函数。
**参见**: [[Harness-Engineering-for-Coding-Agent-Users|面向编码智能体用户的 Harness 工程]]
### Behaviour Harness (行为 Harness)
**英文**: Behaviour Harness
**中文**: 行为 Harness
**定义**: 引导和感知应用程序是否按需要功能运行的 Harness 类别,包括功能规范和测试套件。
**参见**: [[Harness-Engineering-for-Coding-Agent-Users|面向编码智能体用户的 Harness 工程]]
### Rippable Harness (可剥离 Harness)
**英文**: Rippable Harness
**中文**: 可剥离 Harness
**定义**: 设计为可以移除"智能"逻辑的 Harness,当模型变得足够智能不需要时。
**参见**: [[Harness-Engineering-Complete-Guide|Harness 工程完整指南]]
### Three Pillars of Harness Engineering (Harness 工程三大支柱)
**英文**: Three Pillars of Harness Engineering
**中文**: Harness 工程三大支柱
**定义**: OpenAI 框架组织的三个核心类别:上下文工程、架构约束、熵管理(垃圾收集)。
**参见**: [[Harness-Engineering-Complete-Guide|Harness 工程完整指南]]
### Context Engineering (上下文工程)
**英文**: Context Engineering
**中文**: 上下文工程
**定义**: 确保智能体在正确时间拥有正确信息的学科,包括存储库本地文档和动态可观测性数据。
**参见**: [[Harness-Engineering-Complete-Guide|Harness 工程完整指南]]
### Architectural Constraints (架构约束)
**英文**: Architectural Constraints
**中文**: 架构约束
**定义**: 机械地强制执行好代码样子的机制,包括依赖分层规则、确定性 linter、结构测试等。
**参见**: [[Harness-Engineering-Complete-Guide|Harness 工程完整指南]]
### Entropy Management (熵管理)
**英文**: Entropy Management
**中文**: 熵管理
**定义**: 定期清理智能体,用于解决 AI 生成代码库随时间积累的熵(文档漂移、命名约定发散、死代码积累)。
**参见**: [[Harness-Engineering-Complete-Guide|Harness 工程完整指南]]
### Reasoning Sandwich (推理三明治)
**英文**: Reasoning Sandwich
**中文**: 推理三明治
**定义**: 一种推理预算分配策略:规划使用高推理、实现使用中推理、验证使用高推理。
**参见**: [[LangChain-Harness-Engineering|LangChain Harness 工程实践]]
### Self-Verification Loop (自我验证循环)
**英文**: Self-Verification Loop
**中文**: 自我验证循环
**定义**: 让智能体验证其工作的机制:规划与发现、构建、验证、修复。
**参见**: [[LangChain-Harness-Engineering|LangChain Harness 工程实践]]
### Managed Agents (托管智能体)
**英文**: Managed Agents
**中文**: 托管智能体
**定义**: Anthropic 的托管服务,通过一组通用接口虚拟化智能体组件(会话、Harness、沙箱)来运行长 horizon 智能体。
**参见**: [[Managed-Agents-Decoupling-Brain-from-Hands|Managed Agents:将大脑与手分离]]
### Session (会话)
**英文**: Session
**中文**: 会话
**定义**: Managed Agents 中发生的一切的仅追加日志,作为生活在 Claude 上下文窗口之外的上下文对象。
**参见**: [[Managed-Agents-Decoupling-Brain-from-Hands|Managed Agents:将大脑与手分离]]
### Meta-Harness (元 Harness)
**英文**: Meta-Harness
**中文**: 元 Harness
**定义**: 一种系统,具有允许许多不同 Harness 的通用接口,而对 Claude 未来将需要的特定 Harness 没有意见。
**参见**: [[Managed-Agents-Decoupling-Brain-from-Hands|Managed Agents:将大脑与手分离]]
### Model Self-Evolution (模型自我进化)
**英文**: Model Self-Evolution
**中文**: 模型自我进化
**定义**: 模型深度参与迭代自己的过程,包括构建强化学习 Harness、更新记忆、驱动自身的强化学习。
**参见**: [[MiniMax-M27-Self-Evolution|MiniMax M2.7:开启模型的自我进化]]
### Agent Teams (智能体团队)
**英文**: Agent Teams
**中文**: 智能体团队
**定义**: 多智能体协作的原生能力,要求角色边界、对抗性推理、协议遵循、行为分化内化到模型中。
**参见**: [[MiniMax-M27-Self-Evolution|MiniMax M2.7:开启模型的自我进化]]
### Six Stages of AI Adoption (AI 采用的六个阶段)
**英文**: Six Stages of AI Adoption
**中文**: AI 采用的六个阶段
**定义**: Mitchell Hashimoto 的采用路径:放弃聊天机器人界面、重现自己的工作、日终智能体、外包确定的任务、工程化 Harness、始终有一个智能体在运行。
**参见**: [[Mitchellh-AI-Adoption-Journey|Mitchell Hashimoto 的 AI 采用之旅]]
+52 -64
View File
@@ -1,84 +1,72 @@
--- ---
title: WikiLLM 知识库首页 title: "WikiLLM 知识库索引"
tags: [首页, 索引, 导航] source: "Externalization in LLM Agents: A Unified Review"
last_updated: 2026-04-07 last_updated: 2026-04-11
--- ---
# WikiLLM 知识库 # WikiLLM 知识库索引
欢迎来到 **WikiLLM**——一个关于 [[Harness-Engineering|Harness 工程]] 的中文知识库。本 wiki 基于多篇权威来源编译而成,旨在为 AI 智能体时代的软件工程提供系统化的指南 欢迎来到 WikiLLM 知识库!本 wiki 基于论文《Externalization in LLM Agents: A Unified Review of Memory, Skills, Protocols and Harness Engineering》编译而成
> **Harness 工程**是设计和实现使 AI 智能体可靠工作的系统的新学科。如果说 2025 年是 AI 智能体验证它们能够编写代码的一年,那么 2026 年就是我们认识到**智能体不是难点——Harness 才是**的一年。 ## 快速导航
--- - [[Glossary|术语表]] - 核心概念定义与对照
- [[Externalization-in-LLM-Agents|LLM Agent 中的外部化]] - 核心理论框架
- [[Harness-Engineering|Harness 工程]] - 统一集成层
## 📚 核心概念 ## 核心概念
- [[Harness-Engineering|Harness 工程]] - Harness 工程的完整概述,包括三大支柱、为什么现在重要、以及实践中的方法 本知识库围绕 LLM Agent 的**外部化框架**组织,涵盖四大支柱:
- [[Context-Engineering|上下文工程]] - 如何确保智能体在正确的时间获得正确的信息
- [[Architectural-Constraints|架构约束]] - 如何机械地强制执行好代码的样子,而不是仅仅告诉智能体"写好代码"
- [[Anthropic-Harness-Design|Anthropic Harness 设计]] - Anthropic 的三智能体架构:Planner、Generator、Evaluator
- [[Self-Verification|自我验证]] - 让智能体通过构建-验证循环自我改进的技术
--- ### 1. 外部化理论
- [[Externalization-in-LLM-Agents|LLM Agent 中的外部化]] - 外部化作为组织原则
- [[From-Weights-to-Context-to-Harness|从权重到上下文到 Harness]] - 历史演进路径
## 🛠️ 实践指南 ### 2. 三大外部化维度
- [[Memory-Systems|记忆系统]] - 跨时间外部化状态
- [[Skill-Systems|技能系统]] - 外部化程序专长
- [[Agent-Protocols|智能体协议]] - 外部化交互结构
- [[Mitchellh-Adoption-Journey|Mitchellh AI 采用之旅]] - HashiCorp 创始人从怀疑论者到深度用户的六个阶段 ### 3. Harness 工程
- [[Building-Your-First-Harness|构建你的第一个 Harness]] - 从个人开发者到工程组织的三级实用框架 - [[Harness-Engineering|Harness 工程]] - 统一协调层
- [[Harness-Engineering-Complete-Guide|Harness 工程完整指南]] - NxCode 的完整 Harness 工程指南
- [[Harness-Engineering-for-Coding-Agent-Users|面向编码智能体用户的 Harness 工程]] - Martin Fowler 的指南与传感器框架
- [[Harness-Engineering-First-Thoughts|Harness 工程:最初的思考]] - Martin Fowler 团队的早期备忘录
- [[Meta-Harness|Meta-Harness:模型 Harness 的端到端优化]] - 斯坦福/MIT 的自动 Harness 优化研究
--- ### 4. 实践指南
- [[Long-Running-Harness-Design|长运行应用的 Harness 设计]] - Anthropic 团队的多 Agent 架构实践
- [[OpenAI-Codex-Harness-Engineering|OpenAI Codex Harness 工程]] - 完全由智能体生成代码的产品开发实践
- [[Mitchellh-AI-Adoption-Journey|Mitchell Hashimoto 的 AI 采用之旅]] - HashiCorp 创始人从怀疑论者到深度用户的六个阶段
- [[LangChain-Harness-Engineering|LangChain Harness 工程实践]] - 从 Top 30 到 Top 5 的 Harness 优化经验
- [[Managed-Agents-Decoupling-Brain-from-Hands|Managed Agents:将大脑与手分离]] - Anthropic 的托管智能体架构设计
- [[MiniMax-M27-Self-Evolution|MiniMax M2.7:开启模型的自我进化]] - 模型参与迭代自己的实践
## 📖 学习路径 ## 学习路径
### 初学者路径 ### 初学者路径
1. 从 [[Externalization-in-LLM-Agents|LLM Agent 中的外部化]] 开始,理解核心论点
2. 阅读 [[From-Weights-to-Context-to-Harness|从权重到上下文到 Harness]],了解历史背景
3. 深入三大外部化维度:[[Memory-Systems|记忆]]、[[Skill-Systems|技能]]、[[Agent-Protocols|协议]]
4. 最后学习 [[Harness-Engineering|Harness 工程]] 如何将它们统一
1. 首先阅读 [[Harness-Engineering|Harness 工程]] 获得概览 ### 架构师路径
2. 然后阅读 [[Mitchellh-Adoption-Journey|Mitchellh AI 采用之旅]] 了解个人采用路径 1. 直接阅读 [[Harness-Engineering|Harness 工程]] 了解六大分析维度
3. 最后阅读 [[Building-Your-First-Harness|构建你的第一个 Harness]] 开始实践 2. 参考 [[Externalization-in-LLM-Agents|外部化理论]] 作为理论基础
3. 根据需要深入各模块细节
### 深入学习路径 ## 最新研究
1. 从 [[Anthropic-Harness-Design|Anthropic Harness 设计]] 开始了解前沿架构 本知识库基于 2026 年 4 月发表的最新综述论文和实践报告,涵盖:
2. 深入研究 [[Context-Engineering|上下文工程]] 和 [[Architectural-Constraints|架构约束]] - 记忆架构的四代演进(单片上下文 → 检索存储 → 分层编排 → 自适应系统)
3. 学习 [[Self-Verification|自我验证]] 技术让智能体自我改进 - 技能系统从工具使用到能力包的演变
- 协议生态系统(MCP、A2A、ACP、ANP、A2UI 等)
- Harness 工程的六大分析维度
- 多 Agent 架构实践(Planner-Generator-Evaluator 三 Agent 系统)
--- ## 相关研究
## 🔗 快速导航 - 认知人工制品理论 (Norman, 1991)
- 分布式认知 (Hutchins, 1995)
- [[Glossary|术语表]] - 40+ 核心概念的中英对照和解释 - 互补策略 (Kirsh, 1995)
- [概念目录](./concepts/) - 所有核心概念文章 - CoALA 架构
- [实践目录](./practices/) - 所有实践指南文章
---
## 📊 编译来源
本知识库基于以下权威来源编译:
1. **OpenAI** - Harness Engineering:在智能体优先的世界中利用 Codex
2. **Anthropic** - Harness design for long-running application development
3. **Martin Fowler** - Harness engineering for coding agent users
4. **LangChain** - Improving Deep Agents with harness engineering
5. **NxCode** - Harness Engineering: The Complete Guide
6. **MiniMax** - MiniMax M2.7: Early Echoes of Self-Evolution
7. **Mitchell Hashimoto** - My AI Adoption Journey
---
## 💡 关于 WikiLLM
WikiLLM 是一个利用 LLM 构建个人知识库的系统。本项目的核心原则是:
- **LLM 编写和维护所有 wiki 数据**;手动编辑很少见
- **用户探索和查询被归档回 wiki** 以增强它
- **系统专注于 markdown 文件和 Obsidian 兼容格式**
- **图像被下载到本地** 以便 LLM 轻松引用
查看 [[Glossary|术语表]] 了解更多核心概念,或从 [[Harness-Engineering|Harness 工程]] 开始阅读!
---
*最后更新:2026-04-07*
*本文档由 [[WikiLLM]] 自动生成*
Binary file not shown.

After

Width:  |  Height:  |  Size: 19 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 4.6 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 86 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 12 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 115 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 282 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 199 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 284 KiB

After

Width:  |  Height:  |  Size: 295 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 17 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 177 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 5.2 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 378 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 339 KiB

Before

Width:  |  Height:  |  Size: 7.1 MiB

After

Width:  |  Height:  |  Size: 7.1 MiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 136 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 11 KiB

After

Width:  |  Height:  |  Size: 101 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 11 KiB

After

Width:  |  Height:  |  Size: 234 KiB

@@ -1,280 +0,0 @@
<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd">
<html>
<head>
<meta content = 'uft-8' name = 'charset'></meta>
<title>not found</title>
<meta http-equiv="Content-type" content="text/html;charset=UTF-8" /><script defer src="https://cloud.umami.is/script.js" data-website-id="eb12527f-b713-4afa-905a-8a50f8a7f157"></script>
<link href = '/global.css' rel = 'stylesheet' type = 'text/css'></link>
</head>
<body><header id = 'banner' style = 'background-image: url("/banner.png"); background-repeat: no-repeat'>
<div class = 'name-logo'><a href = 'https://martinfowler.com'><img src = '/mf-name-white.png'></img></a></div>
<div class = 'search'>
<!-- SiteSearch Google -->
<form method='GET' action="https://www.google.com/search">
<input type='hidden' name='ie' value='UTF-8'/>
<input type='hidden' name='oe' value='UTF-8'/>
<input class = 'field' type='text'
name='q' size='15' maxlength='255' value=""/>
<button class = 'button' type='submit'
name='btnG' value=" " title = "Search"/>
<input type='hidden' name='domains' value="martinfowler.com"/>
<input type='hidden' name='sitesearch' value=""/>
<input
type='hidden' name='sitesearch' value="martinfowler.com"/>
</form>
</div>
<div class = 'menu-button navmenu-button'><a class = 'icon icon-bars' href = '#navmenu-bottom'></a></div>
<nav class = 'top-menu'>
<ul>
<li><a class = '' href = 'https://refactoring.com'>Refactoring</a></li>
<li><a class = '' href = '/agile.html'>Agile</a></li>
<li><a class = '' href = '/architecture'>Architecture</a></li>
<li><a class = '' href = '/aboutMe.html'>About</a></li>
<li><a class = 'tw' href = 'https://www.thoughtworks.com/engineering'>Thoughtworks</a></li>
<li><a class = 'icon icon-rss' href = '/feed.atom' title = 'feed'></a></li>
<li><a class = 'icon icon-twitter' href = 'https://www.twitter.com/martinfowler' title = 'Twitter stream'></a></li>
<li class = 'icon'><a href = 'https://toot.thoughtworks.com/@mfowler' title = 'Mastodon stream'><svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 24 24" fill="currentColor"><path d="M21.2595 13.9898C20.9852 15.4006 18.8033 16.9446 16.2974 17.2439C14.9907 17.3998 13.7041 17.5431 12.3321 17.4802C10.0885 17.3774 8.31809 16.9446 8.31809 16.9446C8.31809 17.163 8.33156 17.371 8.3585 17.5655C8.65019 19.7797 10.5541 19.9124 12.3576 19.9742C14.1779 20.0365 15.7987 19.5254 15.7987 19.5254L15.8735 21.1711C15.8735 21.1711 14.6003 21.8548 12.3321 21.9805C11.0814 22.0493 9.52849 21.9491 7.71973 21.4703C3.79684 20.432 3.12219 16.2504 3.01896 12.0074C2.98749 10.7477 3.00689 9.55981 3.00689 8.56632C3.00689 4.22771 5.84955 2.95599 5.84955 2.95599C7.2829 2.29772 9.74238 2.0209 12.2993 2H12.3621C14.919 2.0209 17.3801 2.29772 18.8133 2.95599C18.8133 2.95599 21.6559 4.22771 21.6559 8.56632C21.6559 8.56632 21.6916 11.7674 21.2595 13.9898ZM18.3029 8.9029C18.3029 7.82924 18.0295 6.97604 17.4805 6.34482C16.9142 5.71359 16.1726 5.39001 15.2522 5.39001C14.187 5.39001 13.3805 5.79937 12.8473 6.61819L12.3288 7.48723L11.8104 6.61819C11.2771 5.79937 10.4706 5.39001 9.40554 5.39001C8.485 5.39001 7.74344 5.71359 7.17719 6.34482C6.62807 6.97604 6.3547 7.82924 6.3547 8.9029V14.1562H8.43597V9.05731C8.43597 7.98246 8.88822 7.4369 9.79281 7.4369C10.793 7.4369 11.2944 8.08408 11.2944 9.36376V12.1547H13.3634V9.36376C13.3634 8.08408 13.8646 7.4369 14.8648 7.4369C15.7694 7.4369 16.2216 7.98246 16.2216 9.05731V14.1562H18.3029V8.9029Z"></path></svg>
</a></li>
<li class = 'icon'><a href = 'https://www.linkedin.com/in/martin-fowler-com/' title = 'LinkedIn'><svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 24 24" fill="currentColor"><path d="M4.00098 3H20.001C20.5533 3 21.001 3.44772 21.001 4V20C21.001 20.5523 20.5533 21 20.001 21H4.00098C3.44869 21 3.00098 20.5523 3.00098 20V4C3.00098 3.44772 3.44869 3 4.00098 3ZM5.00098 5V19H19.001V5H5.00098ZM7.50098 9C6.67255 9 6.00098 8.32843 6.00098 7.5C6.00098 6.67157 6.67255 6 7.50098 6C8.3294 6 9.00098 6.67157 9.00098 7.5C9.00098 8.32843 8.3294 9 7.50098 9ZM6.50098 10H8.50098V17.5H6.50098V10ZM12.001 10.4295C12.5854 9.86534 13.2665 9.5 14.001 9.5C16.072 9.5 17.501 11.1789 17.501 13.25V17.5H15.501V13.25C15.501 12.2835 14.7175 11.5 13.751 11.5C12.7845 11.5 12.001 12.2835 12.001 13.25V17.5H10.001V10H12.001V10.4295Z"></path></svg>
</a></li>
<li class = 'icon'><a href = 'https://bsky.app/profile/martinfowler.com' title = 'BlueSky'><svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 24 24" fill="currentColor"><path d="M12 11.3884C11.0942 9.62673 8.62833 6.34423 6.335 4.7259C4.13833 3.17506 3.30083 3.4434 2.75167 3.69256C2.11583 3.9784 2 4.95506 2 5.52839C2 6.10339 2.315 10.2367 2.52 10.9276C3.19917 13.2076 5.61417 13.9776 7.83917 13.7309C4.57917 14.2142 1.68333 15.4017 5.48083 19.6292C9.65833 23.9542 11.2058 18.7017 12 16.0392C12.7942 18.7017 13.7083 23.7651 18.4442 19.6292C22 16.0392 19.4208 14.2142 16.1608 13.7309C18.3858 13.9784 20.8008 13.2076 21.48 10.9276C21.685 10.2376 22 6.10256 22 5.52923C22 4.95423 21.8842 3.97839 21.2483 3.6909C20.6992 3.44256 19.8617 3.17423 17.665 4.72423C15.3717 6.34506 12.9058 9.62756 12 11.3884Z"></path></svg></a></li>
</ul>
</nav>
</header>
<nav id = 'top-navmenu'>
<nav class = 'navmenu'>
<div class = 'nav-head'> <div class = 'search'>
<!-- SiteSearch Google -->
<form method='GET' action="https://www.google.com/search">
<input type='hidden' name='ie' value='UTF-8'/>
<input type='hidden' name='oe' value='UTF-8'/>
<input class = 'field' type='text'
name='q' size='15' maxlength='255' value=""/>
<button class = 'button' type='submit'
name='btnG' value=" " title = "Search"/>
<input type='hidden' name='domains' value="martinfowler.com"/>
<input type='hidden' name='sitesearch' value=""/>
<input
type='hidden' name='sitesearch' value="martinfowler.com"/>
</form>
</div>
<div class = 'closediv'>
<span class = 'close' title = 'close'></span>
</div>
</div>
<div class = 'nav-body'>
<div class = 'topics'>
<h2>Topics</h2>
<p><a href = '/architecture'>Architecture</a></p>
<p><a href = 'https://refactoring.com'>Refactoring</a></p>
<p><a href = '/agile.html'>Agile</a></p>
<p><a href = '/delivery.html'>Delivery</a></p>
<p><a href = '/microservices'>Microservices</a></p>
<p><a href = '/data'>Data</a></p>
<p><a href = '/testing'>Testing</a></p>
<p><a href = '/dsl.html'>DSL</a></p>
</div>
<div class = 'about'>
<h2>about me</h2>
<p><a href = '/aboutMe.html'>About</a></p>
<p><a href = '/books'>Books</a></p>
<p><a href = '/faq.html'>FAQ</a></p>
</div>
<div class = 'content'>
<h2>content</h2>
<p><a href = '/videos.html'>Videos</a></p>
<p><a href = '/tags'>Content Index</a></p>
<p><a href = '/boardgames'>Board Games</a></p>
<p><a href = '/photos'>Photography</a></p>
</div>
<div class = 'tw'>
<h2>Thoughtworks</h2>
<p><a href = 'https://thoughtworks.com'>Home</a></p>
<p><a href = 'https://thoughtworks.com/insights'>Insights</a></p>
<p><a href = 'https://thoughtworks.com/careers'>Careers</a></p>
<p><a href = 'https://thoughtworks.com/radar'>Radar</a></p>
<p><a href = 'https://www.thoughtworks.com/engineering'>Engineering</a></p>
</div>
<div class = 'feeds'>
<h2>follow</h2>
<p><a href = '/feed.atom'>RSS</a></p>
<p><a href = 'https://toot.thoughtworks.com/@mfowler'>Mastodon</a></p>
<p><a href = 'https://www.linkedin.com/in/martin-fowler-com/'>LinkedIn</a></p>
<p><a href = 'https://bsky.app/profile/martinfowler.com'>Bluesky</a></p>
<p><a href = 'https://www.twitter.com/martinfowler'>X</a></p>
<p><a href = 'https://boardgamegeek.com/blog/13064/martins-7th-decade'>BGG</a></p>
</div>
</div>
</nav>
</nav>
<main><h1 id="section">404</h1>
<p>Im afraid this is not the document youre looking for. Try using the
search box above, and good luck.</p>
</main>
<nav id = 'bottom-navmenu'>
<nav class = 'navmenu'>
<div class = 'nav-head'> <div class = 'search'>
<!-- SiteSearch Google -->
<form method='GET' action="https://www.google.com/search">
<input type='hidden' name='ie' value='UTF-8'/>
<input type='hidden' name='oe' value='UTF-8'/>
<input class = 'field' type='text'
name='q' size='15' maxlength='255' value=""/>
<button class = 'button' type='submit'
name='btnG' value=" " title = "Search"/>
<input type='hidden' name='domains' value="martinfowler.com"/>
<input type='hidden' name='sitesearch' value=""/>
<input
type='hidden' name='sitesearch' value="martinfowler.com"/>
</form>
</div>
<div class = 'closediv'>
<span class = 'close' title = 'close'></span>
</div>
</div>
<div class = 'nav-body'>
<div class = 'topics'>
<h2>Topics</h2>
<p><a href = '/architecture'>Architecture</a></p>
<p><a href = 'https://refactoring.com'>Refactoring</a></p>
<p><a href = '/agile.html'>Agile</a></p>
<p><a href = '/delivery.html'>Delivery</a></p>
<p><a href = '/microservices'>Microservices</a></p>
<p><a href = '/data'>Data</a></p>
<p><a href = '/testing'>Testing</a></p>
<p><a href = '/dsl.html'>DSL</a></p>
</div>
<div class = 'about'>
<h2>about me</h2>
<p><a href = '/aboutMe.html'>About</a></p>
<p><a href = '/books'>Books</a></p>
<p><a href = '/faq.html'>FAQ</a></p>
</div>
<div class = 'content'>
<h2>content</h2>
<p><a href = '/videos.html'>Videos</a></p>
<p><a href = '/tags'>Content Index</a></p>
<p><a href = '/boardgames'>Board Games</a></p>
<p><a href = '/photos'>Photography</a></p>
</div>
<div class = 'tw'>
<h2>Thoughtworks</h2>
<p><a href = 'https://thoughtworks.com'>Home</a></p>
<p><a href = 'https://thoughtworks.com/insights'>Insights</a></p>
<p><a href = 'https://thoughtworks.com/careers'>Careers</a></p>
<p><a href = 'https://thoughtworks.com/radar'>Radar</a></p>
<p><a href = 'https://www.thoughtworks.com/engineering'>Engineering</a></p>
</div>
<div class = 'feeds'>
<h2>follow</h2>
<p><a href = '/feed.atom'>RSS</a></p>
<p><a href = 'https://toot.thoughtworks.com/@mfowler'>Mastodon</a></p>
<p><a href = 'https://www.linkedin.com/in/martin-fowler-com/'>LinkedIn</a></p>
<p><a href = 'https://bsky.app/profile/martinfowler.com'>Bluesky</a></p>
<p><a href = 'https://www.twitter.com/martinfowler'>X</a></p>
<p><a href = 'https://boardgamegeek.com/blog/13064/martins-7th-decade'>BGG</a></p>
</div>
</div>
</nav>
</nav>
<footer id='page-footer'>
<div class='tw-logo'>
<a href='https://www.thoughtworks.com/engineering'>
<img src='/thoughtworks_white.png'>
</a>
</div>
<div class='menu-button'>
<div class='icon-bars navmenu-button'></div>
</div>
<div class='copyright'>
<p>© Martin Fowler | <a href="/aboutMe.html#disclosures">Disclosures</a></p>
</div>
</footer>
<script src = '/jquery-1.11.3.min.js' type = 'text/javascript'></script>
<script src = '/mfcom.js' type = 'text/javascript'></script>
</body>
</html>
Binary file not shown.

Before

Width:  |  Height:  |  Size: 11 KiB

After

Width:  |  Height:  |  Size: 200 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 11 KiB

After

Width:  |  Height:  |  Size: 175 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 11 KiB

After

Width:  |  Height:  |  Size: 198 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 305 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 286 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 1.6 MiB

File diff suppressed because one or more lines are too long

Before

Width:  |  Height:  |  Size: 139 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 395 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 1.8 MiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 216 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 85 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 34 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 966 KiB

Some files were not shown because too many files have changed in this diff Show More