The debate over whether fully AI-automated software construction should exist has finally erupted. The trigger was
@yetone maintaining magpie, which peers criticized as "fast-food service" and "too risky to use."
Most people agree that AI-automated software construction is the trend. So how does magpie practice this? I pulled down magpie, analyzed the 2,862 commits from the past ten days, and used OpenClaw for comparison.
https://x.com/wquguru/status/2109458639501271530
1204 Versions in 19 Days
magpie is a local LLM gateway that connects coding agents like Claude Code, Codex, and Gemini CLI to any model or subscription. From the first commit on September 22 to today, it has 9,474 stars in 19 days. Of the 2,862 commits, yetone's account accounts for 85%, merging 383 community PRs.
The number of tags (equivalent to versions) reached 1,204 in 19 days, averaging over 60 versions per day—a frequency impossible for humans to maintain manually. In contrast, OpenClaw had 1,423 commits in its first 19 days, 99% submitted by Peter Steinberger himself, with about 10 tags.
magpie has twice as many commits but over a hundred times more versions.
What I wanted to figure out was one thing: How exactly does AI close the loop in this project?
A Routing Table and a Mistake Notebook
magpie's CLAUDE.md only references two files: AGENTS.md and LESSONS.md.
AGENTS.md is a routing table. It doesn't write rules itself; it just tells agents which documents to consult before doing work: check docs/subsystems/ before modifying subsystems, follow docs/code-standards.md for writing/reviewing code, and look at docs/contributors.md for reviewing community PRs.
The first sentence of LESSONS.md answers the initial question:
magpie's code is written, reviewed, merged and released by agents. (magpie's code is written, reviewed, merged, and released by agents)
Writing, reviewing, merging, releasing—all done by agents. Without human oversight, why doesn't it crash? The answer lies in this file. At 479 lines, it's essentially a mistake notebook, written by the agent every night.
Every night, the agent reviews all commits from the day, focusing on three scenarios:
- Which fixes were reworked?
- Which shouldn't have been merged at all?
- Which verifications were too sloppy?
For each issue found, it records a rule, a reason, and attaches commit and issue numbers as evidence. For example:
Reproduce the reporter's exact case, on the surface they use, before you call it fixed.
834 came back 4 times (v0.1.920 → .925 → .931 → .938). Each round fixed only the agent in the screenshot. Seen 1× (2026-10-05).
Issue 834 was reworked four times, each time fixing only the agent shown in the screenshot. So the rule is: before calling it fixed, reproduce the problem exactly on the interface the reporter uses.
The key is the last line: Seen 1×. If the same lesson appears in reviews on three different days, it moves from LESSONS.md to code-standards.md, becoming a hard rule from experience. Every subsequent PR review must check against these rules item by item.
Eight rules have already been promoted this way. Examples include fixing similar bugs together (caught once each on Oct 5, 6, and 7); tests failing on main are immediate bugs to fix; and clearly stating what was tested and on which commit before merging.
Of course, the mistake notebook also records things done right, marked as Done right, so agents read not just "don't do this," but also "do this instead."
The Mistake Notebook Was Introduced Early Morning on October 6
Was this system designed from the start? Checking the commit history of LESSONS.md, the first version was commit 303819c5, created at 3:10 AM on October 6.
One day prior, on October 5, magpie released 168 versions.
The first version of LESSONS.md recorded that day. With 220 commits and about 170 releases, roughly 40 were fixing versions released that same day or the previous day. CI was almost always canceled by the next push. A test that failed only on Linux stayed red from v0.1.877 to v0.1.892 without anyone noticing.
294 tags in 51 hours left no run to finish. (294 tags in 51 hours meant no CI run could complete)
So the first version added a rule: Before tagging, the CI for that commit must finish and be green. Canceled runs due to new pushes don't count.
168 releases on day one, 42 on day two, 15 on day three, then stabilizing between 15 and 26.
No meetings were held, no notifications sent. Nightly reviews found issues, added a rule to the mistake notebook, and the working agents read it, slowing down release frequency.
Thus, the twenty-something releases per day recently cited as problematic were actually the speed after hitting the brakes.
PR Acceptance Criteria
Next, let's look at real PRs:
code-standards.md lists six acceptance criteria. All must pass before merging or releasing. Standard checks like three-platform compilation and sandbox testing aside, three are rare in typical projects:
- Regression Test: Put old code back; new tests must fail explicitly, pointing out which behavior is wrong. Compilation errors or skipped tests don't count.
- Real Usage: If changing a vendor integration, verify with real accounts, configs, and API responses. Features not tested cannot be built upon or claimed usable to users.
- Unrun Checks Must Be Listed:
A check that wasn't run is listed as not run, never left out. (Checks not performed must be listed as not run, never omitted)
I randomly picked #1339 to see the comment left before merging:
Red check means regression test, Break check means intentionally breaking code to see if tests catch it, and the last line says Not run—meaning it wasn't tested on real hardware.
The merge command includes --match-head-commit <reviewed sha>. If someone pushes during review, the merge fails, ensuring unreviewed code doesn't enter.
Human in the Loop
Of course, humans still participate in some work, at least these areas:
- Product decisions: "Don't reverse deliberate decisions." Things like scrolling behavior or button cursors cannot be changed by PRs.
- Security Gates: If risk checks trigger, wait for maintainer approval.
- Community Suggestions: 474 commit titles mention reporters from X or Discord, e.g., JonahFu on X, Jeremy.Zhou on Discord.
In short, humans manage direction and guard gates; writing, reviewing, merging, and releasing are handled by agents.
Is yetone Hiding Something?
At this point, some might ask: Is yetone hiding something? I was curious too and dug deeper.
Indeed, several things aren't in the repo:
- How nightly reviews are triggered isn't documented. None of the 4 workflows are scheduled.
- I extracted the times for 7 LESSONS.md updates: 03:10, 03:49, 00:23, 23:46, 03:48, 14:24, 05:26. They're scattered, looking less like cron jobs and more like manual local runs or custom scripts.
- The prompts used for nightly reviews aren't there either. Which agent, how many parallel sessions—none documented.
Along the way, I found two additional points:
First, the method involves community contributions. Subsystem documentation and semantic change reviews were introduced via community member Junjie Zhou's PR #889. The two tables at the start of code-standards.md are explicitly credited in commit 926bcaa3 to six review rules proposed by Junjie Zhou on Discord.
Second, yetone's interactions with agents are preserved in commits. 52 commit titles contain yetone's original quotes in parentheses:
1(the owner: This is too ugly)2(the owner: Why is this a native selector instead of our abstract ComboBox?)3(the owner: Why did this come back again? Didn't we say it should only appear on hover? Always appearing is too ugly)4(owner: During pr-preview, sometimes there's no data to test. Can you help create some fake data?)
These are mostly one-line complaints; the rest is filled in by the agent, often resulting in hundreds of words in the commit body. The request for fake data later became the mock data set used for CI recordings.
Humans point out what's wrong; agents turn it into verifiable changes. These dozens of commit titles explain the methodology better than any document.
If You Want to Build Your Own
Anyone can write a routing table. The real value to copy is the mistake notebook system. A LESSONS entry looks like this:
1**<One-sentence rule>**2- <Which commit/issue caused the problem, what was the consequence>3- <Fix commit>4- Done right: <Example of doing it correctly>5- Seen N× (YYYY-MM-DD, YYYY-MM-DD).
Have an agent review daily commits every night, looking only for rework, bad merges, and weak verification. If the same issue appears on three different days, move it to code standards. During acceptance, list regression tests, real usage, and unrun items in commit bodies and PR comments.
After reviewing the entire repo, what impressed me most was the Seen 3× threshold. Mistakes are written down, agents read them on startup, and repeating the same error three times promotes it to a hard rule, perfectly embodying the adage "third time's the charm" (or rather, third strike).
Final Thoughts...
A year ago, or even half a year ago, model capabilities rarely reached production-level automation. But now, especially since Opus 5.5 emerged, we can see—the future is here.
This reminds me of @yuanhao's Yoyo, an Agent that has self-iterated for over 200 days: https://yoyo-gasp.yolog.dev/, also worth checking out.
