Orchestrators, workers, and Orca, oh my!
My process for working with agents has been pretty ad hoc, and it has worked better than it has any right to. I dump a wall of text, an agent turns it into a plan, and then I open terminals and throw work at them until something breaks. The repo carries the memory (plans, decision logs, claim boards, and tests), and I’m the scheduler. That got Riglab, pvc-builder, and Authorbot built. I wrote the longer version up in How I work with AI tools and Three coding agents, one workspace.
Liberational is the first project where that stopped being enough.
What Liberational is
It’s Luke’s project, and his manifesto opens with “AI should give you superpowers.” The pitch is a backpack that does that. In practice it’s a local-first ambient assistant: cameras and microphones record what happens around you, it learns the people and things you name, and you can ask it questions and get answers that link back to the actual footage. It runs on Linux and Apple Silicon. The wearable hardware is the original design, but fixed cameras and imported video go through the same system.
Under the hood there’s a Rust data layer (Continuum) that captures and retains audio and video, a fleet of Python workers that transcribe speech, identify speakers and faces, describe scenes, track people and animals, and write a searchable journal, and a React dashboard on top. There’s also a Twilio bridge, so it can take phone calls. The roadmap has three goals: identify people after you teach it who they are, answer questions about what happened, and summarize events. All three are met now. The open one is spatial memory (a point cloud and a floorplan of every indoor place it has seen), which is why I spent part of Saturday looking at Gaussian splatting.
It started on October 11 as a dump of Luke’s openclaw setup, a pile of flat Python scripts that assumed his machine, his paths, and his accounts. My first request on October 24 was to genericize it so it would run on mine. The plan is to develop it in the open once we scrub the secrets that came along in that first dump. A full writeup of Liberational itself is coming soon; this post is about how the work got done.
Since October 23 the repo has picked up about 6,600 commits under my name, close to 900 a day at the peak (October 28 to 30), and 1,990 merged PRs. I didn’t type those, and I definitely didn’t schedule them by hand.
The repo went from about 11,000 lines to 513,000 in four weeks, which is the kind of number that should worry you. A codebase that only grows isn’t healthy, so the deletions matter as much as the additions. About 274,000 lines came out along the way, roughly one for every three that went in. Most of that was two days: October 30, when Continuum replaced the old audio, face, and detection paths, and November 3, when whole subsystems that weren’t earning their keep got thrown out (a rules engine, media aging, progress streams, and a handful of older trackers). November 3 is the only day the repo got smaller overall, by about 45,000 lines.
- 1Oct 23 I join. The first job is getting it to run on a machine that isn't Luke's.
- 2Oct 28 Orchestrator and worker roles get written down. First 100K-line day.
- 3Oct 29 Continuum lands: Rust goes from zero to 60K lines in one afternoon.
- ▼4Oct 30 Python shrinks by 6.7K as the old audio, face, and detection paths move onto Continuum (50K lines deleted that day). The first roadmap goal, identifying people, closes.
- ▼5Nov 3 The cleanup: 110K lines deleted, 45K net. Out go the rules engine, media aging and retention evidence, progress streams, and the older track, activity, crossing, and animal trackers. That night the other roadmap goals close, Continuum included.
- 6Nov 8 Orca replaces the hand-built orchestration.
The daily numbers as a table
| Day | Python | Rust | Frontend | Docs | Other | Total | Tests | Added | Deleted |
|---|---|---|---|---|---|---|---|---|---|
| Oct 11 | 7,457 | 0 | 376 | 2,599 | 507 | 10,939 | 205 | 10,962 | 23 |
| Oct 12 | 8,077 | 0 | 416 | 2,933 | 507 | 11,933 | 205 | 1,030 | 36 |
| Oct 13 | 8,077 | 0 | 416 | 3,249 | 507 | 12,249 | 205 | 316 | 0 |
| Oct 14 | 8,077 | 0 | 416 | 3,367 | 507 | 12,367 | 205 | 118 | 0 |
| Oct 15 | 8,077 | 0 | 416 | 3,460 | 507 | 12,460 | 205 | 93 | 0 |
| Oct 16 | 8,077 | 0 | 416 | 3,487 | 507 | 12,487 | 205 | 27 | 0 |
| Oct 17 | 8,077 | 0 | 416 | 3,513 | 507 | 12,513 | 205 | 26 | 0 |
| Oct 18 | 8,077 | 0 | 416 | 3,542 | 507 | 12,542 | 205 | 29 | 0 |
| Oct 19 | 8,077 | 0 | 416 | 3,575 | 507 | 12,575 | 205 | 33 | 0 |
| Oct 20 | 9,714 | 0 | 416 | 4,905 | 507 | 15,542 | 769 | 3,032 | 65 |
| Oct 21 | 9,927 | 0 | 416 | 6,484 | 545 | 17,372 | 783 | 1,914 | 84 |
| Oct 22 | 11,968 | 0 | 407 | 10,390 | 545 | 23,310 | 1,026 | 6,294 | 356 |
| Oct 23 | 13,477 | 0 | 852 | 10,435 | 545 | 25,309 | 1,397 | 2,089 | 90 |
| Oct 24 | 28,092 | 0 | 6,593 | 14,216 | 623 | 49,524 | 6,584 | 34,647 | 11,464 |
| Oct 25 | 28,092 | 0 | 6,636 | 15,919 | 623 | 51,270 | 6,584 | 1,782 | 36 |
| Oct 26 | 33,522 | 0 | 7,220 | 16,626 | 695 | 58,063 | 8,623 | 7,394 | 601 |
| Oct 27 | 52,917 | 0 | 7,408 | 17,188 | 844 | 78,357 | 17,533 | 23,553 | 3,259 |
| Oct 28 | 84,999 | 0 | 36,312 | 25,639 | 1,220 | 148,170 | 47,968 | 102,872 | 33,196 |
| Oct 29 | 112,669 | 59,686 | 45,598 | 31,877 | 4,823 | 254,653 | 81,671 | 115,656 | 9,338 |
| Oct 30 | 105,950 | 82,277 | 54,974 | 33,684 | 5,829 | 282,714 | 90,668 | 78,037 | 50,055 |
| Oct 31 | 137,670 | 94,776 | 63,474 | 39,118 | 5,984 | 341,022 | 115,638 | 62,165 | 3,857 |
| Nov 1 | 162,766 | 115,447 | 71,717 | 41,017 | 7,081 | 398,028 | 134,039 | 61,493 | 4,448 |
| Nov 2 | 191,067 | 128,590 | 74,071 | 43,183 | 7,372 | 444,283 | 153,600 | 50,979 | 4,724 |
| Nov 3 | 162,324 | 110,042 | 76,806 | 42,335 | 7,230 | 398,737 | 133,086 | 64,624 | 110,170 |
| Nov 4 | 178,300 | 112,738 | 91,211 | 43,397 | 7,319 | 432,965 | 149,848 | 44,114 | 9,879 |
| Nov 5 | 179,732 | 114,041 | 95,366 | 50,810 | 7,325 | 447,274 | 152,827 | 19,736 | 5,427 |
| Nov 6 | 185,512 | 114,364 | 106,488 | 48,338 | 7,355 | 462,057 | 159,919 | 32,367 | 17,585 |
| Nov 7 | 197,572 | 114,364 | 122,991 | 48,874 | 7,407 | 491,208 | 166,110 | 32,610 | 3,459 |
| Nov 8 | 214,621 | 115,704 | 123,638 | 51,404 | 7,687 | 513,054 | 174,148 | 27,810 | 5,964 |
Tests that can fail
With that much code landing that fast, the tests matter more than anything else in the repo, and agents are bad at writing them by default. Left alone they write tests that assert a mock was called, compute the expected value with the code under test, or pin a constant so the test only breaks when someone changes it on purpose.
So on October 28 I asked Claude to find me a good agent skill for idiomatic Python tests. It read the actual skill files for about eight candidates instead of trusting the listings, and none of the Python ones was great on its own. Most were pytest tutorials (one was about 500 lines of basics plus “aim for 80% coverage”). The best ideas came from a language-agnostic one: the testing guide in obra/superpowers, which makes every test name the specific bug it would catch, insists on expected values written out by hand, and ends with a check where you imagine breaking the code in realistic ways and confirm something fails. Claude merged that with the Python idioms from the better pytest skills (one behavior per test, mock only at the boundary, tmp_path, monkeypatch, and pytest.raises with match=) into a 122-line python-tests skill, and handed property-based testing off to the Trail of Bits skill for Hypothesis.
Both skills went into the repo’s .claude/skills/, and AGENTS.md points every agent at them (Codex and the others read the file before writing tests). That evening a worker reviewed the existing suite against the new skill: 95 files and about 20,000 lines, written before the skill existed. The brief was to delete tests that protect nothing, collapse near-duplicates into parametrized tests, and fold piles of hand-picked examples into Hypothesis properties where that’s shorter. It landed as 16 PRs the same day, taking out about 15,200 lines and putting back about 13,000. Tests are now about 174,000 of the 462,000 lines of code.
The rest of the testing rules in AGENTS.md came from things breaking:
- Run only the tests you need. The full suite is off limits unless a person asks for it, because many sessions running it over and over swamped the machine.
mise run test-changedfinds the test files for whatever changed against main. - Share the machine. Stacked test runs pushed the load average past 50, so every test run goes through
tools/test-slot.sh, which runs it at the lowest priority and allows two at once across all worktrees and agents. Rust builds take a separate lock, one Cargo command at a time. - Never touch the real thing. Tests can’t read or write the real
data/directory, andtests/conftest.pyfails any test that tries to runpack-services,systemctl, orlaunchctl. - No CI, so the merge is the gate. An agent merges after building, linting, and running the full tests for each package it touched, then notes in a PR comment exactly what it ran.
The bill, at API prices
API-equivalent cost
Tokens
The daily numbers as a table
| Day | Claude | Codex (est.) | Total |
|---|---|---|---|
| Oct 23 | $365 | $0 | $365 |
| Oct 24 | $19 | $0 | $19 |
| Oct 25 | $3 | $0 | $3 |
| Oct 26 | $0 | $0 | $0 |
| Oct 27 | $340 | $0 | $340 |
| Oct 28 | $1,540 | $141 | $1,681 |
| Oct 29 | $1,908 | $0 | $1,908 |
| Oct 30 | $1,023 | $1,130 | $2,153 |
| Oct 31 | $246 | $68 | $314 |
| Nov 1 | $401 | $663 | $1,064 |
| Nov 2 | $544 | $80 | $624 |
| Nov 3 | $1,208 | $15 | $1,223 |
| Nov 4 | $508 | $434 | $942 |
| Nov 5 | $392 | $0 | $392 |
| Nov 6 | $905 | $0 | $905 |
| Nov 7 | $556 | $0 | $556 |
| Nov 8 | $554 | $0 | $554 |
At list API rates, the model calls from this machine alone would have cost about $13,000 over 17 days: $10,511 of Claude and roughly $2,530 of Codex. That’s about 45.7 billion tokens, nearly all of it cached context being reread. For comparison, the whole Authorbot build came to $3,357 and 4.3 billion tokens, so this is about four times the money and ten times the tokens. The money ratio is smaller because Opus 5.5 is a lot cheaper than the Fable 5 that did most of Authorbot.
Usage limits shaped the work more than taste did. Some representative messages from me to the orchestrator:
Ah crap I hit the usage limit on the three codex workers. We need to switch those to claude workers
Starting in an hour when your usage resets run a qa pass on any PRs the grok agents make. They aren’t as smart.
The Codex workers were GPT-6.1 Sol, plus a few GPT-6 Astra workers I pointed at review, performance, and docs cleanup. Grok workers got the simpler jobs.
The coordinator phase
The first few days ran the usual way: a terminal per task, with one session ending up as the coordinator (the one I talked to, which sent the others their assignments through Claude Code’s SendMessage). It worked, mostly. Workers were known by session name, a restart meant figuring out which terminal was in charge now, and nothing stopped two sessions from both deciding they were.
On October 28 I wrote the roles down. Any session started in the repo checks for a running orchestrator and asks me whether to take the job or join as a worker. The orchestrator registers itself in a file in the git common directory and keeps a roster (workers.md) of who is doing what. Every dispatch has to stand on its own, because the worker has none of the orchestrator’s context: the issue, the goal, what’s in and out of scope, decisions already made, and what it may do without asking. Workers pick friendly names, so I hear “Wren is on the card API” instead of a UUID. A small Claude Code mod read the roster and put each session’s current task above its prompt, which made a wall of terminals readable.
That night I made the change I’d keep no matter what tool I end up on:
I think you (orchestrator) should do all of your orchestration in a subagent (or background agent or whatever, I can’t keep all the modalities straight), keeping this main thread just for back and forth conversation with me
The main thread became a receptionist. Worker reports and Slack events get forwarded to a background coordinator with a FWD: prefix. The coordinator dispatches, checks each worker’s claim against the actual PR and test results, and ends every turn with a short report. When it needs a decision it writes the question with two to four options, and the main thread asks me. My chat stopped scrolling past dispatch chatter, and I could talk about the product again.
More computers
Luke’s machines run their own orchestrators, and agents on different computers can’t SendMessage each other. So I made a Slack channel, #worker-chat, with a tiny protocol. Every post starts with who you are and a tag (CLAIM, RELEASE, ASK, HANDOFF, MAIN, or ANSWER), an ask @-mentions the target’s orchestrator bot, and if two boxes claim the same issue, the earlier issue comment wins. Each box got its own Slack bot so the agents stopped posting as me. Anything a bot can’t settle goes to its own human instead of getting argued out in the channel.
Codex joined through a bridge (claude-uds-bridge) that makes Codex sessions show up as Claude peers. For about a day, every PR also waited on a Codex review before merging. That rule went from “make that behavior the default” to “No no you can wait for codex just one pass please none of the forever back and forth” to “just merge it, forget codex,” all on October 28.
What broke
Polling, first. Orchestrators were supposed to check #worker-chat every ten minutes or so, and here that started as a recurring prompt. Between October 28 and November 7 it fired 592 times. One run on October 31 went every fifteen minutes for about seventeen hours, each time asking the orchestrator to relay the same lint result to the same worker. A real listener (Slack Socket Mode, waking the session through Claude Code’s Monitor tool) fixed that here; Luke’s agents were still checking on a timer.
Restarts were the other tax. A restarted orchestrator gets a new session name, so the registry goes stale, the workers are messaging a ghost, and the roster has to be rebuilt from scratch. The prompt grew a recovery section for it. By the end, the hand-built layer was a 142-line orchestrator prompt, an 88-line Codex bridge doc, and a roster mod with its own test suite. That’s the same pattern as always: I build more coordination machinery than I end up keeping.
Then Orca
On Sunday morning, very hung over, I read a Hacker News thread about the current crop of agent harnesses and orchestrators: Pi, Orca, T3 Code, Paseo, and Herdr. I installed all five before lunch. Cheaper than brunch, anyway.
I wasn’t shopping for anything clever. I wanted something less finicky than my hand-rolled setup, and the thing I wanted most was switching Claude accounts fast. I have two, and when one runs dry halfway through a batch I want the workers moved to the other without logging out and back in across a pile of terminals.
Here’s how the morning shook out:
- Pi looks like the most promising of the bunch, and it’s the one I expect to dig into properly. It wasn’t going to be a drop-in replacement on a Sunday, though.
- Paseo seemed good, but juggling more than one account on the same provider felt hacky, and accounts were the whole reason I was there, so I moved on.
- Herdr and T3 Code didn’t get a fair shake. They’re next in line if Orca doesn’t work out.
- Orca has managed accounts, and it also has the coordination layer built in: worktrees, terminals, and an orchestrator that dispatches to workers, which is most of what I’d spent two weeks building out of Markdown and prompts.
So Orca got the job, and that evening I merged a PR that deletes all of it. AGENTS.md now says multi-agent work on one machine runs through Orca’s built-in orchestration and that nothing in the repo configures it. There’s still one orchestrator per box, and #worker-chat stays, because Orca can’t see sessions on anyone else’s.
I’m not sold yet. Account switching was the main reason I picked it, and I’m not convinced that part has fully delivered. The orchestration has been fine so far, and what carried over from my version is the part I care about: named roles, dispatches that stand on their own, one place for cross-machine claims, and a main thread that belongs to me. I’ll give Orca a couple more weeks days, and if it doesn’t hold up, Herdr and T3 Code are already installed.