How I work with AI tools

I’ve built a stupid amount of stuff with AI tools over the last few months: Riglab, pvc-builder, Motif Forge, terminal games, music tooling, a Bitcoin recovery pipeline, Authorbot, and an increasingly ridiculous homelab.

My side of the process is still simple. I talk until the shape of an idea shows up, dump it into a plan, and start throwing work at agents faster than they can finish it. Underneath that is now a small operations shop: several models, worktrees, directory cards, decisions, hooks, running software the agents can inspect, and files that let a new session continue without replaying my life story.

It starts as a conversation

Before any code exists, the project is usually a chat. Riglab began as a claude.ai conversation titled “Linkage simulator for wearable puppet prototype,” where I opened with:

a small web app that I can use to quickly prototype and mechanically simulate linkages for a physical wearable pvc articulated pseudo-marrionette attached to a pack frame.

That’s me thinking out loud until something useful appears.

Sometimes the raw material is somebody else’s words. I’ll paste a whole conversation with a friend, or a rambling ChatGPT thread, and ask for structure. The point is to get the idea out of my head and into text I can inspect.

One hard-SF brainstorm grew into a cosmology, machine design, addiction mechanism, plot, and ending. I asked for a development brief and a benchmark prompt with our answer removed. Nine days later it had become The Causal Projector, with 12 chapters published through the book platform I built because apparently writing the book wasn’t enough of a side project.

Then I make it durable

Once I roughly know what I want, I open with a wall of text. Bad grammar, unfinished sentences, all the important constraints mixed together. Here’s the real opening prompt for pvc-builder, typos and all:

This project is going to be complementary to riglab (in the parent dir), but geared towards FAST experimentation, development, and visual simplicity. It will be be a web app that acts as a 3d-first PCV creation design studio. […] Pull a library of common fittings from somewhere like home depot or mcmaster, or if you can’t ask me to do it for you.

“PCV.” “be be.” Fine. The prompt still contains the product, constraints, judgment calls, and an instruction to ask instead of making shit up. Then I have the agent turn it into a plan with goals, non-goals, phases, and acceptance criteria. From there, the plan is the source of truth. My rambling did its job and can retire.

The useful parts become files:

the plan says what I’m building;
the decision log says why it has this shape;
directory cards say where the code lives and what must stay true;
findings record investigations that produced no code;
handoffs separate verified state from unfinished work; and
tests and hooks enforce anything too important to leave to memory.

At work I also separate confirmed decisions from recommended defaults. An assistant’s decent guess shouldn’t quietly become something I supposedly approved. When a discovery pass ends, I often turn it into a clean brief for another agent: the mental model, target changes, verification commands, and the files it shouldn’t touch.

Literal cross-session memory matters less than I expected. Most of my durable memory is Markdown and Git. I changed agent stacks three times on one machine in six months without changing the basic process. The tool changed. The repository didn’t care.

Split the work, isolate the files

Once a project is clear enough to divide, I stop treating the model as one assistant. One agent can investigate, another can build, and a third can review the result. Sometimes I ask several to try the same uncertain feature and keep the version that sucks least.

Git worktrees are the basic isolation tool. At work, one scan found 56 agent-created worktree branches over about four months, including days when three to seven appeared at once. In pvc-builder, a claim board says which files each agent owns and names the choke points that need to be handled one at a time. Schema migrations, the editor action layer, and scene composition are bad places for two enthusiastic robots to improvise together.

The repository is the shared conversation. Agents don’t need access to one another’s hidden reasoning. They need current plans, scoped ownership, honest handoffs, and a record of what changed. I wrote up that setup in Three coding agents, one workspace.

Collisions still happen. I’ve made commits that amount to “fix the thing the subagent overwrote.” Worktrees and claims don’t make that impossible. They make it obvious before two agents quietly combine their work into one exciting pile of garbage.

I never let them finish a thought

Once something works, I queue the next task before the first one ends. Then another. Then a bug I noticed. Then “actually, undo the thing from three messages ago.” Across three machines and 132 sessions, I sent a new instruction while the model was still working 700 times.

I’m glad these things aren’t conscious. I would be an absolute monster to work for. There’s no “when you get a second.” I keep dropping work on the pile while the model is chewing the last bite. A human coworker would rightfully throw me into traffic. The model says “on it” and carries on.

The branch history looks calmer than the conversation. Half the sampled sessions at work had gaps of two hours or more because I leave agent work running while I do my actual job. In Git it looks less like frantic pair programming and more like several half-finished pull requests sitting on benches around the room.

Let the software point at itself

Text is a lousy way to explain some bugs. “That pipe moves wrong when I do this” can produce an hour of careful tests for a different pipe doing a different wrong thing.

pvc-builder has a development-only bridge between the coding agent and my browser. The agent can inspect the document, selection, camera, editor state, and simulation, then use the same actions as the interface. I select the broken object and tell it to look. That removes a translation step that was both expensive and unreliable.

Tests are still the durable gate, but live state is evidence. The agent can observe the failure I mean and then write the regression test. Human pointing supplies the bit of information the suite doesn’t have yet.

The same idea works outside a browser. My homelab runs a Dockerized stack with local microphones, speech recognition, Home Assistant, and house speakers. Larger hosted models plan the action. Local services keep the audio, voice cloning, and screen vision on my hardware. The answer comes back in a Werner Herzog voice, often with a comment about the futility of existence or the stunning idiocy of chickens.

Each service owns a small set of tools and local details. The planning model doesn’t need a dissertation on my network every time I ask it to turn off the kitchen lights.

It isn’t only software

I use the same loop for music. Suno Experiments became an album, then a version-control problem, then Motif Forge, then a local music lab that is now almost a replacement for the parts of Suno I care about. The generations aren’t the durable work. Lyrics, motifs, lineage, production notes, skills, and hooks live in the repository.

I use agents for MIDI routing, musical hooks, light control, hardware research, and, on one memorable night, the predicative justification of Mahlo universes. The subject changes. The loop doesn’t: talk until something takes shape, write down the state, split the work, inspect the result, and keep what holds up.

Authorbot is the same idea applied to writing. It puts comments, votes, work claims, revisions, and publication around a Git repository, then gives humans and agents the same API. It doesn’t write. It stops the writing process from dissolving into chats and mystery files.

What I still do

The agents type most of the code. On my Mac mini, about 74 percent of surviving commits had a Claude co-author trailer, and that’s probably low. Squash merges erase much of the attribution at work. Pretending I personally wrote every line would be silly.

I provide direction, source material, constraints, taste, and the final call. I decide which approach is worth trying, which agent gets it, when three versions should compete, and which one survives. I resolve real ambiguity. I do live checks that source code can’t answer. I stop beautiful implementations built on a bad premise.

I also decide when a rule should stop being advice. Important operations at work are blocked by hooks until I approve them. Review requests follow a template. Findings have to say whether a claim came from code or from a live test nobody actually ran. “Remember to ask” is prose. A hook that refuses the action is a process.

The models provide patience, recall, and typing speed I don’t have. They don’t take responsibility. One session contained 179 consecutive agent actions without a message from me. Responsibility didn’t transfer during that streak.

What breaks

This machinery isn’t tidy.

Agents clobber changes when ownership is vague.
Directory cards rot unless every relevant change updates them.
Authentication expires and unattended jobs quietly die.
I build more coordination machinery than I end up using.
The best models eat allowances and money at a spectacular rate.
Generated work gets hard to organize. My Suno account is an experiment graveyard with a search box.
Important projects get plans, tests, hooks, and clean history. Experiments get a directory containing three days of work and optimism.

Discipline follows importance. The flagship work gets guardrails. The rest accumulates abandoned scaffolding and services that are “configured” in the same sense that an empty refrigerator is prepared for dinner.

Coordination is the useful bit

I don’t think the most useful work in agent systems is another model wrapper. It’s the boring layer between people, agents, repositories, and running software.

Chat history is expensive, lossy, and stuck inside one tool. Plain files travel well but go stale. Git records what changed but not which old decision matters right now. Running software knows its state but usually can’t explain it to an external worker.

The pieces I keep rebuilding are:

cheap channels for agent updates;
claims or leases that prevent file and resource collisions;
scoped project notes that survive model changes;
retrieval over decisions, failures, and handoffs;
live connections to running applications; and
a hard split between observed facts, human decisions, and agent guesses.

pvc-builder has crude versions of these. Authorbot has a more formal version for books. My homelab does the same thing across services. None is general, but each saves enough time to make the pattern hard to ignore.

Being a generalist finally paid off

I don’t get good results from these tools because I’m a better programmer. I get them because I’m a generalist, and steering an AI is mostly a generalist’s game.

I’m almost 43. I’ve spent those years poking at an absurd range of stuff. I ran a hardware-recycling startup in Alaska after pulling an ancient function generator from the dump, opening it, and finding nixie tubes and gold. I took apart laser printers old enough to use helium-neon tubes and castoffs from the NOVA project that ended up at the HIPAS observatory. I taught myself to refine gold from scrap electronics. I posted flyers at transfer stations so dumpster divers would sell me their hauls, built things from the pile, and sold power transistors and scintillation crystals to collectors on eBay.

I’ve written code in dozens of languages and don’t remember any one of them well enough to call myself an expert. That used to feel like a liability. Jack of all trades, master of none.

It turned out to be the useful part. The model already types faster and remembers more APIs than I ever will. My job is knowing what to ask, smelling when an approach is going wrong, and dragging experience and taste from ten unrelated subjects into one problem. Twenty-odd years of taking things apart is pretty good preparation for this.

Six months ago I worried AI would take my job. Now I think it rewards the exact scattered generalist I already was.

Claude agrees, for the record. I asked. Claude also calls my half-baked ideas “genuinely great” several times a day, so an AI endorsing my theory about why I’m good at AI isn’t exactly a valid datapoint. I’m keeping that particular bit of flattery anyway.

How this post got made

The first version used the method to document itself. I gave an agent logs from three machines and a claude.ai export. It mined 132 sessions, which is where the 700 interruptions came from.

For the next revision, agents on my work machine and Mac mini audited session logs, Git history, worktrees, plans, hooks, skills, orchestration state, and shell metadata. A third agent combined the sanitized reports with the pvc-builder history, book brainstorm, music posts, and my corrections.

This pass happened after I finished Authorbot and published the first 12 Causal Projector chapters. I rewrote the whole thing again because the earlier version sounded more like an unusually polite quarterly report than me.