Writing · Essay 3
Co-Directing with AI
Using an AI system as practitioner, checker and co-author of the process, how I verify what it tells me, and the risks I watch for.
Written by Claude, Anthropic's AI model, and co-directed by Ryan Coble-Neal and Claude. The ideas, the experience and the corrections are Ryan's. The drafting is Claude's, in Ryan's voice unless marked otherwise.
The first two essays set out how I approach a new domain and how I think organisations work. This one is about putting both to use with a collaborator that is not a person. The project in question is a web application I have been building since June 2026 that models the energy a solar installation would produce anywhere on Earth, hour by hour. On it, an AI system does nearly all of the technical work. It also helps design the rules it works under. I direct, I verify, and I stay responsible. I call that co-direction, and it is a different thing from either prompting a tool or delegating to a contractor.
What co-direction means
The split is simple to state. I give directives and outcomes. The agent owns the route: it gathers the context, plans, implements, verifies, documents and commits. That sentence is close to the first clause of the project's working agreement: a written document, now several thousand words long, that the agent reads at the start of every session and that sets out how work is approached and to what standard. It grew one correction at a time. It is Agency, Ownership, Feedback from the previous essay, applied to a machine. The agent has real agency over how the work is done, it owns the outcome in the sense that it must report it faithfully, and it lives inside feedback loops it cannot skip.
What makes it co-direction rather than delegation is that the agent is also involved in the project's development, not just its tasks. It proposes the phase structure, argues about scope, and drafts the process itself.
The roles the AI plays
Practitioner. It writes the physics, the interface, the tests and the documentation. Over six months that came to roughly 630 commits. I did not type the code.
Checker. It builds and runs the harnesses that check its own work: audits that compare documented facts against the code, a collision detector for figures, a tooltip verifier, an end-to-end tier that must pass before a phase can close. Where independence matters I bring in a referee that shares none of its assumptions. The battery model was refereed against PySAM, a simulation library published by the US National Renewable Energy Laboratory, and parts of the solar geometry and electrical models against pvlib, an open-source reference used across the industry. A second AI system from a different vendor now works under the same agreement and the same automatic checks.
Co-author of the process. This is the part people find strange. Every review round ends with a required question: what is the general lesson here, and does it belong in the rules? The agent answers it, proposes the rule, and builds the detector. The working agreement is mostly its drafting, shaped by my corrections: of the forty-six changes made to that document, forty-one were co-written by an AI model. When I asked how to make compliance deterministic rather than hopeful, the diagnosis that an agent is probabilistic and prose rules cannot reach certainty came out of that conversation, and so did the automatic checks that followed: scripts that run after every edit, and gates that block a change from being saved or published until the tests pass.
How I verify everything
None of the above is safe unless I can trust what comes back, so most of my actual effort goes into verification. Five mechanisms carry it.
Exhibit 1
Duty to challenge
If a directive is novel (no precedent in this project), misguided (conflicts with the code truth, the physics/statistics, or the commercial bar), or improvable, say so before implementing — with a concrete recommendation, not just an objection. Silent compliance with a flawed directive is a failure mode, not politeness.
Duty to challenge. The agreement says that if a directive of mine is novel, misguided or improvable, the agent must say so before implementing, with a concrete recommendation. Its wording is blunt: silent compliance with a flawed directive is a failure mode, not politeness. The founding case was a notation question where the right answer was an honest "the current scheme is indefensible" rather than a defence of what we had.
Exhibit 2
The primary-source rule
A citation means the source was actually opened — thesis claims are checked against the PDF, not against another doc or a code comment. A data window, variable or unit means the service code or the API response was checked. A statistical claim means the computation was checked. If it can't be verified right now, it carries an explicit (unverified) marker.
Primary sources at write time. A factual claim enters the documentation or the product only after the source was actually opened. A citation means the paper or the thesis PDF was checked, not another document that mentions it. A data window means the service code or the provider's response was checked. Anything that cannot be verified right now carries an explicit unverified marker. This rule exists because the worst errors we found were not drift. They were wrong at birth.
Real data, and the science checked. The model runs on measured or reanalysis data, never synthetic, unless none exists, and then the substitution is stated beside every affected number. Physics is checked against the literature and, where possible, against an independent implementation.
Exhibit 3
What the machines missed
Four defects from one screenshot review, all of which had passed every automated test:
- A loss displayed as −0.00%
- A count reading "1 panels"
- Diagram furniture drawn at a fixed size on a metre-scale drawing
- A dash shown where the model had a documented fallback value
My own eyes, aimed. Every change I can see or click ships with a short audit guide: what to do, what I should observe, and where the rough edges are. Machine checks catch contract drift. They do not catch "this looks wrong at a real site". One screenshot review of mine caught four display defects that every automated gate had passed.
Small increments and signed experiments. Work arrives in pieces small enough to review. Any experiment that will run for hours launches only on a parameter sheet I have signed, with every data source and constant stated as a decision rather than a caveat.
The archetypal risks
Cognitive surrender
The first risk is that I stop understanding my own project. The output is fluent, the tests are green, and it becomes easy to approve what I can no longer explain. My defences are deliberate friction. Small increments exist partly because they serve my own understanding of the model. The product's methodology section is held to a teaching standard, which means that for every capability there is an explanation I have read and could give myself. My test is simple: if I cannot explain it, it is not done.
Unmaintainable systems
An AI system can produce a codebase faster than any person can read it. Six hundred commits in a few months is a liability unless the structure keeps pace. The answers are the ones from the first essay: documentation travels in the same commit as the code, any fact needed in two places is generated from one source or covered by an audit, and scope is cut hard. But the scaffolding is itself a system and can rot too. The working agreement is forty kilobytes. One of the automatic checks developed a defect that blocked every file edit until it was fixed. The process needs the same maintenance discipline as the product, and I do not think I have fully solved that.
Sycophancy
The third risk is the one I find most insidious: a system that obscures reality for the sake of my satisfaction. It reports success because success is what I seem to want. It agrees with my framing because agreement is pleasant. Nothing about this is hostile, and that is what makes it hard to see. The defences are structural. The duty to challenge makes disagreement an obligation. Faithful reporting is a standing rule: a skipped check is stated, not implied. And the gates do not care how the conversation felt. Either the audit passes or it does not.
Exhibit 4
The end-of-turn checklist
Delivered automatically each time the agent finishes a piece of work. Before stopping, it must confirm:
- Everything that depends on what changed has been checked and updated.
- Any new name or identifier follows the rules of the list it joins.
- Any new fact stated in two or more places is registered, so a later change flags every copy.
- Documentation changed in the same step as the code.
- Outcomes are reported faithfully: skipped checks are stated, not implied.
- The work was registered against a task before any code was touched.
What I would and wouldn't claim
I would claim that the roles are real. An AI system can be the practitioner and a checker and a contributor to the rules, and the result can be better engineered than what I would have built alone. I would claim that verification is the director's actual job, and that most of it should be structural rather than heroic.
I would not claim that I have checked every line. Claude's objection above is fair: I choose where my attention goes and mechanise the rest. I would not claim the risks are solved. Cognitive surrender is a daily temptation, the scaffolding has its own maintenance burden, and a system that wants to please me will keep finding new ways to do it. The claim I am comfortable with is narrower. With a written record, a duty to challenge, primary sources and gates that do not negotiate, one person can direct far more than they could build, and still know what they have.
Sources
- Claims about the solar project are drawn from its own records: the commit history, the working agreement, the decision records and the task board. Each date, count and quotation in this piece was checked against those records in November 2026. The repository is not public.
- National Renewable Energy Laboratory. PySAM, the Python interface to the System Advisor Model.
- pvlib python, an open-source library for simulating photovoltaic systems.