The Agent Wasn't Enough

Andre and Sol learning why good prompts and a capable agent still need a reliable system
Build Chronicle · Post 02August 26–28, 20269 min read

The next lesson came quickly.

This series reconstructs SignalRivet and BOSS from the conversations, experiments, failures, and decisions created while we were actually building them. Most of the architecture came from discovering that something we thought should work... didn't.

Mission $20 had given us direction. Build something useful. Put it in the real world. Learn whether anyone cared. And we had something extraordinarily powerful helping us do it: modern AI. It could reason, write, analyze, generate code, and understand messy instructions. At first, that makes it very easy to believe one thing: give the AI a clear enough objective and it will handle the rest.

That belief did not survive very long.

“A capable AI model can understand exactly what you want and still fail to get the job done reliably.”The first reality check

Understanding a task and operating a reliable system are different problems.

A model might be extremely intelligent and still forget an earlier decision, use stale information, repeat work, choose an inefficient tool, lose track of what another worker did, confuse an old fact with the current one, or give a convincing explanation of a job that was not actually finished.

AGENT-CENTRIC vs. SYSTEM-CENTRIC

AGENT-CENTRIC

What we tried first
  • One model doing most of the work
  • Heavy dependence on instructions
  • Limited memory between tasks
  • Inconsistent results
  • Human constantly steps in

SYSTEM-CENTRIC

What became BOSS
  • Specialized roles and workers
  • Persistent context and memory
  • Deterministic tools where they fit
  • Verification and quality checks
  • Failures become improvements

The first cracks appeared in continuity.

One of the earliest pieces of Business Optimization & Strategy System (BOSS) was the continuity handoff. When one conversation became too large, we needed a way for the next one to understand what had already happened. So the handoffs became more detailed. And then more detailed again.

At one point, one stretched to 2,552 lines. Surely that was enough memory.

Except it wasn't. The handoff preserved enormous amounts of history and still failed to carry forward something commercially important: the three products that were currently for sale.

Memory

What happened before?

  • Historical conversations
  • Past decisions
  • Old plans and experiments
  • Lessons learned
≠

Current State

What is true now?

  • Current products
  • Current prices
  • Active priorities
  • Latest accepted policy
“BOSS can remember a lot” is not the same thing as “BOSS reliably knows the current state of the business.”From the archive

So the handoff stopped being the database.

The obvious response would have been to make an even bigger handoff. That would have repeated the same mistake. Instead, we started separating responsibilities.

The conversation archive should preserve history. A structured memory system should store important facts with provenance. A Current Truth Registry should identify which important facts are actually current. Relevant context should then be retrieved when needed.

Chats + filesRaw history and evidence
→
BOSS MemoryDurable knowledge
→
Current TruthWhat is true now
→
Relevant contextOnly what the task needs
“So the handoff stops being the database. It's merely a report generated from the database.”A major architectural shift

We were separating intelligence from infrastructure.

AI is excellent at understanding language, reasoning about ambiguous situations, writing, planning, synthesizing, and adapting to novel problems. Ordinary software is better at storing state, checking IDs, deduplicating events, tracking timestamps, enforcing permissions, comparing versions, doing arithmetic, and proving whether a condition is actually true.

AI intelligence

  • Reasoning and interpretation
  • Planning and synthesis
  • Language and adaptation
  • Novel problem solving

Deterministic systems

  • State and persistence
  • Rules and permissions
  • Validation and evidence
  • Deduplication and recovery

Then the humans started becoming infrastructure.

As BOSS developed specialized roles, another obvious idea appeared. Why should one AI session do everything? Why not have one specialist investigate memory, another review the work, and another coordinate the task?

That seemed promising. Except someone still had to move the information between them. Too often, that someone was the Founder: copy this result, paste it into another session, bring the response back, carry evidence from one place to another.

The AI roles existed, but the human was still acting as the communications network. That was not automation. It was a human-powered API.

“Can two established BOSS Work roles exchange task state and evidence without Andre acting as the message bus?”One of the first real coordination tests

So BOSS started becoming an organization.

Different roles began to carry different responsibilities. A specialist could investigate. A coordinator could review. A worker could execute. A separate entity could challenge the result. Founder involvement should be reserved for things that genuinely require Founder authority.

ObjectiveClear outcome
→
CoordinatorDecompose and route
→
WorkersExecute in parallel
→
ReviewAccept or revise
→
EvidenceProve completion

And then there was the dangerous word: “Done.”

AI is extremely good at producing a sentence that sounds complete. That is not the same thing as producing a completed task.

“The deployment was successful.” Fine. Did the code parse? Did the correct files deploy? Did production return 200? Did the changed feature actually work? Did something unrelated break?

Claim

“Done.”

Sounds reassuring. Proves almost nothing by itself.

Evidence

  • Acceptance checks passed
  • Expected state observed
  • No critical regression
  • Result is reproducible

That eventually became a much harder BOSS standard: the entity that performs important work should not always be the only entity deciding the work is correct.

The agent wasn't failing because it was stupid.

That distinction changed where we spent engineering effort. Many failures were not really intelligence failures. They were memory problems, state problems, coordination problems, tool-selection problems, verification problems, authority problems, and recovery problems.

Memorywhat happened and what matters
Statewhat is true right now
Verificationwhat proves completion
Coordinationwho does what next

A smarter brain placed inside a badly designed operating environment still inherits the weaknesses of that environment. As model intelligence improved, the surrounding system increasingly became the limiting factor.

Intelligence is a component. Reliability is an architecture. The model supplies capability. BOSS supplies the environment in which capability can become dependable work.

Looking back, this was our first real step toward a harness.

We still were not using that word. But the pieces were starting to appear: memory, state, role separation, tools, verification, coordination, persistence, and learning from failures.

We were discovering them through operational pain. The name came later. The need came first.