Intelligence at the Edges, Determinism at the Core

Abstract

This paper documents a retrospective, failure-driven evolution of BOSS, SignalRivet's multi-agent operating system, from conversation-heavy and model-mediated coordination toward a hybrid architecture that places probabilistic reasoning at decision boundaries while moving transport, lifecycle state, evidence handling, retries, and completion bookkeeping into deterministic infrastructure. The case study was conducted on an 8 GB Windows laptop under severe budget and memory constraints. Those constraints made otherwise tolerable inefficiencies visible early. The paper does not claim that deterministic infrastructure is novel, nor that BOSS proves a general law of reliable agents. Instead, it asks a narrower systems question: when an LLM-based system repeatedly fails at coordination, recovery, or completion truth, which functions benefit from intelligence and which functions become safer when intelligence is removed?

Internal records provide several bounded observations: five steady-state mailbox pickup probes of 0.901 to 2.475 seconds (median 2.033 seconds), one complete API-worker path of approximately 3.880 seconds from submission to terminal receipt, an unmatched architectural observation in which two resident Python watchers consumed roughly 1.3 GB while a later Go mailman serving eight bindings consumed roughly 20 MB, and a tool-result governor canary that preserved 672,018 raw bytes while exposing 4,941 bytes to the model initially, a 99.26% reduction in model-visible bytes. The case also preserves negative evidence, including an unexplained 13.477-second pickup event and later worker runs whose aggregate token usage remained unexpectedly high. These results support feasibility and design lessons, not broad claims of superiority.

The resulting design principle is selective, not absolute: keep judgment, interpretation, and exceptional recovery intelligent; make identity, state transitions, artifact preservation, evidence contracts, bounded retries, and external effect handling as deterministic and auditable as practical. The paper situates that conclusion against durable-workflow systems, state-machine theory, end-to-end systems design, modern agent frameworks, and recent 2026 research that explicitly separates probabilistic reasoning from deterministic execution.

1. Executive synthesis

The central result is not that "deterministic systems are better than AI." That would be both obvious and wrong. The useful result is about placement. BOSS became easier to reason about when the system stopped asking models to do jobs that could be expressed as explicit state transitions, identity checks, evidence requirements, bounded retries, and narrow transport. Intelligence remained valuable where ambiguity, synthesis, planning, and exception handling actually existed.

• Models remained at the edges of uncertainty: goal interpretation, task decomposition, research, diagnosis, writing, and difficult recovery choices.
• The core moved toward explicit mechanics: task IDs, correlation IDs, monotonic lifecycle states, claims, receipts, durable artifacts, bounded retries, and fail-closed ambiguity.
• Persistent knowledge moved out of long-lived conversations and toward durable files or task state so fresh workers could be disposable.
• Large tool results were preserved as artifacts while model-visible output was bounded, separating evidence retention from immediate context exposure.
• Performance work shifted from "make every worker faster" to "find the cognitive critical path and optimize the delay that actually gates the outcome."

2. Problem statement and research questions

Agent systems fail for reasons that are not reducible to model quality. A strong model can still double-execute a side effect, operate on stale state, misinterpret an acknowledgement as completion, lose the ownership boundary between workers, accumulate unusable context, or spend expensive reasoning on a deterministic task. MAST identifies specification and system-design failures, inter-agent misalignment, and task verification/termination as recurring multi-agent failure classes [9]. Agentless shows that simpler fixed workflows can sometimes outperform more elaborate software-engineering agents [7]. Durable workflow systems such as Temporal separate replay-safe workflow state from nondeterministic external activities [3]. These lines of work point toward the same uncomfortable question: how much of an agent system actually needs to be agentic?

This case study frames four research questions:

1. RQ1. Which recurrent BOSS failures were coordination or lifecycle failures rather than failures of model reasoning?
2. RQ2. Which functions improved when they were moved from model-mediated behavior into deterministic code, state, or explicit contracts?
3. RQ3. What costs or failure modes remained after that shift, especially around context growth, external effects, and human/browser boundaries?
4. RQ4. Which claims are supported by current evidence, and which require controlled experiments before they can be generalized?

3. Case context and method

The study is retrospective and evidence-driven rather than a controlled benchmark. The system evolved continuously between late August and October 2026. Evidence was drawn from BOSS receipts, task-mailbox state, local run logs, saved handoff records, system measurements, and the public SignalRivet Build Chronicle and Research Notes. Two independent GPT-6.1 research passes were then asked to challenge the thesis, map prior work, identify novelty risks, and specify the experiments a skeptical reviewer would demand. External claims were checked against academic papers, official documentation, open-source repositories, Stack Overflow practitioner discussions, and engineering guidance.

3.1 Operating constraints

• Hardware: an 8 GB Windows laptop sharing memory among ChatGPT, browser sessions, local workers, development processes, and BOSS services.
• Budget: very limited operating cash, which made model routing, API cost, and persistent process overhead material engineering constraints rather than optimization trivia.
• Execution mix: local Go and Python components, API workers, ephemeral headless model workers, browser tooling, Google Sheets-backed task exchange, and durable local files.
• Human boundary: the Founder remained final authority but was explicitly not intended to become the message bus, terminal operator, or routine recovery mechanism.

3.2 Evidence classes

4. Architecture evolution: failures first

The architecture did not begin with a clean theory. It grew from operational friction. SignalRivet's own Chronicle describes the recurring loop as failure producing evidence, a narrower boundary, a regression check, and finally a durable rule. That history matters because the paper's strongest contribution is not the final diagram. It is the sequence of failure -> intervention -> evidence -> residual uncertainty.

4.1 From intelligent infrastructure to a boring mailman

An early instinct was to put more intelligence into the control plane. Over time, several always-on Python watchers accumulated memory and coordination overhead while performing work that was mostly mechanical. A small Go process was then reused as a persistent mailman/router. Its job was narrow: watch bounded mailboxes, recognize explicit task state, move envelopes, and hand real work to workers that could start and exit. Historical records describe the design principle as "Eight addresses. One mailman."

The important result is architectural, not a language contest. The Python configuration and later Go configuration were not matched experiments. Two resident Python watchers were observed at roughly 1.3 GB combined, while the later Go mailman serving eight bindings was observed around 20 MB. Because topology, process lifetime, code paths, and workload changed at the same time, the evidence supports a resource-discipline lesson but not a claim that Go is a fixed multiple more efficient than Python.

4.2 From giant persistent context to fresh workers

A second pressure came from conversational baggage. Long-lived specialist threads preserved context but also accumulated obsolete decisions, repeated logs, stale evidence, and high context cost. BOSS therefore moved toward a pattern in which durable specialist knowledge lives in compact files or canonical state while routine workers are fresh and ephemeral. One measured API canary used 2,503 bytes of ROLE, CURRENT, and DECISIONS context, completed in about 1.428 seconds of worker runtime, and reported ephemeral=true with no resumed specialist thread. This is evidence that the pattern can work, not yet evidence that it is always better.

4.3 From "DONE" labels to evidence contracts

A recurring failure class was confusing activity with completion. A queued row was not a worker acceptance. A process ID was not task success. A clean model exit was not proof that the requested external result occurred. The system increasingly separated lifecycle evidence from semantic outcome evidence. That change mirrors end-to-end systems reasoning: lower-layer acknowledgements can support reliability, but application-level correctness still has to be verified where the meaning of success is known [12].

4.4 From raw tool output to recoverable bounded visibility

Large tool and file outputs created a different class of failure: the model could be given so much evidence that context itself became a bottleneck. A BOSS-owned tool-result governor was implemented to preserve full raw output on disk while returning bounded excerpts and artifact handles to the model. In one canary, 672,018 raw bytes were preserved while 4,941 bytes were returned initially, a 99.26% reduction in model-visible bytes. The experiment did not prove a 99.26% token or cost reduction, and it did not test whether decisive evidence could always be recovered later. It did demonstrate a concrete mechanism for separating evidence retention from immediate context exposure.

5. Empirical observations

5.1 Mailbox pickup latency

A generic receiver was instrumented after an earlier 13.477-second pickup appeared inconsistent with a two-second healthy polling interval. An initial hypothesis blamed one-row cursor advancement. That hypothesis fit the timing numerically, but the original slow event occurred at the receiver's initial cursor, so backlog could not explain it. The system was then changed to read a bounded 25-row window per poll and five fresh mailbox-only probes were captured against a healthy running receiver.

Observed summary: min 0.901 s, median 2.033 s, max 2.475 s, mean 1.878 s. A subsequent real API-Luna task was picked up in 2.136 s, the worker ran for 1.428 s, and the complete submission-to-DONE path was approximately 3.880 s. These samples support a statement about observed steady-state behavior under that configuration. They do not support p95 or p99 claims, and they do not explain the earlier 13.477-second event.

5.2 Evidence ledger

5.3 A negative result worth keeping: context inflation

Two October job-application canaries are useful precisely because they did not behave like a clean success story. Their prepared context estimates were only about 7,116 and 6,842 tokens, yet the worker receipts reported 479,471 and 1,169,631 aggregate input tokens. Both runs were fresh workers. The discrepancy suggests that bootstrap size was not the dominant driver of token consumption. Repeated tool observations, large reads, internal replay, or accounting semantics may have contributed. Because the exact source was not isolated, the paper treats these numbers as diagnostic evidence and an open measurement problem. They strengthen the case for governing tool-result ingestion, but they do not prove that the governor would have solved those exact runs.

6. Design principles derived from the case

6.1 Put probabilistic judgment where ambiguity exists

LLMs are valuable where the system must interpret goals, synthesize conflicting evidence, select among plausible approaches, or handle exceptions not captured by the current rules. Replacing these decisions with rigid code can simply make the system deterministically wrong.

6.2 Make execution truth explicit

Task identity, state transitions, claim ownership, process identity, receipts, and artifact locations should be explicit enough that the system can answer "what actually happened?" without asking a model to reconstruct history from prose.

6.3 Treat completion as an evidence contract

A terminal label should mean that task-specific acceptance evidence exists, not merely that a worker process exited. This is especially important when the requested result is external, such as a submitted form, modified account, payment, or browser action.

6.4 Prefer at-least-once plus idempotency to magical exactly-once claims

Messaging systems and retry guidance repeatedly warn that acknowledgements and delivery semantics do not automatically produce exactly-once business effects [13][14][15]. BOSS should therefore preserve idempotency keys, attempt identity, reconciliation states, and explicit UNKNOWN outcomes where the external effect cannot be proven.

6.5 Durable state should outlive disposable workers

Fresh workers reduce idle footprint and conversational baggage only when the state needed to resume work exists somewhere durable and scoped. Otherwise "stateless" becomes another word for forgotten.

6.6 Preserve raw evidence, bound immediate context

The governor pattern separates archival completeness from model attention. The useful contract is not "summarize everything." It is "keep everything, expose only what is currently useful, and provide a bounded retrieval path."

6.7 Optimize the critical path of cognition

Cheap parallel leaf work changes the bottleneck. If a 30-minute management or verification step gates completion, shaving five seconds from a worker can have no effect on end-to-end latency. Measure the dependency graph before optimizing components in isolation.

7. Analytical framing

The mathematics in this note is intentionally modest. It is used to make measurement boundaries explicit, not to manufacture precision that the case data does not support.

8. Related work and novelty boundary

The strongest external challenge to novelty arrived during the research process itself. Huk's 2026 ECSA paper is explicitly titled "Decoupling Reasoning from Execution: A Deterministic Runtime Architecture for Autonomous LLM Agents" [1]. Gill-Lakhowal's 2026 IEEE Access paper similarly proposes deterministic runtime enforcement as the execution authority for autonomous AI agents [2]. These works mean that BOSS cannot credibly claim to have invented the architectural separation. That is useful knowledge, not a setback. It forces the paper toward the contribution it can support: a failure-driven case study showing how that separation emerged through operational evidence, where it helped, where it did not, and what remained unresolved.

8.1 Classic systems lessons are part of the story, not competitors to it

Schneider's state-machine tutorial [11] and the end-to-end argument by Saltzer, Reed, and Clark [12] long predate LLMs. AWS retry guidance [13], Kafka delivery semantics [14], and RabbitMQ reliability documentation [15] likewise emphasize idempotency, acknowledgements, redelivery, and the difficulty of exactly-once effects. The BOSS lesson is therefore not "AI discovered distributed systems." It is that agent builders can accidentally rediscover these failure modes because model-centric prototypes make it easy to hide state, ownership, and side effects inside conversational control. The old lessons become newly important when probabilistic components are given execution authority.

8.2 Context as a systems resource

Lost in the Middle shows that long-context models may use relevant information unevenly depending on its position [16]. ReWOO demonstrates that decoupling reasoning from observations can reduce repeated prompt processing [7]. BOSS adds an operational variant of that concern: the tool-result governor does not merely shorten prompts. It creates a storage/retrieval boundary in which complete evidence is retained independently of what the model sees immediately. That design should now be evaluated for evidence recall, not merely byte reduction.

8.3 Practitioner evidence

Stack Overflow discussions about RabbitMQ and message processing repeatedly converge on a practical version of the same warning: if duplicate delivery is possible, design the consumer or business effect to be idempotent rather than treating "exactly once" as a free property of the queue [17][18]. Community evidence is not peer-reviewed proof, but it is useful for identifying recurring implementation pain that formal delivery semantics can make easy to underestimate.

9. Failure-to-intervention map

10. Testable hypotheses and next experiments

The current evidence is strong enough to motivate controlled experiments, but not strong enough to replace them. The most useful next paper would be an ablation study using fixed workloads, budgets, permissions, and acceptance oracles.

11. Limitations

• Single evolving system. BOSS is one installation under continuous development, not a representative sample of agent deployments.
• Small observational samples. Five pickup probes and one complete API task cannot establish reliability distributions, tail latency, throughput, or scalability.
• Configuration confounding. The Python-to-Go memory observation changed process topology and lifecycle as well as implementation language.
• Outcome quality not jointly measured for every optimization. Lower context exposure or faster pickup could theoretically reduce semantic quality.
• External systems dominate some boundaries. Browser CAPTCHA, authenticated sessions, SaaS behavior, and external side effects can defeat otherwise sound internal control.
• Model and platform versions changed during the study. Some observed behavior may not reproduce under future runtimes.
• AI-assisted research. GPT-6.1 Low and Medium were used as independent research/reviewer passes. Their findings were treated as leads and challenges, not as authoritative sources without verification.

12. Practical recommendations for agent-system builders

Start with the end-state evidence. Define what proves the real task succeeded before choosing workers, queues, or orchestration patterns.

Keep one authoritative owner for consequential mutation. Parallelize research, drafts, analysis, and tests freely; converge mutation through one source of truth.

Use durable attempt identity. Retries need task IDs, attempt IDs, and idempotency keys that survive process death.

Represent uncertainty explicitly. UNKNOWN is often safer than a false DONE when the external effect may have occurred but cannot be confirmed.

Make context a bounded I/O problem. Store raw evidence separately; retrieve ranges or excerpts rather than replaying giant logs.

Use intelligence for exceptions, not as a scheduler-shaped hammer. If a transition can be expressed as a deterministic rule, the model should not be paid to rediscover it each time.

Instrument before optimizing. The plausible explanation for the 13.477-second delay turned out to be wrong. Tracing prevented a neat but false story.

Preserve failures that contradict the thesis. Negative results identify where the architecture is incomplete and prevent the paper from becoming marketing copy.

13. Conclusion

BOSS did not become more reliable by making every component smarter. Some of its most useful changes came from identifying work that should never have required intelligence in the first place. A mailman does not need to reinterpret the organization. A task ID should not need a model to remember what it refers to. A receipt should not depend on persuasive prose. A retry should not silently create a second business effect. An enormous tool result should not have to live inside the model context merely because the evidence must be preserved.

At the same time, deterministic infrastructure is not a substitute for judgment. It can faithfully execute the wrong plan, preserve an incorrect state, or enforce an incomplete policy. The useful architecture is therefore hybrid: probabilistic reasoning at the edges of ambiguity, deterministic execution and evidence handling at the core of repeatable mechanics, and explicit verification at the boundary where real-world meaning is known.

The current evidence is best read as a set of engineering observations and testable hypotheses. It demonstrates feasibility, exposes important confounds, and identifies a path toward stronger evaluation. The next step is not another architectural slogan. It is a matched experiment that asks whether the boundary choices documented here improve accepted outcomes, recovery, resource use, and cost under controlled conditions.

References

[1] Artur Huk. "Decoupling Reasoning from Execution: A Deterministic Runtime Architecture for Autonomous LLM Agents." ECSA 2026 Tracks and Workshops, Springer, 2026. https://link.springer.com/book/10.1007/978-3-032-39143-8?page=2

[2] Abhinandan Gill-Lakhowal. "Deterministic Runtime Enforcement: The Execution Authority for Autonomous AI Agents." IEEE Access 14, 2026. DOI 10.1109/ACCESS.2026.3719838. https://doi.org/10.1109/ACCESS.2026.3719838

[3] Temporal Technologies. "Temporal Workflow Definition" - deterministic workflow replay and external Activities. https://docs.temporal.io/workflow-definition

[4] Temporal Technologies. "Activity Definition" - external, retryable work outside the workflow replay path. https://docs.temporal.io/activity-definition

[5] LangChain. "LangGraph Persistence" and "Interrupts" - checkpointing, durable execution, and human-in-the-loop pause/resume. https://docs.langchain.com/oss/python/langgraph/persistence

[6] Shunyu Yao et al. "ReAct: Synergizing Reasoning and Acting in Language Models." ICLR 2023. https://arxiv.org/abs/2210.03629

[7] Binfeng Xu et al. "ReWOO: Decoupling Reasoning from Observations for Efficient Augmented Language Models." 2023. https://arxiv.org/abs/2305.18323

[8] Chunqiu Steven Xia et al. "Agentless: Demystifying LLM-based Software Engineering Agents." 2024. https://arxiv.org/abs/2407.01489

[9] Mert Cemri et al. "Why Do Multi-Agent LLM Systems Fail?" 2025. https://arxiv.org/abs/2503.13657

[10] Adam Fourney et al. "Magentic-One: A Generalist Multi-Agent System for Solving Complex Tasks." 2024. https://arxiv.org/abs/2411.04468

[11] Fred B. Schneider. "Implementing Fault-Tolerant Services Using the State Machine Approach: A Tutorial." ACM Computing Surveys 22(4), 1990. https://doi.org/10.1145/98163.98167

[12] J. H. Saltzer, D. P. Reed, D. D. Clark. "End-to-End Arguments in System Design." ACM TOCS 2(4), 1984. https://doi.org/10.1145/357401.357402

[13] Malcolm Featonby. "Making retries safe with idempotent APIs." Amazon Builders' Library. https://aws.amazon.com/builders-library/making-retries-safe-with-idempotent-APIs/

[14] Apache Kafka. "Design - Message Delivery Semantics." https://kafka.apache.org/design/

[15] RabbitMQ. "Reliability Guide" and consumer acknowledgements / publisher confirms. https://www.rabbitmq.com/docs/reliability

[16] Nelson F. Liu et al. "Lost in the Middle: How Language Models Use Long Contexts." TACL 12, 2024. https://aclanthology.org/2024.tacl-1.9/

[17] Stack Overflow. "How do I ensure that only one consumer actually consumes a published message?" Practitioner discussion of idempotency and exactly-once assumptions. https://stackoverflow.com/questions/53692091/how-do-i-ensure-that-only-one-consumer-actually-consumes-a-published-message

[18] Stack Overflow. "Rabbit MQ Pulling Message More Than Once After Ack." Practitioner discussion of at-least-once delivery and idempotent consumers. https://stackoverflow.com/questions/52522455/rabbit-mq-pulling-message-more-than-once-after-ack

[19] SignalRivet Research Note 002. "From Conversation to Control Plane." September 2026. https://signalrivet.com/research/from-conversation-to-control-plane/

[20] SignalRivet Build Chronicle 07. "Forged Under Constraint." September 2026. https://signalrivet.com/blog/forged-under-constraint/

[21] SignalRivet Build Chronicle 04. "Every Failure Became Architecture." September 2026. https://signalrivet.com/blog/every-failure-became-architecture/

[22] SignalRivet Research Note 001. "Why Reliable AI Work Needs More Than a Good Model." September 2026. https://signalrivet.com/research/governed-worker-completion/

Appendix A. Internal BOSS evidence used in this note

The following internal records were used as first-party evidence. They are not external publications and were not independently audited. Values are reproduced conservatively and only where the run record was specific enough to support the claim.

• 2026-09-28 generic mailbox latency acceptance: five pickup probes, bounded 25-row reads, 2-second polling cadence, and one API-Luna end-to-end canary.
• 2026-09-28/29 Go mailman reuse discussion: prior two Python watchers approximately 1.3 GB combined; Go shadow approximately 16.5 MiB peak and later approximately 20 MB after expansion to eight mailbox bindings.
• 2026-09-30 tool-result governor canary: 672,018 raw bytes preserved; 4,941 returned JSON bytes; 99.26% reduction in model-visible bytes; bounded range retrieval verified.
• 2026-10-01 job application canaries: fresh EPH workers with 479,471 and 1,169,631 reported aggregate input tokens, despite prepared context estimates under 8,000 tokens; browser/CAPTCHA boundaries remained unresolved.

Appendix B. What would falsify the thesis?

• If matched tests show model-mediated routing achieves equal or better accepted outcomes at equal or lower cost and latency, the case for deterministic dispatch should be narrowed.
• If bounded tool-output exposure causes material loss of evidence coverage or outcome quality even with retrieval available, the governor design should be revised or rejected for those task classes.
• If fresh workers require so much repeated reconstruction that cost, latency, or error rate exceeds resident workers under the same workload, the lifecycle policy should become selective rather than default.
• If deterministic lifecycle machinery adds more failure boundaries than it removes, especially under low-volume workloads, a simpler single-worker architecture should be preferred.
• If external effect reconciliation cannot be made reliable for a task class, the system should explicitly preserve human confirmation or UNKNOWN rather than pretending stronger guarantees.

SIGNALRIVET RESEARCH

Proof before promises. Business problems first. Technology second.