Software Factory

What it is

A planned public demo of a ticket-to-reviewed-change workflow. The underlying Factory tooling exists locally: it moves a job through stages, runs independent reviewers, freezes acceptance criteria once the first milestone passes, and takes owner approval outside the coding session. No public demo or public source yet.

What it does

• A stage can only advance when its gate passes; the session can only request approvals, which the owner grants or denies on a separate channel.
• A grant is bound to content: if criteria, review, demo checklist or handoff change after approval, the gate closes until re-approved.
• Acceptance criteria and checks are frozen once milestone 1 passes; changes go through an explicit revise or change-request path.
• Independent reviewer runs are recorded with the binary that executed them.

The problem

Showing that an AI engineering process works needs more than a good-looking demo. The private Factory workflow has run one practice job end to end (the Website Chatbot). The public version designed here adds a ticket-to-pull-request pipeline on a fictional company, human approval before anything is merged, and an honest test of what the knowledge base and the harness each add. The public version is not running yet.

Design decisions

• Test four arms in a 2 by 2 design: bare, knowledge base only, harness only, and both. The tradeoff is cost, since the proposed study is 24 held-out tickets, 5 repeats and 4 arms, about 480 runs, and it needs the owner's approval to spend.
• Judge by fully verified outcomes per dollar and per hour, not by how much code is produced. Hidden tests and reference patches stay outside the agent's workspace so they cannot be edited.
• Visitors choose from preapproved fixture tickets and cannot run arbitrary commands. Merges need the owner's approval. The tradeoff is less freedom for visitors.
• Label every live step and every replay, and report estimated and billed costs separately. The tradeoff is a more cautious page, in return for numbers that can be checked.

How it works

1. Ticket intakeDesigned: a visitor picks a fixture ticket against a fictional SaaS company, and the text is treated as untrusted data.
2. Requirements and knowledge baseDesigned: the pipeline clarifies requirements and retrieves frozen, sanitized knowledge-base notes with source links.
3. Plan and buildDesigned: coding agents work in an isolated git worktree under a capped budget.
4. Tests and independent reviewDesigned: real tests and hidden graders run outside the agent's reach, then an independent reviewer checks the change.
5. Pull request and owner approvalDesigned: the pipeline opens a draft pull request, and only the owner can approve it.
6. Isolated preview and traceDesigned: an approved change goes to a disposable preview, with a trace and cost ledger the visitor can read.

Security: enforced, simulated and known gaps

• Designed only, not built: sandboxed worktrees, no production credentials, egress limits, prompt-injection defences on ticket text, a per-user quota, a global budget circuit breaker and immutable audit logs.
• Designed only: human sign-off is required for any merge, and a reviewer that is unavailable counts as incomplete, not passed.
• Known gap: none of these public-facing controls has been exercised yet. The design notes list hostile probes to run before launch.

Where it runs

Not deployed. The site lists it as coming next. The design places it as a separate backend service under shawndemos.com, with no production route changed without the owner's approval.

Who did what

The owner conceived the factory idea, the benchmark design and the human approval gates, and directs the work. Coding agents (Claude, Codex, Gemini) assisted with the design write-ups. The public pipeline and the benchmark described here have not been built or measured yet.

Limits and honest status

• No live demo and no public repository yet; the public /factory experience is not built.
• Described from the local README and CODEMAP only; no benchmark results are claimed.
• A practice-run mode exists per the local README; no completed public run has been verified.

Connected to

Knowledge Base Server (Memstalker)Presient SE Harness PackAgent SwarmWebsite Chatbot