Website Chatbot

What it is

A small chatbot for a Shopify store. It answers from the store's FAQ and operating manual and names the source, says so when it does not know and offers a contact form, tells a logged-in customer about their own orders, gives a booking link, and asks for contact details only from visitors asking about a purchase. Built end to end by the Software Factory as a practice brief. Synthetic data only.

What it does

• The model only chooses which content entry answers a question; the reply is built from that entry's own text and names it, so it cannot invent a fact.
• A logged-in customer sees only their own orders, through a short-lived signed customer token; another customer's order is never shown.
• Nine acceptance criteria, each with a check script that exits 0 only when the behaviour works.
• Requirements, spec, plan, acceptance checks, review findings and handoff from the Software Factory run are all in the repository.

The problem

A small Shopify store wants a chatbot that answers from its own FAQ and manual, says so when it does not know, and never shows one customer another customer's order. A chatbot that lets the model write free answers can invent store policy. It was built as a practice brief to test the Software Factory workflow.

Design decisions

• The model only picks which FAQ or manual entry answers a question, and the server writes the reply from that entry's own text. The tradeoff is less natural wording, in return for removing invented facts.
• Intent (order status, booking, purchase, knowledge question) is decided by word patterns, not by the model. This is cheap and predictable, but the handoff lists phrasings it still misreads and says a real build needs a small classifier.
• The customer ID for order lookups comes only from a signed token, never from the message. The tradeoff is that a real store needs a small Shopify app to sign the token, which this build does not include.
• All runtime data lives outside the repo, and the build has no runtime dependencies. This keeps install simple but means the store owner must connect a mail service.

How it works

1. WidgetOne script tag adds a chat button and panel, sends messages and shows replies as plain text.
2. Server and limitsA Node HTTP server applies a message length cap and a per-client rate limit.
3. Intent detectionFixed rules decide whether a message is an order question, a booking request, a purchase question or a knowledge question.
4. Knowledge and grounding guardEntries are ranked by word overlap, the model picks an entry ID, and the guard sends an answer only if that entry was supplied; otherwise the visitor gets an I-don't-know reply and a contact form.
5. Orders, leads and storeOrder lookup uses the verified token, and contact forms and leads are saved to files with a queued notification.
6. Admin pageA password-protected page lets the business edit answers, and chat uses the change on the next question.

Security: enforced, simulated and known gaps

• Enforced in code: grounded-or-silent answers, order lookup scoped by a signed token, leads asked only on purchase intent, rate limits, and no keys in the browser.
• Simulated: orders are sample data and notifications are written to a file, not emailed.
• Known gaps listed in the handoff: intent misreads some phrasings, admin sign-in attempts have no overall cap, and a crash at the wrong moment can save a lead twice.

What broke and how it was fixed

What broke

In the first live review, a model answer such as We open at 3am passed the grounding guard as long as it named a valid entry ID, so an invented fact could be shown as sourced.

How it was found

The first automated independent review of the build, by the Codex reviewer, raised it among 9 findings in total, 3 of them critical.

The fix

Answers became extractive: the model only chooses entry IDs and the server builds the reply from the entries' own text.

What guards it now

tests/unit/chat.test.ts (extractive answers) and tests/acceptance/ac-02.test.ts assert that invented text such as 3am never reaches the reply.

Reference

KB #3586

Where it runs

Not deployed to a real store. It is a practice build; the handoff gives install steps for a Shopify theme and notes that a real store needs a small Shopify app to sign customer tokens.

Who did what

The owner designed the Software Factory workflow and its gates and approved the work. The Factory and coding agents (Claude, Codex, Gemini) did the build and the reviews. The job retro records 7 reviews, with 4 critical findings, all fixed.

Limits and honest status

• A practice build: it has not been deployed to a store, and order lookup uses sample orders until the store's own app is added.
• What the visitor wants is decided by word patterns, so some phrasings are misread; the handoff lists them.
• Admin sign-in attempts are limited per address block, with no overall cap yet.

Connected to

Software Factory