How I Turn AI Prototypes Into Working Products

I give the prototype one learning question

Before I build, I write the uncertainty the prototype needs to remove. For Reskill Navigator, the early question was whether a short assessment could produce several career directions that felt reasoned and actionable. For Hermes Workspace, it was whether a local model could use memory and tools well enough to complete a real task.

One question keeps the prototype honest. I build only enough input, model behavior, interface, and output to observe the answer. I do not add accounts, dashboards, billing, or broad integrations unless the experiment needs them.

I also decide what evidence will change my next move. I can continue, revise the premise, or stop. A prototype with no stopping rule turns into a fragile product by accident.

“The overarching purpose of any form of prototype is to learn something at a much lower cost in terms of time and effort than building out a product.”

Marty Cagan, product builder and author↗

I map the real workflow after the idea works

Once the idea earns another build, I map the person, trigger, input, decision, output, next action, owner, and recovery path. I walk through what happens before and after the model. That exposes the parts a polished demo usually hides.

Chams Pay made this concrete. Recording a payment is not one form submission. Staff identify the student, choose a payment method, allocate the amount, issue the correct document, synchronize the record, reconcile problems, and close the day. The product must preserve that sequence when the network fails.

I write the exceptions beside the normal path. I include incomplete information, duplicate actions, stale data, unsupported input, lost responses, changing permissions, and people who return after time away. The first release does not need every case, but it needs a clear response to each case it accepts.

I decide where AI belongs and where it does not

I use AI for uncertain interpretation, generation, recommendation, and coordination. I keep exact rules deterministic. A model can help explain why a career route fits. It does not own the list of available routes or the learner's progress state. An agent can prepare a payment workflow test. It does not decide the financial totals or invent a receipt identity.

This split gives me a stable product around a variable model. Permissions, identifiers, amounts, statuses, approvals, and audit events stay explicit. Model output passes through a schema or a review step before it changes product state.

When a rule keeps appearing inside a prompt, I move it into data or code. The prompt should guide interpretation. It should not become a hidden database, permission system, or policy manual.

I build the smallest complete operational slice

I choose one end to end task and make it operational. The slice includes valid input, model behavior, deterministic rules, user review, saved state, a useful output, and recovery from the failures that task will meet. I prefer one complete route over five disconnected model features.

For Reskill Navigator, that meant assessment, several explainable routes, curated resources, progress, and local use without a mandatory account. For Chams Pay, it meant a durable local payment, the correct document, a pending sync item, safe retry, and visible recovery. For Hermes, it meant a bounded task, the right local model, narrow tools, useful memory, and a reviewed handoff.

This slice reveals the real architecture. It also gives the person using it something complete enough to judge. Feedback on a finished task is more useful than feedback on a collection of generated outputs.

I turn real failures into evaluations

I build the evaluation set from representative decisions, edge cases, and failures I have already seen. Each case has an expected result or a short rubric. I write that expectation before I change the prompt or implementation.

The evaluation covers the whole workflow. I track task completion, correction, time, recovery, explanation quality, and human review. A strong model answer still fails when the product stores the wrong state or leaves the next action unclear.

Every deployed failure adds a case. A recommendation that overweights one answer becomes a counterexample. A lost network response becomes an idempotency test. An agent that reads too much context becomes a permission test. The evaluation set becomes the memory of the product.

I keep manual review after deployment. Automated checks catch known patterns. A small review sample exposes new ones and shows where the metric stopped matching the real outcome.

• Give the prototype one question and one stopping rule.
• Map the full task before expanding the feature list.
• Keep exact rules outside the model.
• Deploy one complete operational slice.
• Add every useful failure to the evaluation set.

I deploy narrowly and watch what people repeat

I deploy the first useful slice to the smallest real setting that can prove the workflow. I watch where people hesitate, leave, correct the system, use a manual path, or return without being asked. These signals tell me more than enthusiasm during a demo.

The measure follows the task. Reskill Navigator should help a learner compare routes and take a next step. Chams Pay should let staff record, document, synchronize, and reconcile a payment without losing trust. Hermes should complete bounded work while respecting its data and tool boundaries. One generic engagement number cannot describe all three.

I count maintenance with the value. Someone has to update resources, review failures, manage access, monitor costs, support users, and retest model changes. I expand the product when repeated use and completed tasks justify that work. When they do not, I reduce the scope instead of adding features to create activity.