Retrieval had to understand paraphrases without missing exact product names, domain terms, or numbers.

Technology

PostgreSQL · pgvector · FTS/RRF

Why this technology

I combine vector retrieval and keyword search. Meaning-equivalent phrases are caught with vector search, while exact terms (product names, numbers) are captured with keyword search.

Implementation evidence

The two result sets are fused with RRF, and every final answer keeps selected document chunks as evidence so retrieval and generation can be cross-verified.

Tradeoff

Both indexes and fusion weights require tuning, and the retrieval path needs more instrumentation.

Mixing model-call state with the document state shown to users made the current stage and valid next actions difficult to manage consistently.

Technology

OpenAI Agents SDK · PostgreSQL · Prisma

Why this technology

The Agents SDK owns model and tool execution, while product state stored through PostgreSQL and Prisma remains the source of truth for the current writing stage and valid transitions.

Implementation evidence

Each stage stores its input, output, and status as product data, and the debugger reads that history to expose the current state and where a problem began.

Tradeoff

The schema and synchronization rules connecting AI execution results to product state require explicit maintenance.

A final output alone cannot reveal where a failure began or support a reliable before-and-after comparison of process changes.

Technology

TypeScript · Zod · Prisma · revisioned checkpoints

Why this technology

Explicit state transitions preserve per-stage inputs, outputs, and revisions so the current state and before-and-after process results can be compared.

Implementation evidence

Attempt UUIDs, transitions, and checkpoint snapshots are schema-validated and published as immutable run evidence.

Tradeoff

Workflow changes require version compatibility, and both storage volume and transition-management code increase.

A prompt change can look better while deleting useful content or regressing cost and latency.

Technology

Versioned eval sets · baseline/candidate · human gate

Why this technology

I run baseline and candidate prompts against the same evaluation set.

Implementation evidence

I track quality score, cost, and latency together. Even if the average improves, I verify that no specific case regressed. Automated results guide decisions, but a human still approves final changes.

Tradeoff

Evaluation sets need ongoing maintenance and the review gate slows the path from experiment to release.

Connecting nodes by event order could make branches, retries, and missing facts look like authored process routes.

Technology

versioned event contract · authored topology · React Flow

Why this technology

Projects retain authority over definitions and transition choices; the console projects only observed state from validated past-tense fact events.

Implementation evidence

Overview renders only authored nodes and edges as node-edge-node. Repeated events remain distinct occurrences and never create a new transition.

Tradeoff

Projects must publish matching definition versions and event contracts; unmatched observations cannot be attached to the graph by guesswork.

A single failure cannot distinguish an isolated error from a recurring state or guardrail problem.

Technology

PostgreSQL · fixed UTC windows · deterministic time-series aggregation

Why this technology

State events from the same project, process definition, and execution mode are bucketed into fixed windows with explicit denominators for success, reach, and evaluation.

Implementation evidence

The time series places node reach beside guardrail PASS, WARN, and BLOCK. Empty or non-evaluable windows stay ‘—’ instead of being counted as healthy.

Tradeoff

Exact comparison constrains scope and time windows, and small samples cannot support strong anomaly claims.

After locating a guardrail conflict, the same condition had to be reproduced without editing the source project or production state.

Technology

Zod · immutable baseline · isolated TEST replay

Why this technology

The console presents a safe state snapshot for the selected node or edge, copies candidate changes, and asks the project-owned TEST runner to revalidate them.

Implementation evidence

Baseline and candidate results keep separate identities and compare each guardrail's PASS, WARN, BLOCK, and reason codes. LIVE execution and production mutation remain unavailable.

Tradeoff

Each project must provide a safe snapshot and reproducible test adapter; the console cannot execute arbitrary state on its own.

Domain state events alone cannot support future site analysis that combines user behavior with model-run characteristics.

Technology

Read-only PostHog · LangSmith · bounded provider federation

Why this technology

Product behavior remains in PostHog and technical execution remains in LangSmith, while the console collects only allowlisted aggregates and references.

Implementation evidence

Provider metrics do not replace current process-state causality or guardrail verdicts; they remain separate evidence. Shared operation references prepare them for future unified site analysis.

Tradeoff

Provider data is correlation evidence, not causation. Raw payloads and free-form queries are intentionally excluded to reduce privacy and authorization risk.