System architecture

How the pieces fit together

One LangGraph agent, two ways to put it through its paces: a live reviewer console for talking to it directly, and an offline evaluation harness for scoring it at scale — both run the exact same graph and tools.

nginx reverse proxy · /api/* → backend · everything else → frontend

Next.js — Reviewer Consolefrontend/app

TopBarcustomer & docs switcher

ScenarioRailscenario library

ChatPanelstreamed conversation

AgentTracePanellive trace + eval score

SSE stream + REST · /api/chat/stream, /api/customers, /api/scenarios, /api/kb/documents

FastAPI backendbackend/api/main.py

Streams model tokens and tool-call events over SSE for a chat turn; serves REST for customer, order and knowledge-base lookups used by the console chrome.

POST /api/chat/stream (SSE)GET /api/customersGET /api/customers/{email}/ordersGET /api/scenariosGET /api/kb/documentsGET /api/health

graph.astream_events() per turn

LangGraph — agent loopbackend/agent/graph.py

agentGPT-5.4-mini + tools

toolsToolNode executes call

update_trackingrecords retrieval + escalation

END — reply sent

tool_calls?no tool callsloops back to agent until the model stops calling tools

invokes the selected tool function

order_toolslookup_order, check_return_window

customer_toolsget_customer_profile

escalation_toolscreate_escalation_ticket

kb_toolssearch_knowledge_base

Postgres+ pgvector

customersordersescalation_ticketsdocuments (384-dim embeddings)

Knowledge baseall-MiniLM-L6-v2 embeddings via sentence-transformers · cosine search over pgvector

Evaluation harnessruns offline / in CI — not behind nginx

scenarios.jsonbackend/data/scenarios — scripted conversations + expected outcomes

runner.pyReplays every scenario through the same LangGraph agent loop above and collects tool calls, retrievals and replies — it does not score.

5 evaluators score the run

tool_selectionretrievalescalationdecision_correctnessresponse

evaluation_report.jsonper-scenario scores + aggregate pass rate

Same evaluators, live: when a scenario is opened in the console, main.py runs the identical scoring inline and streams it back as an evaluation SSE event — that fills the AgentTracePanel score above.