Durable execution
Ingestion and evaluation pair a fast, synchronous API with a Temporal-backed worker underneath for the heavy, long-running work — built to survive real document volumes and real usage levels, not just a demo. A live conversation turn deliberately doesn't follow this pattern; it stays synchronous end to end instead.
Not a script that happens to work
Fred's ingestion and evaluation services separate the part a user waits on from the part that does the actual work. The synchronous API responds fast: an upload is accepted, an evaluation run is scheduled. The heavy lifting behind each one runs as a durable, Temporal-backed workflow.
That separation matters the moment something takes longer than a request timeout, or fails partway through. A workflow that crashes mid-step resumes from its last completed step, not from scratch — the same guarantee whether the work behind it is OCR on a long document or a multi-step evaluation run against a live model.
Agent execution — a live chat turn — deliberately does not follow this pattern; see Streaming without a broker below for why.
Ingestion: async by design
Uploading a document doesn't block on OCR, chunking, and embedding happening inline. The upload API accepts the file and hands the actual processing to a durable workflow, which does the work in the background and survives a crash mid-way through — so a large or slow document doesn't tie up a request thread, and a worker restart mid-ingestion doesn't lose progress or silently drop the file.
Recurring evaluation, not a one-off suite
The same pattern extends to evaluation: Fred runs recurring evaluation campaigns against your team's agents on a schedule, in production — not a one-off test suite that only ran the day someone remembered to run it.
If an agent's behavior drifts — a model update changes its answers, a prompt regression creeps in — you find out from a scheduled check, not a support ticket. Evaluation is an ongoing operational signal, the same way you'd treat a health check, not a one-time gate before shipping.
Streaming without a broker
Conversations themselves stream over a standard HTTP connection directly between the browser and the runtime pod running the agent — no broker in the middle relaying every token, and no Temporal workflow behind it either. Access is secured by the same per-request authorization used everywhere else in Fred, rather than a fragile session-affinity trick that ties a conversation to one specific pod.
That's a deliberate difference from ingestion and evaluation, not a gap: someone watching an answer stream in cares about sub-second responsiveness far more than about surviving a mid-turn crash, and each turn already gets its own resume point without needing a durable workflow underneath it.