Durable execution
Ingestion, evaluation, and agent execution all pair a fast, synchronous API with a Temporal-backed worker underneath for the heavy, long-running work — built to survive real document volumes and real usage levels, not just a demo.
Not a script that happens to work
Every Fred service — document ingestion, agent evaluation, agent execution itself — separates the part a user waits on from the part that does the actual work. The synchronous API responds fast: an upload is accepted, an evaluation run is scheduled, a conversation turn starts streaming. The heavy lifting behind each of those runs as a durable, Temporal-backed workflow.
That separation matters the moment something takes longer than a request timeout, or fails partway through. A workflow that crashes mid-step resumes from its last completed step, not from scratch — the same guarantee whether the work behind it is OCR on a long document or a multi-step evaluation run against a live model.
Ingestion: async by design
Uploading a document doesn't block on OCR, chunking, and embedding happening inline. The upload API accepts the file and hands the actual processing to a durable workflow, which does the work in the background and survives a crash mid-way through — so a large or slow document doesn't tie up a request thread, and a worker restart mid-ingestion doesn't lose progress or silently drop the file.
Recurring evaluation, not a one-off suite
The same pattern extends to evaluation: Fred runs recurring evaluation campaigns against your team's agents on a schedule, in production — not a one-off test suite that only ran the day someone remembered to run it.
If an agent's behavior drifts — a model update changes its answers, a prompt regression creeps in — you find out from a scheduled check, not a support ticket. Evaluation is an ongoing operational signal, the same way you'd treat a health check, not a one-time gate before shipping.
Streaming without a broker
Conversations themselves stream over a standard HTTP connection directly between the browser and the runtime pod running the agent — no broker in the middle relaying every token. Access is secured by the same per-request authorization used everywhere else in Fred, rather than a fragile session-affinity trick that ties a conversation to one specific pod.