Case Study - Making a frontier model ~52% cheaper and ~74% faster — without downgrading it

Anonymized engineering case study. Client identity, product name, and proprietary system internals are withheld. Cost and latency figures below are from production runs of the same frontier model (Claude Opus 4.8) before and after the change.

The problem

The application generates long, structured technical documents — multi-section proposals with scope classification, costing, schedules, and rendered artifacts — from short free-text briefs. Generation ran on a premium frontier reasoning model (Claude Opus 4.8), and the output quality was exactly what the workflow needed. The model was not the problem. How we were calling it was.

One long serial call. Every proposal ran as a single call that produced the entire document end to end — so latency was the sum of writing every section, and there was no way to bound or shape where the spend went.

Latency. A full run took on the order of ~160 seconds — slow enough to hurt the interactive experience and to bottleneck any batch use.

Cost. At frontier token rates, an output-heavy proposal ran to roughly $0.87 in model spend, all in one monolithic pass.

The usual reflex here is to reach for a cheaper model. We deliberately did not. The question we tested was not “do we need Opus 4.8?” — the quality bar said we did — but “are we calling Opus 4.8 efficiently?” The answer was no.

The approach — re-engineer the call, keep the model

A long proposal is not one indivisible thing. It is a set of largely independent section groups — narrative, technical, and review content — that don’t depend on each other to be written. The serial pipeline was generating them one after another anyway, paying for that ordering in wall-clock time.

1. Parallel section generation. We restructured the pipeline to generate the independent section groups concurrently on the same Opus 4.8 model, bounded by a configurable concurrency limit (an async semaphore), with a per-section timeout and partial-failure salvage so one slow or failed section never sinks the whole document. End-to-end time collapses from the sum of every section to roughly the slowest one.

2. Tighter, scoped calls instead of one monolith. Each concurrent call generates only its own section group against a shared, pre-computed context, rather than one giant call carrying the entire document in a single prompt/response. Shorter, focused generations — combined with prompt caching on the shared context across the concurrent calls — are what drove the per-proposal cost down, on the same model.

3. Compute-once upstream stages. Deterministic upstream work (scope classification, scaffolding, costing) is computed once and passed forward as shared context rather than re-derived, keeping repeated work out of the model’s billing path and making the shared prefix cacheable across the concurrent calls.

No model downgrade. Every number below is on Claude Opus 4.8 before and after — the frontier model is held constant. The savings are entirely from how the model is invoked, not from trading quality for price.

Because the model never changed, the risk was not “is the cheaper model good enough” — it was “did splitting one call into concurrent section groups degrade coherence or coverage?” Every win only counted if it survived two independent checks: a deterministic evaluator (structure, required sections, scope coverage) and a semantic parity judge (an independent LLM scoring the restructured output against the original serial-Opus output).

The results

Measured on production runs of the same Opus 4.8 model, before vs after the parallel-section restructure. Same model, same quality bar — only the call structure changed.

The parallel output held the quality bar — verified by an independent judge, not by the team that built it. Roughly half the cost and roughly a quarter of the wall-clock time, on the same frontier model.

Why it worked (and why teams miss it)

• Serial generation was a self-imposed tax. Writing independent section groups one after another made total time the sum of every call and forced one monolithic, expensive prompt/response. Concurrency made latency the slowest call; scoping made each call cheaper — same model, same output quality.
• The instinct to downgrade the model is often the wrong lever. The most defensible win here was not swapping Opus 4.8 out — it was calling it efficiently. That keeps frontier quality and still roughly halves the cost.
• Bounded concurrency with salvage, not naive fan-out. A semaphore caps parallelism, a per-section timeout stops a single slow call from stalling the document, and partial-failure salvage keeps a completed document even if one section needs a retry.
• Compute deterministic work once. Upstream stages that don’t need the model (scope classification, scaffolding, costing) are derived once and passed as shared, cacheable context — not recomputed or re-sent per call.
• Trust requires independent verification. Cost and latency wins are only real if an independent quality gate confirms the restructure didn’t degrade output — so the savings are defensible to a customer, not just to ourselves.