From FP32 to INT8 on the Edge: A Production Pipeline for Quantization-Aware Vision Models
Quantization is not a checkbox at export time — it is an engineering pipeline that spans training, calibration, compilation and on-device validation.
Automation
Hosting, model calls, monitoring and the occasional human review — an itemised monthly bill from a real deployment, not a pricing-page estimate.
Pricing pages tell you what an automation costs to build. Almost nobody publishes what one costs to run, which is the number that decides whether it survives its second year. So here is the actual monthly bill for one of ours: an order-intake automation at a wholesale distributor, processing about 1,400 orders a month that arrive as emailed PDFs, photographed paper forms and the occasional spreadsheet. Names withheld; numbers real, from the March 2026 invoice.
| Line item | Monthly | Notes |
|---|---|---|
| Hosting | $46 | Two small VMs, storage, backups |
| OCR | $58 | ~2,900 pages; photographed forms cost the most |
| LLM extraction calls | $104 | Includes retries and the nightly eval run |
| Monitoring and alerting | $18 | Uptime, queue depth, parse-failure counter |
| Human review | $308 | ~7% of orders flagged; ~11 hrs at $28 loaded |
| Maintenance retainer share | $150 | Updates, format fixes, small changes |
| Total | $684 | ≈ $0.49 per order |
The team this replaced spent about three hours every morning keying orders — call it 66 hours a month, roughly $1,850 at the same loaded rate, before counting the keying errors that used to ship wrong pallets. The automation pays for itself even in a slow month, but look at the shape of the bill: the two biggest lines are humans. Model calls are 15% of the total. Anyone who prices an automation for you using only API rates has quoted you a sixth of the truth.
One thing deliberately absent from the table: build-cost amortisation. The build was a fixed-scope project, paid once, and we think mixing it into the monthly view hides the decision that actually matters — whether the system is worth keeping now, at today’s volume, against today’s alternative. Sunk cost gets no line item.
Run costs do not stay flat on their own, and in our experience 80% of the creep comes from one place: the review queue. Every new customer with a new form layout initially routes to human review, which is correct behaviour. The trap is that nobody promotes learned formats back out of the queue. At this client, the review share drifted from 7% to 12% over a quarter — a 70% increase in the largest cost line — before anyone noticed, because each individual review felt normal. The fix was procedural, not technical: a 30-minute monthly triage where someone asks “which formats did we review more than five times, and why are they still in the queue?”
The remaining 20% of creep is retries. A model call that fails to parse gets retried; retries hide upstream problems and double-bill you for them. Our parse-failure counter exists mostly to make this visible — when retries exceed 2% of calls, something changed and the bill will say so in three weeks if the alert does not say so today.
We have turned down order-intake automations, and the maths above shows why. Three honest thresholds:
This post describes real client work. Bring us yours