# Assay

> Messy documents in, graded data out: read on local GPUs, every field confidence-scored, on a verifiable record.
>
> https://pravda.systems/work/assay · 2026-07-11

**Problem.** Businesses buried in supplier invoices retype them by hand, and the generic cloud tools are rented, one-size-fits-all, and keep the data off the business's own machine.

**Solution.** A local-GPU pipeline that reads each document twice, grades every field's confidence, routes the doubtful ones to a human bench, and records every step on an append-only hash-chained ledger.

**Outcome.** 98.73% field accuracy over 1,361 ground-truth-reconciled documents; a 1,001-document batch on two consumer GPUs with zero failures at cents per thousand pages.

## Key metrics

- **98.73%** Field accuracy — Over 1,361 ground-truth-reconciled documents, computed from the ledger, not asserted. The count is the corrected one: a 2026-07-27 audit in the product repo found 16 leftover test-account documents had been counted as public corpus. The accuracy figure never moved, because it derives from ground-truth verdicts and no account-owned document ever carried one. _(verified: GET /api/metrics overallAccuracy (6,223 / 6,302 fields); docs/METRICS.md 2026-07-05)_
- **1,001 docs, 0 failures** Scale batch — One recorded batch across both GPUs and all 50 FATURA templates (20 unseen before the run). _(verified: Extractions joined to documents created 2026-07-05, no failed documents; docs/METRICS.md)_
- **413 + 459 docs/hr** Throughput — Two 12 GB consumer GPUs reading in parallel. _(verified: Mean durationMs per workerId over the batch; docs/METRICS.md 2026-07-05)_
- **$0.0000139–0.0000304** Energy per document — Local-GPU electricity at $0.11/kWh. Cents per thousand pages. _(verified: Mean measured energyWh × $0.11/kWh per extraction, per GPU; docs/METRICS.md 2026-07-05)_
- **22,744** Ledger events — Append-only, hash-chained; the chain verifies unbroken. _(verified: GET /api/ledger/verify -> ok, 22,744 events; app-assay docs/METRICS.md 2026-07-27)_

## The problem

Businesses that run on documents retype them by hand. A small importer keys in around 200 supplier invoices a month: scans and phone photos, several languages, dozens of supplier layouts. The generic document-AI tools do not fix this for her. They are cloud-only and rented. They assert an accuracy number with no method behind it, hide or charge extra for per-field confidence, and keep the data on someone else's servers. A business that wants its own fields, its own languages, and its documents kept on its own machine has nowhere to go.

## What I built

Assay turns messy documents into checked rows. It reads each document twice, independently, on local graphics hardware, once from the picture of the document and once from the recognized text, and compares the two reads. Every field gets a computed confidence grade. Fields it is sure of pass straight through. Fields that score low, or fail a check like a total that does not add up or a date that will not parse, are held back and sent to a keyboard-first review bench where a person fixes them in one keystroke. Every read, check, correction, and export is written to an append-only, hash-chained record that cannot be quietly edited. The documents never leave the machine.

## How it works

The reading runs on two 12 GB consumer GPUs (an RTX 3080 Ti and an RTX 4070) through a queue-polling worker. The web app and its thin API sit on Vercel and never call out to the PC. The worker polls for jobs. Extraction is a local Qwen3-VL model under llama.cpp, constrained to the field schema, with RapidOCR grounding the second read and supplying per-line confidence. The grade on each field is a logistic model calibrated on the ledger's own ground-truth corrections, not a raw model score. Hard gates fail closed: a field that fails grounding or a business rule cannot reach an export. One JSON schema is the single source of truth for both the web validators and the worker prompt, so the two halves cannot drift apart.

## Results

One recorded batch put 1,001 documents across both GPUs with zero failures, covering every one of the 50 FATURA templates, 20 of them never seen before the run. Field accuracy was 98.73% over 1,361 ground-truth-reconciled documents, computed from the register rather than asserted. The two cards read 413 and 459 documents an hour. Each document cost between $0.0000139 and $0.0000304 of local electricity, cents per thousand pages. The record now holds 22,744 hash-chained events and the chain verifies unbroken. Auto-accepted fields ran 99.9% accurate against 96.3% for the flagged ones, so the confidence grade earns the routing it drives.

## Tech

- **Languages:** TypeScript, Python, SQL
- **Frameworks:** Next.js, React, llama.cpp, RapidOCR
- **Infrastructure:** Vercel, Neon Postgres, Vercel Blob, Local dual-GPU worker (RTX 3080 Ti + RTX 4070)
- **AI/ML:** Qwen3-VL-8B-Instruct (GGUF, local), Schema-constrained JSON extraction, Cross-read + logprob confidence, calibrated logistic model

## Skills demonstrated

- Local-GPU model serving (llama.cpp + Qwen3-VL)
- Schema-constrained LLM extraction
- Per-field confidence calibration from ground-truth data
- Fail-closed validation and human-in-the-loop routing
- Append-only, hash-chained audit ledger
- Full-stack Next.js on Vercel with a polling GPU worker
- Measured accuracy and cost methodology