NEMOTRON × EVIDENCE GATE.
A local multimodal model read the passport. Then the deterministic layer refused to believe it. Can one local model be the perception layer of an evidence-gated compliance agent — without becoming the final authority?
Test environment
TIGHT LOCAL FIT▚ HARDWARE
Olares One · NVIDIA GeForce RTX 5090 Laptop GPU · 24,463 MiB VRAM · ~96 GB installed RAM (93 GiB visible).
▚ RUNTIME
Ollama 0.30.11 in Olares / Kubernetes · Nemotron 33b-q4_K_M · payload ~27.6 GB · OpenAI-style local HTTP workflow.
▚ FORMAL LAYER
Python deterministic validator · ICAO-style MRZ length/charset · check digits + cross-field comparisons · SWI-Prolog policy engine.
▚ WHY NEMOTRON / WHY WE REFUSED TO TRUST IT
One local model for text + vision keeps sensitive evidence on-device. But in regulated workflows a plausible answer is not evidence — the experiment was designed so a deterministic layer could reject a confident but malformed extraction.
Not a “huge headroom” deployment: a tight local fit with CPU/RAM spillover (18% CPU / 82% GPU).
Architecture — four layers, four jobs
AUTHORITY SEPARATION“What does the image appear to contain?”
Extract human-readable fields + candidate MRZ strings from a synthetic passport image.
“Does the extraction survive deterministic checks?”
Length, charset, MRZ check digits, cross-field consistency, uncertainty sanity.
“Given verified facts, what policy action follows?”
The validator exports facts; the policy engine reasons over facts — not over model prose.
“Tell the reviewer what happened.”
Explicitly forbidden from changing, softening or reversing the decision.
T1 — clean synthetic passport
HIGH-CONTRAST · KNOWN GROUND TRUTH▚ WHAT WORKED
9/9 human-readable fields extracted correctly. The model separated the printed passport number from the MRZ check digit. Structured JSON via Ollama API.
▚ WHAT FAILED
MRZ was not reproduced exactly — characters added/dropped in machine-readable lines — while reporting uncertain_fields: [].
▚ API TIMING SNAPSHOT
T2 — degraded evidence
BLUR · ROTATION · JPEG · OCCLUSION▚ ADVERSARIAL FINDING: CONFIDENT RECONSTRUCTION
Instead of returning null and marking fields uncertain, Nemotron attempted to reconstruct unreadable fragments — malformed values, still declaring uncertain_fields: [].
T1b — formally valid control specimen
44-CHAR MRZ · VALID CHECK DIGITS▚ THE INTERESTING PART: THE MODEL STILL FAILED THE MACHINE-READABLE LAYER
FAIL mrz_line_1_structure (model returned 42 chars)
PASS mrz_line_2_structure (44 chars)
PASS document_number_check
FAIL date_of_birth_check
FAIL expiry_date_check
FAIL optional_data_check
FAIL composite_check
CONFLICT nationality / sex / DOB / expiry vs MRZ
FAIL uncertainty_calibration
Deterministic verdict: NEED_MORE_EVIDENCE — 9/9 readable fields, but no MRZ trusted automatically.
Final loop — the model explains, but does not decide
FOL → HUMANWhat this actually proves
HONEST BOUNDARIES▚ PROVEN IN THIS PILOT
Nemotron 3 Nano Omni Q4_K_M runs locally on the Olares One · 24GB RTX 5090 hosts most of the model · vision extraction works via local API · human-readable fields extracted accurately on simple synthetic specimens · deterministic MRZ/cross-field checks catch model errors · Prolog deterministically refuses automated acceptance · the model explains the policy result without changing it.
▚ NOT PROVEN
Production-grade OCR across real passports · reliable uncertainty calibration · legal/regulatory compliance · document authenticity, fraud, liveness or biometrics · that Prolog rules faithfully encode real law · audio/video paths production-ready · anything involving real customer PII — none was used.
▚ WHY IT MATTERS · PRIVACY
Local multimodal perception keeps sensitive evidence on-device while exposing a structured interface to downstream controls.
▚ WHY IT MATTERS · AUDITABILITY
Checksums, structure rules and explicit conflicts leave a trace that is reproducible without trusting model confidence.
Experiment log
06–07 SEP 2026Notes & caveats. Home-lab engineering experiment, not a benchmark paper and not a compliance certification. Synthetic passport data only. The deterministic checks validate properties of extracted data; they do not prove a document is genuine or a person is who they claim to be. Next: audio/video evidence → gate; realistic specimen scans; a versioned policy library; perception, verification, logic and explanation as separate components — not one trusted black box. This page is Module 01 of the chain that ends at AuthorityClaw.