Claims Document Assistant

01 / Challenge

The problem.

Claims examiners read every submitted document by hand to find the fields they need, check what is missing or wrong, and write back to the billing office. Documents arrive as forms, cover letters, and shorthand notes, so template-based extraction breaks as soon as the layout changes.

Who needed it: Claims-processing teams that review submitted documents and reply to billing offices, and the engineers who would evaluate an assistant like this before using it on real data.

02 / Approach

What I did.

1. 1

   Send each claim PDF to Claude as a document block, with structured outputs constraining the answer to a fixed 17-field claim schema and a prompt that tells the model to return null rather than guess, a guard against hallucinated values.

2. 2

   Validate the extracted fields with deterministic rules, not the model: required fields, NPI check digits, ICD-10 and CPT formats, date order, expired referrals, and non-positive amounts.

3. 3

   Draft a short status reply from the rule findings, with a template fallback when no API key is set or the model call fails.

4. 4

   Queue every result for a person: the reviewer sees the PDF beside the fields, edits fields or the draft, then approves, escalates, or rejects. Corrections are re-validated and every decision is logged.

5. 5

   Route declined requests to manual review instead of returning an empty result.

6. 6

   Generate 60 labelled synthetic claim PDFs in three layouts, using the referral fields from Data Generator & File Processing, and score extractors with an evaluation harness: accuracy on all 17 fields (including fields left blank on purpose, so an invented value counts as an error), issue-flag precision and recall, routing accuracy, latency, and token usage.

03 / Outcome

What changed.

The assistant extracts, checks, and drafts replies end to end on synthetic data, and nothing goes out until a reviewer approves it. A 26-test suite covers the validation rules, the exact request sent to Claude, refusal handling, reply drafts, and the review queue. Claude's extraction accuracy has not been measured yet; the repository includes the evaluation command that will produce it.

• 17 claim fields extracted from PDFs with Claude
• Deterministic rules decide what is missing or invalid
• Status reply drafted for review, never sent automatically
• Review queue with audit log of every correction
• 60 synthetic claims in three layouts with ground truth
• Evaluation harness scores every field against ground truth, so invented values count as errors
• 26 automated tests that run without an API key

04 / Systems & decisions

How it fit together.

A generator produces seeded synthetic claim PDFs and their ground truth; every page is stamped as a synthetic test document.

The Claude extractor sends the PDF as a document block with a JSON schema. An offline rules extractor serves as a baseline and as a fallback without an API key.

A validation module applies the business rules; a drafting module writes the reply from the findings.

A local review server shows the queue, records each decision with the reviewer's name and changed fields in an append-only audit log, and writes only approved replies to an outbox.

Constraints and tradeoffs

• The model extracts and drafts, but rules decide whether a claim is complete, so the outcome is predictable and testable.
• A person approves every reply; the assistant never sends anything itself.
• Synthetic data only. Using real claims would also need a provider agreement covering protected health information, access control on the review queue, retention rules, and an evaluation on a labelled sample of real documents.
• Accuracy is reported only from the evaluation against ground truth. The rules baseline is published; Claude's results will be added once the evaluation runs with an API key.
• No retrieval layer yet, because each claim is a self-contained document. If replies had to cite payer rules or policy text, the next step would be retrieval-augmented generation (embeddings and a vector database over that text), evaluated the same way for answer accuracy and hallucinations.

05 / Evidence

What you can check.

Why it matters for FDE work

This is the kind of AI build I would bring to a healthcare or government team: the model handles messy documents, rules and a person stay in control, and every claim about accuracy comes from a measured evaluation.

06 / Toolkit

Tools used.