Reads English and Arabic documents, labels their fields, fills the owner's document templates, works out whose each one is, and groups them by person. Optionally, a model on the same server describes each one in a few sentences.
Everything runs on the server itself.
Service API
https://docread.srv1283448.hstgr.cloud/v1For other servers, with an API key. Start here to integrate. See Service API (v1).
Page
https://docread.srv1283448.hstgr.cloudThe playground and its own /api, after signing in. See Access.
Local run
http://127.0.0.1:8780On the machine the service runs on; the service API on API_PORT beside it.
Docs
/docs/Where the service serves this page. No sign-in needed.
Spec
/docs/openapi.yamlOpenAPI 3.1, served by the service. Also docs/openapi.yaml in the project.
Version
v0.2
Child pages
Quick startGet a key, send a document, read the fields. One request.
JobsJob states, timing, and how long results are kept.
Live progressFollow a job as it is read: each page, then the owner, the fields and the result, as events.
AccessSigning in, API keys, and the checks on every request.
ReferenceResult shapes, templates, descriptions, grouping, errors and limits.
The OpenAPI 3.1 spec is docs/openapi.yaml in the project, and the service serves it at /docs/openapi.yaml. Its examples were checked against a running server with made-up documents.
Not in this version: callbacks. Poll, or wait on the request (?wait=).
1. Get a key. Sign in to the page, open API keys (/account), choose New key and name it after the system that will use it. Copy the key: it is shown once. If you have no sign-in, ask the owner for a key. See Keys and access.
2. Send a document. One request: wait=60 holds it until the read is done, and forget=1 drops the result from the server once you have it. A read that takes longer answers 202 with an id; fetch it with GET /v1/jobs/{id}.
# OCR_API_KEY holds your key. Keep it out of files and URLs.
curl -s -H "Authorization: Bearer $OCR_API_KEY" \
-F file=@card.png \
"https://docread.srv1283448.hstgr.cloud/v1/read?wait=60&forget=1"
# 200 {"id":"…","state":"done","result":{"fields":[…],…},…}# 202 {"id":"…","state":"reading"}: not done within 60 s; then# curl -s -H "Authorization: Bearer $OCR_API_KEY" "https://docread.srv1283448.hstgr.cloud/v1/jobs/<id>?wait=60&forget=1"
import { readFile } from 'node:fs/promises';
const BASE = 'https://docread.srv1283448.hstgr.cloud/v1';
const auth = { authorization: `Bearer ${process.env.OCR_API_KEY}` };
const form = new FormData();
form.append('file', new Blob([await readFile('card.png')]), 'card.png');
let res = await fetch(`${BASE}/read?wait=60&forget=1`, { method: 'POST', headers: auth, body: form });
let job = await res.json();
if (!res.ok) throw new Error(job.error.message);
// not done within 60 s: wait on the job itself, as long as it takes
while (job.state === 'queued' || job.state === 'reading') {
job = await (await fetch(`${BASE}/jobs/${job.id}?wait=60&forget=1`, { headers: auth })).json();
}
if (job.state === 'failed') throw new Error(job.error.message);
for (const f of job.result.fields) console.log(f.label, '=', f.value);
import os
import requests
BASE = "https://docread.srv1283448.hstgr.cloud/v1"
auth = {"Authorization": f"Bearer {os.environ['OCR_API_KEY']}"}
with open("card.png", "rb") as f:
res = requests.post(f"{BASE}/read", params={"wait": 60, "forget": 1},
headers=auth, files={"file": f}, timeout=90)
job = res.json()
res.raise_for_status()
# not done within 60 s: wait on the job itself, as long as it takes
while job["state"] in ("queued", "reading"):
job = requests.get(f"{BASE}/jobs/{job['id']}", params={"wait": 60, "forget": 1},
headers=auth, timeout=90).json()
if job["state"] == "failed":
raise RuntimeError(job["error"]["message"])
for field in job["result"]["fields"]:
print(field["label"], "=", field["value"])
Output, for a sample card
Name = Sample Person
ID number = 784-1990-1234567-6
Date of birth = 01/01/1990
Nationality = Testland
Expiry date = 31/12/2030
The page's own API, /api, reads the same way but is for the page: it needs a signed-in browser, keeps results for an hour and can follow a job as events (Live progress). Other servers use /v1.
queued→reading→doneorfailed
Two documents are read at once; the rest wait in order. A queued job reports its position, and a reading job reports pagesDone of pagesTotal; its events also carry each page as it is read (see Live progress). On the VPS (4 CPU cores) an ID card photo takes about 30 seconds, a passport photo about 40, and a two-page scanned passport about 35.
A finished job is kept for up to an hour (less when results pile up past 512 MB), then answers 404. Nothing is written to disk: the upload is dropped as soon as its read ends.
Instead of polling a job, follow it. GET /api/jobs/{id}/events sends each step of the read as it happens, as server-sent events. Each page comes as soon as it is read, so the first page of a long scan shows in seconds, not when the last one is done.
Event
Data
When
queued
{ position }
On connect while the job waits, and again each time its place changes. 1 is next.
page
{ page, pagesDone, pagesTotal }
As each page is read. page is the whole page, as in result.ocr.pages[i], its image included.
type
{ documentType }
Right after page 1, when the server works the type out from it. The other pages are read meanwhile.
The read is finished. result is the one GET /api/jobs/{id} returns.
description
{ description }
With descriptions set up and a read with text: pending straight after done, then done or failed once it is written.
failed
{ error: { code, message } }
The read failed (unreadable, unavailable or internal), or the job was deleted or expired (gone).
end
{}
Last: after done and the description, or after failed. The server then ends the response.
The stream
Each event is three lines and a blank one: its id (from 1, one per event, per job), its name, and its data as one line of JSON. A comment line, : ping, goes out every 15 seconds, so a quiet read is not taken for a dropped connection.
On connect, every event so far comes first, in order, so a stream opened late misses nothing; of the queued events only the latest is sent again. With a Last-Event-ID header, which an EventSource sends when it reconnects, only the events after that id come. An EventSource reconnects whenever a response ends, so close it on end. A reconnect that has had every event gets 204, which stops it.
Every request's checks apply (see Access). As the stream carries the document, a browser page must also be this server's own, as for a POST. At most 8 streams can be open on one job; the next gets 503 busy. GET /api/jobs/{id} is unchanged, so polling still works, and is the fallback when a stream fails.
Over plain HTTP, as on a local run, a browser keeps at most six connections to one server, and each open stream holds one. Follow a few jobs at a time and poll the rest. Through Caddy (HTTPS) the streams share one connection.
// on a page this server serves
const source = new EventSource(`api/jobs/${id}/events`);
source.addEventListener('page', (e) => {
const { page, pagesDone, pagesTotal } = JSON.parse(e.data);
showPage(page); // its lines, boxes and image
});
source.addEventListener('done', (e) => showResult(JSON.parse(e.data).result));
source.addEventListener('end', () => source.close());
source.addEventListener('error', () => {
// before end: close it and poll GET api/jobs/{id} instead
});
// Node has no EventSource: read the lines. The server ends each with \n.
const res = await fetch(`${BASE}/api/jobs/${id}/events`);
const decoder = new TextDecoder();
let buffer = '';
for await (const chunk of res.body) {
buffer += decoder.decode(chunk, { stream: true });
for (let end; (end = buffer.indexOf('\n\n')) >= 0; buffer = buffer.slice(end + 2)) {
const lines = buffer.slice(0, end).split('\n');
const event = lines.find((l) => l.startsWith('event: '))?.slice(7);
if (!event) continue; // a ping
const data = JSON.parse(lines.find((l) => l.startsWith('data: ')).slice(6));
if (event === 'page') console.log(`page ${data.pagesDone} of ${data.pagesTotal}`);
if (event === 'done') console.log(data.result.fields);
}
}
# -N: print each event as it comes
curl -sN http://127.0.0.1:8780/api/jobs/6c90f5bd-9a31-449d-a4bf-1d74dff49f2d/events
# after a dropped connection, only what came after event 3
curl -sN -H 'Last-Event-ID: 3' http://127.0.0.1:8780/api/jobs/6c90f5bd-9a31-449d-a4bf-1d74dff49f2d/events
200 · a made-up two-page PDF, shortened. Page 1 came after 4.9 s, page 2 after 12.6 s, the rest at once.
An API key, as Authorization: Bearer <key>. See Keys and access.
A person
The page, and its /api
A username and password, at /login. The browser keeps a session cookie.
Accounts. There is no sign-up. An admin adds people on the account page (/account, Users): each new user gets a password shown once, and changes it under Password. Admins see and revoke every key; a user sees their own.
Sessions. The cookie is HttpOnly, SameSite=Strict and Secure over HTTPS. A session ends after 12 hours unused or 7 days in all, on Sign out, when the password changes, when the user is removed, and when the service restarts. Ten wrong passwords from one address in 15 minutes lock that address out for the rest of the 15.
Dashboard./dashboard shows your own usage over 7, 30 or 90 days: documents and boards read, API calls, read times, document types, how board fields came out, and your keys. An admin sees everyone, can pick one person, and sees the server: the engines, the queue, load, memory and who is signed in. Only counts and timings are kept for it (usage.json beside the accounts): never a document, a value read or a key. The same numbers are at GET /api/usage?days=7|30|90 (and &user= for an admin) and GET /api/system (admins).
Open without signing in: the sign-in page, /api/health, /docs/ and /docs/openapi.yaml. Anything else on the page's side answers 401 unauthenticated (a page sends the browser to /login).
In a shell on the VPS itself, the page is at http://127.0.0.1:8045 and the service API at http://127.0.0.1:8047/v1, without Caddy. A process there that calls http://127.0.0.1:8045 directly, such as the WhatsApp bridge, needs no session; keys and users always need a person signed in. Set DOCREAD_TRUST_LOCAL=0 to make local callers sign in too.
Two checks guard every request. The Host header must be one of the server's own names: localhost, 127.0.0.1, [::1], or a name listed in DOCREAD_PUBLIC_HOSTS. On the VPS that is docread.srv1283448.hstgr.cloud, the name Caddy passes on; without it, every request through Caddy gets 403. A POST or DELETE that carries a browser Origin must come from the same origin, host and port, which in practice means a page this server served. A browser request marked cross-site or same-site is refused. Calls without an Origin, such as curl or a backend service, pass. Anything else gets 403 forbidden.
200 while the web server runs, for a request that names one of its own hosts; any other Host gets 403 forbidden (see Access). A method other than GET gets 405, and a server failure 500. ocr and grouping say whether each engine answers. Without the grouping engine, grouping still runs on identifiers alone.
describe is there only when descriptions are set up: whether the model's proxy answers. It does not ask the model itself.
Send multipart/form-data. Accepted: PDF, PNG, JPEG, WebP, TIFF and BMP, up to 25 MB, with up to 20 pages read. Only the first page of a multi-page TIFF is read. An image over 60 megapixels is accepted, then its job fails as unreadable. A PDF page with a text layer is read from that layer unless forceOcr is on.
A .zip, up to 100 MB, becomes one job per document inside it. Entries that cannot be read, such as a nested zip, a spreadsheet or an encrypted file, come back in skipped with the reason, and so do entries over 25 MB, past the 50th document, past 100 MB of documents unpacked, or past 200 MB read in all. Folders, archive litter (__MACOSX, Thumbs.db, desktop.ini) and any file or folder whose name starts with a dot, even a PDF, are dropped without a mention. Each job keeps its folder, which is often the one thing that tells two people's passport.pdf apart.
A zip is recognised by its first bytes, not its name. A .docx, .xlsx or .odt is a zip, so it is opened as one: the answer is 202, its parts are listed in skipped, and any PNG or JPEG inside it becomes a job. A body over 25 MB is accepted only when it is a zip.
Form field
Type
Meaning
file
file
The document or .zip. Send it as the first part.
forceOcroptional
text
1, true, on or yes OCR every PDF page, even one with a text layer. Anything else leaves it off.
202 · one document
{ "id": "6c90f5bd-9a31-449d-a4bf-1d74dff49f2d" }
202 · a .zip
{
"jobs": [
{ "id": "d89191b4-0493-4ce4-afb0-c92de0d6f629", "filename": "sample-id-card.png", "folder": "alex" }
],
"skipped": [
{ "filename": "notes.txt", "folder": "", "reason": "not a PDF or a supported image" }
]
}
Errors: 400 bad_request, 403 forbidden, 405 method_not_allowed, 413 too_large, 422 unsupported or unreadable (a file that starts like a zip but cannot be opened), 500 internal, 503 busy. A file that is not a document is refused at once, before it waits in the queue.
State, progress and result
result is filled only when state is done, and error only when it is failed. A failed job's error.code is unreadable, unavailable or internal. The full result is described under Read result, its templates under Templates, and its description under Descriptions.
With descriptions set up, a done job may still have "description": { "state": "pending" }: the description is written after the read. To have it, keep polling the done job, every 2 seconds is enough, until description.state is done or failed, and delete the job after; or follow the job's events, where it comes as a description event (see Live progress). A result with no description has none coming.
Each page carries image, a display JPEG as a data URL (null once the document's images reach 12 MB, or when the page cannot be drawn, with imageNote saying why), so the result can be several megabytes. If you only need values, read fields, ownership and templates.
A queued job never runs. A job being read stops before its next page. A finished result is dropped, and a description still waiting or being written for it is stopped. A stream following the job gets a failed event with codegone, then end. Answers 204, or 404 if the job is already gone.
Group documents by person
Send each finished document's ownership as JSON, never its pages, images or OCR text: those are refused with 400. Any other key a document carries, such as fields, is ignored. You get back who is who and the evidence for every link.
If the caller disconnects, the service stops asking the name model on its behalf.
docs[] key
Type
Meaning
id
string · number
Unique within the request, up to 200 characters. A number is turned into a string. The job id works well.
filenameoptional
string · null
result.filename, up to 300 characters. Not used for grouping.
ownershipoptional
object · null
result.ownership as read. Without it the document is not grouped.
Errors: 400 bad_request (the message names the offending docs[n] where there is one), 403 forbidden, 405 method_not_allowed, 413 too_large, 500 internal (including a name-model error that is not an outage).
The owner's document templates
Every template the service has loaded from its templates/ folder, in file-name order and, within a file, in the order the file lists them. A file that does not parse is left out, and the reason goes to the server log. No folder means { "templates": [] }. What a template is, and how to make one, is under Templates.
POST /v1/address-boardA photo of a Qatar address board in; building, zone and street out.
Every /v1 request needs a key. Make one for each system that calls the service, so each can be revoked alone.
Make a key
Sign in at /login and open API keys (/account).
Choose New key, name it after the system that will use it, and choose Create.
Copy the key, ocr_ and 43 more characters. It is shown this once: the server keeps only its SHA-256 hash, so a lost key is replaced, not recovered.
The list shows each key by name and its first characters, with when it was made and last used. A key works the moment it is made, and stops the moment it is revoked: no restart. Removing a user revokes their keys. On the server's shell, node tools/users.js key <user> <name> makes one too.
Keep the key on your server, in its secret store. A request that carries an Origin header, as a browser sends, is refused with 403 even with a good key: no web page may hold one.
Rotation. A user may have up to 20 keys. Make a new one, switch the client to it, check its Last used, then revoke the old one.
Who sees what. A job belongs to the user whose key made it. Anyone else asking for its id, and the page, get 404, the same as for an unknown id. Results are kept for 10 minutes.
Status
Code
When
401
unauthorized
No key, or one the server does not know (revoked, mistyped). Comes with WWW-Authenticate: Bearer.
403
forbidden
The request has an Origin header.
404
not_found
No such job for you, or a path outside /v1.
429
busy
You already have 4 requests open, 8 documents waiting or being read, or 2 uploads arriving. Wait Retry-After seconds.
429
rate_limited
You have sent 60 documents in the last minute. Wait Retry-After seconds.
503
no_keys
The server has no keys at all yet.
503
busy
Too many documents waiting, or too many uploads arriving at once. Wait Retry-After seconds.
The other codes are as on /api; see Errors. The server logs one line per request: time, user, method, route, status, time taken and size. Never the key, a file name or anything read.
Reads on /v1 use credits from the account the key belongs to. A new account starts with 50.
Request
Credits
POST /v1/read
2
POST /v1/address-board
1
Fetching a job, feedback, health, GET /v1/credits
0
Credits are taken when the work starts and given back when it does not happen: a read that fails, one deleted before it was done, a board the reader could not take. Every charged answer says what is left in X-Credits-Remaining. With too few, the answer is 402 out_of_credits and nothing is read. Admin accounts are not charged.
Buying more. Signed in, open API keys (/account): the Credits section sells packs through Dodo Payments' checkout, and the credits are added as soon as the payment goes through. An admin can also add credits to an account by hand.
Read one document, and wait for it if you like
Send multipart/form-data with the document in file: a PDF or an image (PNG, JPEG, WebP, TIFF, BMP), up to 25 MB, with up to 20 pages read. A .zip is refused with 422; send one document per request.
With wait the request holds until the read finishes, for up to that many seconds. Finished in time, the answer is 200 with the job, the same as GET /v1/jobs/{id}. Not finished, it is 202 with the id, and you fetch the job later. If you hang up while waiting, the read is cancelled. On the VPS an ID card takes about 20 to 30 seconds, a scanned page about a minute.
Parameter
In
Meaning
file
form
The document.
forceOcroptional
form
1 to OCR every PDF page, even one with a text layer.
describeoptional
form
1 for a short description written by the server's own model (see Descriptions). It comes after the read: result.description.state is pending until it is done or failed, so fetch the job again. Off by default.
waitoptional
query
Seconds to wait, 0 to 60. Default 0: answer at once.
imagesoptional
query
1 to include each page's display image. Off by default, which keeps a result to a few kilobytes.
forgetoptional
query
1 to forget the job once it has been returned finished (with its description, if one was asked for).
The result is the same as on /api (see Read result), except that pages carry no image unless images=1, and there is no description unless describe=1. A read that fails is not an HTTP error: the job is failed with error.codeunreadable, unavailable or internal.
Errors: 400 bad_request (also a bad wait, images or forget), 401 unauthorized, 403 forbidden, 413 too_large, 422 unsupported, 429 busy or rate_limited, 500 internal, 503 busy or no_keys.
Fetch a job, wait for it, or forget it
GET answers the job: id, state, pagesDone, pagesTotal, result and error, as GET /api/jobs/{id} does. With wait (0 to 60 seconds) it holds until the job finishes, so there is no need to poll. With forget=1 a finished job is forgotten once it has been returned.
Building, zone and street off an address board, in one request
For the blue address board on a building in Qatar. Send a photo of it as file in multipart/form-data (PNG, JPEG, WebP, TIFF or BMP, up to 10 MB). The answer comes when it is read, usually in a few seconds: no job, no polling, and nothing is kept.
The board prints each number twice, in Western and in Arabic-Indic digits. The reader finds the labels (Building No. / رقم المبنى, Zone / منطقة, Street / الشارع), reads the number under each, and checks one copy against the other. verified is true only when both copies say the same. Show an unverified value to a person to check; a field it cannot find is null.
On a worn board, where the paint is gone and only the digits' raised outlines are left, the Arabic-Indic copy rarely reads. There a Zone or Street number is read from its Western copy alone, under the English label, and only when two cuts of it agree with high confidence. It comes back unverified.
Values never run longer than the field can be in Qatar: two digits for a zone, four for a street or a building. A longer reading means the Arabic-Indic copy was read as Western lookalikes after the number; the first digits are kept, unverified.
When a person has checked the numbers, send what they confirmed with the board's id, within 7 days: POST /v1/feedback, as JSON. Leave out a field nobody checked; send "" for one the board does not have. Each board is checked once, by the key that read it. The answer says how each field came out: right, wrong, missed (the reader found nothing) or skipped. The dashboard's accuracy is made of these.
No number is kept for this: the board's answer waits as a hash of each number, salted with the board's id, and only whether each one was right is counted. 404 not_found: an id unknown, someone else's, checked already or older than 7 days.
Values are strings of Western digits, as the board's Western copy reads. On 120 made-up boards in many fonts, photographed at an angle and blurred, all 120 came back with all three right, and 57 of 60 worn ones; every value marked verified was right.
Errors: 400 bad_request (no file), 401 unauthorized, 403 forbidden, 413 too_large, 422 unsupported (not a photo, or a .zip), 429 busy or rate_limited, 503 busy, unavailable (the OCR engine is not answering) or no_keys.
LimitsSizes, counts and how long results are kept.
The service never writes a value. Every field and owner name points at the OCR line it was cut from, so a caller can always check it, and can draw its box on the page image. The one text in a result that is written, not cut, is the optional description, and it is labelled as the model's.
value === ocr.pages[page - 1].lines[line].text.slice(span[0], span[1]). A field that ever fails this is dropped, not patched. span is [start, end) in UTF-16 code units (JavaScript string indexes), so count characters the same way in other languages. bbox is [x0, y0, x1, y1] in the page's own unit: pt for a PDF page read from its text layer, px for an image or an OCRed PDF page (which also gives dpi), matching the page image.
Key
Type
Meaning
filename
string
The upload's name, without any path.
fields
Field[]
The lines that hold a named value. Empty when nothing is certain.
ownership
object · null
Whose document it is, its identifiers and any other parties. Null when that could not be worked out (see ownershipError).
templates
TemplateFill[]
The owner's templates this read fills at least one row of, best first. Empty when none fits. See Templates.
descriptionoptional
object
A few sentences about the document, written by a model: may be wrong. Only when descriptions are set up, and only for a read with text. See Descriptions.
ocr
object
Every line on every page, with box, confidence and script.
ms
integer
Read time in milliseconds.
folderzip only
string
Where the document sat in the .zip.
ownershipError, fieldsErrorrare
string
That step failed. The read itself is still good. With fieldsError, fields is empty.
Field
Key
Type
Meaning
key
string · null
Catalogue key: name, surname, given_names, id_number, passport_number, date_of_birth, nationality, expiry_date, email, occupation and more. Null for a printed label outside the catalogue.
label
string
The field's English name, or the words the document prints.
value
string
Cut verbatim from its line.
page, line, span
int, int, [int, int]
Where the value is. page starts at 1, line at 0.
labelAt
object · null
Where the printed label is, when there is one.
via
string
label, mrz, identifier or owner: what named the value.
verified
true · false · null
True when a check digit passed: passport MRZ, Emirates ID (Luhn), IBAN (mod 97), VIN. False when the machine-readable zone on the same page gives a different number or date. Otherwise null.
confidence
number
The value's line confidence, 0 to 1, to 3 decimals: the OCR engine's for an OCR line; 1 for a PDF text-layer line, 0.5 when its right-to-left text is in visual order.
Document type
ownership.documentType says what kind of document this is, worked out from its first page: { type, label, probability, via, evidence }. Rules come first: cue words and phrases in English and Arabic, a title weighing more than the body ("Statement of Account" at the top, not a mention of an invoice in a receipt's text), and a machine-readable zone deciding a passport or an ID card outright. via is rules and evidence quotes what decided it, as printed: title: Statement of Account, كشف حساب (title), text: Without Prejudice or MRZ (TD3). A tie, or too little to go on, is unknown, with evidence null.
For an unknown, the service can ask Laya, the local model, to choose a type from the file name and the top of the page. Its choice would come back with via: laya, its probability, and evidence such as Laya · 91%, but only at or above a probability measured to keep wrong answers out. On the made-up test documents no probability did (wrong answers came back at 1.00), so for now Laya's choice is not used and the rules decide alone.
voucher (receipt or payment voucher, سند قبض / سند صرف)
payee; whoever paid is the other party
salary-certificate, payslip, employment-contract
employee
offer-letter
candidate: the addressee ("To:", "Dear …", "السيد/ …")
cv
candidate: a labelled name, else the full name the CV opens with
tenancy-contract
tenant
certificate
student
lab-report
patient; never the doctor
legal-letter
client: only the client it names ("Client:", "our client, …", "موكلنا"), else nobody
article
nobody: owner is null
letter, other, unknown
other, when a label names one
Owner
ownership.owner is the holder, employee or vehicle owner the document is about, never its issuer, employer or sponsor; its role follows the document type (see the table above). It is found by rules, not a model, and via says which: mrz (the machine-readable zone), label (a subject label such as "Name:" or "الاسم"), twin (the same name printed in Arabic and Latin letters), or top (on a CV, the full name it opens with). nameAlt is the name in the other script. Only identifiers with belongsToOwner: true count for grouping. An identifier's check names the check applied (luhn, iban-mod97 or icao-check-digit) whether or not it passed; verified says if it passed. check is null for emails, phones and labelled numbers.
Line
text as the PDF embedded it or the engine read it, Unicode-normalised (NFKC) with invisible direction marks, tatweel and extra spaces removed, and right-to-left text put in reading order where possible; spans count against this text, not the raw PDF text. Each line also has confidence 0 to 1 (the engine's for an OCR line; 1 for a text-layer line, 0.5 when its right-to-left text is in visual order), bbox, page, and lang (en or ar). An OCR line may carry alt, the other script's reading. ocr.text is every line's text in the same order: lines joined by \n, pages joined by a form feed (\f). ocr.source is text-layer, ocr or mixed, and truncated is true when pages were left unread: past page 20 of a PDF, or after the first page of a multi-page TIFF.
A template is one of the owner's document types: "ID card" with a name, an ID number, a date of birth, a nationality and an expiry date. Every read fills each template it can from its fields and identifiers, and result.templates lists those it fills at least one row of, best first. The page shows the best one under Details on its Summary tab, with a picker for the others.
Templates are filled by rules, not a model, and a value is never rewritten. A row's value is the field's or identifier's own, with its page, line and span, so it can be checked against its line like any field (see How values are cited). A value of the wrong type is left out, never repaired.
Make one in the Ontology Playground
1. Design
In Microsoft's Ontology Playground (microsoft.github.io/Ontology-Playground), add an entity type for each kind of document, named as people call it ("Residence visa"). Give it a property for each value to fill, named as documents print it: fullName, passportNumber, dateOfIssue. Set a date's type to date and a number's to integer or decimal, and mark an ID number as the identifier.
2. Export
Export the ontology as RDF/XML. Each entity type with at least one property becomes a template; relationships are kept but fill nothing.
3. Add
Put the .rdf file in the service's templates/ folder. A running service reads it on the next read, with no restart, and again whenever it changes. A file that does not parse is left out and logged; the others still load. GET /api/templates lists what loaded.
4. Deploy
On the VPS: commit the file and deploy as usual. Only the committed branch is sent.
The service ships with starter files: banking.rdf (Bank statement), business.rdf (Invoice, Letter), employment.rdf (Offer letter, CV), identity.rdf (ID card, Passport), legal.rdf (Legal letter) and medical.rdf (Lab report, its common tests one row each). They open in the Playground too. Where a row should take a value the read names otherwise, a starter file adds a skos:altLabel: a "Job Title" fills an offer letter's position, and its "Date" the offer date.
Arabic labels
The Playground names a property once. To match a label printed in Arabic, add it by hand under the first label, with xml:lang="ar"; a skos:altLabel works too. Keep the English label first: the Playground shows the first label, and drops the others the next time it exports the file, so add them after the last export.
A property takes the value of a field whose label, or catalogue key, is the property's label once both are normalised: case, camelCase and snake_case, and punctuation do not count.
dateOfBirth = "Date of birth" = date_of_birth
A few words are read as one: "No.", "num" and "#" as "number", "DOB" as "date of birth", "E-mail" as "email", and "Expiry Date" as "date of expiry".
An Arabic label is compared with the letter forms two renderings disagree on folded together (أ إ آ and ا, ة and ه, ى and ي).
رقم الهوية = رقم الهويه
An identifier property also takes an identifier the read found, by its kind or by the label printed beside it.
passportNumber, idNumber or emiratesId, iban, email, phone; "Policy No." for policyNumber
The value must have the property's type. A date takes only a date, a number only a number, a yes/no only yes or no, a list only one of its values. Otherwise the row stays empty, with a note.
31/12/2030, 2030-12-31, 31 Dec 2030, ٣١/١٢/٢٠٣٠
Of several values, a verified one comes first, then the earliest on the page.
A passport number the MRZ confirms
Templates are ranked by the rows they fill, then by the share of their rows filled, then by the order they were loaded in.
TemplateFill
Key
Type
Meaning
id
string
The template's IRI, as its file names it.
label
string
Its name, such as "ID card".
filled, total
int, int
Rows with a value, and rows in all.
rows
Row[]
One per property, in the file's order.
Row
Key
Type
Meaning
property, label
string, string
The property's IRI and its name.
type
string
string, integer, decimal, double, date, datetime, boolean or enum, as the Playground sets it.
value
string · null
The field's or identifier's value, verbatim. Null when nothing fills the row.
page, line, span
int, int, [int, int] · null
Where the value is, as for a field.
via
string · null
field or identifier: what the value came from.
verified
true · false · null
As the field's or identifier's: true when a check digit passed, false when one failed or the MRZ disagrees.
noterare
string
Why an empty row is empty when a value was found: "N/A" is not a date.
When the owner sets it up, every read with text also gets a description: two to four sentences of plain English on what kind of document it is, who or what it concerns, and the dates and numbers it states. It is result.description, and the page shows it under the counts.
A description is written by a model, and may be wrong. It is the one text in a result not cut from a line, so check it against the text. It never changes a field, the owner, a template or grouping, and nothing is worked out from it. The page and the WhatsApp replies label it as the model's.
What runs where
A small open-source model of the owner's choice runs on the same server under Ollama, behind the LiteLLM proxy. Both listen on loopback only, and both are set up apart from this service. The service sends the proxy the read's text page by page, cut at the end of a line past 6,000 characters, with the file name and, as hints the model is told may be wrong, the document type and owner's name the rules found. No image is sent. The service will not start with a proxy address off loopback, so nothing leaves the server.
A model that thinks before it answers takes minutes on a small CPU. Turn its thinking off with DESCRIBE_EXTRA_BODY: Qwen3.5 takes {"think": false}, and an instruct model such as qwen3:4b-instruct needs nothing. Any thinking that comes back anyway is cut from the answer.
Settings
Read from the service's environment at start. Without DESCRIBE_URL or DESCRIBE_MODEL, nothing is described: no result has a description, and /api/health has no describe. A setting that is there but wrong stops the service at start, with a message naming it.
Setting
Meaning
DESCRIBE_URL
The LiteLLM proxy, without /v1, such as http://127.0.0.1:4000. Only http on 127.0.0.1, localhost or [::1].
DESCRIBE_MODEL
The model's name in the proxy's configuration.
DESCRIBE_KEYoptional
The proxy's master key, when it has one. Sent as a bearer token with each description, never logged or returned. Keep it in the service's environment file, not in the project.
DESCRIBE_TIMEOUT_MSoptional
How long one description may take: 1000 to 600000 milliseconds, 120000 by default.
DESCRIBE_EXTRA_BODYoptional
A JSON object added to each request as it is, such as {"think": false}. It may change temperature (0.2) or max_tokens (220), not model, messages or stream.
When it is written
pending→doneorfailed
The job turns done as soon as the read is done, with the description pending. Descriptions are then written one at a time, in the order the reads finished, because the model shares the server's CPU with the OCR engine. To have one, keep polling the done job every 2 seconds or so until description.state is done or failed, then delete the job; or follow the job's events, where it comes as description events (see Live progress). The page waits up to 3 minutes. The WhatsApp bridge replies as soon as the read is done, and sends the description as a message of its own once it is written.
At most 20 wait behind the one being written; past that a description fails at once with "Too many documents are waiting to be described." Deleting a job stops its description, and so does the job expiring. A read that found no text gets no description, and nothing is sent to the model.
Description
Key
Type
Meaning
state
string
pending, done or failed.
textdone
string
The description: plain text, no markdown, at most 1,200 characters. Arabic in the document is put into English; names are kept as written.
modeldone
string
The model that wrote it, as DESCRIBE_MODEL names it.
msdone
integer
How long it took, in milliseconds.
errorfailed
string
Why, in a short sentence fit to show, such as "The description model is not answering." or "The description model took too long." Details stay in the server log.
result.description · done
{
"state": "done",
"text": "An employment letter for Sample Person, who works as an engineer. It gives Emirates ID 784-1990-1234567-6 and the email address sample.person@example.com.",
"model": "qwen3-4b-instruct",
"ms": 6400
}
Putting two different people together gives a complete, plausible and wrong person. Splitting one person in two is visible and harmless. So every rule leans toward keeping documents apart, and heldApart says why a link was not applied.
Tier
Links documents when they share
proven
A checksum-verified Emirates ID, IBAN or passport number.
likely
A personal email address (a shared one such as info@ never links), a passport number with no check digit (one whose check failed never links), a traffic code or a personal labelled number.
names
An owner name, when the name rules and the local name model both agree (p ≥ 0.8 both ways). Reported as likely. Below 0.5 either way vetoes any link between the two.
never alone
Phone numbers and group insurance policy numbers. A shared one that belongs to the owner is listed under supporting; others are dropped.
Name questions have a time budget: 20 seconds by default, which the VPS uses. NAME_BUDGET_MS in the server's environment changes it. Pairs not asked are counted in nameQuestions.cut and pairsSkipped, never dropped silently. judged is false only when names had to be judged and the model was not available.
The web page's Grouping & matching view shows people as cards or as a graph. Its Download graph button saves people.ttl: the latest /api/match answer and every finished document, as RDF in Turtle. The page builds it from what it already holds; no endpoint serves it.
Node
IRI
Holds
Person
person/{key}
The name grouping chose, and the same name in the other script. hasIdentifier and hasLink.
Document
doc/{key}
File name, document type, page count, the owner's name as printed. ownedBy, carries, mentions and differentOwnerFrom (a "Different people" pair).
Identifier
id/{kind}/{normalized}
Kind, every printing of it, the normalized value, verified. One per kind and value, so the documents that carry it share it.
Link
link/{person}/{n}
One link from people[].links: the two documents it joins, its tier, reason and evidence.
Party
doc/{key}/party/{n}
One of the document's otherParties, with the label it was printed under.
IRIs sit under http://example.org/ontology/docread/ and are made from the page's own document keys and from normalized identifier values, percent-encoded, never from a name. Names and values are written exactly as read; Arabic ones are tagged @ar. A key such as d1 lasts as long as the page is open.
people.ttl, for the two sample documents (an excerpt)
The classes and properties are described in docs/ontology/docread.rdf in the project, in the RDF/XML that Microsoft's Ontology Playground imports. Open it there to see the model and change it. The service does not serve that file.
Every error has the same shape. Branch on code; message is a sentence for a person and may change.
Error body
{ "error": { "code": "unsupported", "message": "\"notes.txt\" is not a PDF or a supported image (png, jpeg, webp, tiff, bmp)." } }
Status
Code
When
400
bad_request
Not multipart, no file field, invalid JSON, a docs[n] in the wrong shape, or pages, images or OCR text sent to /api/match.
401
unauthenticated
The page's /api without a signed-in session. See Access.
401
wrong_login
POST /api/login with a wrong username or password. The two read the same, on purpose.
Unknown Host, or a browser page from any other origin (another port counts). See Access.
404
not_found
No such job, or it has expired. Also any unknown path.
405
method_not_allowed
Wrong method. The Allow header lists the right ones.
413
too_large
The upload or request body is too big, or /api/match has too many documents.
422
unsupported
Not a PDF, a supported image or a zip.
422
unreadable
A file that starts like a zip but cannot be opened.
402
out_of_credits
/v1: the account has too few credits for this read. See Credits.
429
rate_limited
Too many wrong passwords from one address, or, on /v1, too many documents a minute. Wait Retry-After seconds.
503
busy
Too many documents waiting, 2 uploads already arriving, or 8 event streams already open on the job. Retry shortly.
500
internal
The server itself failed, on any route; on /api/match also a name-model error that is not an outage. Retry. Details stay in the server log.
A read that fails after upload is not an HTTP error: GET /api/jobs/{id} answers 200 with statefailed and error.codeunreadable (a damaged PDF or image, or an image over 60 megapixels), unavailable (the OCR engine is not running) or internal.
What
Limit
One document
25 MB, 20 pages read
One image
60 megapixels
One .zip
100 MB, 50 documents, 100 MB of documents unpacked, 200 MB read in all
Documents read at once
2
Documents waiting
50, plus 10 for a .zip
Uploads received at once
2
Results kept
1 hour, less past 512 MB held
Event streams on one job
8 at once; a ping every 15 seconds
/api/match request
500 documents, 24 MB
Name pairs judged in one match
500
One template file
2 MB
Descriptions written at once
1, with 20 waiting
Text sent for one description
6,000 characters, cut at a line
One description
1,200 characters; 2 minutes to write it (DESCRIBE_TIMEOUT_MS)
Service API (v1)
What
Limit
One document
25 MB, 20 pages read; no .zip
wait
60 seconds
Results kept
10 minutes; 64 MB of results per user, oldest first
Per user
4 requests open, 8 documents waiting or being read, 2 uploads arriving, 60 documents a minute
All of /v1
25 documents waiting or being read, 3 uploads arriving; the page keeps the rest of the queue
Keys
20 per user
/v1/address-board
10 MB a photo; 2 boards read at once; each counts as one document a minute
Signing in
What
Limit
Wrong passwords
10 per address, 50 per username, in 15 minutes
A session
12 hours unused, 7 days in all
A password
10 to 256 characters
Users
100
Past the .zip's document, unpacked-size or 200 MB read limit, or 25 MB for one entry, the extra entries come back in skipped; the rest are read. Name pairs past 500 are counted in nameQuestions.cut.cap.