Diabetic retinopathy screening
with AI 100% on your device

0 · Context — what problem we solve

DIRD+ is designed primarily as a human-in-the-loop (HITL) tool: a medical technologist or ophthalmologist works with the model, not replaced by it. The model flags, suggests and prioritizes; the human reviews, validates and decides. As a secondary mode, it can run as autonomous screening when context requires it (rural posts, mass screening, no specialist on site).

This validation evaluates the autonomous mode — the model alone, unassisted — because that is the setup measurable against APTOS labels. It is deliberately the most demanding scenario: the worst case from the product's standpoint, since the real use case (HITL) adds clinical judgment on top of the model output. The operational question answered here is the binary screening one: does this image need an ophthalmologist's attention, yes or no?

To validate that we used APTOS 2019 Blindness Detection (Kaggle, n=3,662 images), labeled with the International Clinical Diabetic Retinopathy (ICDR) scale, grades 0 to 4:

We binarize the ground truth: grade 0 → NORMAL, grades 1-4 → PATHOLOGICAL (1,805 healthy vs 1,857 pathological, 49.3% / 50.7% — balanced). The validation task reduces to "given the detector output, which confidence thresholds best separate healthy from pathological?". That is the question the whole experiment chain pursues — and PCT_fpr02 is the calibrated answer.

1 · What is PCT_fpr02?

PCT = Per-Class Thresholds. FPR02 = False Positive Rate ≤ 2% measured on normal fundi.

Instead of applying the same confidence threshold to every lesion class (e.g. uniform 0.25), each lesion has its own threshold, calibrated so that the false positive rate on healthy retinas stays below 2% per class.

Decision rule: ABNORMAL ⇔ ∃ class c with score(c) ≥ τ[c]. If only anatomical landmarks (optic disc, fovea) or nothing is detected, the image is NORMAL.

2 · How did we get there?

It wasn't an a-priori choice. Six chained experiments, each answering the question the previous one left open:

3 · Master table — all points compared

4 · Alternative modes by use case

5 · Scope of this validation

What this experiment measures is DIRD+'s autonomous mode — the model operating alone against APTOS ground truth, unassisted. It's the most demanding scenario for the product, and the numbers hold up: Sens 97.8% / Spec 93.1% / MCC 0.91 at ~9 FPS on CPU, with τ stable under 5-fold CV.

What this experiment does not cover:

• Human-in-the-loop mode: the product's primary use case. Its validation is a separate clinical study — technologists labeling with and without AI, decisions with and without the algorithm, verifier as adjudicator. Another story, planned, not part of this technical validation.
• Cross-cohort: validated on APTOS 2019 (India) and externally on DDR (China, n=12,522, raw) and Messidor-2 (France). The frozen APTOS τ transports across populations (ΔMCC≈0); out-of-domain AUC is 0.84 (DDR) — a real, quantified drop from the 0.95 in-distribution figure.
• Clinical granularity: the operational output is binary "needs attention yes/no". ICDR 0-4 staging is a distinct task, owned by the specialist.
• Image-level metrics: APTOS provides no lesion-level annotations, so validation is reported as image-level binary classification — consistent with the use case.

In short: this validation covers the floor of the product (model alone). The ceiling — performance in real HITL flow — is measured in a dedicated clinical study.