AI medical chronology accuracy, and how it is measured

Every figure on this page comes from test files with a known answer key, graded by code. None comes from a client’s files. The figures were measured on the previous version of Brennick.

Each test file comes with an answer key: every date of service, visit, test and procedure in it, and the page it is on. Sixteen medical case files, from a few pages to 406 pages. The keys are written before Brennick reads a page, so nothing it produces can shape them.

Brennick reads each file the way it reads any other, and code compares the output with the key. A citation counts as right only when the cited page itself contains the fact. An event counts as found only when the timeline has it with the right date.

The visit figure comes from a separate set of 12 case files and 606 visits that were held back from development, so nothing was tuned on them. They include the hard cases: dates written in the body of a note, signature-only dates, scanning noise, interleaved bills and claim forms, fax covers and notes that mention the next visit.

No AI grades another AI here. The graders are ordinary code with their own tests, so the same output always gets the same score.

The same kind of check runs inside the product: an answer in Ask this case, or a non-medical event in the timeline, whose quote is not on its cited page is dropped before anyone sees it.