P0 complete · thesis killed The gate was written before the work. It returned KILL. This page reports that.

Sanctions watchlists are written in Latin script.
The trade corridor under scrutiny is written in Cyrillic.

Between those two facts sits a matching problem with no canonical answer: one Cyrillic name has many legitimate Latin spellings, and no standard governs which one a record used. I built an open benchmark to measure how badly existing matchers handle it, with a numeric kill criterion fixed in advance. Dedicated open-source matchers fail exactly as predicted. Then a general-purpose language model, given a two-sentence prompt and no linguistic resources at all, solved the whole thing — and the criterion fired.

Щербаковone person

ShcherbakovBGN/PCGN
ShcherbakovICAO 9303
ŜerbakovISO 9
Ščerbakovscholarly
ShherbakovGOST 7.79-B
Scherbakovad-hoc

Six Latin forms of one surname, five of them produced by a published standard. A matcher that romanises once picks a lane and mismatches every record written in another.

5,102labelled pairs

19baselines scored

7languages

KILLgate verdict

Why fuzzy matching lies here

Fuzzy string matching assumes edit distance correlates with the probability of being a different entity. Across scripts, it does not — and the failure is not a tuning problem, because legitimate spelling variance and genuine identity difference sit at the same edit distance.

Shcherbakov ↔ Scherbakov

edit distance 1

Same person. Two romanisation systems disagreeing about щ.
Ivanov ↔ Ivanova

edit distance 1

Usually different people. Gendered forms of a surname shared by millions.
Aleksandr ↔ Oleksandr

edit distance 1

Genuinely ambiguous. Russian and Ukrainian forms of one given name; the records do not say which.

Layered on top: Kazakh exists simultaneously in Cyrillic, in competing Latin orthographies and in a Russian-mediated romanisation that collapses ә ұ ө ғ қ ң onto their nearest Russian letters. Patronymics appear, vanish and abbreviate. Turkic particles -ұлы / -қызы coexist with Russified -ович / -овна. Surnames are gendered, and a feminine ending stripped by one system is the same person while the gendered pair is two.

What was built

A labelled corpus of name pairs and a harness that scores any matcher against it. Hard negatives are the whole game: a corpus of easy negatives makes every matcher look excellent and teaches nothing.

Positives

Pairs known to be the same entity.

OFAC SDN alias sets: Cross-script and cross-romanisation variants curated by government analysts. Weak aliases excluded.
Wikidata labels: CC0. The ru/kk/uk and en labels of one item are the same entity by construction.
Rule-generated: Published transliteration tables, for rare-letter coverage. Capped at 30% of positives — synthetics only test what we already modelled.

Hard negatives

Different entities, string-similar on purpose. 72% of negatives.

Same surname: Different given name, distinct entities.
Gendered pairs: Иванов / Иванова, unrelated people.
Romanisation collisions: Surnames like Әбиев and Абиев that collapse to one Latin form — the sharpest negative available.
Shared patronymic: Same surname and patronymic, different person.

Discipline

The parts that decide whether it is honest.

Entity distinctness: Negatives require both dates of birth present and different — distinct identifiers alone are not proof.
Unresolvable pairs excluded: Identical names for different people are correctly labelled but unanswerable from names; including them would add a fixed error floor that measures nothing.
Conservative tagging: A romanisation standard is named only when it reproduces every token exactly. Under-attribution beats a confident wrong label.
Determinism: Byte-identical rebuild; every deterministic baseline reproduces exactly.

Corpus v1.0 · sha256 bdbc4e8522b3a525… · 5,102 pairs, 2,491 positive / 2,611 negative, 1,911 hard negatives, 745 synthetic positives · ru 2926 · uk 694 · be 520 · kk 497 · ky 198 · tg 145 · uz 122

The result that matters

Recall per phenomenon at an operating point holding the false-positive rate at 1% on hard negatives. The left column is the strongest dedicated matcher — OpenSanctions logic-v2, which romanises properly via rigour. The right column is a general-purpose model asked one question in plain English.

Phenomenon	n	Best dedicated matcher	Frontier LLM
`patronymic:abbreviated`	139	0.137	0.993	+0.856
`patronymic:turkic` `-ұлы`/`-қызы`	131	0.153	0.992	+0.840
`patronymic:dropped`	353	0.210	0.983	+0.773
`romanisation:be-bgn` (Belarusian)	29	0.414	1.000	+0.586
`token:dropped`	633	0.419	0.946	+0.528
`patronymic:form-substituted`	105	0.438	0.991	+0.552
`romanisation:uk-kmu55` (Ukrainian)	63	0.460	1.000	+0.540
`romanisation:ad-hoc`	582	0.490	0.988	+0.498
`legal-form` `ТОО`/LLP	530	0.534	0.959	+0.424
`kazakh:cyrillic-latin`	157	0.592	0.994	+0.401
`romanisation:bgn` (Russian)	366	0.757	0.989	+0.232
`romanisation:icao` (passport MRZ)	128	0.773	0.984	+0.211

Recall at the 1% FPR operating point, hard-negative slice. Left: nomenklatura/logic-v2. Right: llm-judge/claude-opus-5. Phenomena with fewer than 8 positives are omitted. Every failure mode this project was designed to fix is closed by the model on the right.

Every baseline, hard-negative slice

Baseline	Family	max-F1	Recall @ 1% FPR	Recall @ 0.1% FPR	ROC-AUC	FPR to reach 95% recall
`llm-judge/claude-opus-5`	LLM	0.9859	0.9775	0.8414	0.9960	0.004
`llm-judge/claude-sonnet-5`	LLM	0.9820	0.9627	0.5680	0.9918	0.007
`nomenklatura/logic-v2`	OpenSanctions	0.8523	0.5853	0.3549	0.8711	0.918
`nomenklatura/name-based`	OpenSanctions	0.8044	0.4111	0.1437	0.8332	0.828
`embedding/LaBSE`	embedding	0.7864	0.4460	0.2489	0.8171	0.916
`icu-any-latin-ascii+levenshtein`	ICU translit	0.7681	0.3770	0.1835	0.7825	0.864
`icu+soundex`	English phonetic	0.7333	0.3035	0.0000	0.7311	1.000
`nomenklatura/ofac`	OpenSanctions	0.7228	0.2268	0.2268	0.6134	1.000
`exact-match-nfkc`	floor	0.7228	0.0000	0.0000	0.5000	1.000

Selected rows; the full 19-baseline table is in the repository. Read max-F1 against 0.7228, not zero — calling every pair a match scores 0.7228 on this slice, which is why exact-match-nfkc ties a real matcher on F1 while its ROC-AUC of 0.5000 shows it never discriminates at all.

The false-positive cost is not close

To hold 95% recall, the LLM needs a false-positive rate of 0.004. The best dedicated matcher needs 0.918 — it must flag almost every hard negative to get there. That is the difference between a usable control and an unusable one, at the same recall.

English phonetics fail, as predicted

icu+soundex reaches 0.0000 recall at 0.1% FPR and 0.7313 ROC-AUC. Soundex and Metaphone encode English phonology; they collapse distinctions that matter in Slavic and Turkic names and preserve ones that don't. Any library defaulting to them for this data is mis-applied.

No transliteration, no chance

The ofac matcher scores non-zero on 4 of 1,898 cross-script positives. It compares Latin tokens and does not romanise, so the cross-script case is essentially invisible to it — ROC-AUC 0.6130, barely above chance.

The verdict: kill it

KILL

The gate was written before any work, with numeric criteria, precisely so a bad result could not be argued away afterwards: kill the idea if the best baseline reaches 0.93 max-F1 and 0.95 recall at 1% FPR. A frontier LLM judge reached 0.9859 and 0.9775. Both criteria are met. The premise does not survive.

What exactly was disproved

The linguistic knowledge is not scarce

The thesis was that this problem needs Slavic and Turkic linguistic knowledge that vendors' teams do not have, and that modelling the transliteration transformation properly is therefore defensible. A general-purpose model has that knowledge already — no transliteration tables, no romanisation standards, no name-structure parser, a two-sentence prompt. It closed Turkic patronymics (0.153 → 0.992), Ukrainian romanisation (0.460 → 1.000) and Kazakh multi-alphabet (0.592 → 0.994) in one shot.
The false-positive story was wrong twice over

The pitch was that incumbents flood compliance teams with false positives. At a 1% FPR operating point the best dedicated matcher is precise — its failure is missed matches, roughly 41% of true pairs. And the LLM is both more precise and more complete: 8 false positives across 341 romanisation collisions, zero across 380 gendered pairs. There was no false-positive gap to sell into.
The obvious escape route was tested and closed

The corpus is built from public figures, so the model might be recalling known aliases rather than reasoning. If that were the mechanism, rule-generated Latin forms — which appear in no real record anywhere — should score worse. They score better: 0.9987 on synthetic pairs against 0.9600 on real OFAC alias sets. Memorisation would have been the more comfortable answer; it is not the answer.

Three things that are true and still do not save it

An LLM cannot be the scorer of record in an audited pipeline. It is non-deterministic, cannot be pinned to greedy decoding — the current frontier models reject a temperature parameter outright — and cannot be version-frozen for replay, which regulators require. But that argues for distilling or caching LLM-quality judgements deterministically. That is a different company from this one.
Cost and latency at screening volume. The LLM took 1,341s of wall clock for 5,102 pairs at 16-way concurrency, against 2.6s locally for the dedicated matcher, and consumed 619k input plus 263k output tokens. A real operational gap — which narrows every year, and is a poor thing to build a company against.
This corpus is not a production screening queue. It is roughly half positives and its negatives were chosen by me. The hard slice is a lower bound on matcher quality, not a false-positive-rate estimate for anyone's live system.

What survives

The benchmark. It is a reproducible, labelled, openly-published measurement of a problem nobody had quantified, and it now carries a finding worth reading: frontier language models solve cross-script name matching that dedicated open-source matchers miss by 39 points of recall at a fixed false-positive rate — with the per-phenomenon taxonomy showing exactly where and why the older methods break.

It cost weeks, not months, and it answered the question it was built to answer. That is the system working, not failing. The engine was never written; src/ is empty and should stay empty.

5,102 pairs, 19 baselines, run 10 August 2026 against the OFAC SDN list published 24 July 2026. Sanctions data decays weekly; treat every figure as of those dates. Method, corpus, harness and raw per-pair scores are in the repository so anyone can re-run this and disagree.

Reproducibility

Source snapshots are committed, so the corpus rebuilds offline and byte-identically. The report is generated from the metrics file rather than written by hand, so no figure in prose can drift from the measurement. Every baseline is open source or textbook — nothing behind a licence that forbids benchmarking, and no commercial product was evaluated.

Method, data and harness on GitHub ↗

Scope: defensive use only

A tamga is the seal Turkic and Mongol peoples stamped to mark identity and lineage — the oldest answer to "is this the same party?".

This work exists to help regulated parties find sanctioned counterparties. It will not become a tool for testing whether a name variant evades screening: no alias generators positioned around evasion, no "will this clear a watchlist?" checkers, no scoring surface that ranks variants by likelihood of not being flagged. Any matcher built on this benchmark is a screening aid; responsibility for compliance and for the disposition of every alert stays with the screener.