Methodology
Your regulatory dependencies, with a changelog. emendrix watches EU legislation, computes provision-level diffs when it is amended, and explains what changed in plain English where every sentence cites a provision you can click.
Finding which provisions an amendment touched scores micro-F1 0.963 over 17 transitions against labels the legislation publishes about itself — that is agreement with a reference set at article-or-annex granularity, and it is not a measure of whether any explanation is good. Every figure below is generated from a committed artifact, the dated report or the changelog documents themselves; none is typed in by hand.
The corpus on this site, counted
Counted over the 446 events and 4,948 changes this site renders, and over nothing else. The measured table below scores emendrix against a small labelled set of transitions instead: a different question over a different denominator, so a figure there is not a better reading of one here, and neither is adjusted for the other.
| Over the published corpus | Result | n | What it means — and what it does not |
|---|---|---|---|
| Changes the sources disagree about | 2,558 (0.517) | 4,948 changes | At least one of the three sources emendrix checks named the change and another did not, or two of them named different kinds of change. disputed is a fact about the detectors and not a statement about the law: nothing is dropped for it and nothing is overruled. The three rows below say what each disagreement is. |
| …of those, found in the text and not listed elsewhere | 1,034 (0.404) | 2,558 disputed changes | The provision's own words are on the page, before and after. What disagrees is a reference set's granularity: a blanket amendment is annotated once and the provisions it lands on are not enumerated. Not a change nobody could corroborate. |
| …of those, named elsewhere with no difference in the text | 1,491 (0.583) | 2,558 disputed changes | Nothing to quote on either side, so the row carries a location and the source that named it. Where neither an amending act nor its notice says when one of its instructions takes effect, that instruction is claimed in every consolidation window the act touches, so some of these are one instruction counted more than once. |
| …of those, found by every source that looked, called different things | 33 (0.013) | 2,558 disputed changes | The outright contradiction of the three: an insertion against a replacement, say. The smallest class in the corpus, and the one worth reading first. |
| Changes with no text on either side | 1,491 (0.301) | 4,948 changes | Neither a before nor an after text. Such a change exists only because a source other than the text comparison named a provision that comparison did not, so every one of them is also a disagreement above. Not a change whose text is withheld: there is none to show, and the row says so instead of disappearing. |
| Changes carrying an explanation | 3,389 (0.685) | 4,948 changes | Prose that survived the citation gate. Of the 3,457 changes carrying text at all, 3,389 (0.980) do; a change with no text is never sent to a model, because there would be no evidence to write about. Coverage, not quality: whether the sentences are faithful is the sampled row below, and prose written before a later parser fix corrected the text it describes is left visible rather than quietly withdrawn. |
Measured, not asserted
Every row carries what it means and what it does not. The rows are the same ones the repository's README publishes, rendered from the same committed report, so the two cannot disagree.
| Measure | Result | n | What it means — and what it does not |
|---|---|---|---|
| Localisation (which provisions changed) | P 0.952 / R 0.975 / F1 0.963 | 17 transitions | Structural diff against the corpus's own modification metadata, at article-or-annex granularity. Not a measure of whether the explanation is any good; precision is dragged down by blanket amendments the reference set annotates only once. |
| Instruction-parse agreement (cross-check) | P 0.917 / R 1.000 / F1 0.957 | 13 transitions | A third, independent reading of the same question, from the amending act's own prose. Weaker by construction: it cannot read a range or an instruction that delegates to an annex, and those are counted as unread, never approximated. |
| Disputed changes (signals disagree) | 0.120 | 100 changes | Changes at least one signal named and another did not. They ship marked disputed; a high rate is a data-quality finding, not a hidden failure. |
| Change classification | accuracy 1.000 | 79 units | Insert / modify / delete against the metadata's role codes, on units both signals named. The role semantics are empirical — the authority tables publish no labels. |
| Citation grounding (gate pass, no retry) | 1.000 | 55 changes | Of the changes with an explanation to check, the share whose first answer cited only provisions it had been offered. Citation validity, not explanation quality — a wrong sentence with a good citation passes. |
| Quote-fallback rate | 0.000 | 55 changes | The share where the gate replaced the model's prose with a verbatim quotation of the provision after one failed retry. Correct by construction and visibly marked; this is where grounding losses go, and no change is ever dropped. |
| Explanation faithfulness (sampled) | 0.800 | n = 20, LLM judge + spot review (pending) | Whether the shipped sentences follow from the evidence the writer was given: the prompt's deterministic-facts header and the two capped texts (the header joined the evidence base on 2026-08-09, so earlier rates answer a narrower question and are not comparable). The weakest number here: a sampled judgement by openrouter:openai/gpt-5.6-sol, which is a different and stronger model than the explainer but not an independent one. The raw fraction, no confidence interval — at this n an interval would be wider than the number is useful. |
Measured on 2026-09-05 at revision 6097989, from reports/eval/2026-09-05-6097989.md. The deterministic rows cover every transition in the labelled evaluation corpus (18 of 18 scored), which is a pinned set of transitions and not the corpus counted above; the model rows cover the pinned explanation subset only, because each change in it is one recorded call to a provider.
How it works
- 1 · WatchA watchlist of acts, matched against the Publications Office notification feed by identifier. Deterministic.
- 2 · FetchBoth consolidated versions and the amending act, as structured Formex 4 XML, through a disk cache. Deterministic.
- 3 · DeltaA structural diff over the provision trees: which articles and annexes were inserted, modified, deleted. No model.
- 4 · ExplainThe only model call in the loop, constrained to the two verbatim texts it is handed. It never decides what changed.
- 5 · GateEvery citation must resolve to a provision that was offered, or the sentence is replaced by a verbatim quotation. Deterministic.
Between Delta and Explain the changes are corroborated against two independent signals — the corpus's own modification metadata and a parse of the amending act's instruction prose — and anything they disagree about ships marked disputed rather than dropped. After Gate the result is committed to a git repository as Markdown and JSON. Every stage except Explain is deterministic Python.
How this site is built
A directory of static files, generated by emendrix site build from artifacts committed in the emendrix repository and in a changelog repository: the changelog documents this tool wrote and the dated evaluation report it was scored against. This deployment publishes that repository as the changelog repository; where it lives on the operator's machine is still never printed here.
That repository holds two layers and licenses them separately. The provision texts, titles, identifiers and dates quoted in it are © European Union, reused under Commission Decision 2011/833/EU, which permits reuse provided the source is acknowledged, and only the versions published in the Official Journal are authentic. Everything emendrix computed rather than quoted, the structural diffs, the classifications, the corroboration verdicts and the explanations, is licensed Creative Commons Attribution 4.0 International (CC BY 4.0): reuse it, commercially or not, provided the project is credited and changes are indicated. LICENCE-NOTICE.md in that repository states both layers and gives a citation form that pins a commit rather than a branch.
Every page is a rendering of things already committed, so the site can answer no question its inputs do not already contain, which is what makes no number here lacks provenance structural rather than aspirational. Two builds of one repository state produce the same bytes: nothing on the site reads a clock, reaches the network, or calls a model. One small script adds search; no link leaves the site except to EUR-Lex and the repositories this page names.
Each change carries a figure such as +1,204 −318. Those are characters: the characters inside the inserted and deleted spans of the word-level comparison shown under that change, or, for a provision too large to compare word by word, the characters in the whole lines that changed. The site computes it at build time from the committed texts, which is why it is not in the table above: it measures this page's own rendering and evaluates nothing. It is not a measure of legal effect. A two-character change can move a deadline by sixteen months and a two-thousand-character one can renumber a list, so nothing on the site calls a change minor, significant or substantial on the strength of it.