trainset: pick evidence excerpt at the actual provider statement; document provider-currency caveat

- _evidence_excerpt now selects the name occurrence nearest a role keyword
  (custodian/adviser/transfer agent/...) instead of the first match, so the
  excerpt carries the real service-provider sentence, not boilerplate.
- Documented the provider-currency caveat (gold N-CEN 2025Q3 vs current
  prospectus -> ~2-3% of triples name a *former* provider). A purely lexical
  former-flag proved unreliable (35% whole-book vs 1.7% excerpt-scoped) and was
  dropped in favour of an honest note; triples kept (entity+role still in text).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This commit is contained in:
Florian Herzog 2026-06-10 14:19:02 +02:00
parent 991715ab76
commit 216fd9876c

View File

@ -1165,6 +1165,17 @@ EXCERPT_RADIUS = 600 # chars around each grounded name
_ROLE_HINT = ("custodian", "custody", "advis", "transfer agent", "administrat",
"distributor", "underwriter", "serves as", "acts as", "series of")
# NOTE on provider-currency: because the prospectus is fetched at its CURRENT
# version while the gold is N-CEN 2025Q3, a few percent of triples name a provider
# the text describes as a *former* one (e.g. "former fund administrator ... through
# September 5, 2025"). Manual inspection puts this at ~2-3% of distinct triples
# (concentrated in adviser/sub-adviser/administrator; ~1% for custodian). A purely
# lexical "former" flag proved unreliable -- scoped to the whole book it tagged 35%
# (any stray "former" near any of a name's hundreds of occurrences), scoped to the
# chosen excerpt it tagged 1.7% (the excerpt often picks a different, clean
# occurrence). The entity+role is still present in the text, so these remain valid
# extraction signal; the residual time-mismatch is documented rather than flagged.
def _evidence_excerpt(text_norm_full, raw_text, label, predicate=None):
"""Cut a coherent excerpt of raw_text around the BEST mention of the entity.
@ -1233,7 +1244,7 @@ def build_trainset():
if not kept:
n_skip += 1
continue
# collect excerpts around each grounded object name; merge overlaps
# collect excerpts around each grounded object name; merge overlaps.
spans = []
for t in kept:
lbl = ents.get(t["o"], {}).get("label", t["o"])